johhnyUA
Legendary
Offline
Activity: 2422
Merit: 1845
Crypto for the Crypto Throne!
|
|
December 05, 2019, 03:59:17 PM |
|
Hello, how can i see deleted posts which belongs to specific user?
I wanna try to find really epic joke which was deleted by mods. Thx for answer.
|
|
|
|
LoyceV (OP)
Legendary
Offline
Activity: 3416
Merit: 17265
Thick-Skinned Gang Leader and Golden Feather 2021
|
|
December 05, 2019, 06:31:46 PM |
|
Hello, how can i see deleted posts which belongs to specific user? Yes. See http://loyce.club/archive/members/It still gets only manual updates, I'm running an update now. This will probably take a few hours.
|
|
|
|
FontSeli
|
|
December 23, 2019, 05:39:26 PM |
|
Hello, how can i see deleted posts which belongs to specific user? Yes. See http://loyce.club/archive/members/It still gets only manual updates, I'm running an update now. This will probably take a few hours. Hello. I can't use your service. After clicking on the link gives 404 error. Is this a temporary problem?
|
Celebrate Julian's freedom!
|
|
|
LoyceV (OP)
Legendary
Offline
Activity: 3416
Merit: 17265
Thick-Skinned Gang Leader and Golden Feather 2021
|
|
December 23, 2019, 06:40:12 PM |
|
Hello. I can't use your service. After clicking on the link gives 404 error. Is this a temporary problem? Yes. I'm still working on restoring my data on another server. I have a temporary fix for this temporary problem: add aws. in front of loyce.club: http://aws.loyce.club/archive/posts/ I don't make it clickable, because the link will expire the moment I get all my data back online. So feel free to view it, but please don't post links to the temporary location. Once done, archived posts will be available in a matter of seconds (instead of a minute in the old situation).
|
|
|
|
nutildah
Legendary
Offline
Activity: 3094
Merit: 8303
Happy 10th Birthday to Dogeparty!
|
|
December 24, 2019, 05:04:50 AM |
|
Hey Loyce, are you aware that BPIP is currently redirecting to your home page? Kind of weird... I know its off-topic here but I thought you should know.
|
|
|
|
LoyceMobile
|
|
December 24, 2019, 06:33:36 AM |
|
Hey Loyce, are you aware that BPIP is currently redirecting to your home page? Kind of weird... I know its off-topic here but I thought you should know.
I didn't know until you told me. Vod got hit by a HP bug that wiped his drives, guess this is a temporary solution until he restores s backup.
|
|
|
|
FontSeli
|
|
December 24, 2019, 11:01:03 AM |
|
I have a temporary fix for this temporary problem: add aws. in front of loyce.club: http://aws.loyce.club/archive/posts/ It works! Thank you for your help and for your great services!
|
Celebrate Julian's freedom!
|
|
|
logfiles
Copper Member
Legendary
Offline
Activity: 2086
Merit: 1755
Top Crypto Casino
|
|
January 11, 2020, 11:35:28 AM |
|
Hi LoyceV I am trying to look for archived posts of a particular user from http://loyce.club/archive/members because his account was nuked but the Profile ID doesn't show up The last update was on 2020-01-04 16:27 but on http://loyce.club/archive/posts the last update was just today, 2020-01-11 12:31. Could it be an irregularity or a technical error preventing an update on the members link.
|
|
|
|
TECSHARE
In memoriam
Legendary
Offline
Activity: 3318
Merit: 2008
First Exclusion Ever
|
|
January 11, 2020, 11:42:49 AM |
|
Bump (I don't want to delete my previous bump, because it's merited).
I think you keep merit even if a post is deleted? Some one correct me if I am wrong.
|
|
|
|
TryNinja
Legendary
Offline
Activity: 2940
Merit: 7368
|
|
January 11, 2020, 11:44:29 AM |
|
Bump (I don't want to delete my previous bump, because it's merited).
I think you keep merit even if a post is deleted? Some one correct me if I am wrong. You do. He’s not worried about that. He probably wants to keep the reference for which post earned him his merit. Otherwise, it says it was for post “Deleted”.
|
|
|
|
LoyceV (OP)
Legendary
Offline
Activity: 3416
Merit: 17265
Thick-Skinned Gang Leader and Golden Feather 2021
|
|
January 11, 2020, 12:00:26 PM Last edit: January 11, 2020, 12:27:23 PM by LoyceV |
|
I still haven't fully tested this part yet, so updates are only manually started. I'm running an update now, it shouldn't take too long to process a week's worth of posts. Update: see http://loyce.club/archive/members/274/2743460.htmlI think you keep merit even if a post is deleted? Some one correct me if I am wrong. Correct. And TryNinja is also right, I just don't like my "Merit earned for deleted posts"-counter to go up.
|
|
|
|
LoyceV (OP)
Legendary
Offline
Activity: 3416
Merit: 17265
Thick-Skinned Gang Leader and Golden Feather 2021
|
|
January 20, 2020, 01:02:51 PM Last edit: January 20, 2020, 03:27:18 PM by LoyceV |
|
I've been thinking about expanding my archived posts to all posts that haven't been deleted yet. It would be useful for a case like this. It requires scraping a couple million pages, and storing 50+ million posts. I can limit the number of files on the server by storing 10 or 100 posts per page. Would this be useful?
|
|
|
|
LoyceV (OP)
Legendary
Offline
Activity: 3416
Merit: 17265
Thick-Skinned Gang Leader and Golden Feather 2021
|
|
February 02, 2020, 06:05:42 PM Last edit: October 08, 2020, 10:02:18 AM by LoyceV |
|
I've been thinking about expanding my archived posts to all posts that haven't been deleted yet. An update: I have started this project! Measured in scraping time, it's the biggest project I ever started. In the past 9 days, I've scraped about 4% of all data, so I expect to complete this around August. There's also a chance I'll run out of disk space because of the millions of large posts made by bounty spammers, but I'll deal with that when it happens. Sneak preview: http://loyce.club/archive/oldposts/How to use: - Find the msgID you need. Let's use 28228
- Remove the last 5 digits from the msgID to get the directory name (if there are 5 or less digits, use 0): 0
- Replace the last 2 digits of the msgID by xx, and add .html (if there are 2 or less digits, use 0xx): 282xx.html
- Add "#msg" and the msgID: #msg28228
- Put everything together and go to http://loyce.club/archive/oldposts/0/282xx.html#msg28228
Limitations- Currently, the first 2.1 million posts are available.
- I'll scrape the first 5.21 million topics and all posts in there.
- That means I'll archive 53.36 million posts, this partially overlaps with my scraper for new posts.
- This is a one-time thing, I won't update it with newer posts (I scrape unedited versions for those).
- The time "scraped on" is Amsterdam time.
If no username is mentioned, it's either "Anonymous" or "random". I forgot those exist when I started scraping, and it's not important enough to start over.
|
|
|
|
LoyceV (OP)
Legendary
Offline
Activity: 3416
Merit: 17265
Thick-Skinned Gang Leader and Golden Feather 2021
|
|
February 03, 2020, 05:52:37 PM Last edit: February 09, 2020, 07:24:26 PM by LoyceV |
|
Something failed in the above scrape, the Wall Observer thread stopped scraping after page 2628. I thought I had the "last page detection" working, but there still seems to be a flaw.
|
|
|
|
Vod
Legendary
Offline
Activity: 3808
Merit: 3115
Licking my boob since 1970
|
|
February 09, 2020, 04:01:57 PM |
|
LoyceV, are you saving the raw data, or just converting it?
|
|
|
|
nutildah
Legendary
Offline
Activity: 3094
Merit: 8303
Happy 10th Birthday to Dogeparty!
|
|
February 09, 2020, 04:06:33 PM |
|
I've been thinking about expanding my archived posts to all posts that haven't been deleted yet. It would be useful for a case like this. It requires scraping a couple million pages, and storing 50+ million posts. I can limit the number of files on the server by storing 10 or 100 posts per page. Would this be useful? Yes, I'm all for it!
|
|
|
|
LoyceV (OP)
Legendary
Offline
Activity: 3416
Merit: 17265
Thick-Skinned Gang Leader and Golden Feather 2021
|
|
February 09, 2020, 07:24:42 PM |
|
LoyceV, are you saving the raw data, or just converting it? I'm only saving the raw post (one line from the raw HTML), and my own header like this one (post number, link to post, link to my archive, by username and scraping time). Yes, I'm all for it! It's started already
|
|
|
|
LoyceV (OP)
Legendary
Offline
Activity: 3416
Merit: 17265
Thick-Skinned Gang Leader and Golden Feather 2021
|
|
February 18, 2020, 08:34:49 PM |
|
I now have the first 6.1 million Bitcointalk posts archived. Data processing took longer than expected, but it's published now. See link above.
|
|
|
|
LoyceV (OP)
Legendary
Offline
Activity: 3416
Merit: 17265
Thick-Skinned Gang Leader and Golden Feather 2021
|
|
February 19, 2020, 08:47:48 PM |
|
Update: I finally got to work on a topic-view. It'll be published at http://loyce.club/archive/topics/, and it's currently crunching data on 1.9 million posts. I'm downloading the thread titles that I don't have yet, and that part takes a lot of time. This viewer should make it much easier to find back all posts made in a certain thread. Obviously this is also limited to posts made after I started scraping.
|
|
|
|
dkbit98
Legendary
Offline
Activity: 2338
Merit: 7384
|
|
February 21, 2020, 04:30:48 PM |
|
If it is not a secret, how much data space is needed for all that millions of posts? And is there a way to use some compression?
|
|
|
|
|