Title: [get yours] Images from imgur + All links to your own posts Post by: LoyceV on May 18, 2023, 05:18:51 PM yehey another imgur topic As you've probably noticed, images from imgur.com no longer work on Bitcointalk. That's why I've downloaded 801,550 images from imgur, which were embedded on the forum. Downloading took 3 days using 4 different servers, and another 3 days to process the data. I used the links from my unedited posts archive (https://bitcointalk.org/index.php?topic=5167469.0), which means I'm missing all images that were edited in later. After some cleaning, I'm left with 5 directories: Code: 4.1M ./img.imgur.com I made a .zip for each user. It includes all images related to that account (including quoted images). After extracting the .zip file, "list_of_posts.html" shows links to all images, posts (some of which have been deleted) and archived posts. Example: tmp.loyce.club/imagebackup/459836.html (http://tmp.loyce.club/imagebackup/459836.html) (note that the image links don't work without downloading the .zip). Replace my userID by your own: Code: tmp.loyce.club/imagebackup/USERID.html If there's no file with your userID, that means I found no embedded imgur-images for your account. 9999999999 (http://tmp.loyce.club/imagebackup/9999999999.html) is for posts without userID. Update: if someone wants the full data, just ask. No need to download all separate .zip files. Bonus: images posted/quoted by most different accounts (full disclosure: I counted 1 more than needed for each file): Code: 292x https://i.imgur.com/ETbGPPP.png Suggested new image host: TalkImg.com - Image hosting for BitcoinTalk (https://bitcointalk.org/index.php?topic=5450546.0). Title: Re: [get yours] Images from imgur Post by: Little Mouse on May 18, 2023, 05:27:21 PM We can get all the images from here and upload them manually one by one by going to respective threads. I have a lot of pages and have uploaded a lot of images. I don't know how it's going to help me. It's going to take a full day bitcointalk schedule from me. Well, I'm aware about the Tryninja script. I will possibly have no choice but to use that one.
Title: Re: [get yours] Images from imgur Post by: LoyceV on May 18, 2023, 05:32:05 PM I don't know how it's going to help me. The main goal was to avoid this:Quote from: https://techcrunch.com/2023/04/20/imgur-will-ban-explicit-images-on-its-platform-this-month/ In addition to explicit images, Imgur is also removing old and unused photos that are not tied to an account. As a Twitter user noted, Imgur has been used for years to upload photos without an account and post links to different kinds of sites. Removing those images might result in a lot of dead links all over the internet. The changes are going to take effect this Month on the 15th.Title: Re: [get yours] Images from imgur Post by: decodx on May 18, 2023, 06:06:19 PM I made a .zip for each user. It includes all images related to that account (including quoted images). After extracting the .zip file, "list_of_posts.html" shows links to all images, posts (some of which have been deleted) and archived posts. It's great that you've provided a full list of all the posts where the images were used. This is incredibly helpful. Thank you so much! Please, don't delete this... At least for a while. ;) Title: Re: [get yours] Images from imgur Post by: un_rank on May 18, 2023, 09:25:38 PM This has to be the forum with some of the most industrious members of any offline forum. Problems arise and there are always hands on deck to try and solve it with no incentive to do so besides helping out other users.
Great work LoyceV, I may have just one or two posted images but this would be useful to many forum users. - Jay - Title: Re: [get yours] Images from imgur Post by: PowerGlove on May 18, 2023, 10:22:10 PM (...) That's why I've downloaded 801,550 images from imgur, which were embedded on the forum. Downloading took 3 days using 4 different servers, and another 3 days to process the data. (...) I expect a lot of people to not fully realize how valuable what you've done is; nice work!Having a copy of these images safely tucked away means that the Imgur issue can be worked on without rushing things. I hope theymos settles on an approach that involves URL substitutions being made by the image proxy, because that would fix things forum-wide without anyone having to do anything. Title: Re: [get yours] Images from imgur Post by: joker_josue on May 18, 2023, 11:14:09 PM First of all congratulations for this excellent work. ;)
I would like to know if you think there would be any way to confirm if the image is still in use in the forum or not? That would perhaps force a new scraping. :-\ Title: Re: [get yours] Images from imgur Post by: Husires on May 19, 2023, 12:18:48 AM Can you display the images by topic ID or is it a little difficult? If this is possible, all that the person needs is to go to your site with the link to the topic to see the old images, and this will be useful for those who wish to track an old non-active topic.
Browsing the forum has become impossible, so thank you for making it possible to read many old threads. Title: Re: [get yours] Images from imgur Post by: vapourminer on May 19, 2023, 12:33:05 AM this is perfect to me as i am lazy af; now i have everything i need to edit at my leisure.
sweet. thanks much Title: Re: [get yours] Images from imgur Post by: Timelord2067 on May 19, 2023, 01:36:26 AM So, I delete a 20Mb gif from a post of mine (quoted from someone else) and that is then deleted from the ZIP file?
Title: Re: [get yours] Images from imgur Post by: LoyceV on May 19, 2023, 05:51:56 AM I would like to know if you think there would be any way to confirm if the image is still in use in the forum or not? That would perhaps force a new scraping. :-\ Scraping everything again would be the only way to know for sure, but I'm not going to do that again. At one page per second, it took me months to download all old posts.Can you display the images by topic ID or is it a little difficult? I could, but I don't think it's worth the effort. It would mean creating another set of .zip files (~150 GB) and takes quite a while to sort it out.So, I delete a 20Mb gif from a post of mine (quoted from someone else) and that is then deleted from the ZIP file? No, I can't update the .zip files. That would require rescraping all posts.Title: Re: [get yours] Images from imgur Post by: joker_josue on May 19, 2023, 11:47:17 AM Scraping everything again would be the only way to know for sure, but I'm not going to do that again. At one page per second, it took me months to download all old posts. Clear! Does not make sense. Just trying to think of ways to try to avoid importing images that are no longer being used. :-\ Title: Re: [get yours] Images from imgur Post by: LoyceV on May 19, 2023, 12:06:56 PM Just trying to think of ways to try to avoid importing images that are no longer being used. :-\ Apart from duplicates, I don't think any of the images is no longer used. Take this post (https://bitcointalk.org/index.php?topic=33031.msg525707#msg525707) for example: chances are nobody has seen it in years, but it's part of the forum's history.Title: Re: [get yours] Images from imgur Post by: Aikidoka on May 19, 2023, 12:15:07 PM Thanks LoyceV, This is really valuable and highly appreciated. I have downloaded the zip file containing all the images I have posted on the forum via Imgur. I'll try to update my previous posts with the broken Imgur links and replace them with TalkImg links. :)
Title: Re: [get yours] Images from imgur Post by: Timelord2067 on May 19, 2023, 12:35:09 PM Just trying to think of ways to try to avoid importing images that are no longer being used. :-\ Threads that aren't locked but are still active would be where I'd start, then if users reach out to request threads that are locked, but are referenced would also be high on my list. Working backwards from today to the dawn of Unless it's specifically requested to be updated, then wait for the request. I wouldn't go looking for work that isn't needed or requested. Title: Re: [get yours] Images from imgur Post by: joker_josue on May 19, 2023, 01:33:54 PM Apart from duplicates, I don't think any of the images is no longer used. Take this post (https://bitcointalk.org/index.php?topic=33031.msg525707#msg525707) for example: chances are nobody has seen it in years, but it's part of the forum's history. This image is still in use, even though it could be weeks before anyone sees this post. I'm talking about images that are no longer on the forum, for example posts that have already been deleted. I wouldn't go looking for work that isn't needed or requested. Initially this is what is intended. But, unfortunately there will be hundreds or thousands of images that users will not do anything about. So what can we do? Title: Re: [get yours] Images from imgur Post by: RickDeckard on May 19, 2023, 02:30:56 PM This is a remarkable work LoyceV, like always. Considering the amount of resources that you had to use, and the amount of space that it currently takes on your server(s), can I kindly ask you for a way to download those files and act as a backup in case something bad happens? At this stage where the forum is at the moment - regarding deciding what solution to employ - I see having a backup of this archive as a really important thing to do.
Title: Re: [get yours] Images from imgur Post by: LoyceV on May 25, 2023, 07:24:53 AM Imgur-images are working again, but we don't know for how long. So get yours :)
Title: Re: [get yours] Images from imgur + All links to your own posts Post by: dkbit98 on May 25, 2023, 08:11:02 PM Imgur-images are working again, but we don't know for how long. So get yours :) I suspect they will bring them down again in near future, so backing everything up was a good idea.This is probably delayed because of feedback they received from Reddit community that is much bigger than bitcointalk. When I first saw images working again I was thinking theymos did some kind of magic trick with your help and with script. :D Title: Re: [get yours] Images from imgur + All links to your own posts Post by: LoyceV on May 27, 2023, 06:37:18 AM From another topic:
I was even trying to make out time someday to start deleting all my posts history that has an image on them because of Imgur's invalid image display. That's the opposite of archiving :oTitle: Re: [get yours] Images from imgur + All links to your own posts Post by: LoyceV on June 02, 2023, 09:42:23 AM For a short while, imgur-images were working again, but it didn't last.
Title: Re: [get yours] Images from imgur + All links to your own posts Post by: dkbit98 on June 02, 2023, 06:26:33 PM Bump! For a short while, imgur-images were working again, but it didn't last. I heard one theory that this could be related with recently publicly released leaked images of Hunter Biden, suddenly bunch of image hosting providers stopped working correctly.I don't know if this is true or not, but it should be checked if issue is again related only with bitcointalk forum or with everything else. Title: Re: [get yours] Images from imgur + All links to your own posts Post by: RickDeckard on June 02, 2023, 07:23:42 PM Bump! For a short while, imgur-images were working again, but it didn't last. I heard one theory that this could be related with recently publicly released leaked images of Hunter Biden, suddenly bunch of image hosting providers stopped working correctly.I don't know if this is true or not, but it should be checked if issue is again related only with bitcointalk forum or with everything else. Title: Re: [get yours] Images from imgur + All links to your own posts Post by: yhiaali3 on June 02, 2023, 07:45:39 PM Imgur can no longer be trusted again even if it works fine again, I personally moved all my old images to talkimg.com and replaced all old links with TryNinja script from this thread:
[Script] Imgur to TalkImg - automatically fix your broken images (https://bitcointalk.org/index.php?topic=5452662.0) This is currently the best option for me. Title: Re: [get yours] Images from imgur + All links to your own posts Post by: LoyceV on June 03, 2023, 05:22:39 AM I heard one theory that this could be related with recently publicly released leaked images of Hunter Biden, suddenly bunch of image hosting providers stopped working correctly. Unlikely. The images are still accessible if you go to the URL directly.Title: Re: [get yours] Images from imgur Post by: RickDeckard on June 14, 2023, 10:34:27 PM This is a remarkable work LoyceV, like always. Considering the amount of resources that you had to use, and the amount of space that it currently takes on your server(s), can I kindly ask you for a way to download those files and act as a backup in case something bad happens? At this stage where the forum is at the moment - regarding deciding what solution to employ - I see having a backup of this archive as a really important thing to do. I've just realized that I didn't update this comment - LoyceV has shared with me a copy of the data which I will keep (indefinitely) (~ 143GB) (thank you!). I do recommend anyone with some GB to spare to also do it in order to increase the safety of the data.Title: Re: [get yours] Images from imgur + All links to your own posts Post by: DYING_S0UL on September 04, 2023, 01:31:13 PM Bump So what image hosting site should we use now? At first I randomly google and choose whatever I though best. Currently I am using https://www.talkimg.com/. I heard it was specifically created for Bitcointalk.Title: Re: [get yours] Images from imgur + All links to your own posts Post by: LoyceV on September 04, 2023, 01:52:02 PM So what image hosting site should we use now? At first I randomly google and choose whatever I though best. Currently I am using https://www.talkimg.com/. I heard it was specifically created for Bitcointalk. When in doubt, read the OP:Suggested new image host: TalkImg.com - Image hosting for BitcoinTalk (https://bitcointalk.org/index.php?topic=5450546.0). Title: Re: [get yours] Images from imgur + All links to your own posts Post by: DYING_S0UL on September 04, 2023, 01:54:13 PM So what image hosting site should we use now? At first I randomly google and choose whatever I though best. Currently I am using https://www.talkimg.com/. I heard it was specifically created for Bitcointalk. When in doubt, read the OP:Suggested new image host: TalkImg.com - Image hosting for BitcoinTalk (https://bitcointalk.org/index.php?topic=5450546.0). Sorry man I missed the last sentence. I just woke up from sleep. Title: Re: [get yours] Images from imgur + All links to your own posts Post by: LoyceV on May 08, 2024, 11:34:58 AM Get yours now! Or, get it now :P |