[get yours] Images from imgur + All links to your own posts

[LoyceV]

yehey another imgur topic

As you've probably noticed, images from imgur.com no longer work on Bitcointalk. That's why I've downloaded 801,550 images from imgur, which were embedded on the forum. Downloading took 3 days using 4 different servers, and another 3 days to process the data. I used the links from my unedited posts archive, which means I'm missing all images that were edited in later.
After some cleaning, I'm left with 5 directories:

Code:

4.1M    ./img.imgur.com
35M     ./i.stack.imgur.com < this one is still working
1.8M    ./www.imgur.com
132M    ./imgur.com
146G    ./i.imgur.com
146G

99.9% of the images come from i.imgur.com.

I made a .zip for each user. It includes all images related to that account (including quoted images). After extracting the .zip file, "list_of_posts.html" shows links to all images, posts (some of which have been deleted) and archived posts.

Example:
tmp.loyce.club/imagebackup/459836.html (note that the image links don't work without downloading the .zip).
Replace my userID by your own:

Code:

tmp.loyce.club/imagebackup/USERID.html

On top of that page, you'll find a link to a .zip file. Or replace "html" by "zip" to go there directly.
If there's no file with your userID, that means I found no embedded imgur-images for your account.
9999999999 is for posts without userID.
Update: if someone wants the full data, just ask. No need to download all separate .zip files.

Bonus: images posted/quoted by most different accounts (full disclosure: I counted 1 more than needed for each file):

Code:

292x https://i.imgur.com/ETbGPPP.png
219x https://i.imgur.com/OhqIX6L.png
161x https://i.imgur.com/kw0dNZL.png
154x https://i.imgur.com/VLuTL9G.png
138x https://i.imgur.com/JC7kWkh.png
133x https://i.imgur.com/7Qld6sd.png
131x https://i.imgur.com/3tjawcx.png
130x https://i.imgur.com/PCufaMX.png
127x https://i.imgur.com/7yIpdEv.png
119x https://i.imgur.com/EDloGrT.png
106x https://i.imgur.com/PeGCkdp.png
105x https://i.imgur.com/epkPFTa.png
103x https://i.imgur.com/pa8YM7i.jpg
100x https://i.imgur.com/pgqaTer.jpg
 99x https://i.imgur.com/ONj4pkf.png
 99x https://i.imgur.com/sIRXPW8.jpg
 98x https://i.imgur.com/UKkuLO6.png
 97x https://i.imgur.com/6p14ByC.jpg
 95x https://i.imgur.com/CF44dTs.png
 92x https://i.imgur.com/9g7fPLN.png
 90x https://i.imgur.com/7drHiqr.gif
 88x https://i.imgur.com/vtMP7UZ.png
 88x https://i.imgur.com/HTfbFXl.png
 88x https://i.imgur.com/65ks2FH.png
 87x https://i.imgur.com/3gug5A7.png
 86x https://i.imgur.com/sMmagMS.png
 86x https://i.imgur.com/2KGMO6k.jpg
 83x https://i.imgur.com/ZkMo9ZC.png
 82x https://i.imgur.com/op1C1hB.png
 80x https://i.imgur.com/FFiSZGG.png
 79x https://i.imgur.com/q6sJpTH.jpg
 79x https://i.imgur.com/UfxjTNm.png
 78x https://i.imgur.com/BDw7lhz.png
 78x https://i.imgur.com/3iL01sL.png
 78x https://i.imgur.com/PfL9bwa.png
 74x https://i.imgur.com/rlXB9Yi.png
 72x https://i.imgur.com/1Ehtfqb.png
 72x https://i.imgur.com/LlQdqQF.png
 71x https://i.imgur.com/JYF9G6p.png
 71x https://i.imgur.com/u5bAUor.png
 68x https://i.imgur.com/jiFfM.jpg
 68x https://i.imgur.com/lWdXGZB.png
 68x https://i.imgur.com/p4sYVj8.png
 67x https://i.imgur.com/bYgWWGN.png
 66x https://i.imgur.com/hy1Ok9q.png
 66x https://i.imgur.com/5MGLhWv.png
 65x https://i.imgur.com/FMVKtb8.jpg
 65x https://i.imgur.com/lLE6iEW.png
 64x https://i.imgur.com/jxEyXop.png
 64x https://i.imgur.com/Biu9igZ.png
 63x https://i.imgur.com/Cc29VCA.png
 62x https://i.imgur.com/5iVkhdR.png
 61x https://i.imgur.com/rX7N7Vc.png
 58x https://i.imgur.com/5kisQjj.png
 57x https://i.imgur.com/dSAlr80.png
 56x https://i.imgur.com/vwMin.gif
 56x https://i.imgur.com/xh2Q8KB.jpg
 55x https://i.imgur.com/8YVVSCd.png
 55x https://i.imgur.com/iJUsS9u.png
 55x https://i.imgur.com/6doUbfO.png
 54x https://i.imgur.com/HiV0n8k.png
 54x https://i.imgur.com/H0cSEZE.png
 53x https://i.imgur.com/ueKxZYN.png
 52x https://i.imgur.com/DDMQjOw.jpg
 52x https://i.imgur.com/EzE4R6s.png
 51x https://i.imgur.com/izGMDEn.png
 51x https://i.imgur.com/1XE1SGK.png
 51x https://i.imgur.com/LFT4Hqi.png
 51x https://i.imgur.com/O01bDDB.png
 50x https://i.imgur.com/6TweJmB.png
 50x https://i.imgur.com/7Wfrdt8.png
 50x https://i.imgur.com/sEYArLN.jpg
 49x https://i.imgur.com/TKYScB5.gif
 48x https://i.imgur.com/IopV9Ux.jpg
 48x https://i.imgur.com/V4jVrsi.png
 48x https://i.imgur.com/diPTtsf.png
 48x https://i.imgur.com/TxR7OWW.png
 48x https://i.imgur.com/039AcDB.png
 47x https://i.imgur.com/QwSqFRt.png
 46x https://i.imgur.com/RopDEIf.png
 46x https://i.imgur.com/BbkzbjP.png
 44x https://i.imgur.com/O8IpMmi.jpg
 44x https://i.imgur.com/pXAIgye.png
 44x https://i.imgur.com/iNcDmVL.png
 44x https://i.imgur.com/jpeS3uj.png
 43x https://i.imgur.com/tCp90.gif
 43x https://i.imgur.com/tJnSJPz.png
 43x https://i.imgur.com/5n8DTcP.png
 43x https://i.imgur.com/0VMGERG.png
 41x https://i.imgur.com/ae9sK3J.jpg
 41x https://i.imgur.com/82jlhEJ.png
 41x https://i.imgur.com/x9tBXxH.png
 40x https://i.imgur.com/jZTMIjx.jpg
 40x https://i.imgur.com/eA1saz4.jpg
 40x https://i.imgur.com/EwJzDtX.png
 39x https://i.imgur.com/keblvKh.png
 39x https://i.imgur.com/6PDrnEu.png
 38x https://i.imgur.com/MpDC8wN.png
 38x https://i.imgur.com/wcAjrHe.png
 38x https://i.imgur.com/ls3gXpe.png
 38x https://i.imgur.com/XpJSUu1.png
 38x https://i.imgur.com/zQQbgtM.png
 38x https://i.imgur.com/56S1ulu.png
 38x https://i.imgur.com/aD2cViq.jpg
 38x https://i.imgur.com/ANUfCm6.png
 37x https://i.imgur.com/SgCrFHa.jpg
 37x https://i.imgur.com/Hp5Jsc9.png
 37x https://i.imgur.com/KvW715W.png
 36x https://i.imgur.com/eSqAsZf.jpg
 36x https://i.imgur.com/dT3l5Us.png
 36x https://i.imgur.com/1FL8pEp.jpg
 36x https://i.imgur.com/BhFfhhh.png
 36x https://i.imgur.com/lt8Y2Ms.png
 36x https://i.imgur.com/5ibApDZ.png
 36x https://i.imgur.com/kHLk5qD.png
 36x https://i.imgur.com/G0owRxs.png
 36x https://i.imgur.com/jvOC9zy.png
 36x https://i.imgur.com/eEbGhHy.png
 36x https://i.imgur.com/kyCuEot.png
 36x https://i.imgur.com/rjnu78l.jpg
 35x https://i.imgur.com/MqTGARn.png
 35x https://i.imgur.com/00lZU8s.jpg
 35x https://i.imgur.com/GI4Ui2E.png
 35x https://i.imgur.com/t5WSnJt.png
 35x https://i.imgur.com/piOwEuI.png
 35x https://i.imgur.com/WlENdQD.png
 35x https://i.imgur.com/rFzTXc3.jpg
 35x https://i.imgur.com/WKq2a17.jpg
 35x https://i.imgur.com/YkposA3.jpg
 34x https://i.imgur.com/73snBNd.png
 34x https://i.imgur.com/Yer95b7.png
 34x https://i.imgur.com/WPRCLfC.png
 34x https://i.imgur.com/66aaYzT.png
 34x https://i.imgur.com/yaJCjdQ.jpg
 34x https://i.imgur.com/xrNdMwF.png
 34x https://i.imgur.com/uQL0VRo.jpg
 34x https://i.imgur.com/c66Rdm2.png
 33x https://i.imgur.com/gE8hDnY.jpg
 33x https://i.imgur.com/UD3R4mP.png
 33x https://i.imgur.com/mrTylXN.png
 32x https://i.imgur.com/T9bhPAP.png
 32x https://i.imgur.com/G5GiwC9.png
 32x https://i.imgur.com/wcMxUyN.png
 32x https://i.imgur.com/KgKI4yr.png
 31x https://i.imgur.com/fYFBsqp.jpg
 31x https://i.imgur.com/8JgXMpk.png
 31x https://i.imgur.com/TjuvEnH.jpg
 31x https://i.imgur.com/rtBRE4d.png
 31x https://i.imgur.com/YrTlCqU.png
 31x https://i.imgur.com/jGO9qqH.png
 31x https://i.imgur.com/bx7dUTp.png
 31x https://i.imgur.com/Ye0x0UL.png
 31x https://i.imgur.com/qpctUK1.jpg
 31x https://i.imgur.com/CZnZtgg.jpg
 31x https://i.imgur.com/mY3wcrR.jpg
 31x https://i.imgur.com/nxPnKfq.png
 31x https://i.imgur.com/zwcJc6D.png
 31x https://i.imgur.com/1osd12y.png
 30x https://i.imgur.com/MoFoYQ4.jpg
 30x https://i.imgur.com/7v3EQvD.png
 30x https://i.imgur.com/s2WGkSi.png
 30x https://i.imgur.com/wKhBVVF.jpg
 30x https://i.imgur.com/dxGUs6q.png
 30x https://i.imgur.com/kiw6Xk8.png
 30x https://i.imgur.com/W8wp6Vo.gif
 30x https://i.imgur.com/wRNRwGX.png
 30x https://i.imgur.com/8XnQbBx.png
 30x https://i.imgur.com/bmHeOpN.png
 30x https://i.imgur.com/qvHPSfd.png
 29x https://i.imgur.com/GivknD1.gif
 29x https://i.imgur.com/kVs1gRE.jpg
 29x https://i.imgur.com/YWE9Jvl.png
 29x https://i.imgur.com/rUlIMGV.png
 29x https://i.imgur.com/XEYY21Y.jpg
 29x https://i.imgur.com/uE22CHZ.png
 29x https://i.imgur.com/cVzBeiZ.png
 29x https://i.imgur.com/pcHaaBv.jpg
 29x https://i.imgur.com/acaXCSm.png
 29x https://i.imgur.com/egSAgOJ.jpg
 29x https://i.imgur.com/B1poZVz.jpg
 29x https://i.imgur.com/HWECNhC.png
 29x https://i.imgur.com/hael4XZ.png
 29x https://i.imgur.com/mupiLfd.jpg
 29x https://i.imgur.com/Q1ewgDE.png
 29x https://i.imgur.com/chhUttM.png
 28x https://i.imgur.com/3IW9d6u.png
 28x https://i.imgur.com/W0c9bUI.png
 28x https://i.imgur.com/kA0l6JH.png
 28x https://i.imgur.com/joysAOQ.png
 28x https://i.imgur.com/ElxHkh2.png
 28x https://i.imgur.com/uS34vI1.gif
 28x https://i.imgur.com/qaVDlDD.png
 28x https://i.imgur.com/qNJsbRy.png
 28x https://i.imgur.com/V0hgK0b.jpg
 28x https://i.imgur.com/eyzhPWN.png
 28x https://i.imgur.com/ITc6D6f.png
 28x https://i.imgur.com/pRxpopB.png
 28x https://i.imgur.com/dallssX.jpg
 28x https://i.imgur.com/0pDzyEb.png
 28x https://i.imgur.com/fZCkwKu.png
 28x https://i.imgur.com/qF92A9r.png
 28x https://i.imgur.com/YDZA3Yu.png
 28x https://i.imgur.com/2WG4lJJ.png
 27x https://i.imgur.com/W9hEG8x.png
 27x https://i.imgur.com/UzvGyGL.png
 27x https://i.imgur.com/cbnQH3s.gif
 27x https://i.imgur.com/ekm3DjQ.png
 27x https://i.imgur.com/6c5u0K3.png
 27x https://i.imgur.com/hCPNZVa.png
 27x https://i.imgur.com/WN3WHF6.png
 27x https://i.imgur.com/sEvaYln.png
 27x https://i.imgur.com/nE9DNpI.png
 27x https://i.imgur.com/plEFlej.png
 27x https://i.imgur.com/dA979MT.png
 27x https://i.imgur.com/kNg63SV.png
 26x https://i.imgur.com/75VzmLr.png
 26x https://i.imgur.com/BszEUl9.png
 26x https://i.imgur.com/hilWiEs.png
 26x https://i.imgur.com/sbx37FT.png
 26x https://i.imgur.com/ZEZJ4iK.png
 26x https://i.imgur.com/zmujliU.jpg
 26x https://i.imgur.com/gnlVXOm.png
 26x https://i.imgur.com/hOfMaDI.png
 26x https://i.imgur.com/AZLpIKy.png
 26x https://i.imgur.com/4TxHtLH.png
 26x https://i.imgur.com/Qt9IGiR.png
 26x https://i.imgur.com/hl3Medw.png
 26x https://i.imgur.com/wI5AsjI.png
 26x https://i.imgur.com/vMV2MZg.png
 26x https://i.imgur.com/upTsvz9.gif
 26x https://i.imgur.com/zBHRuce.jpg
 26x https://i.imgur.com/4vqY7nT.jpg
 26x https://i.imgur.com/4pMY8c9.png
 26x https://i.imgur.com/4cF3HBA.png
 26x https://i.imgur.com/os7GmdH.png
 26x https://i.imgur.com/XICgptV.png
 26x https://i.imgur.com/pyJwWGw.jpg
 26x https://i.imgur.com/Kw2wgyp.png
 26x https://i.imgur.com/wSq8Ivy.png
 26x https://i.imgur.com/l22XUfw.png
 26x https://i.imgur.com/8qLWHPo.png
 26x https://i.imgur.com/khW8B0W.png
 26x https://i.imgur.com/6t1NOwB.png
 26x https://i.imgur.com/tGtKIlp.png
 26x https://i.imgur.com/JbVHrog.png
 26x https://i.imgur.com/UCSAlsH.png
 25x https://i.imgur.com/i32XS0e.png
 25x https://i.imgur.com/WzP4Z4a.jpg
 25x https://i.imgur.com/LVz1c8z.jpg
 25x https://i.imgur.com/fhiMrH5.jpg
 25x https://i.imgur.com/ahvAqjV.png
 25x https://i.imgur.com/g32EfIL.png
 25x https://i.imgur.com/Ww88c12.png
 25x https://i.imgur.com/pEIeBKV.png
 25x https://i.imgur.com/pXsHG1r.png
 25x https://i.imgur.com/lueL5OU.png
 25x https://i.imgur.com/tHSwjSf.png
 25x https://i.imgur.com/ODm6ofW.png
 25x https://i.imgur.com/vZ1yIhd.jpg
 25x https://i.imgur.com/eFE6OFU.jpg
 25x https://i.imgur.com/W5YerDE.png
 25x https://i.imgur.com/qk8uObk.png
 25x https://i.imgur.com/3Hj74yO.png
 25x https://i.imgur.com/xSjFi3L.png
 25x https://i.imgur.com/utj9lF3.jpg
 25x https://i.imgur.com/GtS1tYP.png
 25x https://i.imgur.com/pplGBmJ.png
 25x https://i.imgur.com/QyFiIjB.png
 25x https://i.imgur.com/k7c98UF.png
 25x https://i.imgur.com/lJ9YKmg.jpg
 25x https://i.imgur.com/IZJnRnD.png
 25x https://i.imgur.com/2x53Og0.jpg
 25x https://i.imgur.com/qEg4ihx.jpg
 25x https://i.imgur.com/sV2A2WL.jpg
 25x https://i.imgur.com/vztqBOr.jpg
 25x https://i.imgur.com/zcgd1lw.png
 24x https://i.imgur.com/gQrUpE2.gif
 24x https://i.imgur.com/TblK8RP.png
 24x https://i.imgur.com/XnvT26Y.jpg
 24x https://i.imgur.com/UvparyB.png
 24x https://i.imgur.com/MdUFbhf.jpg
 24x https://i.imgur.com/3mtKNVH.png
 24x https://i.imgur.com/DhsfjQQ.png
 24x https://i.imgur.com/YCSIJIJ.jpg
 24x https://i.imgur.com/TA7dSVN.png
 24x https://i.imgur.com/fzaf292.png
 24x https://i.imgur.com/mBadliI.png
 24x https://i.imgur.com/jQ3GJDC.png
 24x https://i.imgur.com/z2asE7Y.png
 24x https://i.imgur.com/Ib2Zf87.png
 24x https://i.imgur.com/v36K43E.png
 24x https://i.imgur.com/MWCNyMj.png
 24x https://i.imgur.com/99uGgUD.png
 24x https://i.imgur.com/l83VzV6.png
 24x https://i.imgur.com/6uSD65z.png
 24x https://i.imgur.com/7x2CO0F.png
 24x https://i.imgur.com/nnB9hT5.png
 24x https://i.imgur.com/1wlF8ws.jpg
 24x https://i.imgur.com/eYlbx70.jpg

As expected, ChartBuddy is quoted the most.

Suggested new image host: TalkImg.com - Image hosting for BitcoinTalk.

[1714536048]

1714536048

[1714536048]

1714536048

[1714536048]

1714536048

[Little Mouse]

We can get all the images from here and upload them manually one by one by going to respective threads. I have a lot of pages and have uploaded a lot of images. I don't know how it's going to help me. It's going to take a full day bitcointalk schedule from me. Well, I'm aware about the Tryninja script. I will possibly have no choice but to use that one.

[LoyceV]

I don't know how it's going to help me.

The main goal was to avoid this:

In addition to explicit images, Imgur is also removing old and unused photos that are not tied to an account. As a Twitter user noted, Imgur has been used for years to upload photos without an account and post links to different kinds of sites. Removing those images might result in a lot of dead links all over the internet.

The changes are going to take effect this Month on the 15th.

[decodx]

I made a .zip for each user. It includes all images related to that account (including quoted images). After extracting the .zip file, "list_of_posts.html" shows links to all images, posts (some of which have been deleted) and archived posts.

It's great that you've provided a full list of all the posts where the images were used. This is incredibly helpful. Thank you so much!

Please, don't delete this... At least for a while. 

[un_rank]

This has to be the forum with some of the most industrious members of any offline forum. Problems arise and there are always hands on deck to try and solve it with no incentive to do so besides helping out other users.

Great work LoyceV, I may have just one or two posted images but this would be useful to many forum users.

- Jay -

[PowerGlove]

(...) That's why I've downloaded 801,550 images from imgur, which were embedded on the forum. Downloading took 3 days using 4 different servers, and another 3 days to process the data. (...)

I expect a lot of people to not fully realize how valuable what you've done is; nice work!

Having a copy of these images safely tucked away means that the Imgur issue can be worked on without rushing things.

I hope theymos settles on an approach that involves URL substitutions being made by the image proxy, because that would fix things forum-wide without anyone having to do anything.

[joker_josue]

First of all congratulations for this excellent work.

I would like to know if you think there would be any way to confirm if the image is still in use in the forum or not? That would perhaps force a new scraping.

[Husires]

Can you display the images by topic ID or is it a little difficult? If this is possible, all that the person needs is to go to your site with the link to the topic to see the old images, and this will be useful for those who wish to track an old non-active topic.

Browsing the forum has become impossible, so thank you for making it possible to read many old threads.

[vapourminer]

this is perfect to me as i am lazy af; now i have everything i need to edit at my leisure.

sweet. thanks much

[Timelord2067]

So, I delete a 20Mb gif from a post of mine (quoted from someone else) and that is then deleted from the ZIP file?

[LoyceV]

I would like to know if you think there would be any way to confirm if the image is still in use in the forum or not? That would perhaps force a new scraping.

Scraping everything again would be the only way to know for sure, but I'm not going to do that again. At one page per second, it took me months to download all old posts.

Can you display the images by topic ID or is it a little difficult?

I could, but I don't think it's worth the effort. It would mean creating another set of .zip files (~150 GB) and takes quite a while to sort it out.

So, I delete a 20Mb gif from a post of mine (quoted from someone else) and that is then deleted from the ZIP file?

No, I can't update the .zip files. That would require rescraping all posts.

[joker_josue]

Scraping everything again would be the only way to know for sure, but I'm not going to do that again. At one page per second, it took me months to download all old posts.

Clear! Does not make sense.

Just trying to think of ways to try to avoid importing images that are no longer being used. 

[LoyceV]

Just trying to think of ways to try to avoid importing images that are no longer being used. 

Apart from duplicates, I don't think any of the images is no longer used. Take this post for example: chances are nobody has seen it in years, but it's part of the forum's history.

[Aikidoka]

Thanks LoyceV, This is really valuable and highly appreciated. I have downloaded the zip file containing all the images I have posted on the forum via Imgur. I'll try to update my previous posts with the broken Imgur links and replace them with TalkImg links. 

[Timelord2067]

Just trying to think of ways to try to avoid importing images that are no longer being used. 

Threads that aren't locked but are still active would be where I'd start, then if users reach out to request threads that are locked, but are referenced would also be high on my list. Working backwards from today to the dawn of time the forum.

Unless it's specifically requested to be updated, then wait for the request.

I wouldn't go looking for work that isn't needed or requested.

[joker_josue]

Apart from duplicates, I don't think any of the images is no longer used. Take this post for example: chances are nobody has seen it in years, but it's part of the forum's history.

This image is still in use, even though it could be weeks before anyone sees this post.
I'm talking about images that are no longer on the forum, for example posts that have already been deleted.

I wouldn't go looking for work that isn't needed or requested.

Initially this is what is intended. But, unfortunately there will be hundreds or thousands of images that users will not do anything about.

So what can we do?

[RickDeckard]

This is a remarkable work LoyceV, like always. Considering the amount of resources that you had to use, and the amount of space that it currently takes on your server(s), can I kindly ask you for a way to download those files and act as a backup in case something bad happens? At this stage where the forum is at the moment - regarding deciding what solution to employ - I see having a backup of this archive as a really important thing to do.

[LoyceV]

Imgur-images are working again, but we don't know for how long. So get yours

[dkbit98]

Imgur-images are working again, but we don't know for how long. So get yours

I suspect they will bring  them down again in near future, so backing everything up was a good idea.
This is probably delayed because of feedback they received from Reddit community that is much bigger than bitcointalk.
When I first saw images working again I was thinking theymos did some kind of magic trick with your help and with script. 

[LoyceV]

From another topic:

I was even trying to make out time someday to start deleting all my posts history that has an image on them because of Imgur's invalid image display.

That's the opposite of archiving