Poll on solving the imgur issue

[NeuroticFish]

For consideration: some images are posted by many different users, it wouldn't make sense to have all of them upload the data again with their own unique filename

Imho this is a non-problem. An image-website owner would notice in less than a month he can optimize his space by checking the images (maybe their checksums at first) and not store the duplicates while providing same or different (by his choice) links to them. And before the question arises, where deletion is possible, the actual file will be removed only when reference count goes to 0.

[1714526042]

1714526042

[1714526042]

1714526042

[1714526042]

1714526042

[joker_josue]

I'm done processing the data, see [get yours] Images from imgur.
@joker_josue: I can get you one file with all images if you want.

For consideration: some images are posted by many different users, it wouldn't make sense to have all of them upload the data again with their own unique filename:

I think that's fabulous. I believe we don't need to load repeated images.
This would help a lot in managing files on the server.

One thing you can't do is order from oldest to newest, can you? This could be very useful.

[ibminer]

This isn't a long-term solution once imgur starts deleting anonymous uploads:

None of the options deal with deleted images very well, might depend on how big of an issue that actually is. For these, option #2 would create broken links all over the forum, which isn't good for SEO.

Right now, technically the image proxy at least displays a legit image showing the words "invalid image", so it's not really a broken link when seen from a spider/search bot. Again, I don't really know how many of the images have been deleted vs. how many are still hosted under imgur accounts, so I'm not sure how much of an impact it would be.

Instead of "expand on click", could you make this a profile option? Something like: "Tick to enable all imgur images without proxy".

What about users who aren't logged in?

I had mentioned (privately) the thought of the forum itself hosting/archiving sets of images from particular domains through the image proxy, potentially for future situations like this. Hosting sets of images on the forum itself is probably the best case scenario for historical preservation purposes, but based on the options presented in the OP, I don't think this is really something theymos is willing to entertain, probably for good reasons.

I'm not excited about the idea of redirecting everything to any particular image host though, as it feels like that could build up to be a larger problem when that image host has even temporary issues or needs to shut things down entirely for whatever reason.

I don't think I've ever used imgur, but I have quoted hundreds (link doesn't work yet) of imgur images. I wasn't planning on editing those posts, but it would be nice if the images still work.

Good point on the quoted images.

[joker_josue]

Imho this is a non-problem. An image-website owner would notice in less than a month he can optimize his space by checking the images (maybe their checksums at first) and not store the duplicates while providing same or different (by his choice) links to them. And before the question arises, where deletion is possible, the actual file will be removed only when reference count goes to 0.

Doing this is not exactly easy. It would take a system that compares the image posted with the image on the server. This would require constant scraping to see if the image link is the one in today's post or not.

When I created my image service, I thought of doing something so that if an image was not posted on the forum, it would be deleted. But, I quickly realized that this was not going to work. Because there was no minimally viable way to monitor this. The image could be posted once and never viewed again; it could be posted days after it was uploaded... in short, many possibilities. Best solution: uploaded image, cannot be deleted.

[Timelord2067]

For consideration: some images are posted by many different users, it wouldn't make sense to have all of them upload the data again with their own unique filename

Imho this is a non-problem. An image-website owner would notice in less than a month he can optimize his space by checking the images (maybe their checksums at first) and not store the duplicates while providing same or different (by his choice) links to them. And before the question arises, where deletion is possible, the actual file will be removed only when reference count goes to 0.

Nice to see my suggestion of checksums and duplicate files is starting to gain traction. 

In any event, it looks like option #2 is remaining out in front.  Its not the best option and other users suggestions are probably going to fall on deaf ears until IMGUR pulls the plug then this will start again.  As a couple of users have pointed out, none of this will make a difference to users who are not logged into the Forum and will just see a whole lot of broken links.

[LoyceV]

One thing you can't do is order from oldest to newest, can you? This could be very useful.

See tmp.loyce.club/imagebackup/sorted_by_date.txt. I don't see how this helps though: images could be posted/quoted years after they were uploaded.

Instead of "expand on click", could you make this a profile option? Something like: "Tick to enable all imgur images without proxy".

What about users who aren't logged in?

Maybe they can still get the "click to expand" feature.

Imho this is a non-problem. An image-website owner would notice in less than a month he can optimize his space by checking the images (maybe their checksums at first) and not store the duplicates while providing same or different (by his choice) links to them. And before the question arises, where deletion is possible, the actual file will be removed only when reference count goes to 0.

Doing this is not exactly easy. It would take a system that compares the image posted with the image on the server. This would require constant scraping to see if the image link is the one in today's post or not.

A checksum makes this quick: create a checksum for each file, and add it to a long list (or database). For each new upload, see if the checksum already exists. If it does, hard link the new file to the old file instead of creating a new one.
Out of curiosity, I'm now checking how many duplicates there are in the 800k imgur images I have. Results are surprising: there's one image uploaded 32752 times, which turns out to be a removed image: "The image you are requesting does not exist or is no longer available". In total, 10124 images are uploaded once or more, and in total 55932 times. Hard linking those files could save a few GB of data.

[hugeblack]

EDIT:

The ideal solution, is to have an unofficial hosting service (non-profit organization), so that the forum pays a little to ensure that the service will last for several years and bear the volume of traffic, with a monthly payment to them like staff.
thus reducing the damage to intellectual property rights and the existence of a service that the forum members can hold accountable.

Does the TalkImg service fit this idea!?

Unfortunately not, I trust you and thank you for your great effort, but at some point hosting these images will be expensive and therefore you will not be able to pay them out of your own pocket, in addition to that you must be held accountable in case of misuse (such as IP addresses), which is something we cannot do for you.

We need an unofficial party, but it is from the forum and the forum pays for it to be able to do that.

[Timelord2067]

Kind of strange someone turning down help after the hard yards have been done getting the replacement service up and running.

My donation hasn't been confirmed but I hope it helps cover your expenses if only in a small way @joker_jouse


Any one know where @theymos got to?

[rdluffy]

I voted for option 2, I believe that among the options, this would be the best
The images of old topics would still be accessible, especially tutorials and more important things, easily accessible to any user. And any thread would be included in this fix, since if the user needs to do some action, there are many threads that are good but wouldn't be updated by the users in case they need to do something.

Would this be an option for you (@theymos)? If so, I have 801,550 images waiting

It's a good idea too, combined with Talkimg from user joker_josue.
I just don't know what would be the impact on the server of having to host these 801,550 images like that quickly and the costs in the long run  


Maybe option 2 would be the simplest and most effective in order not to generate problems for anyone in the future.

[joker_josue]

Unfortunately not, I trust you and thank you for your great effort, but at some point hosting these images will be expensive and therefore you will not be able to pay them out of your own pocket, in addition to that you must be held accountable in case of misuse (such as IP addresses), which is something we cannot do for you.

We need an unofficial party, but it is from the forum and the forum pays for it to be able to do that.

I understand what you mean.

But whatever you're saying, you're in exactly the same situation. How is this forum, being in the hands of @theymos.


Regarding the current service, I have already explained in more detail what is involved and my commitment to the project. Of course, not everyone has to believe me. But, I believe that I am very committed to the project. If it wasn't, that wouldn't have scaled the project 6 months earlier than anticipated, and at a 3x cost increase, like I did.

And if it gets to the point where I can't take it anymore, I'm totally available to transfer the project to someone else. But believe me, I will always make the effort to keep it going. And if the costs become unbearable, I will only let the project fall if the community gives zero support.

Kind of strange someone turning down help after the hard yards have been done getting the replacement service up and running.

My donation hasn't been confirmed but I hope it helps cover your expenses if only in a small way @joker_jouse

Thanks for the donation. It can take up to a year to confirm, no problem. 

[rby]

2: Make it so that all past imgur embedded images show up as a clickable link instead of as an embedded image, as if the poster had just posted the imgur link instead of using the [img] tag. I could also maybe make it so that if you click the link, it will JavaScript-expand into an embedded image (using the specified width etc.) without going through the image proxy.

In as much as all the options appear to me as a temporary measure, I went for option 2. Although I am skeptical in clicking any link that will take me out of the forum, especially if the profile behind the link isn't a reputable member of the forum.
For future purposes, the geeks in the forum, is there no way to link the forum images in imgur server to talkImg by Joker_josue. I don't know if this sounds silly or unachievable, but I know imgur is an open source project. Anything is possible.

[Synchronice]

I'm missing option 5:
5. Someone creates a website with all most of the imgur images, and you forward the image proxy to that site for any imgur picture. It's probably a good idea to combine it with option 4:

4: ~prevent using imgur embedded images in any future posts, to sort of phase it out.

Would this be an option for you (@theymos)? If so, I have 801,550 images waiting

I came here to write this but since Loycev already wrote, I support his idea. This is the best option out there because @LoyceV already managed to get all the images, @joker_josue has already created TalkImg and is more than happy to host images and make his service popular, has proven past experience, is one of the oldest member, isn't going to abandon his service and overall, I personally believe in him.

@theymos this is the best option!

[Wind_FURY]

IMO I'm not sure it should matter as much on what the user can do themselves, some of the users may not even be here anymore.

The forum itself should want the images as they were in the thread, to preserve the historical nature.

I certainly will not be mass-editing my prior posts, with or without a script, so hopefully there will be something that can be done here with one of the options in the OP, even if it doesn't end up the most optimal option for the forum and future readers.

But what else can be done?

I believe the second option is the best compromise without making the forum give up its users' privacy. It's the right move in principle, although it won't make many threads look very good.

[Stalker22]

IMO I'm not sure it should matter as much on what the user can do themselves, some of the users may not even be here anymore.

The forum itself should want the images as they were in the thread, to preserve the historical nature.

I certainly will not be mass-editing my prior posts, with or without a script, so hopefully there will be something that can be done here with one of the options in the OP, even if it doesn't end up the most optimal option for the forum and future readers.

But what else can be done?

I believe the second option is the best compromise without making the forum give up its users' privacy. It's the right move in principle, although it won't make many threads look very good.

I am just throwing out an idea here, but maybe TryNinja or some of the other developers could possibly create a userscript that allows embedding images within posts directly from the client side. It is just a thought off the top of my head!


EDIT: Thank you LoyceV! This is a very useful script.

[LoyceV]

maybe TryNinja ~ could ~ create a userscript that allows embedding images within posts directly from the client side.

You're looking for [Script] Fix broken Imgur images forum-wide (client only).

[DdmrDdmr]

From a user perspective, the ideal solution would be transparent to the him, preserving the display of the images whilst not involving him in the process (many can’t be). Bearing that in mind, and having, I figure, discarded the obvious (having a solution on imgur’s end to the mess they’ve created) the best solution would seem to be this one:

<…> 5. Someone creates a website with all most of the imgur images, and you forward the image proxy to that site for any imgur picture. It's probably a good idea to combine it with option 4 <…>

I would be concerned though with which image repository to use, that minimizes (not eliminates) the risk in the long run of having to shift yet again. The closer the repository is to the forum the more control over it one can have, both in terms of the team that runs it and supports it financially.

[Hispo]

Since in this case I am personally interested in archival purposes, my vote goes for the second option.

There has been occasions when I would like to check on past threads and discussion which are significantly old in the forum and I am afraid I end up losing much of the context, due to the lack of the images referenced. At least with the clickable link the context and the content is still accessible to all of us. 

Since months ago I stopped using Imgur in favor of better alternatives, btw.

[ibminer]

But what else can be done?

With imgur?  Out of the options given in the OP, I would still probably lean towards #3 or #4.

These allow the best view for threads for any images still hosted at imgur and would still allow the image proxy to put up the "invalid image" image for any deleted images, and not a broken link.

People using TOR are usually the ones concerned with their IP privacy, so why does an exemption here matter to them if they're IP is already being covered by Tor? Imgur would only see the Tor node's IP.

For people not using TOR, what would be the main concerns with imgur knowing your IP address viewed an image on the forum?

For #5, this would certainly deal with the deleted imgur images nicely. Are there maybe trust/liability issues here when theymos picks a specific place to force redirect and/or edit the posts of other users image(s) to a particular host?  Options in the OP are just sort of using what the user posted, but displaying them differently.

For the future?

~
I had mentioned (privately) the thought of the forum itself hosting/archiving sets of images from particular domains through the image proxy, potentially for future situations like this. Hosting sets of images on the forum itself is probably the best case scenario for historical preservation purposes, but based on the options presented in the OP, I don't think this is really something theymos is willing to entertain, probably for good reasons.
~

Essentially that the forum would save a backup copy of images as they are seen through the image proxy. Not for [immediate] display purposes, but for archiving, so that if/when an image host/domain goes offline, the forum could then display its archived copy of those images.

It may be too complicated to create, and would overall obviously take some more resources in terms of storage space to save an image the first time it is seen, and bandwidth to display any images that are using the archived copy, aaand probably some slight increases in things like processor/memory/DB activity to manage it all.  

Maybe overkill here but I was thinking more long-term, or in situations where users or the community may not get a chance to save the images. I believe images are a big part of the forum, so losing them and/or access to them in large batches seems bad for the forum.

[theymos]

It looks like imgur is no longer blocking the image proxy. I didn't change anything on the forum's end, so imgur must've loosened up their limits. I guess it's not necessary to convert img images into links as long as imgur keeps working. But to prevent this from happening again, you'll get an error if you try to embed imgur images in posts from now on.

I'd like to create a page which recommends services for hosting forum images, and mention this in the error message you get when you try to use imgur. Other than TalkImg, which ones are good?

[Learn Bitcoin]

Other than TalkImg, which ones are good?

Well, Talkimg should be the first one to suggest since it's an dedicated service for forum members only. Other than this one, we have imgbb.com and postimages.org/postimg.cc (both are the same). These two images hosting are fast for me. Most of the other services are works slowly and not available 24/7. Some of them went down at some point and then get back again after a few hours. But, I am satisfied with imgbb and postimg so far. I hope others will suggest them as well.

I did not check any old Imgur images yet. Is it started working again?