Bitcointalk Offline

[dkbit98]

I was wondering is there a way to have Bitcointalk forum working offline without internet connection?
That would probably take a lot of space if downloading images are included, but maybe we could have slim version that only include links for images.
This could work as a backup, and used only for browsing or history check, but it could have periodic updates.
I know we can download single page that could work offline, but it would be complicated to do this manually for entire forum.

If we can have offline books in digital format, than we could have the Bitcointalk forum, but I am not sure if this breaks any legal terms or not.
There is also a problem how to compress all this data to a reasonable and acceptable size.

PS
I did some basic research and I found some free software like HTTrack or Cyotek WebCopy, and there are some paid alternatives.

[PX-Z]

Technically, it's possible, but talking about how practical it is, i say it's not. It looks like you will spend more internet data resources than using the forum in normal way since you will need to download the threads' replies to your device everytime you visit a thread. Most feature will be affected is the search feature, it will be different way and it will be very so slow to do that. Even merits, and edits will be inconsistent too.

[LTU_btc]

I guess it's poosible, but I'm wondering is it really worth to have such thing? As I understand, such backup version might be useful when Bitcointalk goes offline, forexample when there is an issue with Cloudflare or something similar. But from what I recall Bitcointalk downtime in recent years issomething what doesn't happens often and it takes up to few hours. And I guess such backup would require lot of resources.
It's not that I'm against such thing, but I guess that it would have very minimal use.

[mcdouglasx]

Yes, it can be done, but you'd have to configure it to extract only simple things to save space. By simple things, I mean avoiding avatars, images, links you obviously won't be able to review, and user signatures. Regarding its legality, you'd likely need permission, as we wouldn't want to slow down the user experience with server requests. Also, some users with posts deleted after your backup might be concerned about their privacy.

I think if you wanted a read-only version of the site for historical context, you should simply focus on the boards that provide historical context, and not on others that might contain content we can do without.

[Welsh]

I do recall theymos publishing backups of the forum once upon a time...but maybe I'm imagining that. If it did exist it would be more of a backup of all posts rather than a offline version of Bitcointalk that operates exactly the same, i.e running a local server. I'll see if I can find it, but again this is going back years ago if this ever was a thing. Maybe, it wasn't theymos and it might have been one of the external tools or something. My memory is failing me bad and quick searches for it aren't bringing up anything. So I guess I'm talking through my arse.



I wasn't talking through my arse: https://bitcointalk.org/index.php?topic=75207.msg833973#msg833973

Someone asked for a dump of all forum posts. Here it is (minus posts in restricted sections):
http://uploadmirrors.com/download/2JWDLDGR/Bitcoin_Forum_posts_dump_Apr_3_2012.7z
SHA-256 hash: 92C12F9FC75B3D58C87B6802C6F6392F22615BDAF24D22A1E83CE0109347103D

I'm not going to post updated versions of this regularly.

Okay, not very relevant since its from years ago, but hey my memory hasn't completely failed me yet!




I imagine saving pages via print function or just using curl to retrieve certain pages might be a better option? Sort of a offline reference rather than the whole thing.

[TryNinja]

I could teorically do that with the bitlist archive data, creating my own offline app that downloads a few topics for offline reading.

But why? I don't think anyone would really use this.

[DYING_S0UL]

Only images? Or like the whole forum but totally offline?

I'm a little confused (probably because of my language barrier ) as well as curious! What do you intent to do with a backup of "links of images"? If it's only links for images, then afaik talkimg has a public backup of that, available to download at any given time.

I have personally used HTTrack before. It would download/clone/copy (whatever you call it) every page, every image, links, section by section, folder by folder and make a backup of it categorized. A very powerful tool that is, but the problem is it's very time consuming and resource hungry. So without a proper setup it might take days or weeks based on the site you are tying to copy. I had tried it with altcointalk once, and even after hours there weren't any significant progress (maybe because of my internet is the shittiest).

After reading a couple more times, I think I confused with actual images and links of images.

[hosemary]

I'm a little confused (probably because of my language barrier ) as well as curious! What do you intent to do with a backup of "links of images"?

dkbit98 means that to reduce the data size, the saved pages can contain only links to images instead of storing the images themselves.

I have personally used HTTrack before.

I also used HTTrack once for saving another website once and I can confirm that it works perfectly.

[promise444c5]

You will need the entire current data as at the time you choose to do that stored locally to use without internet.. Which means you have to keep downloading every single data probably at a specific period of a timeframe to stay current.


Never tried this though but what DYING SOUL mentioned seem like a solution, you can search  other tools more like it.. And maybe checkout YT vid if available .

BTW, TryNinja can give you a solution from what he posted.

[Stalker22]

I have personally used HTTrack before.

I also used HTTrack once for saving another website once and I can confirm that it works perfectly.

Yes, HTTrack works if you are trying to mirror a static HTML websites.  But I think in this case it would be better to ask for raw data (posts) from TryNinja or LoyceV because they are already scraping the data regularly.  It would be an unnecessary burden to download all the HTML, when one could just get clean data in archive format or via an API call.  As far as I remember, ninjastic.space had such capabilities.  But I do not know if bitlist.co is similar.

[Churchillvv]

I’m quite ignorant on this but however, I imagine the offline version to be already available data’s and not the continuous data so if we make a post after your last update, I mean individual updates then you will not be able to access the new post? Someone school me on this, in ELI5.

I think it’s not totally a convincing to have the offline version but it could be cool when one experience an internet outage like those who are in countries with situations like wars.

[Stalker22]

I’m quite ignorant on this but however, I imagine the offline version to be already available data’s and not the continuous data so if we make a post after your last update, I mean individual updates then you will not be able to access the new post? Someone school me on this, in ELI5.

Yes, you are 100% correct.  without a live internet connection, it is essentially a database frozen in time, and any new posts made after the last update will not exist in offline versoin of the database.

I think it’s not totally a convincing to have the offline version but it could be cool when one experience an internet outage like those who are in countries with situations like wars.

It definitely could have its uses.  For example, lets say you want to devote more time to reading some older megathreads, having them offline means you can do it anywhere, without needing a constant internet connection.

[Mia Chloe]

~snip

It's a fine idea especially for a case where you just want to surf the forum mostly old posts without any intention of writing any. I haven't really thought of a nice work around yet but I don't know how much data it will consume to store a certain amount of data of the forum for offline view.

In the past I've tried to use 2G on the forum and to my suprise it worked really easy and it was so fast you would barely realise it's 2G unlike most sites that won't even load up.

[Churchillvv]

I think it’s not totally a convincing to have the offline version but it could be cool when one experience an internet outage like those who are in countries with situations like wars.

It definitely could have its uses.  For example, lets say you want to devote more time to reading some older megathreads, having them offline means you can do it anywhere, without needing a constant internet connection.

Right, makes more sense to me now, I think I once had a conversation about this with Greg, about learnmeabitcoin, so you could get to make it an offline book or some PDF kind of file but he suggested I print and download each page manually which took me a very long time to get all the information that I needed for the main time but that lead me not to read all at once like I would have loved to, going back and front to download them got me distracted, lol but that’s on my end. However it’s a good one.

[dkbit98]

Technically, it's possible, but talking about how practical it is, i say it's not. It looks like you will spend more internet data resources than using the forum in normal way since you will need to download the threads' replies to your device everytime you visit a thread. Most feature will be affected is the search feature, it will be different way and it will be very so slow to do that. Even merits, and edits will be inconsistent too.

I would be more than happy to have update every six months or once every year, and we don't really need to have information about merits.
If we can have offline PDF books or offline movies than we could have something like The Book of Bitcointalk.

I could teorically do that with the bitlist archive data, creating my own offline app that downloads a few topics for offline reading.

But why? I don't think anyone would really use this.

Not many people would use it now, but it could be very valuable thing in the future.
Can you give us some estimate how much data space would this use?

[UchihaSarada]

The forum website is just down seconds ago. Does anyone else get this issue recently?