Suppose there are boards B1 and B2. B1 has child boards B11 and B12. Each board is represented as a folder with the same name. The main foder, F, could be structured as follows (every instance of content.txt represents a file; the rest are folders).

F
├───B1
│   ├───content.txt
│   ├───B11
│   │   └───content.txt (*)
│   └───B12
│       └───content.txt
└───B2
    └───content.txt

Now here's what a content.txt file could look like. Suppose we're looking at (*).

{
    "name": "B11",
    "topics": [
        {
            "topicId": "1111",
            "subject": "Help me out plz",
            "op": {"userId": "3596085", "username": "ltcltcltc", "activity": 26, "merit": 60},
            "time": <timestamp of the original post>,
            "messages": [
                {
                    "msgId": "6666",
                    "author": {"userId": "3596085", "username": "ltcltcltc", "activity": 26, "merit": 60},
                    "time": <timestamp of this message (in this case the original post)>,
                    "merited": "2",
                    "message": "Hey does anyone know how to speed up ecdsa signature bruteforcing?"
                },
                {
                    "msgId": "6699",
                    "author": {"userId": "3597570", "username": "aleph1", "activity": 1, "merit": 23},
                    "time": <timestamp of this message>,
                    "merited": 0,
                    "message": "Stop wasting your time."
                }
            ]
        },
        {
            "topicId": "2222",
            "subject": "Test. Do not answer.",
            "op": {"userId": "3597570", "username": "aleph1", "activity": 1, "merit": 23},
            "time": <timestamp of the original post>,
            "messages": [
                {
                    "msgId": "8008",
                    "author": {"userId": "3597570", "username": "aleph1", "activity": 1, "merit": 23},
                    "time": <timestamp of this message (in this case the original post)>,
                    "merited": "3",
                    "message": "Testy test."
                }
            ]
        }
    ]
}


I didn't give any example of timestamp because I don't know what your time format is, but I think I'd prefer Unix time. Also note the redundancy: the topic's timestamp is the same as the timestamp on the first message of said topic. The topics inside each board are ordered chronologically (older first) and the messages inside each topic too.

What do you think about this format?

It's a good idea too. I'll .append() it to the list. Ltc stands for other than litecoin.

Ok. Then perhaps TryNinja's database fits better my purposes.

I think a tree-like structure (board/subboard/topic/message) would work best so as to study conversations as a whole more than individual messages, since I don't care about individual opinions as much as I do about global sentiments.
So maybe JSON? Does this work for you? I mentioned the Economy board as an example; ideally I'd want the whole data.

It would be a super favour you'd be doing me.

I sometimes fit in the penultimate profile you put forward. But definitely the last one.

---

Not really.

CS means computer science.


So, please tell me if I'm wrong: you just scrape content with limited data treatment, so the filtering functionalities you offer are the same that the forum offers, i.e. sorting by chronological order, viewing the posts inside a topic and filtering by user.


Ok! I thought by the previous message that you were declining. But if that's not the case then I'd be super grateful if you shared your database with me to avoid the undesirable task of rescraping the scraped!

Ninja's website looks handy too. Harder to scrape though.

Thanks, btw what do you mean by WYSIWYG? I get what it stands for, and that it's CS slang, but how does it apply here? Also, I've seen that your website offers the functionality of showing any given user's messages. Is there an analogous way of filtering messages by board? Like: "showing Economy messages".

PD. I've seen your other work, quite impressive!

I can't make a joke now... You might be confounding me with someone? Anyways I'm really innocuous fellas but whatever keep it going. Por dios.

How is your data classified? Tree-structure or raw recent-first stack? In the second case, I'd probably reorganize it myself into a tree-like structure. This way should be quicker to filter out some data. Maybe start with the Bitcoin discussion board, then scale up.

Also I can always chop those 100GB into various time series. Perhaps the 2020-2022 time period contains jucier data than the rest (due to the rise and drop of BTC). Everything can be explored.

Haha I didn't think about that indeed.
I came across this sentimental analysis of BTT. It aims to infer a correlation between the temperature/feeling of this forum and the tendency of cryptos like Bitcoin. I found it interesting so I thought I'd try it myself, play around with the data, see what comes out. Might even be an intro to ML. My goal: learning. Oftentimes that leads to interesting results but one can never be certain. Still, if the least comes out of this you'll be the first to read about it.

I know LoyceV has put together a nice scrapable archive of the topics of this forum.

I want to do a analysis of the BTT forum. I could write a script to scrape the data from LoyceV's archive but I was really wishing someone could point me towards a fully downloadable database to speed things up. Does anyone have a reference?

Cheers!

Digaran bro/sis, satoshi dreams of that fifty percent. Wait for them pizza txes to come.

Anyone know how much bitcoin be owns?

Yeah, good but that's easy to rev engineer. Decompiling that code should be straightforward. @garlonicon link is so accurate, thanks! And @BTCapsule: it's cool that someone else thought the same thing as me. Cognitive convergence...

Thanks to everyone who answered. I don't intend to buy any account. I'll keep asking whatever I want. And I'm newer to this than I was to s** at 16.

I don't know what things are illegal in this forum. So please tell me if I broke any rules.
I discovered that you don't lose merit when sending merit. I read how smerit works. To test it out I created aleph1 to dump all my smerit in and recovered half of it.

@aleph1 was just an experiment.

I tend to post the first thing that goes through my mind, so I respect and like when other people do so as well!

Is there a way to safely buy/sell accounts? A dedicated auction site or something? How is the transaction done?

I was surprised to see that buying and selling accounts is not prohibited in this forum. But looking into it I only saw offers that looked like scams (based on the red alert that flags on top of them).

So I'm wondering if there truly are account sellers. Or even buyers! Is this a liquid market? How much would someone pay on average for a M/Full M/Sr. M/Hero M/L account?

Some days ago I asked for help with the building of a trusted clock using BTC. I got a few replies that helped a lot and led me to a solution to the problem. But my true goal is to code a program that acts like a time capsule. Its task is to safely encrypt data during a set period of time and decrypt it afterwards. Here it is in a more detailed form.

• Input (at t=now): release_date (UNIX time value), encrypt=True (bool).
• Output (at t=now): encrypt_key (type key).
• Input (at t≥release_date): encrypt_key (type key), encrypt=False (bool).
• Output (at t≥release_date): decrypt_key (type key).

Take some data you want to send to the future. Give the program a release date, encrypt your data with encrypt_key and throw the raw data and the encryption key away. Safely store the encrypted data. Wait. After the specified date, give the program back its encrypt_key and get back your decrypt_key to decrypt your data with.

Everything must be run locally.

Critical assumption: the user is good-intentioned right until they throw away the raw data, after which they feel an immense regret and become malicious.

The problem: make it as hard as possible to get decrypt_key before release_date

First idea (which led to the previous post): 1) make the program get the current date from a trusted source and compare it to release_date; 2) if time is up proceed to outputting decrypt_key.

This can be beautifully solved by BTC (or any other PoW-based cryptocurrency, really): after release_date, give the program the longest chain of block headers; the program will check the PoW, that the difficulty increase is coherent and that the time intervals are not suspicious (i.e. 10 mins on average, also if the last time interval is 2 days, it might suggest that a malicious user mined their own last block and faked the timestamp... a few details should be taken into account despite compromising accuracy). The beauty for me is that the program doesn't care if the chain is the longest or whether it's been confirmed by the network: it only wants the proof of work; and the good-intentioned user doesn't need to bother about calculating that PoW: miners do that for them!

This already makes for long-term reliable (though potentially very rough) time-checking method, but my task is to build a time capsule. I still face the issue of having to obfuscate the code responsible for producing decrypt_key. Sadly there is no way to avoid the threat of reverse engineering, especially for locally run programs. If a machine can follow the instructions in your executable, so can a talented enough human.

Second idea (here you BTC geeks might help out): using the lock_time parameter. The program can issue a transaction that's time-locked to release_date. How could I exploit this feature? Or to put it another way: what kind of information does the user gain when a transaction is processed by the network?

I came across some posts about a certain puzzle transaction that sent some good 32 BTC to 256 addresses.

What's the point in brute forcing all of these addresses. Aren't they empty?

Yeah, in some way, but no network involved. I'll start from the beginning. My goal is to code a time capsule. The program should work as follows.

• Input (at t=now): release_date (UNIX time value), encrypt=True (bool).
• Output (at t=now): encrypt_key (type key).
• Input (at t≥release_date): encrypt_key (type key), encrypt=False (bool).
• Output (at t≥release_date): decrypt_key (type key).

Take some data you want to send to the future. Give the program a release date, encrypt your data with encrypt_key and throw the raw data and the encryption key away. Safely store the encrypted data. Wait. After the specified date, give the program back its encrypt_key and get back your decrypt_key to decrypt your data with.

Everything must be run locally.

Critical assumption: the user is good-intentioned right until they throw away the raw data, after which they feel an immense regret and become malicious.

The problem: make it as hard as possible to get decrypt_key before release_date

First idea (which led to this post): 1) make the program get the current date from a trusted source and compare it to release_date; 2) if time is up proceed to outputting decrypt_key.

This can be beautifully solved by BTC (or any other PoW-based cryptocurrency, really): give the program the whole chain of block headers; the program will check the PoW, that the difficulty increase is coherent and that the time intervals are not suspicious (i.e. 10 mins on average, also if the last time interval is 2 days, it might suggest that a malicious user mined their own last block and faked the timestamp... a few details should be taken into account despite compromising accuracy). The beauty for me is that the program doesn't care if the chain is the longest or whether it's been confirmed by the network: it only wants the proof of work; and the good-intentioned user doesn't need to bother about calculating that PoW: miners do that for them!

This already makes for long-term reliable (though potentially very rough) time-checking method, but my task is to build a time capsule. I still face the issue of having to obfuscate the code responsible for producing decrypt_key. Sadly there is no way to avoid the threat of reverse engineering, especially for locally run programs. If a machine can follow the instructions in your executable, so can a talented enough human.

Second idea (here you BTC geeks might help out): using the lock_time parameter. The program can issue a transaction that's time-locked to release_date. How could I exploit this feature? Or to put it another way: what kind of information does the user gain when a transaction is processed by the network?

PD: should I mark this topic as solved and post the time-capsule-related question as a new one? Whatever gathers more attention.