Ninjastic.space - BitcoinTalk Post/Address archive + API

[Csmiami]

----

But it doesn't show it inside the profile overview; it only does once you search only that address. It does only display the number of times a user has posted that address, and show only those posts.

It'd be a lot faster not to hand check every address to see if it has been used by someone else by just displaying that

[dkbit98]

@TryNinja is it possible to add option for searching Boards the same way like we are searching Topics?
For example, I would like to see all accounts that ever posted in Croatian local board or Scam Accusations board.
That would mean adding new Board ID option with all current (Post ID, Topic ID, Address, Username, User ID).

[Stalker22]

@TryNinja is it possible to add option for searching Boards the same way like we are searching Topics?

I think we already have that option.

For example, I would like to see all accounts that ever posted in Croatian local board or Scam Accusations board.
That would mean adding new Board ID option with all current (Post ID, Topic ID, Address, Username, User ID).

All you have to do is go to the Posts serach page and under 'Search params' select the board you want to search. Very practical here is also the ability to select a date range for the search. In the results, you have a second tab with a list of all users.

For example, here is a list of all users who posted in Croatian board today. You can also include sub-boards if you wish.



I hope this helps.

[LoyceV]

Feature request: Search Titles only (instead of Content).

[TryNinja]

Feature request: Search Titles only (instead of Content).

It's already planned and included in the API refactor. I just need to find some time to finish it.

[Stalker22]

Feature request (or Bug report): Multi-word usernames in Posts Search params don't work properly.

When I fill in the 'Author' field with a multi-word username, there are no search results. I tried to put in quotes but to no effect.

EDIT: The same thing happens with usernames that contain some special characters at the beginning or at the end.
For example, if we search for posts from "zasad@" or "-ck", there are no results. (Sorry guys, I had to mention you here.)

[TryNinja]

Feature request (or Bug report): Multi-word usernames in Posts Search params don't work properly.

Thanks. It should be working now. The new version coming out in a few weeks should fix most issues and add some new features.

[Stalker22]

Feature request (or Bug report): Multi-word usernames in Posts Search params don't work properly.

Thanks. It should be working now. The new version coming out in a few weeks should fix most issues and add some new features.

The fastest customer support I've ever experienced. Great job!
I'm very much looking forward to the new version and new features.

[LoyceV]

Do you have any idea what happened to post 55860583? My version doesn't make sense.

[TryNinja]

Do you have any idea what happened to post 55860583? My version doesn't make sense.

It wasn't scrapped. My script only saved the posts which had an author (with a correct UID), so all Anonymous and Guest posts are missing.

[LoyceV]

My script only saved the posts which had an author (with a correct UID), so all Anonymous and Guest posts are missing.

I clearly failed to check for this, but as far as I know anonymous posts aren't possible. I didn't save the patrol snapshot so I can't check what was really there.

[Csmiami]

----

IIRC deleted accounts show as Anonymous. Why do you think your version is wrong? that's exactly what I see when I search for the actual post, made on 2011: https://bitcointalk.org/index.php?topic=3110.msg55860583#msg55860583

[TryNinja]

I clearly failed to check for this, but as far as I know anonymous posts aren't possible. I didn't save the patrol snapshot so I can't check what was really there.

Oh, wait. When was this scraped? I just noticed that's the post ID 55860583 and the link/content leads to post ID 43455 from 2011.

Maybe it's possible that theymos anonymized the user recently and somehow the post got a new id (long shot)? I could see this happening due to the recent Ledger leak... But that probably doesn't make any sense.

[LoyceV]

IIRC deleted accounts show as Anonymous.

Do accounts get deleted?

Why do you think your version is wrong? that's exactly what I see when I search for the actual post, made on 2011: https://bitcointalk.org/index.php?topic=3110.msg55860583#msg55860583

That's post 43455, post 55860583 was deleted.

When was this scraped?

Last Friday. My guess would be it was a spambot copying an old post.

Maybe it's possible that theymos anonymized the user recently and somehow the post got a new id (long shot)?

Nope. That post has always been anonymous.

[Csmiami]

IIRC deleted accounts show as Anonymous.

Do accounts get deleted?

I read something that in the past you could request your account to be deleted, it amy take a while before I can get to search the reference tho, many IRL things going on before thursday....

Why do you think your version is wrong? that's exactly what I see when I search for the actual post, made on 2011: https://bitcointalk.org/index.php?topic=3110.msg55860583#msg55860583

That's post 43455, post 55860583 was deleted.

I think that means enough studying for today.... I don't even read simple numbers

[hosseinimr93]

Do you have any idea what happened to post 55860583? My version doesn't make sense.

According to modlog, that post had been made by jorenabanu19.



Your scrapper failed to scrap the user ID of the poster, probably because it was deleted so fast.

[suchmoon]

Do you have any idea what happened to post 55860583? My version doesn't make sense.

According to modlog, that post had been made by jorenabanu19.



Your scrapper failed to scrap the user ID of the poster, probably because it was deleted so fast.

It was a copy-pasta post that had a bunch of those "watch live something or other" spam links, possibly edited in after the post was created.

[BlackHatCoiner]

Wow, that's great. I guess that's a huge database, similar with Loyce's club. The design it awesome, was this a template or did our Ninja write it by himself? I suck on front-end, so I don't know if was that hard.

I don't understand, though, how you guys scrape all this giant information. @TryNinja, did you wrote a script that goes on every single post and saves it on a database? Once I search for my account does it search among all these millions posts? Is this ran on a VPS? Is it expensive? I'm curious, because I get my account's result within a second.

Edit: (Nevermind, just saw the 18$/month, that must be pretty worthy)

[TryNinja]

The design it awesome, was this a template or did our Ninja write it by himself? I suck on front-end, so I don't know if was that hard.

Thanks. I made it myself with the help of the Ant Design library. I also suck on front-end, but I Try.

I don't understand, though, how you guys scrape all this giant information. @TryNinja, did you wrote a script that goes on every single post and saves it on a database?

Old posts = courtesy of our favorite bot human, LoyceV. I did write a script to scrape all of his posts.
New posts = scraped from the recent posts page.
Merits = scraped from the recent merits page.

Once I search for my account does it search among all these millions posts?

Yes. Everything is stored in a PostgreSQL database and indexed for searching in a Elastic Search database (life saver).

Is this ran on a VPS?

Yes. Contabo's "VPS L SSD".

Is it expensive? I'm curious, because I get my account's result within a second.

The VPS costs ~€15 ($18) per month. It's not that powerful, but it does have 30 GB of RAM to help with the searching.

You should have seen how slow it was on the first version when the search was done through the PostgreSQL database. I'm talking about 3+ minutes, terrible results and some server crashing.

[Rizzrack]

@TryNinja I have a small request
On page https://ninjastic.space/search can you make the "author" field set to exact matches or add a dropdown to input uid
If the username has a space it thinks there are multiple keywords and may display hundreds of usernames
Otherwise very nice