Talksearch.io - Advanced Bitcointalk Search Engine

[LoyceV]

That may not be necessary anymore - theymos suggested I should use the sitemap.xml page. It's a little convoluted, but I managed to hack together a script that will check for deleted/updated posts.

Does this allow you to track all edited posts? More specifically: can you track all posts in which a (spam) URL was added later? I've seen many spambots do this to create a backlink without quickly getting caught.

[NotATether]

That may not be necessary anymore - theymos suggested I should use the sitemap.xml page. It's a little convoluted, but I managed to hack together a script that will check for deleted/updated posts.

Does this allow you to track all edited posts? More specifically: can you track all posts in which a (spam) URL was added later? I've seen many spambots do this to create a backlink without quickly getting caught.

Yes, absolutely. In fact, the reason why I haven't released any updates for the search engine is because I'm waiting for my scraper to catch up on the back-log of edited posts. Admittedly, it could take several months before it's finished. I just hope a firmware update doesn't force me to reboot before then.

[NotATether]

Bump!

[NotATether]

Talksearch is a easiest way to post photos in Bitcointalk. Easy to post photos and a helpful post.

I have been thinking of creating an image search for quite some time.


The updating runner is currently processing topic IDs around the 500k range, which means that topics made before 2014 are now completely in sync.

[GazetaBitcoin]

Hey NotATether, please be aware that a new translation was made by AOBT for your topic:

- Japanese translation, made by Akuma_

Are you so kind, please, to add it to OP as well?

[NotATether]

Hey NotATether, please be aware that a new translation was made by AOBT for your topic:

- Japanese translation, made by Akuma_

Are you so kind, please, to add it to OP as well?

Tomorrow this user already asked me for permission.

[NotATether]

New content update v1.1.2 published

This update improves the search algorithm. Previously, titles were disproportionately boosted. This lead to spam topics such as "sell sell sell" appearing near the top of results when you search for "sell bitcoin". Also, each post message has a title field, but titles are really only applicable for OPs. In the future, I plan to add a new field for distinguishing between title post and not title post.

Also, filler words like "a", "the", "or" etc used to be filtered out of search queries - this has been fixed now, so when your query contains these words, you should now see results instead of the message "no results found".

[NotATether]

What are some features you guys would like to see in Talksearch? (besides translations, which I'm already working on)

Edit: I am planning on caching recent search results for a very short time (maybe 10 minutes or something), so that clicking on Page 2/3/4/5/etc. does not trigger an additional search call on the backend.

[NotATether]

We're now at the 1.6 millionth topic indexed out of about 5.5 million.

[NotATether]

Bump

[NotATether]

I can confirm that posts made on April 1 in boards like "Bitcoin Discussion" and "Bitcoin Technical Support" will incorrectly display the board name containing "Dollar" instead of "Bitcoin" in the search results, however this is a visual issue only - searching and indexing are not broken and still work normally.

(Amazon customer support on Christmas Day vibes etc etc  )

[NotATether]

The Talksearch scraper was temporarily disrupted today by an expired Elasticsearch certificate. I renewed that, and am patching up the posts and updates that were missed by the scraper.

[NotATether]

New app update v1.0.8 published

Technically, v1.0.7, but that release was blocked by CVE-2025-55182 in the Next.JS dependency (the infamous React2Shell - sorry hackers not this time).

This release refreshes the frontend. The new style makes it look more like Google Search. Search results are displayed like Google, with topic name, link name, and some metadata at the bottom.

Pages now look like this:



Note: I am aware of a bug that makes some results say "Untitled Bitcointalk Post". I am working on fixing that.

Note2: The copyright will be updated soon

[NotATether]

We are experiencing temporary problems with the Elasticsearch cluster again, which impacts searches. Please stand by while I fix it.

Update: It seems to be stable now.

[NotATether]

I have invented an MCP server for fetching Bitcointalk posts.

You can use it inside AI tools like ChatGPT when you want it to search for Bitcointalk posts without being rate-limited by the forum or Google.

It is currently beta, and not publicly available - the main topic update still hasn't finished yet, it's at the 3.5 millionth topic or something, so not all posts are indexed yet - so PM me if you want access.

[NotATether]

Massive hardware upgrade planned

Hi everyone,

I have found a great deal for a new dedicated server and I plan to migrate Talksearch to the new server over the coming months.

Because the reindexing still hasn't finished yet, there will be a minimum of several weeks before this happened. It's currently indexed over 4.6 million topics.

Without giving away too much information, here are some of the specs I managed to get a hold of:

- 2 x Intel Xeon Gold 6262 (That's almost 100 threads!)
- 128GB of RAM (4 x 32 GB - DDR4 - RDIMM)
- 2 x 1 TB U.2 NVMe SSDs
- 10Gbps uplink speed

It even has a GPU included! RTX 3050 6GB with 2304 shader cores. This will be a huge boost for implementing natural language search.

All this only for the modest price of $189 a month.

Major app upgrades will be coming to Talksearch over the next few months, possibly with the capability of indexing other content beyond Bitcointalk.

[shahzadafzal]

Massive hardware upgrade planned

Hi everyone,

I have found a great deal for a new dedicated server and I plan to migrate Talksearch to the new server over the coming months.

Because the reindexing still hasn't finished yet, there will be a minimum of several weeks before this happened. It's currently indexed over 4.6 million topics.

Without giving away too much information, here are some of the specs I managed to get a hold of:

- 2 x Intel Xeon Gold 6262 (That's almost 100 threads!)
- 128GB of RAM (4 x 32 GB - DDR4 - RDIMM)
- 2 x 1 TB U.2 NVMe SSDs
- 10Gbps uplink speed

It even has a GPU included! RTX 3050 6GB with 2304 shader cores. This will be a huge boost for implementing natural language search.

All this only for the modest price of $189 a month.

Major app upgrades will be coming to Talksearch over the next few months, possibly with the capability of indexing other content beyond Bitcointalk.

Wow, that is an excellent deal indeed. With 128GB RAM and 2 x Intel Xeon Gold 6262 should normally cost $250-$400/m and this $189/month is definitely a very good price.

I think with this power TalkSearch can go fully AI and current 4.6+ million indexed topics can easily fit in the RAM give it a lighting fast speed.

Would you mind sharing the hosting provider? That's an impressive deal for the hardware specs you've listed. I'm curious whether this is a publicly available offer or a special/custom arrangement for yourself. DM is also fine if you'd prefer not to post the details publicly.

[NotATether]

Wow, that is an excellent deal indeed. With 128GB RAM and 2 x Intel Xeon Gold 6262 should normally cost $250-$400/m and this $189/month is definitely a very good price.

I think with this power TalkSearch can go fully AI and current 4.6+ million indexed topics can easily fit in the RAM give it a lighting fast speed.

That's not even factoring in the retail cost of the GPU.

In about a month or maybe two, all topics will be fully indexed and up-to-date.

The MCP server prototype is already ready - once I move Talksearch to the newer hardware, people will be able to integrate bitcointalk data into ChatGPT, Claude etc. but in order to prevent DDoS attacks using the MCP server, people are going to have to use API keys and credits.

I'm still trying to think of a use for the GPU since I just remembered that ELSER doesn't have a GPU version of the model. It shouldn't really be a problem, but there is also spam/AI detection classification that clearly has a GPU version.

Would you mind sharing the hosting provider? That's an impressive deal for the hardware specs you've listed. I'm curious whether this is a publicly available offer or a special/custom arrangement for yourself. DM is also fine if you'd prefer not to post the details publicly.

The server is hosted by Enzu but I bought it through budgetvm.com