[ANN] BTCplex: an open source block chain explorer written in Go

[tomt0m]

I've spent the last 6 months working on a new open source block chain explorer/browser written in Go: https://btcplex.com.
It's an early release (still not production ready), but I would love to hear any feedback!

Some features:

• Browse blocks and transactions.
• Check address balance history.
• Search for blocks, transactions, and addresses.

It also provides a full-featured JSON API, a plain-text query API and few server-sent events endpoint to access data pragmatically.

GitHub repository: https://github.com/tsileo/btcplex
Demo website: https://btcplex.com
Hosted documentation: http://docs.btcplex.com
Donation address: 19gzwTuuZDec8JZEddQUZH9kwzqkBfFtDa

Let me know if you have any suggestion/idea/question !

[1713864573]

1713864573

[1713864573]

1713864573

[btc_enigma]

Hey,

Nice Work !

Is there are reason you are storing block json again in leveldb ? Shouldn't just maintaining the indexes suffice ? You can always get the raw block from bitcoind

[9inety7even]

I would say you need to do some optimization, waiting a week for the database to be built seems unreasonable.

[tomt0m]

@btc_enigma  thanks! I'm storing block data in SSDB (which is backed by LevelDB internally) for performance, by design the webapp never call directly bitcoind.

@9inety7even, I spent a lot of time working on optimization, it takes one week on a small server (1.2Ghz dual core/6GB RAM).
On a more high-end server (i5/16GB RAM, the server btcplex.com is currently hosted), it took 48hours, and I store a lot of data (since disk is cheap, the database is 100+GB), but request are fast, even for querying the balance of an address with thousands of transactions.

[9inety7even]

What seems to be the bottleneck? Do you think you could run it in parallel, or is IO speed the problem?

[SomeoneInNeed]

It's pretty neat and functional. Me like.

[tomt0m]

@9inety7even You're right, the real bottleneck seems to be IO, I think running BTCplex on a server with a SSD should really makes a difference.

I already run everything I can in parallel, but I must parse blocks and transactions chronologically since I need to fetch data about previous TxOuts.
I think 2 days to load the entire block chain is not that bad (when comparing the time it take to start a new bitcoind node), and given the fact that BTCplex keeps track of every known addresses balance.

@SomeoneInNeed Thanks!

[btc_enigma]

@btc_enigma  thanks! I'm storing block data in SSDB (which is backed by LevelDB internally) for performance, by design the webapp never call directly bitcoind.

Hi,

Can you throw some insights/figures on how this is faster than directly calling bitcoind. This is because I am also writing a similar block explorer and I am thinking about design issues. I was trying to  avoid exporting out the whole blocks seeing the huge amount of time its taking.   For example , I did

On testnet3

bitcoind getblock 00000000373403049c5fff2cd653590e8cbe6f7ac639db270e7d1a7503d698df >/tmp/log.txt 2>&1

This completed in 0.013 sec which gives me around 71 requests/second . As far as I know, even bitcoind  would just read its leveldb database stored in ~/.bitcoin and give you the output. So it should give similar performance. Actually I see that c++ code is able to read leveldb faster than python.

[9inety7even]

@9inety7even You're right, the real bottleneck seems to be IO, I think running BTCplex on a server with a SSD should really makes a difference.

I already run everything I can in parallel, but I must parse blocks and transactions chronologically since I need to fetch data about previous TxOuts.
I think 2 days to load the entire block chain is not that bad (when comparing the time it take to start a new bitcoind node), and given the fact that BTCplex keeps track of every known addresses balance.

Yeah I would definitely grab a DO vps and see how much of a difference it makes. Chances are you'll see fairly huge speedups.

[tomt0m]

@btc_enigma Calling bitcoind on itself is fast, but when you add you own server call:

bitcoind <-> your app <-> client
your app <-> client

Even if you can perform multiple RPC calls at once to prevent HTTP round-trip time, I prefer maintaining my own database, when I request a block, I'll want to also fetch every related transactions (sometimes 1000+ txs), that's why BTCplex save the block with every belonging transactions in a single key, so I think it's faster in my use case, but depending on what you want to accomplish, you may achieve better performance calling bitcoind RPC API rather than maintaining your own database.

@9inety7even I would love to but sadly I can't afford a 160USD/month server (database is 120+GB), maybe I'll give it a try if I get enough donation.

[9inety7even]

@9inety7even I would love to but sadly I can't afford a 160USD/month server (database is 120+GB), maybe I'll give it a try if I get enough donation.

You could buy it for a few hours just to test the speed; I think the lowest of the high-end plans is like 30 cents an hour.

[tomt0m]

Bump!