Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: Bitdev181 on September 20, 2017, 04:55:57 AM



Title: Is bitcoinJ a viable Server side bitcoin library?
Post by: Bitdev181 on September 20, 2017, 04:55:57 AM
I recently posted a thread looking for help finding a proper alternative to Bitcoind for a web app.

I began using bitcoinj after it was recommended in the thread, steering away from fully writing everything. i finally have a base understanding of it, but a very small base.

Its been working great, but i see some issues. Wallets must be saved all the time, if a save is missed due to some issue, the wallet will show messed up balances and give restrictions on available inputs. i prefer a more live approach where i can verify the inputs/outputs against the chain that is already downloaded, but bitcoinj seems to only allow a one time update so to speak. if you dont save the wallet to disk when the tx comes in or is replayed, its non existent to your wallet. no "quick check" of the block store to be sure. maybe i am wrong, but its what it seems like. i spent days trying to get a balance to persist without saving the wallet to disk when the TX comes in, and only saving the SPV store and the wallet keys, i dont think its possible with the current setup of bitcoinj for the balance to persist using a live sync without re-download. it seems i must completely delete the chain file, re-download block chain and finally re-populate wallet. could be very time consuming on mainnet correct?

I like bitcoin j, i like how its set up and give a lot of control, but i dont like how i am unable to have a full node. i want a node that i can scan TXs against the downloaded chain, update immediate, but also replay without issues. the point of bitcoinj is that its a light client, so i think i have the wrong tool for the job.

this being said, does any of what i said make sense? is it a valid concern or do i just have no understanding of what i actually need?

if this is a valid concern, is bitcoinj still a viable library for a server side app? or is there a tool better suited for this?

i read some material on writing a client from "scratch", its just too involved. So if i shouldn't continue to use bitcoinj server-side, is there an equivilent that is also pretty low level and allows for "infinite" wallet creation?


Title: Re: Is bitcoinJ a viable Server side bitcoin library?
Post by: ScripterRon on September 20, 2017, 03:28:51 PM
I started using BitcoinJ but eventually ended up writing my own support (see my BitcoinCore project on GitHub).  One of the problems I had was the fact that the entire wallet was rewritten on a checkpoint basis and I needed to integrate the wallet transactions into a database commit/rollback mechanism.  I also needed to handle a large number of private keys and wanted to use a hash lookup when matching incoming transactions to account addresses.  You can see an example in my TokenExchange project (which handles Bitcoin <-> Nxt exchanges).

I wrote a Java bitcoin node (JavaBitcoin).  But I haven't maintained it since it was just a learning experience but it might provide some useful tips.  I now run Bitcoin Core for the server but I still use my own wallet (BitcoinWallet).


Title: Re: Is bitcoinJ a viable Server side bitcoin library?
Post by: Bitdev181 on September 21, 2017, 03:29:12 AM
Wow you really know what you are doing, im checking through your github now. ill spend a few hours reading through code to see if its something i could tackle in a reasonable amount of time or not.

My biggest issue with bitcoinj s that i cant manage to persist a balance without deleting and re-downloading the block-chain. in main net this would be horribly slow for a server side app. maybe i am wrong and im doing it incorrectly, but i have about a week experience with bitcoinj.


My plan b was, use bitcoin-j, rewrite the source a bit to allow for a more "direct" approach. but your classes look a lot like whats in bitcoinj too, did you use bitcoinj as a base? how did you go about writing that?

can you outline a bit about what was involved in writing yoru bitcoin core framework/library or whatever you may call it.

also i notice a lot of bloom filtering and merkel root classes, is this doing the same as bitcoinj by only using spv wallets? or are those classes just carried over?


Title: Re: Is bitcoinJ a viable Server side bitcoin library?
Post by: ScripterRon on September 21, 2017, 03:07:41 PM
Wow you really know what you are doing, im checking through your github now. ill spend a few hours reading through code to see if its something i could tackle in a reasonable amount of time or not.

My biggest issue with bitcoinj s that i cant manage to persist a balance without deleting and re-downloading the block-chain. in main net this would be horribly slow for a server side app. maybe i am wrong and im doing it incorrectly, but i have about a week experience with bitcoinj.


My plan b was, use bitcoin-j, rewrite the source a bit to allow for a more "direct" approach. but your classes look a lot like whats in bitcoinj too, did you use bitcoinj as a base? how did you go about writing that?

can you outline a bit about what was involved in writing yoru bitcoin core framework/library or whatever you may call it.

also i notice a lot of bloom filtering and merkel root classes, is this doing the same as bitcoinj by only using spv wallets? or are those classes just carried over?
My BitcoinCore library supports both servers and wallets.  Thus it has support for bloom filters which are used by SPV wallets.  You need the Merkel root support to validate incoming transactions.

I started using BitcoinJ.  When I decided to go my own way (mainly because BitcoinJ at that time did not support servers), I kept the same class names and parameters so that my existing application code would continue to work.  Since then, the design has diverged significantly, especially with the addition of HD keys and segregated witness transactions.

I'm not sure what you mean when you say you cannot persist a balance.  Do you mean the wallet has no transactions after a restart?  If you look at the BitcoinWallet class in the version-2 branch of TokenExchange, that uses the BitcoinJ wallet support.  So that might give you some pointers on maintaining a wallet using BitcoinJ.  The BitcoinWallet class in the master branch of TokenExchange stores the wallet transactions in a RDBM (H2) but still uses BitcoinJ for the network support.  My BitcoinWallet and JavaBitcoin projects uses my BitcoinCore library for everything (no BitcoinJ dependency).

If you are interested in using the Bitcoin Core RPC interface, take a look at my BitcoinMonitor project.  This displays the status of a Bitcoin Core node.  I use this to monitor my Bitcoin Core VPS node from my desktop.  You can also look at the version-1 branch of TokenExchange which used a Bitcoin Core node for all of the Bitcoin support (as you can guess, I went through multiple design iterations for TokenExchange until I found one that I was happy with).


Title: Re: Is bitcoinJ a viable Server side bitcoin library?
Post by: nicosey on September 28, 2017, 07:33:53 AM
Yes, I believe RSK are using it as part of there Side Chain implementation.


Title: Re: Is bitcoinJ a viable Server side bitcoin library?
Post by: Artimunor on September 28, 2017, 07:59:51 PM
I started using BitcoinJ but eventually ended up writing my own support (see my BitcoinCore project on GitHub).  One of the problems I had was the fact that the entire wallet was rewritten on a checkpoint basis and I needed to integrate the wallet transactions into a database commit/rollback mechanism.  I also needed to handle a large number of private keys and wanted to use a hash lookup when matching incoming transactions to account addresses.  You can see an example in my TokenExchange project (which handles Bitcoin <-> Nxt exchanges).

I wrote a Java bitcoin node (JavaBitcoin).  But I haven't maintained it since it was just a learning experience but it might provide some useful tips.  I now run Bitcoin Core for the server but I still use my own wallet (BitcoinWallet).

Do you have some links to those projects?
Sounds really interesting, and i would love to take a look.


Title: Re: Is bitcoinJ a viable Server side bitcoin library?
Post by: BitcoinPC on November 11, 2017, 02:43:21 PM
I recently posted a thread looking for help finding a proper alternative to Bitcoind for a web app.

I began using bitcoinj after it was recommended in the thread, steering away from fully writing everything. i finally have a base understanding of it, but a very small base.

Its been working great, but i see some issues. Wallets must be saved all the time, if a save is missed due to some issue, the wallet will show messed up balances and give restrictions on available inputs. i prefer a more live approach where i can verify the inputs/outputs against the chain that is already downloaded, but bitcoinj seems to only allow a one time update so to speak. if you dont save the wallet to disk when the tx comes in or is replayed, its non existent to your wallet. no "quick check" of the block store to be sure. maybe i am wrong, but its what it seems like. i spent days trying to get a balance to persist without saving the wallet to disk when the TX comes in, and only saving the SPV store and the wallet keys, i dont think its possible with the current setup of bitcoinj for the balance to persist using a live sync without re-download. it seems i must completely delete the chain file, re-download block chain and finally re-populate wallet. could be very time consuming on mainnet correct?

I like bitcoin j, i like how its set up and give a lot of control, but i dont like how i am unable to have a full node. i want a node that i can scan TXs against the downloaded chain, update immediate, but also replay without issues. the point of bitcoinj is that its a light client, so i think i have the wrong tool for the job.

this being said, does any of what i said make sense? is it a valid concern or do i just have no understanding of what i actually need?

if this is a valid concern, is bitcoinj still a viable library for a server side app? or is there a tool better suited for this?

i read some material on writing a client from "scratch", its just too involved. So if i shouldn't continue to use bitcoinj server-side, is there an equivilent that is also pretty low level and allows for "infinite" wallet creation?

Viable and useful are two different things. There is no reason to use that for anything when there are plenty of other options out there that are better coded than that one. I had to pick it apart the first time track more than six bugs to get it going and when I have to re-write the software for it to work, that is not worth it for me or you.