[ANNOUNCE] BitCoinJ/BitCoinSharp v0.2

[Mike Hearn]

[also sent to bitcoin-development@]

I'm pleased to announce BitCoinJ v0.2. There have been over 100 commits since the first release back in March, which have added:

• Full support for block chain re-orgs, including recognition of dead transactions (that will never be included in the chain).
• Persistence of the block chain using multiple, pluggable stores. A BoundedOverheadBlockStore is provided that is suitable for usage on mobile devices where low memory usage and instant startup time are requirements.
• A much larger test suite
• IRC, DNS  and seed list peer discovery
• ASN.1 key export
• Many many bugfixes and minor API improvements.

This release represents the work of many people. In particular I'd like to thank:

• Andreas Schildbach
• Miron Cuperman
• Gary Rowe
• Thilo Planz
• Micheal Swiggs
• Noa Resare
• John Sample
• Xiaofeng Guo

You can get it from the release-0.2 branch in Subversion or the downloads page.

BitCoinSharp

I'm also very pleased to announce that Nathan Baulch has created BitCoinSharp, a complete port of the library into C# which opens up low level Bitcoin development to .NET developers. You can start reading at the PingService:

  http://code.google.com/p/bitcoinsharp/source/browse/src/Examples/PingService.cs

[TierNolan]

Sorry for so many questions, but is a pretty useful library.

What is the thread safety model for this library?

Is everything thread unsafe unless told otherwise?

If you connect to 2 peers, can you set them to the same block chain?  That way if a block arrives from either peer, it will be incorporated into the chain.

How does the getBlock() method in Peer work.  Does it send a request to the peer and the Future resolves when it gets an answer?  If the peer dies, then it would never complete?

A getTransaction() method would be great for the Peer class.  That would require a new message though .  In theory, you could add it and if bitcoinj gets popular, it would probably be incorporated into the official spec.  Bitcoinj enabled nodes would support the message.  If 1% of clients were bitcoinj nodes, then by, ahem, spamming 100 nodes, you would probably get a hit.

Transaction.verifyInput() has a "warning: not finished" message.  How much functionality has it?  Can it verify the 2 standard scripts?

The Block object doesn't have a way to get at the transactions in the block.  It could have an isHeader() method to indicate that it doesn't have all the information. 

A Block.attemptMine(int attempts) would be nice too, but that is probably outside the scope of the project.  That would need to be combined with a way to add transactions to a block.

It isn't possible to create transactions directly.  The constructor isn't public.  It could be done by sending a fake payload of.

I think "01 00 00 00 00 00 FF FF FF FF" would work. 

version = 00000001
inputs = 00
outputs = 00
lock time = FFFFFFFF

It is a hack though.

Script could also do with a default constructor and some script processing methods.  The script could be broken into pieces of some super class.  The OP_PUSHDATA1 opcode could be combined with the actual data.

[Mike Hearn]

Sorry for so many questions, but is a pretty useful library.

What is the thread safety model for this library?

The intention is that multiple threads can access block chain and wallet instances concurrently. However currently there's only ever one peer thread, so the thread safety hasn't been well tested and there are surely bugs.

If you connect to 2 peers, can you set them to the same block chain?  That way if a block arrives from either peer, it will be incorporated into the chain.

It's intended that will be possible. It may even work today. I haven't tried it yet.

How does the getBlock() method in Peer work.  Does it send a request to the peer and the Future resolves when it gets an answer?  If the peer dies, then it would never complete?

Correct. There are no timeouts on the future.

A getTransaction() method would be great for the Peer class.  That would require a new message though . 

I think you can use getdata on a transaction if it's in memory. Otherwise yes, you're right. But what would it be used for?

Transaction.verifyInput() has a "warning: not finished" message.  How much functionality has it?  Can it verify the 2 standard scripts?

Lightweight/SPV clients don't verify transactions. They take inclusion in the chain as proof of validity. See here:

http://code.google.com/p/bitcoinj/wiki/SecurityModel

The verifyInput code dates from a time before I decided to implement this model and I'll delete it at some point. Verifying transactions means (at minimum) storing them all, which it does not do.

The Block object doesn't have a way to get at the transactions in the block.  It could have an isHeader() method to indicate that it doesn't have all the information. 

Right, that would be useful. Patches welcome.

A Block.attemptMine(int attempts) would be nice too, but that is probably outside the scope of the project.  That would need to be combined with a way to add transactions to a block.

There is already a solve() method which does mining, but it doesn't update the extraNonce so it's only useful for unit-test difficulties.

It isn't possible to create transactions directly.  The constructor isn't public.  It could be done by sending a fake payload.

We can make the bytestream constructor public.

Script could also do with a default constructor and some script processing methods.  The script could be broken into pieces of some super class.  The OP_PUSHDATA1 opcode could be combined with the actual data.

Yes, the Script class needs a lot of work. It'll probably get done in the context of supporting contracts.

[TierNolan]

It's intended that will be possible. It may even work today. I haven't tried it yet.

Great.

A getTransaction() method would be great for the Peer class.  That would require a new message though . 

I think you can use getdata on a transaction if it's in memory. Otherwise yes, you're right. But what would it be used for?

I was thinking that it would allow lightweight clients to verify transactions.  However, on further thought, it would need to indicate which is the matching block.

The message could return how many confirms there are for a particular transaction.  A transaction that was incorporated into the chain 100 blocks ago and was just spent but the spend hasn't been confirmed would return 100,0.  -1 could be used for "unknown transaction".

If the peer was trusted, then that would show the status of the transaction.

The verifyInput code dates from a time before I decided to implement this model and I'll delete it at some point. Verifying transactions means (at minimum) storing them all, which it does not do.

Not, if there is a way to ask the full nodes how many confirms a transaction has.

The method could require that you provide an array of the input transactions.

The Block object doesn't have a way to get at the transactions in the block.  It could have an isHeader() method to indicate that it doesn't have all the information. 

Right, that would be useful. Patches welcome.

What is your procedure for that?  Do contributors just submit their code as a patch file attachment on the issue list?  I know some projects require that contributors have to sign something first (to formally license the submission).

It isn't possible to create transactions directly.  The constructor isn't public.  It could be done by sending a fake payload.

We can make the bytestream constructor public.

I think the byte stream one is already public.  I was thinking of the one that just takes params and generates an empty transaction.

Why do you only have a readback for pending confirmations?  Is this to allow the client to resend them periodically?

Speaking of contracts, you could include a Wallet method that allows coins to be spent to a transaction, rather than broadcasting to the network.  This would allow a contract to be easily built up and give an already formed transaction.

Maybe there could be a "combine inputs" transaction which creates a transaction which has all the same inputs as a given transaction.  This would allow spending a transaction and then adding an arbitrary payload as output.

This could be combined with a "pay to wallet" method which takes a transaction and pays the output to the wallet.

[Mike Hearn]

Not, if there is a way to ask the full nodes how many confirms a transaction has.

You can't verify a transaction by merely checking the inputs are valid. It could be a double spend. The only way to prove it's not a double spend is to be aware of all transactions, so you can track which outputs are spent and which aren't.

What is your procedure for that?  Do contributors just submit their code as a patch file attachment on the issue list?  I know some projects require that contributors have to sign something first (to formally license the submission).

You have to (electronically) sign the Google CLA:

http://code.google.com/legal/individual-cla-v1.0.html

Why do you only have a readback for pending confirmations?  Is this to allow the client to resend them periodically?

I don't understand this question, sorry.

Speaking of contracts, you could include a Wallet method that allows coins to be spent to a transaction, rather than broadcasting to the network.  This would allow a contract to be easily built up and give an already formed transaction.

Maybe there could be a "combine inputs" transaction which creates a transaction which has all the same inputs as a given transaction.  This would allow spending a transaction and then adding an arbitrary payload as output.

This could be combined with a "pay to wallet" method which takes a transaction and pays the output to the wallet.

Yes, there are many ways to do it. That way sounds reasonable. Gary Rowe is interested in contracts as well, you could maybe team up with him.

[TierNolan]

You can't verify a transaction by merely checking the inputs are valid. It could be a double spend. The only way to prove it's not a double spend is to be aware of all transactions, so you can track which outputs are spent and which aren't.

If the node tells you which block has it, you could scan back to just that block, if it isn't to far.  That could be practical for a lightweight client, if the transaction didn't occur to far back the chain.

Why do you only have a readback for pending confirmations?  Is this to allow the client to resend them periodically?

I don't understand this question, sorry.
[/quote]

Sorry, I meant that in Wallet, the only transactions you can read-back are the pending ones with getPendingTransactions(), it doesn't have a method for finding any other transactions.

[Mike Hearn]

The pools are an implementation detail of how the wallet works, that's why. There should be an API for reading all transactions in the wallet together, but just exposing the pools isn't the right way to do it.

[BitterTea]

You can't verify a transaction by merely checking the inputs are valid. It could be a double spend. The only way to prove it's not a double spend is to be aware of all transactions, so you can track which outputs are spent and which aren't.

If the node tells you which block has it, you could scan back to just that block, if it isn't to far.  That could be practical for a lightweight client, if the transaction didn't occur to far back the chain.

BitcoinJ stores serialized StoredBlock objects to disk, without transaction data. The transactions are scanned and discarded in the ConnectBlock method.

[TierNolan]

BitcoinJ stores serialized StoredBlock objects to disk, without transaction data. The transactions are scanned and discarded in the ConnectBlock method.

Right, realisation dawns , because it is purely for tracking your own transactions. 

The block chain headers are the only thing that is stored.  When it starts up again, it just checks new blocks for references to those transactions.

[BitterTea]

BitcoinJ stores serialized StoredBlock objects to disk, without transaction data. The transactions are scanned and discarded in the ConnectBlock method.

Right, realisation dawns , because it is purely for tracking your own transactions. 

The block chain headers are the only thing that is stored.  When it starts up again, it just checks new blocks for references to those transactions.

Yeah, that's right. I'm wondering if it wouldn't be that difficult to create a new block type (FullBlock?) and associated storage mechanism. Mike would know better than I, though, I'm only just starting to play with BitcoinSharp.

[Mike Hearn]

Full mode is a lot of work. You have to store all transactions in a database, load all the dependencies for each transaction, run the scripts, update the spent pointers and more.

There seems to be a lot of interest in making BitCoinJ a full Bitcoin implementation. But there's plenty of work to do on light mode before those two visions diverge.

[BitterTea]

Full mode is a lot of work. You have to store all transactions in a database, load all the dependencies for each transaction, run the scripts, update the spent pointers and more.

There seems to be a lot of interest in making BitCoinJ a full Bitcoin implementation. But there's plenty of work to do on light mode before those two visions diverge.

Shortly after I wrote that I was reading the issue tracker and came across the discussion on the matter. I didn't realize storing transactions would be that difficult, but using a graph DB sounded interesting.

[Mike Hearn]

It's not difficult really. It's just a bunch of work to make sure all the loose ends are handled. There are quite a few rules that are not checked/enforced today because we assume the most difficult chain is valid.

[TYDIRocks]

I'm not the best coder so I have a question with bitcoinsharp

Do I need to include all 3 dlls in my project? I'm also having trouble with this line:

final NetworkParameters params = testNet ? NetworkParameters.testNet() : NetworkParameters.prodNet();

I'm coding this in vb.net so the code isn't exactly the same.

[titeuf_87]

In vb.net that would be the following:

Code:

dim params as NetworkParameters = if(testNet, NetworkParameters.testNet(), NetworkParameters.prodNet())

This is what is called a ternary operation.

[TYDIRocks]

In vb.net that would be the following:

Code:

dim params as NetworkParameters = if(testNet, NetworkParameters.testNet(), NetworkParameters.prodNet())

This is what is called a ternary operation.

Okay and could you help me with the line before that:

boolean testNet = args.length > 0 && args[0].equalsIgnoreCase("testnet");

Also, do I need to include all 3 dlls?

[titeuf_87]

&& in C# is the and operator in vb.net. Also vb.net uses round brackets for arrays instead of square ones.

Code:

dim testNet as boolean = args.length > 0 and args(0).equalsIgnoreCase("testnet")

As for the dlls: you'll certainly need to add BitCoinSharp.dll to your project. I think that just adding that one and keeping the other dll's in the same folder should be good enough.
I can't test it though since I don't have a vb.net or even a c# compiler here at home.

[TYDIRocks]

&& in C# is the and operator in vb.net. Also vb.net uses round brackets for arrays instead of square ones.

Code:

dim testNet as boolean = args.length > 0 and args(0).equalsIgnoreCase("testnet")

As for the dlls: you'll certainly need to add BitCoinSharp.dll to your project. I think that just adding that one and keeping the other dll's in the same folder should be good enough.
I can't test it though since I don't have a vb.net or even a c# compiler here at home.

"args.length" was actually the only problem I had. For some reason i can't put that.

[titeuf_87]

"args.length" was actually the only problem I had. For some reason i can't put that.

Length is the standard way in .NET to get the size of an array back. Paste the error you get and the code surrounding it? The error is probably where the args variable is declared.

Also I just realized something: in this case you should be using AndAlso instead of and. This short-circuits the and so that if the first part is False (the array is empty), the second part will never be evaluated.

Also, please PM me this or start a new topic for this? This isn't really related to BitCoinSharp, so let's not derail the topic