NBitcoin : Almost full bitcoin implementation in .NET

[Nicolas Dorier]

Hi all,

I passed the two last weeks, full time, porting the C++ code of bitcoin on C#.
It is the most complete and faithful porting I know of bitcoin.
Here is a copy paste of the info I put on the github page.

For those familiar with visual studio, it is available on nuget:
Install-Package NBitcoin

Why is it a big deal ? Because you can run it and debug into it without any linux-voodoo-setup to make it run.
Visual studio express for free, XUnit and you are up to go.

• Full port of the test suite of bitcoin core with their own data
• Full script evaluation and parsing
• Recognize standard script and permit to create them
• Object model faithful to the C++ API but with C# goodness
• Simpler API (here is how to generate a key and get the address : new Key().PubKey.Address.ToString())
• Bloom filter, partial merkle tree
• Serialization of Blocks, Transactions, Script
• Signing/verification with private keys, support compact signature for prooving ownership
• Deterministic wallet : BIP 32

Then the RPC client part. Which I need your help because I don't have any satoshi myself, so I can't emit transactions for testing. If you like my work, submit the testing satoshis to 15sYbVpRh6dyWycZMwPdxJWD4xbfxReeHe

Public classes are clean and easy to use, but the implementation is a little messy due to the fact that I'm using C# convention and C++ and the same time. (I will clean that up after the RPC client implementation)

I ported directly from the C++, except the OpenSSL part where I'm using BouncyCaslte instead. (BitcoinJ helped me a lot on the implementation)
I also ported OpenSSL bugs (you can't believe how much time it took me)

Please, use the code to explore/learn/debug/play/sharing/create the licence is LGPL v3, so you should be good to go.
This is the simple way and most complete way to see the internal of bitcoin without going to C++ madness.

With no so much work, it should be Mono compliant. I don't have a lot of dependency on the Windows.

If you like my work, send some satoshi I can crucify for the testing of the RPC client.

Info :
github : https://github.com/NicolasDorier/NBitcoin
bitcoin address : 15sYbVpRh6dyWycZMwPdxJWD4xbfxReeHe

Useful link :
Nuget site : https://www.nuget.org/packages/NBitcoin/
Visual studio express : http://www.visualstudio.com/en-us/products/visual-studio-express-vs.aspx
XUnit runner (only if using vs express) : http://xunit.codeplex.com/releases

[DeathAndTaxes]

Wow got to take a look at this.

Still you are kidding about needing coins to do testing right?
https://tpfaucet.appspot.com/

[Nicolas Dorier]

I did not know about the test network .(seen in the code that we could change, but had no idea about an existing test network)
Thanks for the tips, I can now start implementing the rpc client.

Even if I ported the code, I am relatively new to bitcoin, I only knew the basics before starting to coding.

[DeathAndTaxes]

Well glad I could help.   Testnet exists for that exact purpose.  Depending on your testing needs you may also want to look into "testnet in a box" which is a self contained testnet that exists only on your system.

BTW I have no problem with you asking for tips just didn't want you wasting them thinking they need to be used for testing.  Glad to see some .net development.  Will take a look at your code this weekend.

[mcaizgk2]

Very nice initiative and as @DeathAndTaxes said it's good to see more people getting involved with C# for Bitcoin.

I can now start implementing the rpc client.

Before you decide to reinvent the wheel I would suggest you take a look at this C# RPC API library & wrapper for Bitcoin, Litecoin and Bitcoin-based ALTs:

https://github.com/GeorgeKimionis/BitcoinLib

Some of BitcoinLib's features:

- Fully compatible and up-to-date with Bitcoin 0.9.1 RPC API.
- Design-by-contract, service-oriented architecture.
- Strongly-typed structures for complex RPC requests and responses.
- Implicit JSON casting for all RPC messages.
- Extended methods for every-day scenarios where the built-in methods fall short.
- Exposure of all RPC API's functionality as well as the extended methods through a single interface.
- Fallback mechanism for timed-out RPC requests.
- Custom RPC exceptions.
- Supports all Bitcoin clones.
- Can operate on unlimited daemons with a single library reference.
- Litecoin integration included.
- Each coin instance can be fully parametrized at run-time and implement its own constants.
- Console and web test clients out of the box with demo methods implemented in them.
- Testnet ready.
- Fully configurable.

[Nicolas Dorier]

Thanks, it will definitively help me for the implementation. I note that.
I will keep reimplementing that in NBitcoin, since my goal is to have a full implementation of bitcoin, and not just the RPC part.

[Nicolas Dorier]

Just added different network support.
Implementing bitcoin and sharing with this forum is the best way to learn for sure.

[AirBreather]

Ahh damn, you beat me to it.  I wanted to have the first (mostly) independent full-node reimplementation in .NET.  I guess I'll have to focus on making mine the best.

Some things I've learned that may be relevant to your experiences too:

• If you're willing to experiment with an x64-specific build, I would highly recommend checking out https://github.com/joshlang/Secp256k1.NET.  It made a world of difference in the time for my client to do the initial sync.  My super early WIP code uses it, with BouncyCastle as a fallback, like so (obviously doesn't have any special hashtype-related behavior).
• There's no good LevelDB library on .NET; I think it would be a really big value-add to be able to run off the standard database from the Satoshi client, so just a couple of days ago, I dipped my feet into bitcoin's leveldb tree and started working on https://github.com/airbreather/LevelDB.NET.  It won't work on Mono, it's just a C++/CLI bridge, so I've got a fallback for some other things for the IL-only version.

EDIT: I accidentally a word.

[Nicolas Dorier]

Air

Ahh damn, you beat me to it.  I wanted to have the first (mostly) independent full-node reimplementation in .NET.  I guess I'll have to focus on making mine the best.

Some things I've learned that may be relevant to your experiences too:

• If you're willing to experiment with an x64-specific build, I would highly recommend checking out https://github.com/joshlang/Secp256k1.NET.  It made a world of difference in the time for my client to do the initial sync.  My super early WIP code uses it, with BouncyCastle as a fallback, like so (obviously doesn't have any special hashtype-related behavior).
• There's no good LevelDB library on .NET; I think it would be a really big value-add to be able to run off the standard database from the Satoshi client, so just a couple of days ago, I dipped my feet into bitcoin's leveldb tree and started working on https://github.com/airbreather/LevelDB.NET.  It won't work on Mono, it's just a C++/CLI bridge, so I've got a fallback for some other things for the IL-only version.

EDIT: I accidentally a word.

Keep your implementation, maybe I am lacking feature that only you have. The ECDSA alternative is very intesting, I might try to make it works on x86. I add that to my check list..

For the DB, I am using SqLite as a NoSql store, it works well. The indexing of the 300 000 blocks take only 5 minutes. (Compared to the 5 hours of the core implementation when lvldb is corrupt)
The size of the db afterward is 35 MB. I would have a problem though if I intend to index transactions. (It would take 170 GB given that 1 block = 5000 Tx)
The block files of the satoshi client are not from LevelDB, but raw blocks with a small header. I created the code to parse it (StoredBlock) as well to write it.

I created an article on only the crypto part I have coded : http://www.codeproject.com/Articles/768412/NBitcoin-The-most-complete-Bitcoin-port-Part-Crypt, maybe it can interest you.

Today, I am banging my head on the parsing of their wallet which is not a leveldb structure. I also need to make the RPC client/server part.

For LevelDB... I stopped to try when I could not compile that stuff on windows without installing X dependency.

[5flags]

What problem are you trying to solve here? What does your .Net implementation give us that that Bitcoin-QT or talking to bitcoind over RPC doesn't?

I'm totally in favour of alternative clients, but all of them seem to solve a particular problem. Are you planning to simply keep up to date with the bitcoin-core changes? Are you planning to head in a different direction?

[DeathAndTaxes]

What problem are you trying to solve here? What does your .Net implementation give us that that Bitcoin-QT or talking to bitcoind over RPC doesn't?

I'm totally in favour of alternative clients, but all of them seem to solve a particular problem. Are you planning to simply keep up to date with the bitcoin-core changes? Are you planning to head in a different direction?

It isn't a client, it is a library.

[DeathAndTaxes]

Keep your implementation, maybe I am lacking feature that only you have. The ECDSA alternative is very intesting, I might try to make it works on x86. I add that to my check list..

I wouldn't kill yourself on x86 compatibility.  Honestly who has a machine that is running .net which isn't x64 at this point. Even in QT devs started using x64 builds because of performance and memory issues.  As the blockchain grows the memory demands will also grow with it.

[5flags]

It isn't a client, it is a library.

Aar, I see.

For the benefit of other .Net developers, I'm currently using BitcoinJ in a .Net application (https://github.com/XCoinr/Metrobit) by using IKVM to create a .Net dll from a BitcoinJ bundled jar. I will be writing a how-to when I've nailed the very basic functionality, but happy to share tips with anyone interested.

[Nicolas Dorier]

What problem are you trying to solve here? What does your .Net implementation give us that that Bitcoin-QT or talking to bitcoind over RPC doesn't?

I'm totally in favour of alternative clients, but all of them seem to solve a particular problem. Are you planning to simply keep up to date with the bitcoin-core changes? Are you planning to head in a different direction?

It depends on if you are talking about the crypto part goal, the node implementation goal, or the overall goal.

For the overall goal :
Actually Bitcoin code is in C++. Compiling it on a windows computer, or even linux, and playing by debugging tests is a big battle that will cost you lots of time.
In other words, you can't participate into the bitcoin code if you are not a linux wizard in C++.

With NBitcoin, you clone the project, open with visual studio, and you can directly compile and run tests.
There is no dependencies to compile and fetch on the net.
To use the NBitcoin library, you just use the Nuget Package manager that will automatically download and add reference to NBitcoin with dependencies so you can directly code without any hassle.
In clear : If you have visual studio, it takes 1 minutes to start using the lib in your code, and 1 minute to compile it yourself and run tets.
Also the C++ code is not always obvious to read for business developers (I guess that's ok enough for hackers though), the C# have a cleaner model. (But I try to get as close as I can to the C++ model)

For the crypto part :
You can generate keys or use BIP 32 without the need of any node server to install, which is huge depending on where you deploy your code. (Hosted website will not permit you to run a node server, and you should not need that if you just want to generate keys)

For the node client part :
I think that developers need an easy way to talk to the network.
Why should I be obliged to install bitcoind to use a JSON API, download the whole blockchain, to get informations about a block, when I can just connect to the network and ask for it by using the raw protocol ?
Why if I want just to parse a raw transaction, I need to host a server ?

For the node server part :
Well, this come for free once you coded the client part.
I just permit you to create a indexes on blocks, and soon, transactions without the need to install a dedicated server.
The goal will be to manually create transactions in code from your bitcoin wallet.

Anyway, for now, I'm always a little late on the node part behind bitcoin core, and I intend to follow their direction because they will always be used by the majority. (NBitcoin only limits to C# developers)

[PeanutPower]

Nicolas Dorier you have my respect, thank you!

[Nicolas Dorier]

Nicolas Dorier you have my respect, thank you!

Thanks, don't hesitate to share, ask features or tell issues on github

[tprogex]

Great work thanks.I'll join to project in github.

[DeathAndTaxes]

So I finally got around to taking a look at this library as I need to do some BIP32 work.  All I can say is I wish I had taken a closer look earlier.   This is solid work man.  I am surprised it hasn't been given more attention.  Bonus points for making it a nuget package. 

Is the tip address in your profile valid? 

[Nicolas Dorier]

So I finally got around to taking a look at this library as I need to do some BIP32 work.  All I can say is I wish I had taken a closer look earlier.   This is solid work man.  I am surprised it hasn't been given more attention.  Bonus points for making it a nuget package.  

Is the tip address in your profile valid?  

Yes, this is valid

This lib is very new I started in April I think.

Here is some article to get you started. (some methods might be outdated)
Basic of Bitcoin  http://www.codeproject.com/Articles/768412/NBitcoin-The-most-complete-Bitcoin-port-Part-Crypt (maybe not for you)
Usage of Stealth Address and BIP38 http://www.codeproject.com/Articles/775226/NBitcoin-Cryptography-Part
Scanning of Blockchain http://www.codeproject.com/Articles/784519/NBitcoin-How-to-scan-the-Blockchain

I am working nowadays on a open source JSON service that would permit you to track any Account you want on the blockchain from the simple implementation of a "Scanner" talked in the article. (Colored Coin, stealth payment included)
This will be awesome !

For the BIP32, look at the bip32_tests.cs tests, I'm using the same tests vectors as core.
The base58 representation is called BitcoinExtKey.

[DeathAndTaxes]

A tip sent.  Please continue to development you got something good started here.  The library is very robust for something that was only started a few months ago.  I am also glad to see some good use of C#.  I prefer the syntax of C# or Java C++.  I assume you write C# professionally?   I was also glad to see unit testing, I assume you are a proponent of test driven development?  I don't have much experience with xUnit (mainly used NUnit and when forced Microsoft's test platform) but the syntax was easy to pick up.  Well at least the basis using a little google-fu as needed.