Documentation resources for SPV client development

[andreas_huber69]

I'm considering to implement an SPV client from scratch. Where should I start?

I've had a look at the following links ...

https://en.bitcoin.it/wiki/Protocol_rules
https://en.bitcoin.it/wiki/Category:Technical

... but it seems much of the information revolves around full nodes and isn't very in-depth to begin with.

Thanks!

[1715000817]

1715000817

[1715000817]

1715000817

[1715000817]

1715000817

[Mike Hearn]

You can look at the source code of bitcoinj. Also look at the BIP for bloom filtering.

[R2D221]

I'm considering to implement an SPV client from scratch. Where should I start?

In which language will you write it?

[R2D221]

In which language will you write it?

C# 5.0

If you want to write an application that uses SPV, you can port bitcoinj to .NET easily (see this post). If you want to actually write the library from scratch, then I don't know much about it. I think you can study bitcoinj's source code.

[R2D221]

I also ran into the Portable Class Library issue. And in fact I already started to write a port based on bitcoinj (although with both school and work I have almost no time for this). Maybe we can join forces to get it done faster

[Mike Hearn]

Works on the Windows desktop only. Portable class libraries, Windows Store libraries and Windows Phone libraries don't seem to be possible.

That's unfortunate - I asked before but don't think I got an answer. Is this some policy limitation by Microsoft or is it a technical limitation of IKVM/.NET

There's no way to exploit the much better scalability that comes with the use of async and await.

What do you mean by this? I am not aware of any magic scalability technique that requires C# but would be interested to learn more if there is one.

I don't particularly want to, but it seems there's really no well-documented and well-tested alternative that lets client code take advantage of all the benefits of C# 5.0.

Then you would have to consider if the benefits are really worth all that effort? There's also a C SPV client now, check this forum to find it (it's linux/ncurses).

I think the Mono guys have a tool that auto converts Java to C#, by the way.

[Mike Hearn]

Well-written asynchronous code scales much better than synchronous code, because the former lets you use one thread to talk to e.g. 100 server simultaneously, while the latter requires one thread per server. So much is true for any language. However, for languages that do not support something like async / await, writing asynchronous code tends to become very difficult very quickly, which is why it is often avoided. Async / await lets you write code that looks synchronous but is then automatically turned into asynchronous code by the compiler.

A neat feature for sure, but bitcoinj already uses async IO and has been used in high-scalability scenarios like p2p network crawlers and BlueMatt's relay network.

Right, I haven't decided yet. BTW, what would you estimate is the effort write a minimal SPV client library from scratch?

Define minimal. bitcoinj is the result of years of work but that wasn't full time effort, and some of that time was spent on things like documentation, example apps, lots of unit tests, performance optimisations, implementing Bloom filtering on the C++ side etc. You could ask Maxime who did the C client as he did it more recently, though his code is less complete than bitcoinj.

[R2D221]

Works on the Windows desktop only. Portable class libraries, Windows Store libraries and Windows Phone libraries don't seem to be possible.

That's unfortunate - I asked before but don't think I got an answer. Is this some policy limitation by Microsoft or is it a technical limitation of IKVM/.NET

I think it was me who didn't give a complete answer. It's more of a technical limitation. Since .NET Portable Class Libraries are designed to work on Windows Store, Windows Phone and Silverlight at the same time, they have access to a limited subset of the .NET Framework. This means that anything specific to one of those platforms (files, system settings, etc) are unavailable.

bitcoinj is written in Java, which is totally unaware of all this, and IKVM provides a conversion to .NET, but as far as I know, IKVM has not implemented yet a conversion to Portable Class Library. I don't how difficult that might be, but obviously I couldn't even do it.

[R2D221]

The code you have already ported from bitcoinj, is it open-source?

I plan to release it in the same Apache License that bitcoinj is (but I haven't looked at all the legal implications).

[Mike Hearn]

If the "portable class library" doesn't include files, then yeah, I'm not surprised it doesn't work .... googling around I see this:

http://sourceforge.net/mailarchive/forum.php?thread_name=CAJmyD7Q%3DO%3Dnitf39yF%2BGQewOQ-X%3DoP8CK14TaJ_TahH38Z%2BQ2g%40mail.gmail.com&forum_name=ikvm-developers

So it seems like Microsoft copied Apple's braindead policies around JIT compilers. Runtime code generation isn't allowed and IKVM uses that. Too bad, WP is pretty crippled indeed.

Source to source transpilation is possibly the right way to go for sure.

[fnsa1]

[...]

Right, I haven't decided yet. BTW, what would you estimate is the effort write a minimal SPV client library from scratch?

Define minimal. bitcoinj is the result of years of work but that wasn't full time effort, and some of that time was spent on things like documentation, example apps, lots of unit tests, performance optimisations, implementing Bloom filtering on the C++ side etc. You could ask Maxime who did the C client as he did it more recently, though his code is less complete than bitcoinj.

That would be me -- he, I'm not quite done and I prefer not to quantify the time I spent on this I tried to not cut too many corners for the most part and I wasted time (i.e. had fun) re-implementing a poll loop, an async network i/o stack and various generic libraries. But assuming you re-use existing building blocks, have copious spare time and are proficient in C#, you could get there in a few weeks.

You'll soon be able to look at/re-use bitc code.  In any case, feel free to ping me if you have any question.  Cheers.

[Mike Hearn]

To do that with SPV requires you to implement almost all the core functionality.