Bitcoin Forum
October 20, 2018, 10:04:08 AM *
News: Make sure you are not using versions of Bitcoin Core other than 0.17.0 [Torrent], 0.16.3, 0.15.2, or 0.14.3. More info.
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: Documentation resources for SPV client development  (Read 1196 times)
andreas_huber69
Newbie
*
Offline Offline

Activity: 31
Merit: 0


View Profile
January 19, 2014, 09:56:57 PM
 #1

I'm considering to implement an SPV client from scratch. Where should I start?

I've had a look at the following links ...

https://en.bitcoin.it/wiki/Protocol_rules
https://en.bitcoin.it/wiki/Category:Technical

... but it seems much of the information revolves around full nodes and isn't very in-depth to begin with.

Thanks!
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1540029848
Hero Member
*
Offline Offline

Posts: 1540029848

View Profile Personal Message (Offline)

Ignore
1540029848
Reply with quote  #2

1540029848
Report to moderator
Mike Hearn
Legendary
*
Offline Offline

Activity: 1526
Merit: 1006


View Profile
January 20, 2014, 09:03:38 PM
 #2

You can look at the source code of bitcoinj. Also look at the BIP for bloom filtering.
R2D221
Hero Member
*****
Offline Offline

Activity: 658
Merit: 500



View Profile
January 22, 2014, 11:21:53 PM
 #3

I'm considering to implement an SPV client from scratch. Where should I start?
In which language will you write it?

An economy based on endless growth is unsustainable.
andreas_huber69
Newbie
*
Offline Offline

Activity: 31
Merit: 0


View Profile
January 24, 2014, 03:32:09 PM
 #4

You can look at the source code of bitcoinj. Also look at the BIP for bloom filtering.

So I guess this means that existing docs are out of date?

I wanted to avoid looking at existing implementations as that always has the danger of reimplementing existing bugs. I'm saying this out of a theoretical consideration, not because I think that bitcoinj is buggy or something.

Thanks,

Andreas
andreas_huber69
Newbie
*
Offline Offline

Activity: 31
Merit: 0


View Profile
January 24, 2014, 03:39:04 PM
 #5

In which language will you write it?

C# 5.0
R2D221
Hero Member
*****
Offline Offline

Activity: 658
Merit: 500



View Profile
January 24, 2014, 05:38:12 PM
 #6

In which language will you write it?
C# 5.0
If you want to write an application that uses SPV, you can port bitcoinj to .NET easily (see this post). If you want to actually write the library from scratch, then I don't know much about it. I think you can study bitcoinj's source code.

An economy based on endless growth is unsustainable.
andreas_huber69
Newbie
*
Offline Offline

Activity: 31
Merit: 0


View Profile
January 24, 2014, 07:41:37 PM
 #7

If you want to write an application that uses SPV, you can port bitcoinj to .NET easily (see this post).

An IKVM approach has at least the following drawbacks:

  • Works on the Windows desktop only. Portable class libraries, Windows Store libraries and Windows Phone libraries don't seem to be possible.
  • There's no way to exploit the much better scalability that comes with the use of async and await.

Quote
If you want to actually write the library from scratch, then I don't know much about it. I think you can study bitcoinj's source code.

I don't particularly want to, but it seems there's really no well-documented and well-tested alternative that lets client code take advantage of all the benefits of C# 5.0.
R2D221
Hero Member
*****
Offline Offline

Activity: 658
Merit: 500



View Profile
January 24, 2014, 07:51:41 PM
 #8

I also ran into the Portable Class Library issue. And in fact I already started to write a port based on bitcoinj (although with both school and work I have almost no time for this). Maybe we can join forces to get it done faster Smiley

An economy based on endless growth is unsustainable.
andreas_huber69
Newbie
*
Offline Offline

Activity: 31
Merit: 0


View Profile
January 24, 2014, 08:21:14 PM
 #9

I also ran into the Portable Class Library issue. And in fact I already started to write a port based on bitcoinj (although with both school and work I have almost no time for this). Maybe we can join forces to get it done faster Smiley

The code you have already ported from bitcoinj, is it open-source?
Mike Hearn
Legendary
*
Offline Offline

Activity: 1526
Merit: 1006


View Profile
January 24, 2014, 08:58:02 PM
 #10

Works on the Windows desktop only. Portable class libraries, Windows Store libraries and Windows Phone libraries don't seem to be possible.

That's unfortunate - I asked before but don't think I got an answer. Is this some policy limitation by Microsoft or is it a technical limitation of IKVM/.NET

Quote
There's no way to exploit the much better scalability that comes with the use of async and await.

What do you mean by this? I am not aware of any magic scalability technique that requires C# but would be interested to learn more if there is one.

Quote
I don't particularly want to, but it seems there's really no well-documented and well-tested alternative that lets client code take advantage of all the benefits of C# 5.0.

Then you would have to consider if the benefits are really worth all that effort? There's also a C SPV client now, check this forum to find it (it's linux/ncurses).

I think the Mono guys have a tool that auto converts Java to C#, by the way.
andreas_huber69
Newbie
*
Offline Offline

Activity: 31
Merit: 0


View Profile
January 24, 2014, 09:38:12 PM
 #11

Works on the Windows desktop only. Portable class libraries, Windows Store libraries and Windows Phone libraries don't seem to be possible.

That's unfortunate - I asked before but don't think I got an answer. Is this some policy limitation by Microsoft or is it a technical limitation of IKVM/.NET

I would guess it's mainly the latter.

Quote
Quote
There's no way to exploit the much better scalability that comes with the use of async and await.

What do you mean by this? I am not aware of any magic scalability technique that requires C# but would be interested to learn more if there is one.

Well-written asynchronous code scales much better than synchronous code, because the former lets you use one thread to talk to e.g. 100 server simultaneously, while the latter requires one thread per server. So much is true for any language. However, for languages that do not support something like async / await, writing asynchronous code tends to become very difficult very quickly, which is why it is often avoided. Async / await lets you write code that looks synchronous but is then automatically turned into asynchronous code by the compiler.

Quote
Quote
I don't particularly want to, but it seems there's really no well-documented and well-tested alternative that lets client code take advantage of all the benefits of C# 5.0.

Then you would have to consider if the benefits are really worth all that effort? There's also a C SPV client now, check this forum to find it (it's linux/ncurses).

Right, I haven't decided yet. BTW, what would you estimate is the effort write a minimal SPV client library from scratch?

Quote
I think the Mono guys have a tool that auto converts Java to C#, by the way.

I know, I guess I would be using that extensively.
Mike Hearn
Legendary
*
Offline Offline

Activity: 1526
Merit: 1006


View Profile
January 24, 2014, 09:42:49 PM
 #12

Well-written asynchronous code scales much better than synchronous code, because the former lets you use one thread to talk to e.g. 100 server simultaneously, while the latter requires one thread per server. So much is true for any language. However, for languages that do not support something like async / await, writing asynchronous code tends to become very difficult very quickly, which is why it is often avoided. Async / await lets you write code that looks synchronous but is then automatically turned into asynchronous code by the compiler.

A neat feature for sure, but bitcoinj already uses async IO and has been used in high-scalability scenarios like p2p network crawlers and BlueMatt's relay network.

Quote
Right, I haven't decided yet. BTW, what would you estimate is the effort write a minimal SPV client library from scratch?

Define minimal. bitcoinj is the result of years of work but that wasn't full time effort, and some of that time was spent on things like documentation, example apps, lots of unit tests, performance optimisations, implementing Bloom filtering on the C++ side etc. You could ask Maxime who did the C client as he did it more recently, though his code is less complete than bitcoinj.
R2D221
Hero Member
*****
Offline Offline

Activity: 658
Merit: 500



View Profile
January 24, 2014, 09:44:39 PM
 #13

Works on the Windows desktop only. Portable class libraries, Windows Store libraries and Windows Phone libraries don't seem to be possible.
That's unfortunate - I asked before but don't think I got an answer. Is this some policy limitation by Microsoft or is it a technical limitation of IKVM/.NET
I think it was me who didn't give a complete answer. It's more of a technical limitation. Since .NET Portable Class Libraries are designed to work on Windows Store, Windows Phone and Silverlight at the same time, they have access to a limited subset of the .NET Framework. This means that anything specific to one of those platforms (files, system settings, etc) are unavailable.

bitcoinj is written in Java, which is totally unaware of all this, and IKVM provides a conversion to .NET, but as far as I know, IKVM has not implemented yet a conversion to Portable Class Library. I don't how difficult that might be, but obviously I couldn't even do it.

An economy based on endless growth is unsustainable.
R2D221
Hero Member
*****
Offline Offline

Activity: 658
Merit: 500



View Profile
January 24, 2014, 09:46:02 PM
 #14

The code you have already ported from bitcoinj, is it open-source?
I plan to release it in the same Apache License that bitcoinj is (but I haven't looked at all the legal implications).

An economy based on endless growth is unsustainable.
Mike Hearn
Legendary
*
Offline Offline

Activity: 1526
Merit: 1006


View Profile
January 24, 2014, 09:57:09 PM
 #15

If the "portable class library" doesn't include files, then yeah, I'm not surprised it doesn't work .... googling around I see this:

http://sourceforge.net/mailarchive/forum.php?thread_name=CAJmyD7Q%3DO%3Dnitf39yF%2BGQewOQ-X%3DoP8CK14TaJ_TahH38Z%2BQ2g%40mail.gmail.com&forum_name=ikvm-developers

So it seems like Microsoft copied Apple's braindead policies around JIT compilers. Runtime code generation isn't allowed and IKVM uses that. Too bad, WP is pretty crippled indeed.

Source to source transpilation is possibly the right way to go for sure.
fnsa1
Newbie
*
Offline Offline

Activity: 19
Merit: 0


View Profile
January 25, 2014, 12:10:40 AM
 #16

[...]
Quote
Right, I haven't decided yet. BTW, what would you estimate is the effort write a minimal SPV client library from scratch?

Define minimal. bitcoinj is the result of years of work but that wasn't full time effort, and some of that time was spent on things like documentation, example apps, lots of unit tests, performance optimisations, implementing Bloom filtering on the C++ side etc. You could ask Maxime who did the C client as he did it more recently, though his code is less complete than bitcoinj.

That would be me -- he, I'm not quite done and I prefer not to quantify the time I spent on this Smiley I tried to not cut too many corners for the most part and I wasted time (i.e. had fun) re-implementing a poll loop, an async network i/o stack and various generic libraries. But assuming you re-use existing building blocks, have copious spare time and are proficient in C#, you could get there in a few weeks.

You'll soon be able to look at/re-use bitc code.  In any case, feel free to ping me if you have any question.  Cheers.
andreas_huber69
Newbie
*
Offline Offline

Activity: 31
Merit: 0


View Profile
January 25, 2014, 09:53:09 AM
 #17

Define minimal.

The first version of the library will most likely not do more than the following:
  • Return the current balance for a given address
  • Send coins from a given address to two other addresses
Mike Hearn
Legendary
*
Offline Offline

Activity: 1526
Merit: 1006


View Profile
January 25, 2014, 04:00:50 PM
 #18

To do that with SPV requires you to implement almost all the core functionality.
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!