Bitcoin Forum
May 09, 2024, 02:01:57 AM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1] 2 3 4 5 6 7 »  All
  Print  
Author Topic: btcd: a bitcoind alternative written in Go  (Read 20927 times)
behindtext (OP)
Full Member
***
Offline Offline

Activity: 121
Merit: 103


View Profile WWW
May 01, 2013, 10:04:13 PM
Merited by ABCbits (1)
 #1

NOTE: i have syndicated this content from https://blog.conformal.com/ so readers don't need to visit the site.

btcd is an alternative full-node implementation of the bitcoin protocol written in Go and is currently under active development. btcd has been under development for 10 weeks and the initial code is nearly ready for public release. We feel that by providing an alternative to bitcoind we can substantially improve the diversity and resilience of the bitcoin ecosystem and infrastructure.

A number of us at Conformal Systems had been keeping an eye on bitcoin as passive observers for the past couple years since bitcoin combines technologies that are already of interest to us: practical use of cryptography, distributed systems, and electronic payments. In January 2013 I had one of our developers, David Hill, attempt to port bitcoind and its GUI to Bitrig, an OS that several of our developers forked from OpenBSD. David encountered several problems with porting to Bitrig and in the process found issues with unit tests, non-portable functions and seeding of a PRNG. While pushing to get the port complete, it was clear that it would take a lot more effort than usual to complete this port. After seeing these issues with the porting, I felt that the bitcoin ecosystem could use an alternative to bitcoind.

Until starting on btcd, most of our developers had written almost exclusively in C. Our CTO, Marco Peereboom, and developers had been pushing for a new project written entirely in Go. Writing a bitcoind replacement seemed like an interesting project that would take less than 6 months to complete. Once discussion on btcd began in earnest, it became clear that using Go offered a number of advantages over C or C++, especially for financial software:

  • integrated test infrastructure
  • no active memory management
  • standard formatting
  • platform independent code
  • simpler parallelism
  • virtually crash-proof
  • built-in profiling and documentation facilities

The most important Go feature in the context of btcd is that of the test infrastructure: by having robust test infrastructure from the outset, all code can and shall have test coverage. Having full test coverage will ensure that most bugs are caught early in the development cycle, before they cause widespread problems. Since btcd is financial software and errors could lead to someone losing money, we take test coverage particularly seriously.

btcd is a work-in-progress and we will be making the initial source code release in the next couple weeks. At the moment, we have approximately 40% test coverage on our code but we will be expanding our test coverage once more of the core functionality is complete. We have setup btcd at our colocation facility and are using it to generate output similar to blockexplorer.com, have a look at blocksafari.com to see btcd-generated output.

Currently, the following components of btcd are tested as working:

  • Discovery
  • Protocol
  • Crypto, hashing, base58 etc
  • Populating the block database
  • Serve blocks from the database
  • Peer-to-peer manager
  • IPv6 and IPv4 connectivity
  • Execute all transaction scripts currently in use
  • JSON RPC that deals with blocks and transactions
  • Verification of transaction signatures

Most of the code is in pretty good shape however some pieces are incomplete. Within 2 weeks we should have enough core functionality written and at that point we are going to release parts of the code to the general community. After this initial release, we plan on adding new functionality to handle wallets, Tor connectivity, etc.

If you are interested in talking to our developers about btcd, come chat on our public IRC server in channel #btcd.

Activity + Trust + Earned Merit == The Most Recognized Users on Bitcointalk
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
Herbert
Hero Member
*****
Offline Offline

Activity: 488
Merit: 500



View Profile WWW
May 02, 2013, 03:25:30 PM
 #2

We have setup btcd at our colocation facility and are using it to generate output similar to blockexplorer.com, have a look at blocksafari.com to see btcd-generated output.
OMG. Thought my display is broken untill I realized that you have an amazing background there :-)

Ontopic: Very interesting! Do you plan the rpc-interface to be a possbile drop-in replacement for bitcoind?

www.bitcoinmonitor.net - Free payment notification via email, newsfeed, xpmm/jabber, url callback and full API access!
Send SMS with www.txt4coins.net! No registration, pay-per-use, full API access, bulk messages - All inclusive!
gmaxwell
Moderator
Legendary
*
expert
Offline Offline

Activity: 4172
Merit: 8418



View Profile WWW
May 02, 2013, 03:43:14 PM
 #3

Greetings,  sounds interesting.

What strategy are you using to make sure that you're implementing the distributed algorithm consistently with the reference software?
behindtext (OP)
Full Member
***
Offline Offline

Activity: 121
Merit: 103


View Profile WWW
May 02, 2013, 04:45:20 PM
 #4

We have setup btcd at our colocation facility and are using it to generate output similar to blockexplorer.com, have a look at blocksafari.com to see btcd-generated output.
OMG. Thought my display is broken untill I realized that you have an amazing background there :-)

Ontopic: Very interesting! Do you plan the rpc-interface to be a possbile drop-in replacement for bitcoind?

yes, we have got the bulk of the json bitcoind uses working and plan to have full coverage for that. in theory, it will act as a drop-in replacement.

behindtext (OP)
Full Member
***
Offline Offline

Activity: 121
Merit: 103


View Profile WWW
May 02, 2013, 04:56:39 PM
 #5

Greetings,  sounds interesting.

What strategy are you using to make sure that you're implementing the distributed algorithm consistently with the reference software?

our devs have implemented the protocol per the docs at https://en.bitcoin.it/wiki/Protocol_specification . additionally, they have been reading bitcoind code as necessary to make everything work. there are a couple scenarios where bitcoind does not behave according to the published spec. we are currently quirking around this but we can document it if you think it helpful.

the devs have been testing against bitcoind since starting work. at the moment, we have a local bitcoind at our colo and are restricting btcd to interacting with only that bitcoind instance. we will open it up and start communicating with other bitcoind servers soon.

NOTE: i am not a developer. if you want to get into details, i'll need help whitelisting our devs who are registering right now (they don't have bitcointalk accts)

Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 2216


Chief Scientist


View Profile WWW
May 02, 2013, 05:25:58 PM
 #6

our devs have implemented the protocol per the docs at https://en.bitcoin.it/wiki/Protocol_specification . additionally, they have been reading bitcoind code as necessary to make everything work. there are a couple scenarios where bitcoind does not behave according to the published spec. we are currently quirking around this but we can document it if you think it helpful.

Yes, please, feedback from re-implementors is very helpful.



How often do you get the chance to work on a potentially world-changing project?
gmaxwell
Moderator
Legendary
*
expert
Offline Offline

Activity: 4172
Merit: 8418



View Profile WWW
May 03, 2013, 08:49:34 AM
 #7

our devs have implemented the protocol per the docs at https://en.bitcoin.it/wiki/Protocol_specification . additionally, they have been reading bitcoind code as necessary to make everything work. there are a couple scenarios where bitcoind does not behave according to the published spec. we are currently quirking around this but we can document it if you think it helpful.
That text is far from complete. Help improving it would be most welcome.

Any implementation needs to specifically test for uniformity with the network: Bitcoin is a distributed consensus algorithm and differences in what nodes accept or reject in the blockchain— things which would be minor harmless behavioral differences in most software—  can often result in fatal security flaws where an attacker can move the nodes in question onto a separate fork and double-spend their funds away or partition the network. This requires a unusual level of care and system level tests.

Many of the most interesting cases are the great many things which must be rejected as no amount of exposure to the live network will trigger those cases (until an attacker exploits them to partition the network).

Quote
NOTE: i am not a developer. if you want to get into details, i'll need help whitelisting our devs who are registering right now (they don't have bitcointalk accts)
PM me their account names and I'll gladly do so.
behindtext (OP)
Full Member
***
Offline Offline

Activity: 121
Merit: 103


View Profile WWW
May 03, 2013, 05:59:15 PM
 #8

our devs have implemented the protocol per the docs at https://en.bitcoin.it/wiki/Protocol_specification . additionally, they have been reading bitcoind code as necessary to make everything work. there are a couple scenarios where bitcoind does not behave according to the published spec. we are currently quirking around this but we can document it if you think it helpful.
That text is far from complete. Help improving it would be most welcome.

10-4, i'll have the guys put together a list of things that may need updating.

Quote
Any implementation needs to specifically test for uniformity with the network: Bitcoin is a distributed consensus algorithm and differences in what nodes accept or reject in the blockchain— things which would be minor harmless behavioral differences in most software—  can often result in fatal security flaws where an attacker can move the nodes in question onto a separate fork and double-spend their funds away or partition the network. This requires a unusual level of care and system level tests.

davec had noticed some inconsistencies between the protocol spec and behavior which may vary by version. had to do with requesting blocks while building blockchain.

Quote
Many of the most interesting cases are the great many things which must be rejected as no amount of exposure to the live network will trigger those cases (until an attacker exploits them to partition the network).

Quote
NOTE: i am not a developer. if you want to get into details, i'll need help whitelisting our devs who are registering right now (they don't have bitcointalk accts)
PM me their account names and I'll gladly do so.


btcd currently drops all messages that do not match spec, so the approach is to block until it is specifically allowed.

Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 2216


Chief Scientist


View Profile WWW
May 03, 2013, 11:31:28 PM
 #9

10-4, i'll have the guys put together a list of things that may need updating.

Ummm... it is a wiki.  They should just update it themselves (after asking questions if they have any), it is much more efficient.

How often do you get the chance to work on a potentially world-changing project?
Gyrsur
Legendary
*
Offline Offline

Activity: 2856
Merit: 1520


Bitcoin Legal Tender Countries: 2 of 206


View Profile WWW
May 04, 2013, 09:11:10 AM
Last edit: May 09, 2013, 03:50:27 PM by Gyrsur
 #10

very exciting! every additional full implementation will make Bitcoin stronger and more independent in the long run.


EDIT:


killerstorm
Legendary
*
Offline Offline

Activity: 1022
Merit: 1015



View Profile
May 04, 2013, 03:35:46 PM
 #11

Also full node implementations (or are close to it):

https://github.com/bitcoinjs/bitcoinjs-server


https://github.com/bitsofproof/supernode

Chromia: a better dapp platform
gmaxwell
Moderator
Legendary
*
expert
Offline Offline

Activity: 4172
Merit: 8418



View Profile WWW
May 09, 2013, 12:37:36 AM
 #12

Apparently this has been announced: https://blog.conformal.com/btcwire-the-bitcoin-wire-protocol-package-from-btcd/


Quote
David encountered several problems with porting to Bitrig and in the process found issues with unit tests, non-portable functions and seeding of a PRNG.
Ah. This is the folks with the rand() that only returned even values? That was fun. Smiley
matt4054
Legendary
*
Offline Offline

Activity: 1946
Merit: 1035



View Profile
May 09, 2013, 10:01:50 AM
 #13

It is the post on top of Slashdot right now, congratulations Smiley
behindtext (OP)
Full Member
***
Offline Offline

Activity: 121
Merit: 103


View Profile WWW
May 09, 2013, 12:06:18 PM
 #14

It is the post on top of Slashdot right now, congratulations Smiley

thanks for the heads up.

mike hearn's comment running down the project was particularly amusing Smiley

Mike Hearn
Legendary
*
expert
Offline Offline

Activity: 1526
Merit: 1129


View Profile
May 09, 2013, 03:09:04 PM
 #15

Wink Well, like I said, hopefully Conformal is different to the others. I was just pointing out that there's a graveyard of projects that intended to reimplement Bitcoin and never made it to the finish line. Best of luck!
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1596
Merit: 1091


View Profile
May 09, 2013, 03:19:16 PM
 #16

very exciting! every additional full implementation will make Bitcoin stronger and more independent in the long run.


EDIT:


Don't forget

C: https://github.com/jgarzik/picocoin/
Python: https://github.com/jgarzik/python-bitcoinlib and https://github.com/jgarzik/pynode/

Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
Gyrsur
Legendary
*
Offline Offline

Activity: 2856
Merit: 1520


Bitcoin Legal Tender Countries: 2 of 206


View Profile WWW
May 09, 2013, 03:43:55 PM
 #17

thank you, jeff. appreciate your hard work! :-) and the hard work of the other Bitcoin developers too, of course.

jcv
Jr. Member
*
Offline Offline

Activity: 34
Merit: 1



View Profile WWW
May 13, 2013, 05:39:55 PM
 #18

We've just released the second btcd component, btcjson, the JSON-RPC library.

Here is the blog post about it.

https://blog.conformal.com/btcjson-the-bitcoin-json-rpc-package-from-btcd/

Or if you want the code directly, it is on github:

https://github.com/conformal/btcjson
grau
Hero Member
*****
Offline Offline

Activity: 836
Merit: 1021


bits of proof


View Profile WWW
May 13, 2013, 06:54:02 PM
 #19

Welcome to the club!

Having a choice of implementations unlocks innovation and will benefit all of us. Good Luck.

I cant wait to reveal you all features of the BOP Enterprise Bitcoin Server (formerly known as the bits of proof supernode)
and related offers of my company at the conference.

piotr_n
Legendary
*
Offline Offline

Activity: 2053
Merit: 1354


aka tonikt


View Profile WWW
May 13, 2013, 07:36:18 PM
 #20

We've just released the second btcd component, btcjson, the JSON-RPC library.

Here is the blog post about it.

https://blog.conformal.com/btcjson-the-bitcoin-json-rpc-package-from-btcd/

Or if you want the code directly, it is on github:

https://github.com/conformal/btcjson
May I ask, why don't you just release the whole client at once?
Do you have it already and testing - or not yet finished, but you think it will be finished soon?

Unless it's not 'ask a question' type of topic, in which case: sorry, it will probably pop up my ignore digit Smiley

Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.
PGP fingerprint: AB9E A551 E262 A87A 13BB  9059 1BE7 B545 CDF3 FD0E
Pages: [1] 2 3 4 5 6 7 »  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!