|
Title: btcd: a bitcoind alternative written in Go Post by: behindtext on May 01, 2013, 10:04:13 PM NOTE: i have syndicated this content from https://blog.conformal.com/ so readers don't need to visit the site.
btcd is an alternative full-node implementation of the bitcoin protocol written in Go and is currently under active development. btcd has been under development for 10 weeks and the initial code is nearly ready for public release. We feel that by providing an alternative to bitcoind we can substantially improve the diversity and resilience of the bitcoin ecosystem and infrastructure. A number of us at Conformal Systems had been keeping an eye on bitcoin as passive observers for the past couple years since bitcoin combines technologies that are already of interest to us: practical use of cryptography, distributed systems, and electronic payments. In January 2013 I had one of our developers, David Hill, attempt to port bitcoind and its GUI to Bitrig, an OS that several of our developers forked from OpenBSD. David encountered several problems with porting to Bitrig and in the process found issues with unit tests, non-portable functions and seeding of a PRNG. While pushing to get the port complete, it was clear that it would take a lot more effort than usual to complete this port. After seeing these issues with the porting, I felt that the bitcoin ecosystem could use an alternative to bitcoind. Until starting on btcd, most of our developers had written almost exclusively in C. Our CTO, Marco Peereboom, and developers had been pushing for a new project written entirely in Go. Writing a bitcoind replacement seemed like an interesting project that would take less than 6 months to complete. Once discussion on btcd began in earnest, it became clear that using Go offered a number of advantages over C or C++, especially for financial software:
The most important Go feature in the context of btcd is that of the test infrastructure: by having robust test infrastructure from the outset, all code can and shall have test coverage. Having full test coverage will ensure that most bugs are caught early in the development cycle, before they cause widespread problems. Since btcd is financial software and errors could lead to someone losing money, we take test coverage particularly seriously. btcd is a work-in-progress and we will be making the initial source code release in the next couple weeks. At the moment, we have approximately 40% test coverage on our code but we will be expanding our test coverage once more of the core functionality is complete. We have setup btcd at our colocation facility and are using it to generate output similar to blockexplorer.com (http://blockexplorer.com/), have a look at blocksafari.com (http://blocksafari.com/) to see btcd-generated output. Currently, the following components of btcd are tested as working:
Most of the code is in pretty good shape however some pieces are incomplete. Within 2 weeks we should have enough core functionality written and at that point we are going to release parts of the code to the general community. After this initial release, we plan on adding new functionality to handle wallets, Tor connectivity, etc. If you are interested in talking to our developers about btcd, come chat on our public IRC server in channel #btcd (https://opensource.conformal.com/wiki/IRC_server). Title: Re: btcd: a bitcoind alternative written in Go Post by: Herbert on May 02, 2013, 03:25:30 PM We have setup btcd at our colocation facility and are using it to generate output similar to blockexplorer.com (http://blockexplorer.com/), have a look at blocksafari.com (http://blocksafari.com/) to see btcd-generated output. OMG. Thought my display is broken untill I realized that you have an amazing background there :-)Ontopic: Very interesting! Do you plan the rpc-interface to be a possbile drop-in replacement for bitcoind? Title: Re: btcd: a bitcoind alternative written in Go Post by: gmaxwell on May 02, 2013, 03:43:14 PM Greetings, sounds interesting.
What strategy are you using to make sure that you're implementing the distributed algorithm consistently with the reference software? Title: Re: btcd: a bitcoind alternative written in Go Post by: behindtext on May 02, 2013, 04:45:20 PM We have setup btcd at our colocation facility and are using it to generate output similar to blockexplorer.com (http://blockexplorer.com/), have a look at blocksafari.com (http://blocksafari.com/) to see btcd-generated output. OMG. Thought my display is broken untill I realized that you have an amazing background there :-)Ontopic: Very interesting! Do you plan the rpc-interface to be a possbile drop-in replacement for bitcoind? yes, we have got the bulk of the json bitcoind uses working and plan to have full coverage for that. in theory, it will act as a drop-in replacement. Title: Re: btcd: a bitcoind alternative written in Go Post by: behindtext on May 02, 2013, 04:56:39 PM Greetings, sounds interesting. What strategy are you using to make sure that you're implementing the distributed algorithm consistently with the reference software? our devs have implemented the protocol per the docs at https://en.bitcoin.it/wiki/Protocol_specification . additionally, they have been reading bitcoind code as necessary to make everything work. there are a couple scenarios where bitcoind does not behave according to the published spec. we are currently quirking around this but we can document it if you think it helpful. the devs have been testing against bitcoind since starting work. at the moment, we have a local bitcoind at our colo and are restricting btcd to interacting with only that bitcoind instance. we will open it up and start communicating with other bitcoind servers soon. NOTE: i am not a developer. if you want to get into details, i'll need help whitelisting our devs who are registering right now (they don't have bitcointalk accts) Title: Re: btcd: a bitcoind alternative written in Go Post by: Gavin Andresen on May 02, 2013, 05:25:58 PM our devs have implemented the protocol per the docs at https://en.bitcoin.it/wiki/Protocol_specification . additionally, they have been reading bitcoind code as necessary to make everything work. there are a couple scenarios where bitcoind does not behave according to the published spec. we are currently quirking around this but we can document it if you think it helpful. Yes, please, feedback from re-implementors is very helpful. Title: Re: btcd: a bitcoind alternative written in Go Post by: gmaxwell on May 03, 2013, 08:49:34 AM our devs have implemented the protocol per the docs at https://en.bitcoin.it/wiki/Protocol_specification . additionally, they have been reading bitcoind code as necessary to make everything work. there are a couple scenarios where bitcoind does not behave according to the published spec. we are currently quirking around this but we can document it if you think it helpful. That text is far from complete. Help improving it would be most welcome.Any implementation needs to specifically test for uniformity with the network: Bitcoin is a distributed consensus algorithm and differences in what nodes accept or reject in the blockchain— things which would be minor harmless behavioral differences in most software— can often result in fatal security flaws where an attacker can move the nodes in question onto a separate fork and double-spend their funds away or partition the network. This requires a unusual level of care and system level tests. Many of the most interesting cases are the great many things which must be rejected as no amount of exposure to the live network will trigger those cases (until an attacker exploits them to partition the network). Quote NOTE: i am not a developer. if you want to get into details, i'll need help whitelisting our devs who are registering right now (they don't have bitcointalk accts) PM me their account names and I'll gladly do so.Title: Re: btcd: a bitcoind alternative written in Go Post by: behindtext on May 03, 2013, 05:59:15 PM our devs have implemented the protocol per the docs at https://en.bitcoin.it/wiki/Protocol_specification . additionally, they have been reading bitcoind code as necessary to make everything work. there are a couple scenarios where bitcoind does not behave according to the published spec. we are currently quirking around this but we can document it if you think it helpful. That text is far from complete. Help improving it would be most welcome.10-4, i'll have the guys put together a list of things that may need updating. Quote Any implementation needs to specifically test for uniformity with the network: Bitcoin is a distributed consensus algorithm and differences in what nodes accept or reject in the blockchain— things which would be minor harmless behavioral differences in most software— can often result in fatal security flaws where an attacker can move the nodes in question onto a separate fork and double-spend their funds away or partition the network. This requires a unusual level of care and system level tests. davec had noticed some inconsistencies between the protocol spec and behavior which may vary by version. had to do with requesting blocks while building blockchain. Quote Many of the most interesting cases are the great many things which must be rejected as no amount of exposure to the live network will trigger those cases (until an attacker exploits them to partition the network). Quote NOTE: i am not a developer. if you want to get into details, i'll need help whitelisting our devs who are registering right now (they don't have bitcointalk accts) PM me their account names and I'll gladly do so.btcd currently drops all messages that do not match spec, so the approach is to block until it is specifically allowed. Title: Re: btcd: a bitcoind alternative written in Go Post by: Gavin Andresen on May 03, 2013, 11:31:28 PM 10-4, i'll have the guys put together a list of things that may need updating. Ummm... it is a wiki. They should just update it themselves (after asking questions if they have any), it is much more efficient. Title: Re: btcd: a bitcoind alternative written in Go Post by: Gyrsur on May 04, 2013, 09:11:10 AM very exciting! every additional full implementation will make Bitcoin stronger and more independent in the long run.
EDIT:
Title: Re: btcd: a bitcoind alternative written in Go Post by: killerstorm on May 04, 2013, 03:35:46 PM Also full node implementations (or are close to it):
https://github.com/bitcoinjs/bitcoinjs-server https://github.com/bitsofproof/supernode Title: Re: btcd: a bitcoind alternative written in Go Post by: gmaxwell on May 09, 2013, 12:37:36 AM Apparently this has been announced: https://blog.conformal.com/btcwire-the-bitcoin-wire-protocol-package-from-btcd/
Quote David encountered several problems with porting to Bitrig and in the process found issues with unit tests, non-portable functions and seeding of a PRNG. Ah. This is the folks with the rand() that only returned even values? That was fun. :)Title: Re: btcd: a bitcoind alternative written in Go Post by: matt4054 on May 09, 2013, 10:01:50 AM It is the post on top of Slashdot right now, congratulations :)
Title: Re: btcd: a bitcoind alternative written in Go Post by: behindtext on May 09, 2013, 12:06:18 PM It is the post on top of Slashdot right now, congratulations :) thanks for the heads up. mike hearn's comment running down the project was particularly amusing :) Title: Re: btcd: a bitcoind alternative written in Go Post by: Mike Hearn on May 09, 2013, 03:09:04 PM ;) Well, like I said, hopefully Conformal is different to the others. I was just pointing out that there's a graveyard of projects that intended to reimplement Bitcoin and never made it to the finish line. Best of luck!
Title: Re: btcd: a bitcoind alternative written in Go Post by: jgarzik on May 09, 2013, 03:19:16 PM very exciting! every additional full implementation will make Bitcoin stronger and more independent in the long run.
EDIT:
Don't forget C: https://github.com/jgarzik/picocoin/ Python: https://github.com/jgarzik/python-bitcoinlib and https://github.com/jgarzik/pynode/ Title: Re: btcd: a bitcoind alternative written in Go Post by: Gyrsur on May 09, 2013, 03:43:55 PM thank you, jeff. appreciate your hard work! :-) and the hard work of the other Bitcoin developers too, of course.
Title: Re: btcd: a bitcoind alternative written in Go Post by: jcv on May 13, 2013, 05:39:55 PM We've just released the second btcd component, btcjson, the JSON-RPC library.
Here is the blog post about it. https://blog.conformal.com/btcjson-the-bitcoin-json-rpc-package-from-btcd/ Or if you want the code directly, it is on github: https://github.com/conformal/btcjson Title: Re: btcd: a bitcoind alternative written in Go Post by: grau on May 13, 2013, 06:54:02 PM Welcome to the club!
Having a choice of implementations unlocks innovation and will benefit all of us. Good Luck. I cant wait to reveal you all features of the BOP Enterprise Bitcoin Server (formerly known as the bits of proof supernode) and related offers of my company at the conference. Title: Re: btcd: a bitcoind alternative written in Go Post by: piotr_n on May 13, 2013, 07:36:18 PM We've just released the second btcd component, btcjson, the JSON-RPC library. May I ask, why don't you just release the whole client at once?Here is the blog post about it. https://blog.conformal.com/btcjson-the-bitcoin-json-rpc-package-from-btcd/ Or if you want the code directly, it is on github: https://github.com/conformal/btcjson Do you have it already and testing - or not yet finished, but you think it will be finished soon? Unless it's not 'ask a question' type of topic, in which case: sorry, it will probably pop up my ignore digit :) Title: Re: btcd: a bitcoind alternative written in Go Post by: behindtext on May 13, 2013, 10:45:41 PM We've just released the second btcd component, btcjson, the JSON-RPC library. May I ask, why don't you just release the whole client at once?Do you have it already and testing - or not yet finished, but you think it will be finished soon? Unless it's not 'ask a question' type of topic, in which case: sorry, it will probably pop up my ignore digit :) we chose not to release all the code at once because it was put together rather quickly. one of the main goals with btcd is to have easy-to-understand code along with full test coverage. having full test coverage runs directly in the face of the "making it work", so we made it work first, and are now polishing the individual pieces and releasing them. i'm not sure if you have checked out the test coverage on btcwire but it exercises every single line of code in the package in the tests. this includes negative testing in order to give all the code a run-through. the btcjson package is similar in quality: 88% of the code has test coverage and that should improve to 100% in the next several days. per my earlier posts in this thread, btcd is up and running against a single local bitcoind node. it can pull the whole blockchain down but you may have noticed that it cannot handle chain forking properly yet. we expect this to be working well and release it soon. full tx verification is under development, a number of experiments are being done to determine how to make it run faster. since these experiments involve changing (sqlite) db schema, it doesn't make sense to release in such a half-baked form, especially when it means users would have to rebuild/restructure their whole db. we expect to have btcd interoperating nicely with bitcoind before June 1st. note that this will not include wallet functionality, we expect that to be ready before July 1st and likely before June 15th. since you're a developer yourself you know how timelines can slip, so be aware these are estimates. Title: Re: btcd: a bitcoind alternative written in Go Post by: Cyrus on May 13, 2013, 11:04:17 PM Congrats and also, best of luck with taking btcd to the finish line.
Title: Re: btcd: a bitcoind alternative written in Go Post by: jcv on May 14, 2013, 12:14:35 AM We've just released the second btcd component, btcjson, the JSON-RPC library. May I ask, why don't you just release the whole client at once?Do you have it already and testing - or not yet finished, but you think it will be finished soon? Unless it's not 'ask a question' type of topic, in which case: sorry, it will probably pop up my ignore digit :) we chose not to release all the code at once because it was put together rather quickly. one of the main goals with btcd is to have easy-to-understand code along with full test coverage. having full test coverage runs directly in the face of the "making it work", so we made it work first, and are now polishing the individual pieces and releasing them. i'm not sure if you have checked out the test coverage on btcwire but it exercises every single line of code in the package in the tests. this includes negative testing in order to give all the code a run-through. the btcjson package is similar in quality: 88% of the code has test coverage and that should improve to 100% in the next several days. Just to build on behindtext's answer, by releasing btcd in pieces, we can get the code out there much faster as parts that are more mature don't have to wait for other parts to come out. Doing it this way also forces us to keep the code in distinct, reusable pieces rather than us building one monolithic thing, which we think will help the overall quality. And it means that if someone else had a project where they want some of the code, they only have to understand the component that they need and not worry about the others at all since they are all separate Go packages. Title: Re: btcd: a bitcoind alternative written in Go Post by: piotr_n on May 14, 2013, 06:14:34 PM We've just released the second btcd component, btcjson, the JSON-RPC library. May I ask, why don't you just release the whole client at once?Do you have it already and testing - or not yet finished, but you think it will be finished soon? Unless it's not 'ask a question' type of topic, in which case: sorry, it will probably pop up my ignore digit :) we chose not to release all the code at once because it was put together rather quickly. one of the main goals with btcd is to have easy-to-understand code along with full test coverage. having full test coverage runs directly in the face of the "making it work", so we made it work first, and are now polishing the individual pieces and releasing them. i'm not sure if you have checked out the test coverage on btcwire but it exercises every single line of code in the package in the tests. this includes negative testing in order to give all the code a run-through. the btcjson package is similar in quality: 88% of the code has test coverage and that should improve to 100% in the next several days. per my earlier posts in this thread, btcd is up and running against a single local bitcoind node. it can pull the whole blockchain down but you may have noticed that it cannot handle chain forking properly yet. we expect this to be working well and release it soon. full tx verification is under development, a number of experiments are being done to determine how to make it run faster. since these experiments involve changing (sqlite) db schema, it doesn't make sense to release in such a half-baked form, especially when it means users would have to rebuild/restructure their whole db. we expect to have btcd interoperating nicely with bitcoind before June 1st. note that this will not include wallet functionality, we expect that to be ready before July 1st and likely before June 15th. since you're a developer yourself you know how timelines can slip, so be aware these are estimates. I wish you all the best with the project guys, though if you do not have tx verification fully implemented yet, I think your dates might be a bit too optimistic. Also, from my own experience, I think sqlite might not be the best choice, as for a DB backend for bitcoin. I would rather advise you to look into Go ports of LevelDB. Unfortunately none of them is quite finished yet. Myself, I was trying everything; from mysql to each available port of leveldb - but at the end I decided that they all suck and created my own database engine (https://github.com/piotrnar/qdb). You're welcome to use it, if you want, though I should warn you that it has about 0% of test coverage, and a great appetite for system memory. You can order it to free the mem, but then it will be much slower when you need the data. And that's why I recently bought 8 more gigs of RAM, just so I would not need to free the mem... at least for the next couple of months ;) Title: Re: btcd: a bitcoind alternative written in Go Post by: jcv on May 15, 2013, 02:28:49 PM Also, from my own experience, I think sqlite might not be the best choice, as for a DB backend for bitcoin. I would rather advise you to look into Go ports of LevelDB. Unfortunately none of them is quite finished yet. We've definitely looked at LevelDB and seem to alternate between being excited about it and disappointed that the Go ports aren't done enough yet. Those will be pretty interesting once they get a little more mature though. Title: Re: btcd: a bitcoind alternative written in Go Post by: piotr_n on May 15, 2013, 02:51:21 PM We've definitely looked at LevelDB and seem to alternate between being excited about it and disappointed that the Go ports aren't done enough yet. Yep, I know exactly what you mean :)And it's even more disappointing when you know that both; Go and LevelDB are products developed by Google. Title: Re: btcd: a bitcoind alternative written in Go Post by: Mike Hearn on May 16, 2013, 01:04:06 AM What do you mean by port? Aren't the bindings sufficient?
https://github.com/jmhodges/levigo#readme The README makes that sound reasonably complete. "Port" to me implies "rewrite". Why would anyone do that? Title: Re: btcd: a bitcoind alternative written in Go Post by: Zeilap on May 16, 2013, 01:51:26 AM Hang on - why are you trying to put the blockchain in a leveldb database?
Title: Re: btcd: a bitcoind alternative written in Go Post by: jcv on May 16, 2013, 01:41:43 PM What do you mean by port? Aren't the bindings sufficient? https://github.com/jmhodges/levigo#readme The README makes that sound reasonably complete. "Port" to me implies "rewrite". Why would anyone do that? "Package" is the word I meant, not "Port". Sorry about that. Title: Re: btcd: a bitcoind alternative written in Go Post by: jcv on May 16, 2013, 01:44:42 PM Hang on - why are you trying to put the blockchain in a leveldb database? We are actually putting things in an sqlite database, but we've been evaluating all sorts of other things, including leveldb. The ideal case of course would be a native go implementation of something so we wouldn't need cgo, but we haven't had much luck finding something like that (although I have not done any of the database work, so I can't say what problems we've seen or what missing features any of the other packages had). Title: Re: btcd: a bitcoind alternative written in Go Post by: ElectricMucus on May 16, 2013, 01:47:19 PM How long will it take for a functional release?
I myself am learning go and this will go a long way helping me. Thanks! Title: Re: btcd: a bitcoind alternative written in Go Post by: jcv on May 16, 2013, 02:24:16 PM How long will it take for a functional release? I myself am learning go and this will go a long way helping me. Thanks! If you mean a full release of btcd, I can't really say (that's more behindtext's place). But, we do have two functioning packages on github (btcwire and btcjson) which you might want to checkout. Hopefully those can help you get started on something in Go a bit faster. We've really tried to be good with both documenting and adding tests to both of them (of course any comments or bugs you have with them are very welcome). There is also a third component that will be out soon. Title: Re: btcd: a bitcoind alternative written in Go Post by: jcv on June 14, 2013, 01:37:17 PM Just to update folks on this, we've released our next component, btcscript, the script package for btcd:
https://blog.conformal.com/btcscript-the-script-package-from-btcd/ As always, comments, idea, and complaints are always welcome. Title: Re: btcd: a bitcoind alternative written in Go Post by: hf on June 15, 2013, 12:39:40 AM Awesome clean, documented code.
Keep up the awesome work! Title: Re: btcd: a bitcoind alternative written in Go Post by: jcv on July 19, 2013, 02:19:28 PM We've just released the chain selection/block handling code from btcd.
Here is the link to btcchain: https://blog.conformal.com/btcchain-the-bitcoin-chain-package-from-bctd/ The plan is for btcd itself to come out next. Just like before, any comments or questions on the code or documentation are always welcome. We've tried very hard to match the rules as implemented by bitcoind exactly, but as this is very important, we'd appreciate any reviews or comments on the code. Enjoy. Title: Re: btcd: a bitcoind alternative written in Go Post by: LightRider on July 21, 2013, 07:32:39 AM -----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1 This is awesome. Keep up the great work! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (MingW32) iQIcBAEBAgAGBQJR647tAAoJENpjtAbv5/sY080P/2wJbztJ5iLHQujloFiH4vZh dLV3lJ8mU8xZXhJ7qfUL59LdRD7Id3O91hyn6WpkKaWsUXHRl7P1/ZAnXkkMrHIP EZgIKeybxGgHxEV/lfIj5vlw3hBEOMcBjCGzWDA5budnbAYmUJoSFDNQQSgiHjVc QMwPK9viaUbv35gtPrYbpuuPzSPxX+cvxcly8xwehqx5Uu994WmPmXBJj9sJvCM8 EWi3jtTR6o527hwJECaYHuGMvUsP7DbDMT6OyzD1xgWvr8XcR3qrOfl4wSnJx5PU 4jMlCF4v56P4DLtVyHiki7Zr2fpnChYh/rxYMlCoDi3K9bzTidNkrm7Xd4GZ/W5X q/LsyupEVFeFwDdIlyyekV8HYDtc3IRLC8HjlEwc2zwjT3jKRGPZeoXVDLAZ4KQN o8hjDaEdIT5qtRL2VXalbfpauaStj+GZL5vdhf6zshX0ZyQOHh4H58OuUN3OLFqh CdJUtL3YU+DHQSrX5m8H2bgK6Hy/8waTLRsVfoMTF8jewjZRl+F4HH/UfMJTp5Qa ZaSCrrmYidjj94e1Ww0/bYkYFUUneUrq7vdHQi6NwoTA86BS8C5PFpIkVTb+BzSo O5PF7RY9N7ybvw/oyzzSJBygvhYgAsRqg3AWMem+sEPA0Vcrhj2N9hVbu8+Vz4Ma vsUR5/PoKcO2Qql+WU9/ =+tO/ -----END PGP SIGNATURE----- Title: Re: btcd: a bitcoind alternative written in Go Post by: hazek on September 04, 2013, 09:41:48 PM This is starting to get really exciting: https://blog.conformal.com/btcchain-the-bitcoin-chain-package-from-bctd/
Title: Re: btcd: a bitcoind alternative written in Go Post by: behindtext on October 14, 2013, 06:49:59 PM figured i would just update this thread: btcd has been released and is getting tested
https://blog.conformal.com/btcd-not-your-moms-bitcoin-daemon/ currently grinding out bugs with goleveldb and getting btcwallet + btcgui into shape. Title: Re: btcd: a bitcoind alternative written in Go Post by: justusranvier on December 17, 2013, 06:24:54 PM Is it normal for inbound peers to consistently quit after approximately 20 seconds?
Code: 08:16:36 2013-12-17 [INF] BTCD: Version 0.4.0-alpha Title: Re: btcd: a bitcoind alternative written in Go Post by: cczarek123 on December 17, 2013, 09:23:04 PM What strategy are you using to make sure that you're implementing the distributed algorithm consistently with the reference software?
Title: Re: btcd: a bitcoind alternative written in Go Post by: behindtext on December 17, 2013, 10:12:18 PM Is it normal for inbound peers to consistently quit after approximately 20 seconds? Code: 08:22:04 2013-12-17 [INF] BMGR: Lost peer 95.154.245.77:62066 (inbound) that is indeed interesting. i haven't seen anything like that to date. did you file a GH issue? Title: Re: btcd: a bitcoind alternative written in Go Post by: justusranvier on December 17, 2013, 10:17:15 PM that is indeed interesting. i haven't seen anything like that to date. Not yet - I wanted to make sure it wasn't a known issue.did you file a GH issue? If you check the log file here, the same behaviour shows up: https://github.com/conformal/btcd/issues/62 Title: Re: btcd: a bitcoind alternative written in Go Post by: behindtext on December 17, 2013, 10:21:12 PM oh, and for an update:
there is surely more but those are the big ones :) Title: Re: btcd: a bitcoind alternative written in Go Post by: jcv on December 17, 2013, 10:34:36 PM What strategy are you using to make sure that you're implementing the distributed algorithm consistently with the reference software? Hi cczarek, I assume you mean the verification of the blockchain (but if you mean something else, just correct me and I'll try to answer that). We did a number of things to ensure that we match the behavior of bitcoind/-qt. For starters, there is a regression test with a number of test blocks and reorganizations of the chain that we made sure btcd could pass (we talk a little more about that in one of our blog posts https://blog.conformal.com/btcd-not-your-moms-bitcoin-daemon/#more-421). We've spent a lot of time writing test code (which doesn't exactly show that we match bitcoind, but it does help show that the code matches what we think it does which is usually half the battle). Largely, this has come down to testing. We've collectively run btcd for a huge number of hours on a huge number of blocks to ensure it behaves the way bitcoind does. We also pay close attention to comments people have on github about to help catch bugs. Hopefully that gives you some more confidence. And you can always look at the code if so inclined, that is part of why it is open source. Title: Re: btcd: a bitcoind alternative written in Go Post by: coinrevo on December 28, 2013, 12:21:01 PM For future reference I am saving Mike's critique on slashdot below, which raises some interesting questions
Quote A full node is a really, really large amount of work. I feel that lots of people don't realise this, get enthusiastic and think, "I love Bitcoin! I love Go! I'll write Bitcoin in Go" where for Go you can substitute basically any language that's fun or popular. Then they write the easy bits (like wire marshalling) and eventually the project dies around the time that it's time to implement the wallet or Bloom filtering or robust test suites. Possibly Conformal is different, we'll have to wait and see, but the feature set they advertised in their blog is very much what has been seen many times before. In particular there's no handling of the block chain, re-orgs, no wallet and they haven't got any infrastructure to test edge cases. One reason implementing Bitcoin properly is not fun is an entire class of bugs that doesn't exist in normal software - chain splitting bugs - which can be summed up as "Your software behaves how you thought it's supposed to work rather than how the original bitcoind actually does work". Bitcoin is highly unusual in that it implements group consensus - lots of nodes have to perform extremely complicated calculations and arrive at exactly the same result in lockstep, to a far far higher degree of accuracy than other network protocols. This means that you have to replicate the same set of bugs bitcoind has. Failure to do so can lead to opening up security holes via consensus failure which can in turn lead to double spending (and thus your users lose money!). Being compatible with the way bitcoind is written (bugs and all) may require you to break whatever abstractions you have introduced to make the code cleaner or more elegant or whatever reason you have for reimplementing Bitcoin. Here's a trivial example - signatures in Bitcoin have an additional byte that basically selects between one of a few different modes. It's actually one of three modes plus a flag. So a natural way to implement this is as an enum representing the three modes plus a boolean for the flag. But that won't work. There is a transaction in the block chain which has a sighash flag that doesn't fit any of the pre-defined values (it's zero) and because Satoshi's code uses bit testing it still works. But if you turn the flag into an enum, when you re-serialise the mode flags you'll re-serialise it wrong and arrive at an incorrect result. So you have to pass these flags around as integers and select via bit testing as well. Bitcoin is full of these kinds of weird edge cases. Eventually you come to realise that reimplementing it is dangerous and probably whatever benefits you thought it had, it probably doesn't. Some people believe there should be independent reimplementations anyway and I can understand and respect that, but doing it safely is an absolutely massive piece of work. You have to really, really, really believe in diversity to do it - the features of language-of-the-day aren't good enough to justify the effort. Title: Re: btcd: a bitcoind alternative written in Go Post by: coinrevo on December 28, 2013, 12:53:06 PM I think this work is very valuable for several reasons, most importantly to have a clearer distinction between the software (reference client) and the protocol. I would second this comment of an anonymous poster on slashdot. One can acknowledge all the great work done and still be paranoid about backdoors. Ultimately adopters might know nothing about who implements bitcoin, but if there are many checks & balances in the process that is much more secure than one development tree.
Quote Your post is extremely interesting, but the mandatory conclusion I make from it is the exact opposite of yours. If the original code is so full of idiosyncracies and gotchas then it's an extreme liability to everyone who values Bitcoin, and quite likely contains backdoors or deliberate weaknesses that are hidden by the obscurity. There can be no more important task for the Bitcoin community I think than to specify all elements of the static protocol and dynamic behavior of all parts, and reimplement them in other languages, especially safe languages. Go is certainly a good candidate for this large body of work, safe, clean, and fast. I would like to understand more about the points raised in these comments. In how far does a re-implementation have to reproduce weird behaviour (not quite the same as bugs)? In how far are these documented at all? Does btcd have a different concurrency model? How much of the security depends on the go-lang libs themselves? Title: Re: btcd: a bitcoind alternative written in Go Post by: coinrevo on December 28, 2013, 12:55:34 PM Here is Dave's talk on the 2013 conference http://www.youtube.com/watch?v=d1IYvJs5GGw
Title: Re: btcd: a bitcoind alternative written in Go Post by: behindtext on December 28, 2013, 06:01:10 PM For future reference I am saving Mike's critique on slashdot below, which raises some interesting questions to keep this in context, note that this comment was made by mike when all we had was a wire protocol and the rest of the project was incomplete. the 'story' that went up on slashdot was misleading in its title and suggested we were nearly done with the project. pretty much everything except a chunk of the JSON RPC commands is done at this point and we're at the 10 month mark from starting the project. i'll quickly address each point. Quote A full node is a really, really large amount of work. I feel that lots of people don't realise this, get enthusiastic and think, "I love Bitcoin! I love Go! I'll write Bitcoin in Go" where for Go you can substitute basically any language that's fun or popular. Then they write the easy bits (like wire marshalling) and eventually the project dies around the time that it's time to implement the wallet or Bloom filtering or robust test suites. Possibly Conformal is different, we'll have to wait and see, but the feature set they advertised in their blog is very much what has been seen many times before. In particular there's no handling of the block chain, re-orgs, no wallet and they haven't got any infrastructure to test edge cases. all our devs used to code in POSIX-like C and that is really slow. writing this in Go allowed us to produce code much more quickly and get high test coverage for the code without needing a system external to the language. several comments here are no longer relevant because everything mentioned, except bloom filters, is implemented. Quote One reason implementing Bitcoin properly is not fun is an entire class of bugs that doesn't exist in normal software - chain splitting bugs - which can be summed up as "Your software behaves how you thought it's supposed to work rather than how the original bitcoind actually does work". Bitcoin is highly unusual in that it implements group consensus - lots of nodes have to perform extremely complicated calculations and arrive at exactly the same result in lockstep, to a far far higher degree of accuracy than other network protocols. This means that you have to replicate the same set of bugs bitcoind has. Failure to do so can lead to opening up security holes via consensus failure which can in turn lead to double spending (and thus your users lose money!). Being compatible with the way bitcoind is written (bugs and all) may require you to break whatever abstractions you have introduced to make the code cleaner or more elegant or whatever reason you have for reimplementing Bitcoin. Here's a trivial example - signatures in Bitcoin have an additional byte that basically selects between one of a few different modes. It's actually one of three modes plus a flag. So a natural way to implement this is as an enum representing the three modes plus a boolean for the flag. But that won't work. There is a transaction in the block chain which has a sighash flag that doesn't fit any of the pre-defined values (it's zero) and because Satoshi's code uses bit testing it still works. But if you turn the flag into an enum, when you re-serialise the mode flags you'll re-serialise it wrong and arrive at an incorrect result. So you have to pass these flags around as integers and select via bit testing as well. Bitcoin is full of these kinds of weird edge cases. Eventually you come to realise that reimplementing it is dangerous and probably whatever benefits you thought it had, it probably doesn't. Some people believe there should be independent reimplementations anyway and I can understand and respect that, but doing it safely is an absolutely massive piece of work. You have to really, really, really believe in diversity to do it - the features of language-of-the-day aren't good enough to justify the effort. this is a bit of a strawman argument that suggests it is not worthwhile to develop a proper alternative full node implementation because it *might* fork the blockchain. davec, our lead dev, has taken great care to include all of the bitcoind behaviors for chain selection exactly. check the blog entry https://blog.conformal.com/btcchain-the-bitcoin-chain-package-from-bctd/ for more info here. it really wasn't that massive a piece of work. sure, it was a lot of work, but it took less than 12 months to catch up with 5 years of development plus whatever time the SN crew took to do their initial dev work. doing this in Go made it a lot quicker than C or C++, but we did have to work through several nasty bugs in Go. it took less than 7 manyears labor to hack out over the past 10 months and we'll be at feature parity within the next 2 months, at most. last friday davec finished the initial secp256k1 optimizations and the entire chain downloads from scratch in roughly 3 hours (1.75 hrs to last checkpoint, 1.25 hrs from checkpoint to last block) on an i7 + ssd linux machine. Title: Re: btcd: a bitcoind alternative written in Go Post by: hazek on December 28, 2013, 11:55:36 PM ..this is a bit of a strawman argument.. I think it's fantastic what you are doing and that while it's great that you defend against criticism, you should really pay no attention to that thinks he knows it all because he works for that company guy. Title: Re: btcd: a bitcoind alternative written in Go Post by: grau on December 29, 2013, 09:31:54 AM it took less than 7 manyears labor to hack out over the past 10 months and we'll be at feature parity within the next 2 months, at most. Congratulations. I know how that work feels. It took me about half a man year to reach feature parity in Java though - passing the block tests used by satoshi and bitcoinj - thereafter I worked mostly on higher level projects using the code in production. Regarding 100% security in distributed consensus I think in the meanwhile that it is a moving target we will not reach until we split out and freeze that part of bitcoind. I hope you succeed, as there is no push in the core team to isolate concerns. Title: Re: btcd: a bitcoind alternative written in Go Post by: justusranvier on December 29, 2013, 09:36:14 AM Regarding 100% security in distributed consensus I think in the meanwhile that it is a moving target we will not reach until we split out and freeze that part of bitcoind. Why could there not be a crowdfunding effort to pay someone to push improvements upstream from the reimplementations?I hope you succeed, as there is no push in the core team to isolate concerns. Do BoP and btcd both have more comprehensive unit tests than the Satoshi client? Surely everyone would benefit from getting that same level of coverage in the reference implementation. At some point all there needs to be inter-implementation coordination of features and specs, kind of like the W3C for HTML. Title: Re: btcd: a bitcoind alternative written in Go Post by: grau on December 29, 2013, 10:11:15 AM Regarding 100% security in distributed consensus I think in the meanwhile that it is a moving target we will not reach until we split out and freeze that part of bitcoind. Why could there not be a crowdfunding effort to pay someone to push improvements upstream from the reimplementations?I hope you succeed, as there is no push in the core team to isolate concerns. Do BoP and btcd both have more comprehensive unit tests than the Satoshi client? Surely everyone would benefit from getting that same level of coverage in the reference implementation. At some point all there needs to be inter-implementation coordination of features and specs, kind of like the W3C for HTML. Unit tests of satoshi that were portable (means defined in JSON input and expected output) are all supported by BOP. It has some additional unit tests but is not as impressive as btcd sounds. I have lots of tests on much higher level, testing behavior of production systems that build on BOP, these however do not help the community as not open source. I did expect that the Bitcoin foundation would fill the role you describe, but seen nothing noteworthy. They instead focus on US lobbyism and building a profitable international franchise. Title: Re: btcd: a bitcoind alternative written in Go Post by: justusranvier on December 29, 2013, 10:17:17 AM I did expect that the Bitcoin foundation would fill the role you describe, but seen nothing noteworthy. They instead focus on US lobbyism and building a profitable international franchise. I'm not really impressed with them either. Apparently some other organization is going to have to take on that role instead.Title: Re: btcd: a bitcoind alternative written in Go Post by: coinrevo on December 29, 2013, 12:17:50 PM At some point all there needs to be inter-implementation coordination of features and specs, kind of like the W3C for HTML. I think this is exactly the discussion that is needed. We have the BIPs, which I understand something Amir pushed for and is now part of a formal procedure: https://github.com/bitcoin/bips , They are like RFC or W3C specs, but can't cover everything. It's a good first step, but there are several weakness how these are structured. I think there also has to be stronger firewalls and more transparency in terms of the relationship between the users of the network, and how development interacts with the users or developers who are not core, but build on top of the network. It seems with so many people having made so much money of Bitcoin, there would/should be some capital re-investment into the economy. Instead we are seeing big VC money for online wallets, and more and more corporate elitists getting into this. Over time this will have its effects. The interactions with the legal system are much more complex than HTML. Title: Re: btcd: a bitcoind alternative written in Go Post by: behindtext on December 29, 2013, 03:33:25 PM it took less than 7 manyears labor to hack out over the past 10 months and we'll be at feature parity within the next 2 months, at most. Congratulations. I know how that work feels. It took me about half a man year to reach feature parity in Java though - passing the block tests used by satoshi and bitcoinj - thereafter I worked mostly on higher level projects using the code in production. Regarding 100% security in distributed consensus I think in the meanwhile that it is a moving target we will not reach until we split out and freeze that part of bitcoind. I hope you succeed, as there is no push in the core team to isolate concerns. thanks grau :) btcwallet + btcgui is coming along and is totally usable but has some bugs. the wallet format is deterministic (iirc) so if you backup your wallet every so often, especially after importing any new privkeys, it's totally safe. if it weren't for all the test coverage on the various component packages, we would have been able to build a lot quicker. totally agree with you that it's not feasible to reach distributed consensus until the entire protocol is properly documented, which it is most definitely not atm. having to dig through the tightly coupled bitcoind code is not exactly the proper way to document a protocol. it makes you wonder if it is thinly documented on purpose... Title: Re: btcd: a bitcoind alternative written in Go Post by: behindtext on December 29, 2013, 03:40:20 PM I think there also has to be stronger firewalls and more transparency in terms of the relationship between the users of the network, and how development interacts with the users or developers who are not core, but build on top of the network. It seems with so many people having made so much money of Bitcoin, there would/should be some capital re-investment into the economy. Instead we are seeing big VC money for online wallets, and more and more corporate elitists getting into this. Over time this will have its effects. The interactions with the legal system are much more complex than HTML. the developer network is definitely in need of more transparency imo. i probably could not agree with you more about the reinvestment angle and VC involvement. every time VCs get involved it is only a matter of time before there is (1) a race to the bottom and (2) centralization driven by heavy marketing. i figured i'd flip the script and preemptively contribute to the community before trying to build businesses on top of it. Title: Re: btcd: a bitcoind alternative written in Go Post by: davec on December 29, 2013, 07:05:32 PM In how far does a re-implementation have to reproduce weird behaviour (not quite the same as bugs)? In order to avoid chain forks, multiple implementations have to reproduce the same behavior exactly. Even the slightest difference can cause a chain fork. There are certainly some cases that can be considered weird behavior, but there are also some issues I would consider outright bugs. For example, the transaction script language has a well-defined set of allowed opcodes. However, it doesn't check the scripts for invalid opcodes until it actually executes them. This means that you can craft valid scripts which have invalid and nearly arbitrary data in dead execution branches. This is an outright bug in my mind. There are also a couple of other things in the transaction scripts which are indisputable bugs. In how far are these documented at all? The exact rules are not really documented anywhere from what I could find. The wiki entry for the wire protocol is fairly accurate, but the block acceptance rules are largely undocumented. There is a wiki entry for the block acceptance rules, but it's not very accurate and doesn't provide sufficient depth. However, I don't believe a publicly editable wiki is the right place for something as important as the protocol and block acceptance rules anyways. This is an area I believe really needs to be improved. Unfortunately, the general consensus I have seen amongst the core bitcoind devs seems to be "the code is the spec". I would be happy if my take on what the general consensus appears to be is wrong. I fully understand that writing documentation and specs is not glamorous nor fun work, but I think it's paramount for the longtime survival and growth of Bitcoin as it directly plays into multiple implementations. I can't imagine the internet would be anywhere near what it is today if there weren't legitimate specs for things like TCP/IP and instead everyone that implemented a stack had to just go read and properly interpret some mostly undocumented code. Unfortunately, even if a spec is written, unless the majority implementations (realistically this is only bitcoind currently) choose to follow it, it is meaningless. On the plus side, there is a block tester tool that does a pretty good job of testing the block acceptance rules programatically. While this doesn't eliminate the need for a spec by any means, it is definitely an excellent tool that helps support multiple implementations and would be a fantastic supplement to a real spec. Does btcd have a different concurrency model? Yes. Since it is written in Go, btcd uses CSP (Communicating Sequential Processes) heavily for concurrency. The Go mantra is "Do not communicate by sharing memory; instead, share memory by communicating", which is the model btcd strives to follow. For example, in btcd, the server, block manager, and peers are all separate entities running in different goroutines that use channels to communicate with one another. This provides a clear concurrency model with a clean and concise data flow. How much of the security depends on the go-lang libs themselves? I'm not really sure what area of security you are referring to here. If you're referring to security issues such as exploitability of the software, I would argue that libraries written in Go eliminate an entire class of the most common causes of vulnerabilities such as stack corruption, heap corruption, uninitialized variable use, use of freed variables, and invalid frees. This is because Go is garbage collected, all variables are initialized to their respective zero values by default, all array accesses are bounds checked by the run-time, and pointer arithmetic is disallowed. That is not to say they're fully immune to exploits of course, but eliminating the vast majority of the most common ones by default is quite nice. On the other hand, if you're referring more to the security issues such as coin theft, that has less to do with the implementation language and more to do with things like private key management, trusting third parties with your wallet, etc. It took me about half a man year to reach feature parity in Java though - passing the block tests used by satoshi and bitcoinj - thereafter I worked mostly on higher level projects using the code in production. Thanks grau. Reaching feature parity in that time frame is an impressive one-man effort. Well done! That sounds very close to what it would have taken us as well to just implement everything without the architectural changes, drive for 100% test coverage in the core packages, and heavy focus on producing fully-documented reusable development components for the community as a part of the process. I'm really glad you posted your stats as I think this goes to show that while it's definitely complicated, it's not something that is so complicated that nobody else should bother as some comments have suggested. Title: Re: btcd: a bitcoind alternative written in Go Post by: justusranvier on December 29, 2013, 09:18:57 PM Unfortunately, even if a spec is written, unless the majority implementations (realistically this is only bitcoind currently), it is meaningless. I am confident there is sufficient willingness in the community to fund the development time it would take to push the required cleanups upstream. The only problem appears to be that the organization which you'd naturally expect to be taking that effort on does not consider it to be a priority.Title: Re: btcd: a bitcoind alternative written in Go Post by: LightRider on December 31, 2013, 06:52:44 AM I appreciate what everyone at conformal is doing. Thanks for taking on this important work and producing a viable alternative.
Title: Re: btcd: a bitcoind alternative written in Go Post by: genjix on December 31, 2013, 03:42:03 PM that async principle is a solid one to create for a bitcoin impl.
been checking in on btcd every so often, and the design/arch decisions are solid. the code is the spec is a bit of a crap base for the future of finance and something we'll leave behind. the more of us there are, the quicker bitcoin will move forward. decentralise everything and distribute the power. we need a solid infrastructure and the bitcoind is really poor. Quote For example, the transaction script language has a well-defined set of allowed opcodes. However, it doesn't check the scripts for invalid opcodes until it actually executes them. This means that you can craft valid scripts which have invalid and nearly arbitrary data in dead execution branches. This is an outright bug in my mind. There are also a couple of other things in the transaction scripts which are indisputable bugs. yes omg this is terrible. output scripts dont even need to parse. but there's plenty more like the copy paste serialization code, deeply nested locking mechanisms, cscript inheriting an std:: container (forbidden by standard), undefined behaviour (using integer loop around / invalid casts), non-modular code, ... more i can't recall right now Title: Re: btcd: a bitcoind alternative written in Go Post by: coinrevo on December 31, 2013, 04:24:49 PM the code is the spec is a bit of a crap base for the future of finance and something we'll leave behind. the more of us there are, the quicker bitcoin will move forward. decentralise everything and distribute the power. we need a solid infrastructure and the bitcoind is really poor. Is it even possible to have two major reference clients in parallel? Imagine we have a full spec (bitcoinP) and two clients - bitcoinA and bitcoinX with each 50% distribution. each are SSH locked repos (that's the access model we have currently). Then somebody has a good idea on how to improve bitcoinP. The bitcoinX developers think its a great idea, bitoinA developers think its terrible. who gets to decide? very likely bitcoinX would be build as an Alt-Coin. Now you have the question, is all the infrastructure transportable? it seems to me that there is very little evolution of the primitives, and very little development activity compared to the number of people talking about bitcoin now, like silicon valley hotshots, pumping millions into inferior infrastructure. I guess if you have x millions $ in BTC you are going to be rather conservative in introducing changes. basically the argument goes like this. we have opensource and everybody can suggest changes. therefore its free and anybody can join. this assumes that software is simply the sum of small proposed changes, not of architectural decisions. in reality the development has centralized somewhat. but the Linux/Python/whatever is not good enough as a model. we can't have the equivalent of a Linus Torvalds merging all the changes. but coming up with a better model is very hard. it seems to me that very little profits are being reused to build useful stuff. there is not even the possibility of hosting a server for 100$/year, and funding that through public means, when marketcap has reached 10 billion $. instead we have perhaps a thousand people now trying to produce a Scam AltCoin. Title: Re: btcd: a bitcoind alternative written in Go Post by: justusranvier on December 31, 2013, 07:24:28 PM the more of us there are, the quicker bitcoin will move forward. Unfortunately this is not the case.Bitcoin can only move forward as quickly as the slowest implementation. This is even more true with the slowest implementation is the one use by virtually all the network. There is no path forward that does not involved either fixing the reference implementation, or convincing the network to abandon it en masse. Title: Re: btcd: a bitcoind alternative written in Go Post by: grau on December 31, 2013, 09:01:31 PM Bitcoin can only move forward as quickly as the slowest implementation. This is even more true with the slowest implementation is the one use by virtually all the network. Since we can not eliminate the bottleneck for now, we should seek to control the damage to productivity by isolating and freezing consensus relevant code base of the reference implementation. We should take out the wallet and replace the badly designed RPC with a bus (e.g. zeromq) that allows wallets and all higher level functions to be built without touching or even understanding the P2P consensus kernel. Added: I used the word kernel intentionally. The P2P consensus is the operating system of the network for which we need an API that is high performance, language agnostic, simple but correct representation of the current consensus for application developer who create all sorts of wallet and business functions thereby eliminating the bottleneck at those levels. Title: Re: btcd: a bitcoind alternative written in Go Post by: justusranvier on December 31, 2013, 10:33:06 PM Since we can not eliminate the bottleneck for now, we should seek to control the damage to productivity by isolating and freezing consensus relevant code base of the reference implementation. "Freezing" is the opposite of progress.Title: Re: btcd: a bitcoind alternative written in Go Post by: U1TRA_L0RD on January 01, 2014, 05:26:27 AM This can be a great program for the future use of storing bitcoins, thanks OP!
Title: Re: btcd: a bitcoind alternative written in Go Post by: genjix on January 01, 2014, 07:18:19 AM Since we can not eliminate the bottleneck for now, we should seek to control the damage to productivity by isolating and freezing consensus relevant code base of the reference implementation. "Freezing" is the opposite of progress.I don't want Bitcoin to change. Changing and extending the protocol/standard is the worst thing we can do. There's far more important stuff that needs to be done implementation side. Moving forwards means freezing the standard, that's why I want to diversify. Title: Re: btcd: a bitcoind alternative written in Go Post by: grau on January 01, 2014, 08:56:33 AM Since we can not eliminate the bottleneck for now, we should seek to control the damage to productivity by isolating and freezing consensus relevant code base of the reference implementation. "Freezing" is the opposite of progress.Title: Re: btcd: a bitcoind alternative written in Go Post by: coinrevo on January 01, 2014, 09:09:47 AM satoshi said this about alternative clients:
Quote I don't believe [for] a second, compatible implementation of Bitcoin will ever be a good idea. So much of the design depends on all nodes getting exactly identical results in lockstep that a second implementation would be a menace to the network. In his 575 posts, that is about the strongest statement he ever made. I suppose Gavin and Mike don't want to put in such strong words, but in the end the algorithm is tied up to the current developers. Which in my mind is a huge flaw in the design of the currency. If Gavin wants to design a payment protocol, he can just go ahead (BIP70-72 are terrible IMO). Any solution to this problem of decision making will be a different cryptocurrency. Quote I don't want Bitcoin to change. Changing and extending the protocol/standard is the worst thing we can do. In other words you want the current devs to continue with the process as it is? All design decision will be stuck forever? I certainly hope not. As soon as one wants to do useful things, which are not possible with fiat, one is kind of stuck. Colored Coin and Mastercoin will not work, for very good and actually quite obvious reasons. One can't transfer property on the bitcoin network. Title: Re: btcd: a bitcoind alternative written in Go Post by: U1TRA_L0RD on January 01, 2014, 09:12:19 AM What about other crypto currencies
Title: Re: btcd: a bitcoind alternative written in Go Post by: grau on January 01, 2014, 09:59:13 AM Yes, the design depends on getting identical results in a lockstep on what the longest chain and the set of unconfirmed transactions is.
I see no argument in this against competing wallet implementations just like we have seen a variety of miner. The bottleneck in higher level feature development is imposed by poor design, not fundamental reasons. Added the lockstep is endangered by any new version of the reference implementation itself. Recognize that the most dangerous fork we have seen was between two versions of satoshi's code. Do you need any better argument for an isolation and freeze of that ? We can however only freeze after we create an interface that enables feature development on a higher level. Title: Re: btcd: a bitcoind alternative written in Go Post by: coinrevo on January 01, 2014, 10:25:53 AM grau, I believe you don't see the depth of the above statement. The fundamental question is: who makes decisions about bitcoin? The current bitcoin developers - granted by what power? They have the SSH keys and can implement more or less whatever they think is right, with rather vague references to the community. The trust and consensus system is very far from perfect. Think about it - from an economic perspective all what bitcoin does is enforce consensus for the balances. It does not enforce consensus about the properties of the network itself (devs, miners, community, users are all seperate groups). The process of development is not formalized in any way. Say an alternative implementation tracks all the features at the beginning. It slowly grows to 51% marketshare. Now the developers have the power to force new changes. There is no way to resolve a conflict.
Title: Re: btcd: a bitcoind alternative written in Go Post by: grau on January 01, 2014, 11:26:15 AM Do not overestimate the power of the core team.
They can not introduce a change that breaks the lockstep immediately, since that would fork the network if not upgraded instantaneously (which is not feasible). At max they can introduce a change that takes effect well ahead from some some future block on. If the change is highly controversial, then and this opens the time window to build alternate core team that steps up to maintain the version before the change. Title: Re: btcd: a bitcoind alternative written in Go Post by: coinrevo on January 01, 2014, 12:19:42 PM I think you, like almost all, are seeing this very much from the perspective of bitcoin as it is today, which is a very limited view. Bitcoin in the future can and should be infinitely more, than what it is now. As this is not necessarily btcd related, I will post longer comments elsewhere. But I think you haven't thought this through. Again, imagine two alternative implementations with 50% share. How to resolve conflicts? bitcoin's consensus mechanism is not just about the calculations. it's not just about hashes, IP addresses, ASIC's, etc. we are talking about the future of money. Unfortunately the set of people who can combine both views, the social and technological aspects, is extremely small.
Title: Re: btcd: a bitcoind alternative written in Go Post by: grau on January 01, 2014, 01:22:35 PM I think you, like almost all, are seeing this very much from the perspective of bitcoin as it is today, which is a very limited view. ..... Unfortunately the set of people who can combine both views, the social and technological aspects, is extremely small. Do you count yourself to that chosen few ? If yes, show me small piece your wisdom that I am eventually capable to grasp, as I have not seen it yet. Title: Re: btcd: a bitcoind alternative written in Go Post by: coinrevo on January 01, 2014, 02:20:46 PM If you're interested in these kinds of things I would suggest this resource: http://szabo.best.vwh.net/
When you're saying the bitcoin developers have no power, that doesn't hold up in argument. They have they keys, and they are designing the protocols. What is the input from outside of that group? where is that discussion even taking place? If you approach one of the devs with these questions you wouldn't get a straight answer. I can make a proposal, but this is subject to the lead devs views, which I don't share in many ways. We need much better trust models and infrastructure for collective decision making. we can do better than this messageboard and some mailing list. Title: Re: btcd: a bitcoind alternative written in Go Post by: piotr_n on January 01, 2014, 04:02:08 PM 11 PH/s of the hashing power - how much of it are the devs actually holding?
And do you think whoever is holding it do not realize what kind of power they have? A sooner you realize that devs have no calls on protocol changes anymore, the sooner you will avoid any further disappointments. :) Title: Re: btcd: a bitcoind alternative written in Go Post by: genjix on January 01, 2014, 06:06:42 PM any dev who makes the argument about having 1 bitcoin is talking rubbish. bitcoind itself has changed massively since its inception and will continue to do so. the problem is people are too trigger happy to go and make destructive changes to the bitcoin consensus core which is both dangerous and harms consensus. bitcoin should be kept pure and simple. sorry but i don't want all the contracts and new prototype theoretical features thrown into bitcoin. by all means use the blockchain (people will anyway), but this stuff is so new. bitcoin should be left alone because it works, and needs to be protected.
for example: https://github.com/bitcoin/bitcoin/commit/f5857e5c the sighash is one of the hardest parts of bitcoin. this is not a place I would optimise. compare both versions: https://github.com/bitcoin/bitcoin/blob/f5857e5cb5fb03bee9c05d1dd6ba2621cac49179/src/script.cpp#L978 https://github.com/bitcoin/bitcoin/blob/881a85a22d76c875f519cd54388a419ec6f70857/src/script.cpp#L976 don't mean to criticise sipa since he is working hard. but this is your precious code by satoshi having a radical redesign. Title: Re: btcd: a bitcoind alternative written in Go Post by: genjix on January 01, 2014, 06:11:20 PM there needs to be a incredible amount of friction against modifying bitcoin's rules. some of these issues are very complex and less than a handful of people understand them worldwide. what happens when you as a user, have no-one representing your interest in that circle.
Title: Re: btcd: a bitcoind alternative written in Go Post by: justusranvier on January 01, 2014, 07:37:09 PM sorry but i don't want all the contracts and new prototype theoretical features thrown into bitcoin. Sorry, but Bitcoin is not your personal toy.Quite a few people are waiting for this capability to be returned to the protocol. Title: Re: btcd: a bitcoind alternative written in Go Post by: grau on January 01, 2014, 08:32:31 PM If you're interested in these kinds of things I would suggest this resource: http://szabo.best.vwh.net/ When you're saying the bitcoin developers have no power, that doesn't hold up in argument. They have they keys, and they are designing the protocols. What is the input from outside of that group? where is that discussion even taking place? If you approach one of the devs with these questions you wouldn't get a straight answer. I can make a proposal, but this is subject to the lead devs views, which I don't share in many ways. We need much better trust models and infrastructure for collective decision making. we can do better than this messageboard and some mailing list. I said, do not overestimate their power, not that they would not have one. They have to be very careful exercising the power otherwise they could lose it. Should a significant part of the community disagree with their decisions, then the code could be forked and their keys would become worthless with the majority moving away. We see the trend that non-miner move to SPV slowly erodes the control on that group as a whole. This combined with heavy weight of a few pools concerns me more than the power of core developer. Title: Re: btcd: a bitcoind alternative written in Go Post by: U1TRA_L0RD on January 02, 2014, 11:41:04 AM What language is the program written in?
Title: Re: btcd: a bitcoind alternative written in Go Post by: coinrevo on January 02, 2014, 01:07:25 PM I said, do not overestimate their power, not that they would not have one. They have to be very careful exercising the power otherwise they could lose it. Should a significant part of the community disagree with their decisions, then the code could be forked and their keys would become worthless with the majority moving away. What does "community" stand for here? Marketcap is 10B$ and users are perhaps 5-10M people or more. They have nothing to say about this. What I said is that alternative implementations can potentially defeat the network. If 50% of the "community"/ miners go with A and 50% goes with B, well that's a fork then. And so the devs pretend that anyone can contribute, whereas in reality its a very small group with inherent biases (they are human beings) which does not care that much about outside opinion. Besides all the large players have fortunes to lose by now. If your networth is 8-9 figure $ BTC equivalent you might have some biases. So even if somebody like conformal comes a long and makes a very valuable contribution to architecting the software, the response is less than muted. now think what would happen if they would suggest to make changes to core protocol issues. devs have way too much power, and there are no trust models in place to fix it. Title: Re: btcd: a bitcoind alternative written in Go Post by: behindtext on January 02, 2014, 02:06:37 PM Besides all the large players have fortunes to lose by now. If your networth is 8-9 figure $ BTC equivalent you might have some biases. So even if somebody like conformal comes a long and makes a very valuable contribution to architecting the software, the response is less than muted. now think what would happen if they would suggest to make changes to core protocol issues. devs have way too much power, and there are no trust models in place to fix it. the incentive structure is more than a little fucked up as-is. i will have more to say about this soon. Title: Re: btcd: a bitcoind alternative written in Go Post by: piotr_n on January 02, 2014, 02:36:51 PM I said, do not overestimate their power, not that they would not have one. They have to be very careful exercising the power otherwise they could lose it. Should a significant part of the community disagree with their decisions, then the code could be forked and their keys would become worthless with the majority moving away. What does "community" stand for here? Marketcap is 10B$ and users are perhaps 5-10M people or more. They have nothing to say about this. What I said is that alternative implementations can potentially defeat the network. If 50% of the "community"/ miners go with A and 50% goes with B, well that's a fork then. And so the devs pretend that anyone can contribute, whereas in reality its a very small group with inherent biases (they are human beings) which does not care that much about outside opinion. Besides all the large players have fortunes to lose by now. If your networth is 8-9 figure $ BTC equivalent you might have some biases. So even if somebody like conformal comes a long and makes a very valuable contribution to architecting the software, the response is less than muted. now think what would happen if they would suggest to make changes to core protocol issues. devs have way too much power, and there are no trust models in place to fix it. It is sad and funny at the same time to see how people who think they can make a better bitcoin do not even understand what bitcoin is really about. 50:50 fork is practically impossible and any other fork would be suicidal for the minority branch. So there will not be any fork, unless along with a change in the POW function. People who hold the hashing power do not give a shit about architecture of the software. They do care very much though about security and profits of their mining enterprises. Anyone who will try to do anything against their interest is going to face a huge disappointment. No matter whether he is a brilliant developer, or yet another genocidal president of some united states. And since we have already established that this worldwide bitcoin development community does not seem to work (thus consensus is impossible) a chance for any possible change in the blockchain protocol is very slim, at least in a foreseeable future. And if any will come later on, they will rather be driven by the miners, while developers will only be there to implement them on request. The era of changing the blockchain protocol driven by a dev team is long gone, and it will never come back, so don't try to tilt at windmills for your own sake. Title: Re: btcd: a bitcoind alternative written in Go Post by: genjix on January 02, 2014, 03:34:11 PM sorry but i don't want all the contracts and new prototype theoretical features thrown into bitcoin. Sorry, but Bitcoin is not your personal toy.Quite a few people are waiting for this capability to be returned to the protocol. i want limitations to be removed too. i just don't want new extensions to the protocols (make them rarer). or huge changes/rework stuff. if you think blockchain = consensus, then lets make it a constitution that resists being revised. Title: Re: btcd: a bitcoind alternative written in Go Post by: genjix on January 02, 2014, 03:39:38 PM piotr, its not just about backdoors but about bias in tech decision making. if legit corporations are the ones paying devs salaries, then they will choose the best decisions to benefit corporations and optimise for their use case. and who would be there representing the enterprising individuals and making the software better for them. you're looking too black and white at the things. i'm saying decentralise everything is good for a healthy ecosystem.
note these are tech decisions. i didnt even get into stuff like blacklists, drm, tracking which actually attack users. those can also happen if we're not vigilant. justusranvier explained it well: http://bitcoinism.blogspot.co.uk/2013/12/did-you-get-good-price-when-you-sold-out.html Title: Re: btcd: a bitcoind alternative written in Go Post by: coinrevo on January 02, 2014, 04:02:49 PM so which one is it - financial innovation or stagnation? your own friend Mihai Alisie said: "after 5 years of bitcoin we still don't use the basic features that lie dormant in the bitcoin core". http://www.youtube.com/watch?v=Hbu2KrHx9-0
Title: Re: btcd: a bitcoind alternative written in Go Post by: piotr_n on January 02, 2014, 04:05:41 PM piotr, its not just about backdoors but about bias in tech decision making. if legit corporations are the ones paying devs salaries, then they will choose the best decisions to benefit corporations and optimise for their use case. and who would be there representing the enterprising individuals and making the software better for them. you're looking too black and white at the things. i'm saying decentralise everything is good for a healthy ecosystem. what I meant is that it is totally irrelevant how I look at the things.miners are not going to ask me for an opinion, nor they are going to ask you or any other kind of authority. they have invested lots of money in this infrastructure and they are certainly not stupid, since it has turned out to be a great investment. and that is exactly why I am pretty sure that they will think it through much more carefully this time, before connecting to a mining pool that mines blocks version 3. as an opposite to how ver 2 was introduced, when people still had a lot of trust in the devs, which had been before some of them started inventing black and red lists for their queen, without even a shadow of an embarrassment Title: Re: btcd: a bitcoind alternative written in Go Post by: coinrevo on January 02, 2014, 04:12:14 PM well, it's a flaw in the protocol of economic majority, not a feature. as if dependence on a bunch of profiteers would be a good thing. this makes the system completely static and opaque.
Title: Re: btcd: a bitcoind alternative written in Go Post by: piotr_n on January 02, 2014, 04:13:56 PM well, it's a flaw in the protocol of economic majority, not a feature. as if dependence on a bunch of profiteers would be a good thing. this makes the system completely static and opaque. but bitcoin is a real money and not yet another computer game. it needs to be static and boring because only then it can do its job right.and it definitely isn't a flaw - it's a genius invention that has a very important purpose, but if you don't get it then you are probably at a wrong forum and you should probably go back to developing computer games (no offence - it also takes skills to make games) Title: Re: btcd: a bitcoind alternative written in Go Post by: piotr_n on January 02, 2014, 04:32:07 PM I could not disagree more. if smart contracts were made possible this would be the biggest innovation in the history of finance. at the moment its only used for cash transactions. finance is mostly about non-cash transactions: loans, insurance, equity agreements, taxes, donations, swaps. satoshi was very well aware of this, which is why all the stuff is already there, but not used at the moment. I am pretty sure that the current blockchain protocol already supports all kind of contracts.The limit is only your imagination. That some nodes don't route certain transaction, because they find them non-standard - it doesn't matter at all, since they can still be mined into blocks. All it takes is finding a miner that agrees to do it for a certain fee, which isn't impossible already today. If any changes are really needed in the protocol, the first would be additional signature algorithms (non ECDSA based) and some sort of a blockchain compression so no full node would ever need to download an entire blockchain anymore, though still could be equally secure full node. Not sure if the later even requires a change in the blockchain protocol. But I am quite sure that contracts are already there - you just don't seem to know how to use them. Title: Re: btcd: a bitcoind alternative written in Go Post by: coinrevo on January 02, 2014, 05:11:23 PM But I am quite sure that contracts are already there - you just don't seem to know how to use them. The problem is this: Quote Scripts are, by design, pure functions. They cannot poll external servers or import any state that may change as it would allow an attacker to outrun the block chain. What's more, the scripting language is extremely limited in what it can do. https://en.bitcoin.it/wiki/Contracts The example Mike then goes on to describe will not work IMO, because external API's can change. I am not aware of any implementations of the described oracle scheme, but if you know some please let me know. Title: Re: btcd: a bitcoind alternative written in Go Post by: piotr_n on January 02, 2014, 05:16:39 PM But I am quite sure that contracts are already there - you just don't seem to know how to use them. There are a lot of things which you have not considered carefully enough. The problem is this: Quote Scripts are, by design, pure functions. They cannot poll external servers or import any state that may change as it would allow an attacker to outrun the block chain. What's more, the scripting language is extremely limited in what it can do. https://en.bitcoin.it/wiki/Contracts You would like payment scripts, mined into the blockchain, being able to talk to external servers? ::) Well, I guess I've used all my rudeness limit for today, so no comments on that. :) EDIT: The example Mike then goes on to describe will not work IMO, because external API's can change. I am not aware of any implementations of the described oracle scheme, but if you know some please let me know. Mike has been number one on my shit list, ever since he introduced the concept of "lists" to the world of bitcoin. Ever since then, whatever Mr Google thinks, my natural instincts pushes me to argue for the opposite. So if you want to convince me into some idea, I'd advise you to not mention this NSA cocksucker's name. Title: Re: btcd: a bitcoind alternative written in Go Post by: coinrevo on January 24, 2014, 01:30:10 PM Thanks for your work!
https://blog.conformal.com/redecentralization-robust-developer-network/ Quote As it was originally proposed by Satoshi Nakamoto, Bitcoin was conceived of as a peer-to-peer system that was fundamentally decentralized. Many past and current discussions about the future of Bitcoin acknowledge that too much centralization is a threat to the Bitcoin network. To some extent this process of avoiding centralization has been successful, but in several key areas it has not been very successful and it is in need of redecentralization. The most pressing case for redecentralization in Bitcoin (and cryptocurrencies more generally) is the current development community: the number of developers who have experience developing cryptocurrencies is very low, I would estimate there are less than 100 such developers the majority of the developers are located in the US and other “Western” nations the developer incentive structure with successful cryptocurrencies creates conflicts of interest In what follows, I propose solutions to these issues cited above. Since we develop our own full-node Bitcoin implementation, btcd, the criticism of the developer community status quo that follows applies not only to other developers, but also our developers who work on btcd. ... Title: Re: btcd: a bitcoind alternative written in Go Post by: behindtext on January 25, 2014, 02:20:33 PM Thanks for your work! i'm glad you enjoyed the blog post.https://blog.conformal.com/redecentralization-robust-developer-network/ we're 20-25 rpc calls away from being a drop-in replacement for bitcoind :-) Title: Re: btcd: a bitcoind alternative written in Go Post by: OhShei8e on August 17, 2014, 07:42:55 AM we're 20-25 rpc calls away from being a drop-in replacement for bitcoind :-) What the actual status of this? Does anyone of you using it as replacement (especially as simple full node without wallet functionality)? Title: Re: btcd: a bitcoind alternative written in Go Post by: Newar on August 17, 2014, 07:56:24 AM At least 13 or so:
https://getaddr.bitnodes.io/nodes/1408261895/?q=/btcwire:0.1.4/btcd:0.8.0/ I have used it for a while and if anything it seemed to use less memory. Title: Re: btcd: a bitcoind alternative written in Go Post by: PRab on August 18, 2014, 01:23:35 AM What the actual status of this? Does anyone of you using it as replacement (especially as simple full node without wallet functionality)? I have used it as a successful drop-in replacement for bitcoind-ncurses. https://bitcointalk.org/index.php?topic=668475.msg8190154#msg8190154 It is very nice having a unified interface so that I can monitor both simultaneously. Title: Re: btcd: a bitcoind alternative written in Go Post by: T-rage_11 on September 17, 2014, 09:29:48 PM What is going on here? Is this a BUG?
https://i.imgur.com/73HqNzd.png?1 It just keeps adding orphan blocks !!!!! Title: Re: btcd: a bitcoind alternative written in Go Post by: iron man on September 18, 2014, 01:58:39 AM Very cool stuff! This is great news. What motivated the project? maybe the competition from bitcoin. Title: Re: btcd: a bitcoind alternative written in Go Post by: davec on September 18, 2014, 03:13:00 AM What is going on here? Is this a BUG? There was a bug in the upstream leveldb that could rarely lead to what you're seeing and was fixed a while back. There hasn't been a new Windows msi released yet. You can build from source quite easily with Go on Windows if you want to build with the latest code. There will be another msi release soon which includes that fix along with several other enhancements. Title: Re: btcd: a bitcoind alternative written in Go Post by: T-rage_11 on September 18, 2014, 12:08:30 PM What is going on here? Is this a BUG? There was a bug in the upstream leveldb that could rarely lead to what you're seeing and was fixed a while back. There hasn't been a new Windows msi released yet. You can build from source quite easily with Go on Windows if you want to build with the latest code. There will be another msi release soon which includes that fix along with several other enhancements. I just built on Windows XP x86 AND Windows 8.1 x64 .. BUT the bug is STILL HERE !? Is it normal that my new .exe builds are smaller than the "original" ones? they are just 12 MB (x86) and 15MB (x64) big! original x64 = 18MB Title: Re: btcd: a bitcoind alternative written in Go Post by: davec on September 18, 2014, 02:54:34 PM I just built on Windows XP x86 AND Windows 8.1 x64 .. BUT the bug is STILL HERE !? Is it normal that my new .exe builds are smaller than the "original" ones? they are just 12 MB (x86) and 15MB (x64) big! original x64 = 18MB You'll need to delete your database from "%LOCALAPPDATA%\Btcd\data\mainnet", so the chain download starts over. The issue is your existing database is corrupt (hence why you're seeing the issue looking up a specific transaction), so only upgrading the code alone won't fix it. However, the root cause that led to the corruption to begin with is what updating the code resolves. I should have mentioned that in my original response. As far as the size, the last released msi was built with the Go 1.2 runtime, and I'm guessing you built the latest with the most recent Go 1.3.1 run-time, so a difference in size is expected. 12MB (x86) and 15MB (x64) is what I have as well with the go 1.3 runtime. EDIT: I just rebuilt to verify: x64 09/18/2014 10:13 AM 15,415,296 btcd.exe x86 09/18/2014 10:15 AM 12,524,544 btcd.exe Title: Re: btcd: a bitcoind alternative written in Go Post by: T-rage_11 on September 18, 2014, 03:23:40 PM I just built on Windows XP x86 AND Windows 8.1 x64 .. BUT the bug is STILL HERE !? Is it normal that my new .exe builds are smaller than the "original" ones? they are just 12 MB (x86) and 15MB (x64) big! original x64 = 18MB You'll need to delete your database from "%LOCALAPPDATA%\Btcd\data\mainnet", so the chain download starts over. The issue is your existing database is corrupt (hence why you're seeing the issue looking up a specific transaction), so only upgrading the code alone won't fix it. However, the root cause that led to the corruption to begin with is what updating the code resolves. I should have mentioned that in my original response. As far as the size, the last released msi was built with the Go 1.2 runtime, and I'm guessing you built the latest with the most recent Go 1.3.1 run-time, so a difference in size is expected. 12MB (x86) and 15MB (x64) is what I have as well with the go 1.3 runtime. EDIT: I just rebuilt to verify: x64 09/18/2014 10:13 AM 15,415,296 btcd.exe x86 09/18/2014 10:15 AM 12,524,544 btcd.exe thanks :) my x64 build https://i.imgur.com/L5tkGdK.png EDIT: AFTER SOME HOURS I GET THE "Adding orphan blocks"-BUG AGAIN >:( :'( Title: Re: btcd: a bitcoind alternative written in Go Post by: pmorris on September 18, 2014, 05:59:08 PM I recently switched from running bitcoind to btcd on linux.
Once golang was installed it’s pretty much an automated installation process – 1 command to auto download and install the btcd package. A well-documented sample config file is provided that was easy to tailor. Launched it and done. Full network sync from scratch took 26 hours. Memory usage is in the 200MB range. Couple of days in it’s been rock solid and seems to be working great. I’m hugely impressed with what the devs have achieved. Their code appears to be clean, modular, well documented and comprehensively covered by tests. So a great base to build on. Title: Re: btcd: a bitcoind alternative written in Go Post by: T-rage_11 on September 20, 2014, 11:23:52 PM I am trying the new btcd v0.9.0-beta now .. https://github.com/conformal/btcd/releases
EDIT: x64 version v0.9.0-beta https://i.imgur.com/ngze49q.png EDIT2: PROBLEM SOLVED ! NO BUG ! JUST A BAD HARD DISK ! Title: Re: btcd: a bitcoind alternative written in Go Post by: davec on September 21, 2014, 07:04:19 PM I am trying the new btcd v0.9.0-beta now .. https://github.com/conformal/btcd/releases Would you please file an issue on the btcd issue tracker (https://github.com/conformal/btcd/issues)? We'll need to get your database logs to track down your issue. Title: Re: btcd: a bitcoind alternative written in Go Post by: zznbey on September 21, 2014, 07:09:05 PM as if dependence on a bunch of profiteers would be a good thing.
Title: Re: btcd: a bitcoind alternative written in Go Post by: davec on September 21, 2014, 08:23:58 PM zznbey:
I'm not sure what point you're trying to make, but I think perhaps you are confused. The btcd code is ISC licensed and freely available for all to fork and modify as they see fit. It is no different than Bitcoin Core in that respect. Title: Re: btcd: a bitcoind alternative written in Go Post by: johnnyrocket on September 21, 2014, 09:57:54 PM I recently switched from running bitcoind to btcd on linux. Once golang was installed it’s pretty much an automated installation process – 1 command to auto download and install the btcd package. A well-documented sample config file is provided that was easy to tailor. Launched it and done. Full network sync from scratch took 26 hours. Memory usage is in the 200MB range. Couple of days in it’s been rock solid and seems to be working great. I’m hugely impressed with what the devs have achieved. Their code appears to be clean, modular, well documented and comprehensively covered by tests. So a great base to build on. I've also recently tested btcd out, and the code looks clean + tests are present. I do see better performance than bitcoind, as well. Keep up the good work! Title: Re: btcd: a bitcoind alternative written in Go Post by: T-rage_11 on September 26, 2014, 02:12:30 PM Can you subscribe to new unconfirmed transactions with btcd ?
like the blockchain API: https://blockchain.info/de/api/api_websocket I currently use the blockchain API for my site: btcaudio.tk/live (http://btcaudio.tk/live) but I want to use my own bitcoin node for this .. can I do this with btcd? Title: Re: btcd: a bitcoind alternative written in Go Post by: davec on September 26, 2014, 04:54:36 PM Can you subscribe to new unconfirmed transactions with btcd ? Yes. See https://github.com/conformal/btcd/wiki/JSON-RPC-API#WSExtMethods. In particular the notifynewtransactions (https://github.com/conformal/btcd/wiki/JSON-RPC-API#notifynewtransactions) method. Title: Re: btcd: a bitcoind alternative written in Go Post by: Sanquin on September 26, 2014, 11:49:20 PM Always been interested in GO, Looks a very promising project, block safari looks good also.
Il drop by and see you on IRC :) Title: Re: btcd: a bitcoind alternative written in Go Post by: T-rage_11 on October 30, 2014, 02:06:59 AM Can you subscribe to new unconfirmed transactions with btcd ? Yes. See https://github.com/conformal/btcd/wiki/JSON-RPC-API#WSExtMethods. In particular the notifynewtransactions (https://github.com/conformal/btcd/wiki/JSON-RPC-API#notifynewtransactions) method. I want to connect to my btcd-server with javascript Code: <!DOCTYPE html> http://www.websocket.org/echo.html Can I disable the RPC-CERTIFICATES or how can I connect with Javascript over TLS? Title: Re: btcd: a bitcoind alternative written in Go Post by: work2heat on November 01, 2014, 01:58:20 AM Started playing with this today. Great work conformal team!
Title: Re: btcd: a bitcoind alternative written in Go Post by: davec on November 06, 2014, 05:30:34 PM I want to connect to my btcd-server with javascript Can I disable the RPC-CERTIFICATES or how can I connect with Javascript over TLS? I just saw this. You can't disable TLS, but I have a gist up on my github which demonstrates how to connect via TLS that I've tested with firefox and chrome: https://gist.github.com/davecgh/8a16b1048fe0fb406d29 Make sure you modify the username and password near the top of the file in the javascript to the correct values or you'll get an undefined error since btcd will immediately disconnect you upon receiving invalid credentials. Title: Re: btcd: a bitcoind alternative written in Go Post by: T-rage_11 on November 06, 2014, 09:34:20 PM I want to connect to my btcd-server with javascript Can I disable the RPC-CERTIFICATES or how can I connect with Javascript over TLS? I just saw this. You can't disable TLS, but I have a gist up on my github which demonstrates how to connect via TLS that I've tested with firefox and chrome: https://gist.github.com/davecgh/8a16b1048fe0fb406d29 Make sure you modify the username and password near the top of the file in the javascript to the correct values or you'll get an undefined error since btcd will immediately disconnect you upon receiving invalid credentials. Awesome! Thank you! :) Title: Re: btcd: a bitcoind alternative written in Go Post by: micax1 on November 18, 2014, 01:44:28 PM Hi. first of all Thank you for great software! I`m using it and fully satisfied!
Want to ask a question: Is there any guide on how to convert BTCD or complete solution for litecoin and dogecoin based on BTCD?? Thank you! Title: Re: btcd: a bitcoind alternative written in Go Post by: allgoodthings1 on November 19, 2014, 03:11:54 PM Is this normal?
I am installing btcd on a Widows 7 (64-bit) machine with the .msi istaller. On the initial sync, I'm up to block 324k -- and the HDD is reading/writing constantly! The green light on my box hardly even flashes. It's much more ON than OFF. I tried stopping the program, rebooting the computer and starting the program again. But the behaviour continues, and this has gone on like this for several hours now. Will this go away or get better when the blockchain is fully synced, or is this normal behaviour for btcd? Title: Re: btcd: a bitcoind alternative written in Go Post by: davec on November 19, 2014, 07:46:11 PM Hi. first of all Thank you for great software! I`m using it and fully satisfied! Want to ask a question: Is there any guide on how to convert BTCD or complete solution for litecoin and dogecoin based on BTCD?? Thank you! I don't know of any guides on converting the code base for LTC/DOGE. I know there has been some work to create ppcd which is for peercoin, so that is probably a pretty good place to see some conversion work that has been done. Is this normal? I am installing btcd on a Widows 7 (64-bit) machine with the .msi istaller. On the initial sync, I'm up to block 324k -- and the HDD is reading/writing constantly! The green light on my box hardly even flashes. It's much more ON than OFF. I tried stopping the program, rebooting the computer and starting the program again. But the behaviour continues, and this has gone on like this for several hours now. Will this go away or get better when the blockchain is fully synced, or is this normal behaviour for btcd? After the final checkpoint (currently at 319400), it starts to do full verification on the blocks, which is why you see the heavy HDD usage (it's loading every single input transaction to every transaction in every new block). This is also true of all full nodes that must do verification and is normal until you're fully synced at which point the usage drops significantly. Title: Re: btcd: a bitcoind alternative written in Go Post by: Gyrsur on November 22, 2014, 07:39:46 PM it's the better Bitcoin implementation.
Title: Re: btcd: a bitcoind alternative written in Go Post by: allgoodthings1 on November 29, 2014, 03:12:48 PM Q: Is there a way to monitor my btcd connections -- something like what you can do with the command "bitcoind getinfo" in that program? I have btcd synced up and running on a Windows machine. But I'd really like a way to see at a glance especially how many peers (outbound & inbound) that it has at any moment.
When I try to count how many there might be -- from looking at the long message list in the window -- it seems like there are woefully few connections. Maybe that's normal? But I'd like a way to quickly check that I'm maintaining active live connections (both in and out) at all times. Title: Re: btcd: a bitcoind alternative written in Go Post by: arnuschky on December 16, 2014, 09:38:34 PM This looks like the bitcoin implementation I have always been looking for!
Hope I have the time to check this out over the next days... Even more so if this issue https://github.com/conformal/btcd/issues/190 actually gets implemented! (The Counterparty team offers a 1500$ bounty for the the creation of address index ŕ la Insight API in btcd. Great move, Counterparty!) Title: Re: btcd: a bitcoind alternative written in Go Post by: QuantumQrack on December 19, 2014, 10:20:06 PM Q: Is there a way to monitor my btcd connections -- something like what you can do with the command "bitcoind getinfo" in that program? I have btcd synced up and running on a Windows machine. But I'd really like a way to see at a glance especially how many peers (outbound & inbound) that it has at any moment. When I try to count how many there might be -- from looking at the long message list in the window -- it seems like there are woefully few connections. Maybe that's normal? But I'd like a way to quickly check that I'm maintaining active live connections (both in and out) at all times. You use a program called btcctl.exe to query btcd. It is in the same folder that btcd.exe is in. There are many commands available. e.g. C:\program files\btcd suite\btcd>btcctl getpeerinfo Title: Re: btcd: a bitcoind alternative written in Go Post by: azeteki on December 19, 2014, 10:40:58 PM Q: Is there a way to monitor my btcd connections -- something like what you can do with the command "bitcoind getinfo" in that program? I have btcd synced up and running on a Windows machine. But I'd really like a way to see at a glance especially how many peers (outbound & inbound) that it has at any moment. When I try to count how many there might be -- from looking at the long message list in the window -- it seems like there are woefully few connections. Maybe that's normal? But I'd like a way to quickly check that I'm maintaining active live connections (both in and out) at all times. You may find the tool in my signature (bitcoind-ncurses) useful. I have not performed any testing with btcd however other users have reported success with some tweaking (I believe all that is needed is to produce a config file that is formatted in the same way as Bitcoin Core). It refreshes the number of peers at regular intervals. There is a peer display screen which shows all peers individually however this does not auto-refresh - may do so in the future. Screenshot: https://raw.githubusercontent.com/azeteki/bitcoind-ncurses/master/screenshots/bitcoind-ncurses-monitor.png Title: Re: btcd: a bitcoind alternative written in Go Post by: arnuschky on January 31, 2015, 01:42:54 PM What is the status of the btcd wallet? We're quite happy with btcd as a node, but haven't gathered experience with btcd's wallet yet. Is it ready for production? Or, let's rephrase this: is someone using it for production purposes already? Title: Re: btcd: a bitcoind alternative written in Go Post by: justusranvier on January 31, 2015, 02:54:02 PM What is the status of the btcd wallet? We're quite happy with btcd as a node, but haven't gathered experience with btcd's wallet yet. Is it ready for production? Or, let's rephrase this: is someone using it for production purposes already? As far as I know, it work fine as traditional wallet. Title: Re: btcd: a bitcoind alternative written in Go Post by: tacotime on February 01, 2015, 12:37:12 AM What is the status of the btcd wallet? We're quite happy with btcd as a node, but haven't gathered experience with btcd's wallet yet. Is it ready for production? Or, let's rephrase this: is someone using it for production purposes already? There has been significant efforts to overhaul it for performance considerations recently. Within a month or two these optimizations should be merged to master, and wallet should be good to go for high throughput uses while consuming small amounts of system resources. Previously smaller projects I have heard of using it in production. Title: Re: btcd: a bitcoind alternative written in Go Post by: arnuschky on February 01, 2015, 11:22:57 AM Monetas has two full-time programmers developing new features for btcwallet so that it can be used in production in our system, but that's just because we need features that don't exist in any wallet yet. So I heard, thank's for confirming that this is still the case. To clarify: are you contributing to the open source project, or are you privately patching the software? What are these features you are talking about, if I may ask? Just trying to understand whether your two full-time programmers affect our use at all. ;) Title: Re: btcd: a bitcoind alternative written in Go Post by: arnuschky on February 01, 2015, 11:28:35 AM There has been significant efforts to overhaul it for performance considerations recently. Within a month or two these optimizations should be merged to master, and wallet should be good to go for high throughput uses while consuming small amounts of system resources. Previously smaller projects I have heard of using it in production. Ok, thanks for the update. Are you involved in the development (if I may ask)? Where can I follow this development - is this happening inside conformal? Just curious where to track the progress of this new wallet implementation, and learn about its features. Title: Re: btcd: a bitcoind alternative written in Go Post by: davec on February 02, 2015, 02:21:18 AM Where can I follow this development - is this happening inside conformal? Just curious where to track the progress of this new wallet implementation, and learn about its features. It's a mix of internal and external contributors. Features show up as pull requests before they go in. The main things coming up are outlined in the following PRs: https://github.com/btcsuite/btcwallet/pull/125 - This has already been merged to master, but it's just a package that the wallet is not yet using. https://github.com/btcsuite/btcwallet/pull/152 - Like 125, this has already been merged to master, but the wallet is not yet using it. https://github.com/btcsuite/btcwallet/pull/147 - This will integrate the packages mentioned in 125 and 152 making everything they discussed available. Also, there are the following two PRs which aren't well documented yet, but are being worked on: https://github.com/btcsuite/btcwallet/pull/155 - This brings accounts support to btcwallet, but will need documentation because they are real accounts as opposed to the bitcoin core notion of accounts (completely independent, no commingling of funds, require on-chain transactions to transfer funds between them, etc). If you read between the lines on how pr 125 works, you can probably figure out how the accounts will work based on BIP0032. https://github.com/btcsuite/btcwallet/pull/154 - This will update the transaction store to use the new walletdb bits covered in 152. The current plan is to update 147 to include integration of this package once it's complete. Finally, Monetas has been working on bringing voting pool support and that is covered in the following PRs: https://github.com/btcsuite/btcwallet/pull/151 - This has already been merged, but it's just the package which is currently not being used. https://github.com/btcsuite/btcwallet/pull/178 - This will provide the withdrawal side of voting pools, but like 151, won't actually be used until the rest of the code needed to use it is complete Title: Re: btcd: a bitcoind alternative written in Go Post by: arnuschky on February 02, 2015, 08:08:14 AM Where can I follow this development - is this happening inside conformal? Just curious where to track the progress of this new wallet implementation, and learn about its features. It's a mix of internal and external contributors. Features show up as pull requests before they go in. The main things coming up are outlined in the following PRs: https://github.com/btcsuite/btcwallet/pull/125 - This has already been merged to master, but it's just a package that the wallet is not yet using. https://github.com/btcsuite/btcwallet/pull/152 - Like 125, this has already been merged to master, but the wallet is not yet using it. https://github.com/btcsuite/btcwallet/pull/147 - This will integrate the packages mentioned in 125 and 152 making everything they discussed available. Also, there are the following two PRs which aren't well documented yet, but are being worked on: https://github.com/btcsuite/btcwallet/pull/155 - This brings accounts support to btcwallet, but will need documentation because they are real accounts as opposed to the bitcoin core notion of accounts (completely independent, no commingling of funds, require on-chain transactions to transfer funds between them, etc). If you read between the lines on how pr 125 works, you can probably figure out how the accounts will work based on BIP0032. https://github.com/btcsuite/btcwallet/pull/154 - This will update the transaction store to use the new walletdb bits covered in 152. The current plan is to update 147 to include integration of this package once it's complete. Finally, Monetas has been working on bringing voting pool support and that is covered in the following PRs: https://github.com/btcsuite/btcwallet/pull/151 - This has already been merged, but it's just the package which is currently not being used. https://github.com/btcsuite/btcwallet/pull/178 - This will provide the withdrawal side of voting pools, but like 151, won't actually be used until the rest of the code needed to use it is complete Awesome, thank you for taking the time for listing these PRs. I'll wade through them :) Title: Re: btcd: a bitcoind alternative written in Go Post by: arnuschky on April 23, 2015, 08:14:47 PM Is there a place to discuss the btcd suite and ask questions? The current strategy of creating issues on github for every question seems to be rather cumbersome, especially for the devs... ;) Title: Re: btcd: a bitcoind alternative written in Go Post by: davec on April 23, 2015, 09:09:35 PM We do the vast majority of our communication through IRC (https://github.com/btcsuite/btcd/blob/master/README.md#irc-server).
Title: Re: btcd: a bitcoind alternative written in Go Post by: arnuschky on April 24, 2015, 07:43:55 AM We do the vast majority of our communication through IRC (https://github.com/btcsuite/btcd/blob/master/README.md#irc-server). Ok, thanks. I assume you posting this here means that the channel is not dev-only but also open for user questions? Title: Re: btcd: a bitcoind alternative written in Go Post by: davec on April 24, 2015, 04:18:54 PM Ok, thanks. I assume you posting this here means that the channel is not dev-only but also open for user questions? Correct. There is some development discussion in there, but typically we use other channels for that purpose. Feel free to stop by! |
