I believe that wallet metadata like balances need to be updated during re-orgs as well as any time an incoming transaction is received that affects this information...
With the normalized block database you can derive balances up-to the last node using sql even in an other process. Even build materialized views to cache if you have the database support for that. The client serving bean (once there will be one) could then only care to listen to recent transactions and query the balance after block update (and evtl. reorg) from the database, that maintains transactional consistency etc... Key store is a matter of taste and paranoia. I hope people contribute different flavors. |
|
|
|
By the way, OP, why not reuse BitcoinJ code if it fits?
Because it does not. Ok then. I thought that you had done it on purpose, as if sharing code was something bad. I actually started using bitcoinj, but decided that it needs significant refactoring (as Mike also admits). It was also an exercise of real understanding the protocol and uncovering its ambiguities. The code I wrote is probably the easiest readable but precise existing documentation of what is really going on under the hood, and I am sharing it with you. Isn't this good? Wow, I'm really impressed!  I haven't tested it yet, but if it works, this could become a really important alternative to the satoshi client. There will be an out-of-the-box executable jar sometime down the road for demonstration purposes, but the audience for this is not the end user but the tech savvy implementing new enterprises. |
|
|
|
Nice! The code looks clean.
caveden, there have been several other full reimplementations. BitcoinJS is one, I think UfaSoft made one, and actually I just finished merging code from Matt that makes bitcoinj a fully verifying ultra-prune style implementation as well (of course, it can still run in SPV mode too).
So this implementation joins the ranks.
Thanks for the warm welcome. Was nice to talk to you in London Mike. I released the code since it reached the coverage needed to prove that it is for real. Not for production but to build on it as a platform. Your points on testing are valid. They are clearly not sufficient. I plan to introduce an interface aimed for testing. That would enable implementations to test each other. The Wallet is not there but everything you need to stick it together in a couple hours or days depending on what you want. I did not yet figured what I wanted. But the point is that this is a feature that does not have to be common. Do your own at your own taste and stick it in. I'm not sure you understand what ultraprune does or why the performance is higher.
I do and JPA certainly does not solve for that. My poorly made point is that normalized database schema allows services built next to the server that does not need to know about p2p or binary formats. The feasibility of pruning or different levels of that are now in the hand of the implementor of the persistence layer. |
|
|
|
What so "super" in this code that make it better from standard bitcoind? No idea if it's "better than bitcoind", actually it'd be quite presumptuous to say so.
Because one size does not fit all. Bitcoind is a monolithic hard to maintain code base. But usage of JPA is already an interesting point... Different databases could be plugged. Databases that support clustering or partitioning could be used.
"Fully normalized schema" doesn't always go well with high performance though...
Yes, and you can stick in whatever high performance provider your budget allows. This is for the terabytes coming. Normalization allows to build services on top of the database that do not have to deal with P2P or binary formats. I wonder how the author plans to keep up with all the development done in bitcoind though... they have many competent people improving the code and adding functionality. Just to keep up would be an enormous effort. But anyway, writing such a thing was certainly an enormous effort already!
The effort of bitcoind team made all this possible. I have highest regards, but I do not think the code they inherited is for the future. I would rather worry if bitcoind can keep up with innovation, now that a modular design is available people can easily stick their modules into. You do not have to wait for a new RPC call to add a new type of signature. You do not have to wait for ultraprune, just stick in a higher performance database. BTW: I buit enterprise systems, quant code and algo trading for decades. Look me up on linkedin if curious. By the way, OP, why not reuse BitcoinJ code if it fits?
Because it does not. |
|
|
|
TL;DR Announcing code release of the bitsofproof supernode. A modular Bitcoin protocol implementation to build bitcoin enterprises and server nodes on. WHY Satoshi is a genius, but his implementation of the protocol is hard to maintain and extend. I believe that peers will specialize and this implementation is suited for the big server, tailored for some heavy lifting tasks, not for a mobile. WHERE https://github.com/bitsofproof/supernode.git FEATURES Pure Java source code released under Apache 2.0 license. Radically modular Assembly using Spring. Extensible and reducible to what you need. High performance P2P engine using Java NIO. Block storage using JPA into a fully normalized schema. Full support for multisig and pay to hash, including address maintanance. Parallel validation of transactions of a block. DEPENDENCIES Spring, Bouncy Castle, SL4J Does NOT share any code with Satoshi or bitcoinj clients. LIMITATIONS Not tested enough for production, therefore does not yet answer getheader and getblock and does not feature a wallet. CONTRIBUTIONS Contributions are welcome. Add your own features to the modular design. WHO Tamas Blummer alias grau tamas@bitsofproof.com wrote this implementation in his spare time at home. DONATIONS Please honor the time I invested and motivate me to continue by donating to: 13xhxy21to93Lrb7d4ssZVaMnFtQVvRkSk |
|
|
|
This goes into the right direction. Will this become a full-node client?
Yes, this will be a full server. I target with it the server installations that will deal with the volume magnitudes above today's. It has a radically modular/extensible architecture. I will release the code before end of this week. |
|
|
|
This is an excerpt of my implementations's configuration. Is this the kind of modularization you are looking for ? <bean id="dnsDiscovery" class="com.bitsofproof.supernode.core.DNSDiscovery"/>
<bean id="bitcoinNetwork" class="com.bitsofproof.supernode.core.BitcoinNetwork">
<constructor-arg><value>10</value></constructor-arg> <!-- Number of connections desired -->
<property name="discovery" ref="dnsDiscovery"/>
<property name="chain" ref="${chain}"/>
<property name="store" ref="${store}" />
<property name="transactionManager" ref="transactionManager" />
</bean>
<bean id="chainloader" class="com.bitsofproof.supernode.core.ChainLoader">
<constructor-arg><ref bean="bitcoinNetwork"/></constructor-arg>
<constructor-arg><ref bean="${store}"/></constructor-arg>
<property name="inventoryTimeout" value="30"/>
</bean>
<bean id="pingpong" class="com.bitsofproof.supernode.core.PingPongHandler">
<constructor-arg><ref bean="bitcoinNetwork"/></constructor-arg>
</bean>
<bean id="heartbeat" class="com.bitsofproof.supernode.core.HeartbeatHandler">
<constructor-arg><ref bean="bitcoinNetwork"/></constructor-arg>
<property name="delay" value="60"/>
<property name="timeout" value="60"/>
</bean>
<bean id="address" class="com.bitsofproof.supernode.core.AddressHandler">
<constructor-arg><ref bean="bitcoinNetwork"/></constructor-arg>
</bean>
<bean id="transaction" class="com.bitsofproof.supernode.core.TransactionHandler">
<constructor-arg><ref bean="bitcoinNetwork"/></constructor-arg>
<property name="store" ref="${store}"/>
<property name="loader" ref="chainloader"/>
</bean>
<bean id="getblocks" class="com.bitsofproof.supernode.core.GetBlocksHandler">
<constructor-arg><ref bean="bitcoinNetwork"/></constructor-arg>
</bean>
<bean id="getheaders" class="com.bitsofproof.supernode.core.GetHeadersHandler">
<constructor-arg><ref bean="bitcoinNetwork"/></constructor-arg>
</bean>
<bean id="addressSeeder" class="com.bitsofproof.supernode.core.AddressSeeder">
<constructor-arg><ref bean="bitcoinNetwork"/></constructor-arg>
<property name="delay" value="60"/>
<property name="startDelay" value="60"/>
</bean>
<bean id="application" class="com.bitsofproof.supernode.main.Application">
<property name="chain" ref="${chain}"/>
<property name="network" ref="bitcoinNetwork"/>
<property name="store" ref="${store}" />
<property name="transactionManager" ref="transactionManager" />
</bean>
|
|
|
|
While revalidating the entire chain using my independent implementation I found that transaction: 70f7c15c6f62139cc41afa858894650344eda9975b46656d893ee59df8914a3d validates only if the S value of the signature is forced positive, it seems the ASN.1 coding of the signature in the Satoshi client (probably OpenSSL) is not precise. Here the wire dump of the involved transactions: spend: 0100000001289eb02e8ddc1ee3486aadc1cd1335fba22a8e3e87e3f41b7c5bbe7fb4391d81010000008a47304402206b5c3b1c86748dcf328b9f3a65e10085afcf5d1af5b40970d8ce3a9355e06b5b0220cdbdc23e6d3618e47056fccc60c5f73d1a542186705197e5791e97f0e6582a32014104f25ec495fa21ad14d69f45bf277129488cfb1a339aba1fed3c5099bb6d8e9716491a14050fbc0b2fed2963dc1e56264b3adf52a81b953222a2180d48b54d1e18ffffffff0140420f00000000001976a914e6ba8cc407375ce1623ec17b2f1a59f2503afc6788ac00000000
source: 01000000014213d2fe8f942dd7a72df14e656baab0e8b2b7f59571771ddf170b588379a2b6010000008b483045022037183e3e47b23634eeebe6fd155f0adbde756bf00a6843a1317b6548a03f3cfe0221009f96bec8759837f844478a35e102618918662869188f99d32dffe6ef7f81427e014104a7d3b0dda6d4d0a44b137a65105cdfed890b09ce2d283d5683029f46a00e531bff1deb3ad3862e0648dca953a4250b83610c4f20861555a2f5638bd3d7aff93dffffffff02ddfb1100000000001976a9142256ff6b9b9fea32bfa8e64aed10ee695ffe100988ac40420f00000000001976a914c62301ef02dfeec757fb8dedb8a45eda5fb5ee4d88ac00000000
|
|
|
|
|
T1 refers to the output of certain previous transactions not some "current" balance of A.
|
|
|
|
|
I am here to help, not to fork.
Using your example: for a while HTML was defined by Netscape but that did not save us from the Microsoft fork.
Hundred pages of standard did not work there either.
What might work is a set of test suites that major implementations and new releases have to pass. I am willing to contribute to such.
|
|
|
|
Thanks  Yes, I am done with a high performance P2P kernel using Java NIO, do all verifications of blocks I found on wiki, in Satoshi and bitcoinj code base and some more as this thread illustrates. I store the blockchain in a normalized cross linked and indexed relational database using JPA, Support for BCCAPI, servlet interface is in works, assemly using Spring. Expect to be impressed should I choose to release. |
|
|
|
|
Thanks, that is I was afraid of. I know, since this is old we can only declare it a "feature".
To uncover those "quirks" is exactly the reason I do tho effort.
I think the protocol should not be defined by a particular implementation, even if that one was written by Satoshi. I will add this "feature" to the wiki.
I re-implement the protocol independently since I believe a bug in the dominant implementation is the most present danger to the ecosystem.
I hope the community understands that diversity of implementations is what we need.
|
|
|
|
|
While bootstrapping and verifying the chain using my implementation of the protocol (that shares no code with the Satoshi client) I found that transaction
c99c49da4c38af669dea436d3e73780dfdb6c1ecf9958baa52960e8baee30e73
does not verify. Since it looks regular and my implementation verified all 110299 block before this one, I am concerned.
This transaction is at chain height 110300, block 000000000000b8c3ad583c44e2655f5384f3e4e6a1f2a932b512ecc69a07cc24
Here my JSON dump of the transaction:
{hash:"c99c49da4c38af669dea436d3e73780dfdb6c1ecf9958baa52960e8baee30e73",version:1,inputs:[{sourceHash:"406b2b06bcd34d3c8733e6b79f7a394c8a431fbf4ff5ac705c93f4076bb77602",sourceIx:0,script:"OP_PUSH73 3046022100d23459d03ed7e9511a47d13292d3430a04627de6235b6e51a40f9cd386f2abe302210 0e7d25b080f0bb8d8d5f878bba7d54ad2fda650ea8d158a33ee3cbd11768191fd00 OP_PUSH65 04b0e2c879e4daf7b9ab68350228c159766676a14f5815084ba166432aab46198d4cca98fa3e998 1d0a90b2effc514b76279476550ba3663fdcaff94c38420e9d5",sequence:0}],outputs:[{value:4000000,script:"OP_DUP OP_HASH160 OP_PUSH20 9a7b0f3b80c6baaeedce0a0842553800f832ba1f OP_EQUALVERIFY OP_CHECKSIG"}],lockTime:0}
I think the problem is that the bold byte at the end of the signature, that should be 01 for SIGHASH_ALL. The value 00 does not map to any flavor. IMHO this is/was a "bug" in the Satoshi client.
|
|
|
|
|
Solved: I was talking too fast to the standard client... I am using Java NIO.
Breaking down my output to 8K buffers was the only change needed.
|
|
|
|
|
Would you please have a hint what goes wrong here:
Standard peers (Satoshi 6.x, 7.x) answer my self coded version of verack, inv and getdata if running on my laptop behind a router connected to the internet, but if I run the same code on my server on the internet, the getdata requests are ignored. I get version, verack, inv and addr messages but no blocks I request.
The messages I send seem to be identical (besides address and random nonce).
Thanks for any pointer.
|
|
|
|
|
I freshly built from github on OSX and get endless exceptions:
ProcessMessages(tx, 226 bytes) : Exception 'CDataStream::read() : end of data' caught, normally caused by a message being shorter than its stated length
|
|
|
|
It's not an either-or. 100% of all Bitcoins in existence are saved all the time. 100% of all USD in existence are saved all the time. The only difference is how fast they change the hands of savers.
Well observed Timo. If velocity of money is lower than needed to lubricate trade, then people will start settling bills by other means, e.g. trade of debt and alternate currencies. |
|
|
|
The idea that they must make the smaller units a worse deal is false.
cmon, the incremental production cost of a further chip on the board is less than building its own box, interface, cables around it. That BFL prices them equally on $/hash today is just their idea of how to maximize profit until they get competition. |
|
|
|
It opens up an arb opportunity if the gamma passthrough trades lower than S.DICE, so I expect the passthrough to be at least equivalent to S.DICE, if not higher due to increased demand on GLBSE.
arb is riskless profit you locked in, this is not since you can not transfer the asset from glbse to mpex. at best you get a discount for taking additional counterparty risk. the pass through might also feature higher supply if things arent going well with the underlying asset and is bound to have higher bid-ask spread. |
|
|
|
|
How do you plan to prove, that the rolls are truly random and the house edge is constant?
|
|
|
|
|
Show Posts
