Bitcoin Forum
September 26, 2018, 11:44:17 AM *
News: ♦♦ New info! Bitcoin Core users absolutely must upgrade to previously-announced 0.16.3 [Torrent]. All Bitcoin users should temporarily trust confirmations slightly less. More info.
  Home Help Search Donate Login Register  
  Show Posts
Pages: [1] 2 3 4 5 »
1  Bitcoin / Development & Technical Discussion / MOVED: 2x is the end of bitcoin as we knew it. Coinbase, US, banks are friends on: September 25, 2017, 09:52:04 PM
This (non-technical) topic has been moved to Bitcoin Discussion.
2  Bitcoin / Development & Technical Discussion / Bitcoin Core 0.14.0 release candidate 1 available on: February 19, 2017, 01:42:38 PM

Hash: SHA512

Release candidate 1 of a new major bitcoin Core release, version 0.14.0, has
been made available.

This is a release candidate for a new major version release, including new
features, various bugfixes and performance improvements.

Preliminary release notes for the release can be found here:

Binaries can be downloaded from:

Source code can be found on github under the signed tag

Release candidates are test versions for releases. When no critical problems
are found, this release candidate will be tagged as 0.14.0 final, otherwise
a new rc will be made available after these are solved.

Please report bugs using the issue tracker at github:

Version: GnuPG v1

3  Bitcoin / Bitcoin Discussion / Bitcoin Core 0.13.2 Released on: January 03, 2017, 09:06:08 AM
0.13.2 is out and here is the release announcement:

Hash: SHA512

Bitcoin Core version 0.13.2 is now available from:


Or by bittorrent:


This is a new minor version release, including various bugfixes and
performance improvements, as well as updated translations.

Please report bugs using the issue tracker at github:


To receive security and update notifications, please subscribe to:



Microsoft ended support for Windows XP on [April 8th, 2014](,
an OS initially released in 2001. This means that not even critical security
updates will be released anymore. Without security updates, using a bitcoin
wallet on a XP machine is irresponsible at least.

In addition to that, with 0.12.x there have been varied reports of Bitcoin Core
randomly crashing on Windows XP. It is [not clear](
what the source of these crashes is, but it is likely that upstream
libraries such as Qt are no longer being tested on XP.

We do not have time nor resources to provide support for an OS that is
end-of-life. From 0.13.0 on, Windows XP is no longer supported. Users are
suggested to upgrade to a newer version of Windows, or install an alternative OS
that is supported.

No attempt is made to prevent installing or running the software on Windows XP,
you can still do so at your own risk, but do not expect it to work: do not
report issues about Windows XP to the issue tracker.

- From 0.13.1 onwards OS X 10.7 is no longer supported. 0.13.0 was intended to work on 10.7+,
but severe issues with the libc++ version on 10.7.x keep it from running reliably.
0.13.1 now requires 10.8+, and will communicate that to 10.7 users, rather than crashing unexpectedly.

Notable changes

Change to wallet handling of mempool rejection
- -----------------------------------------------

When a newly created transaction failed to enter the mempool due to
the limits on chains of unconfirmed transactions the sending RPC
calls would return an error.  The transaction would still be queued
in the wallet and, once some of the parent transactions were
confirmed, broadcast after the software was restarted.

This behavior has been changed to return success and to reattempt
mempool insertion at the same time transaction rebroadcast is
attempted, avoiding a need for a restart.

Transactions in the wallet which cannot be accepted into the mempool
can be abandoned with the previously existing abandontransaction RPC
(or in the GUI via a context menu on the transaction).

0.13.2 Change log

Detailed release notes follow. This overview includes changes that affect
behavior, not code moves, refactors and string updates. For convenience in locating
the code changes and accompanying discussion, both the pull request and
git merge commit are mentioned.

### Consensus
- - #9293 `e591c10` [0.13 Backport #9053] IBD using chainwork instead of height and not using header timestamp (gmaxwell)
- - #9053 `5b93eee` IBD using chainwork instead of height and not using header timestamps (gmaxwell)

### RPC and other APIs
- - #8845 `1d048b9` Don't return the address of a P2SH of a P2SH (jnewbery)
- - #9041 `87fbced` keypoololdest denote Unix epoch, not GMT (s-matthew-english)
- - #9122 `f82c81b` fix getnettotals RPC description about timemillis (visvirial)
- - #9042 `5bcb05d` [rpc] ParseHash: Fail when length is not 64 (MarcoFalke)
- - #9194 `f26dab7` Add option to return non-segwit serialization via rpc (instagibbs)
- - #9347 `b711390` [0.13.2] wallet/rpc backports (MarcoFalke)
- - #9292 `c365556` Complain when unknown rpcserialversion is specified (sipa)
- - #9322 `49a612f` [qa] Don't set unknown rpcserialversion (MarcoFalke)

### Block and transaction handling
- - #8357 `ce0d817` [mempool] Fix relaypriority calculation error (maiiz)
- - #9267 `0a4aa87` [0.13 backport #9239] Disable fee estimates for a confirm target of 1 block (morcos)
- - #9196 `0c09d9f` Send tip change notification from invalidateblock (ryanofsky)

### P2P protocol and network code
- - #8995 `9ef3875` Add missing cs_main lock to ::GETBLOCKTXN processing (TheBlueMatt)
- - #9234 `94531b5` torcontrol: Explicitly request RSA1024 private key (laanwj)
- - #8637 `2cad5db` Compact Block Tweaks (rebase of #8235) (sipa)
- - #9058 `286e548` Fixes for test timeouts on travis (#8842) (ryanofsky)
- - #8865 `4c71fc4` Decouple peer-processing-logic from block-connection-logic (TheBlueMatt)
- - #9117 `6fe3981` net: don't send feefilter messages before the version handshake is complete (theuni)
- - #9188 `ca1fd75` Make orphan parent fetching ask for witnesses (gmaxwell)
- - #9052 `3a3bcbf` Use RelevantServices instead of node_network in AttemptToEvict (gmaxwell)
- - #9048 `9460771` [0.13 backport #9026] Fix handling of invalid compact blocks (sdaftuar)
- - #9357 `03b6f62` [0.13 backport #9352] Attempt reconstruction from all compact block announcements (sdaftuar)
- - #9189 `b96a8f7` Always add default_witness_commitment with GBT client support (sipa)
- - #9253 `28d0f22` Fix calculation of number of bound sockets to use (TheBlueMatt)
- - #9199 `da5a16b` Always drop the least preferred HB peer when adding a new one (gmaxwell)

### Build system
- - #9169 `d1b4da9` build: fix qt5.7 build under macOS (theuni)
- - #9326 `a0f7ece` Update for OpenSSL 1.1 API (gmaxwell)
- - #9224 `396c405` Prevent FD_SETSIZE error building on OpenBSD (ivdsangen)

### GUI
- - #8972 `6f86b53` Make warnings label selectable (jonasschnelli) (MarcoFalke)
- - #9185 `6d70a73` Fix coincontrol sort issue (jonasschnelli)
- - #9094 `5f3a12c` Use correct conversion function for boost::path datadir (laanwj)
- - #8908 `4a974b2` Update bitcoin-qt.desktop (s-matthew-english)
- - #9190 `dc46b10` Plug many memory leaks (laanwj)

### Wallet
- - #9290 `35174a0` Make RelayWalletTransaction attempt to AcceptToMemoryPool (gmaxwell)
- - #9295 `43bcfca` Bugfix: Fundrawtransaction: don't terminate when keypool is empty (jonasschnelli)
- - #9302 `f5d606e` Return txid even if ATMP fails for new transaction (sipa)
- - #9262 `fe39f26` Prefer coins that have fewer ancestors, sanity check txn before ATMP (instagibbs)

### Tests and QA
- - #9159 `eca9b46` Wait for specific block announcement in p2p-compactblocks (ryanofsky)
- - #9186 `dccdc3a` Fix use-after-free in scheduler tests (laanwj)
- - #9168 `3107280` Add assert_raises_message to check specific error message (mrbandrews)
- - #9191 `29435db` 0.13.2 Backports (MarcoFalke)
- - #9077 `1d4c884` Increase wallet-dump RPC timeout (ryanofsky)
- - #9098 `ecd7db5` Handle zombies and cluttered tmpdirs (MarcoFalke)
- - #8927 `387ec9d` Add script tests for FindAndDelete in pre-segwit and segwit scripts (jl2012)
- - #9200 `eebc699` bench: Fix subtle counting issue when rescaling iteration count (laanwj)

### Miscellaneous
- - #8838 `094848b` Calculate size and weight of block correctly in CreateNewBlock() (jnewbery)
- - #8920 `40169dc` Set minimum required Boost to 1.47.0 (fanquake)
- - #9251 `a710a43` Improvement of documentation of command line parameter 'whitelist' (wodry)
- - #8932 `106da69` Allow bitcoin-tx to create v2 transactions (btcdrak)
- - #8929 `12428b4` add software-properties-common (sigwo)
- - #9120 `08d1c90` bug: Missed one "return false" in recent refactoring in #9067 (UdjinM6)
- - #9067 `f85ee01` Fix exit codes (UdjinM6)
- - #9340 `fb987b3` [0.13] Update secp256k1 subtree (MarcoFalke)
- - #9229 `b172377` Remove calls to getaddrinfo_a (TheBlueMatt)


Thanks to everyone who directly contributed to this release:

- - Alex Morcos
- - BtcDrak
- - Cory Fields
- - fanquake
- - Gregory Maxwell
- - Gregory Sanders
- - instagibbs
- - Ivo van der Sangen
- - jnewbery
- - Johnson Lau
- - Jonas Schnelli
- - Luke Dashjr
- - maiiz
- - MarcoFalke
- - Masahiko Hyuga
- - Matt Corallo
- - matthias
- - mrbandrews
- - Pavel Janík
- - Pieter Wuille
- - randy-waterhouse
- - Russell Yanofsky
- - S. Matthew English
- - Steven
- - Suhas Daftuar
- - UdjinM6
- - Wladimir J. van der Laan
- - wodry

As well as everyone that helped translating on [Transifex](

Version: GnuPG v1

4  Bitcoin / Bitcoin Discussion / How have fungiblity problems affected you in Bitcoin? on: July 29, 2016, 02:33:27 AM
Privacy and fungiblity are essential components for any money-like system.
Without them, your transactions leak information about your private
activities and leave you at risk of discriminatory treatment. Without them your security is reduced due to selective targeting and your commercial negotiations can be undermined.

They're important and were consideration's in Bitcoin's design since day one. But Bitcoin's initial approach to preserving privacy and fungiblity -- pseudonymous addresses-- is limited, and full exploitation of it requires less convenient usage patterns that have fallen out of favor.

There are many technologies people have been working on to improve fungiblity and privacy in different ways-- coinjoins[url=http://and [url=]swaps] and [url=]swaps, confidential
, encrypted/committed transactions, schnorr
multisignature, MAST, better wallet input selection logic, private wallet scanning, tools for address reuse avoidance, P2P encryption], ECDH-derived addresses, P2P surveillance resistance, to name a few.

Having some more in-the-field examples will help prioritize these efforts. So I'm asking here for more examples of where privacy and fungiblity loss have hurt Bitcoin users or just discouraged Bitcoin use-- and, if known, the specifics about how those situations came about.

Please feel free to provide links to other people's examples too, and also feel free to contact me privately ( GPG: 0xAC859362B0413BFA ).

I also posted this question on Reddit, but though I might get a broader audience here.
5  Bitcoin / Development & Technical Discussion / High-S transactions, help finding origins? on: July 20, 2016, 10:40:18 PM
There is still a steady flow of high-s transactions hitting the network. The only reason they get mined now is because of some people running special nodes that mutate them to make them acceptable.

Is anyone interested in trying to find the sources?

Here is a list of txids for transactions which appear to have originally been high-S (these are the IDs for the mutated forms that had a chance to get confirmed):


6  Bitcoin / Development & Technical Discussion / The first successful Zero-Knowledge Contingent Payment on: February 26, 2016, 10:40:35 PM
I am happy to announce the first successful Zero-Knowledge Contingent Payment (ZKCP) on the Bitcoin network.

ZKCP is a transaction protocol that allows a buyer to purchase information from a seller using Bitcoin in a manner which is private, scalable, secure, and which doesn’t require trusting anyone: the expected information is transferred if and only if the payment is made. The buyer and seller do not need to trust each other or depend on arbitration by a third party.

Imagine a movie-style “briefcase swap” (one party with a briefcase full of cash, another containing secret documents), but without the potential scenario of one of the cases being filled with shredded newspaper and the resulting exciting chase scene.

An example application would be the owners of a particular make of e-book reader cooperating to purchase the DRM master keys from a failing manufacturer, so that they could load their own documents on their readers after the vendor’s servers go offline. This type of sale is inherently irreversible, potentially crosses multiple jurisdictions, and involves parties whose financial stability is uncertain–meaning that both parties either take a great deal of risk or have to make difficult arrangement. Using a ZKCP avoids the significant transactional costs involved in a sale which can otherwise easily go wrong.

In today’s transaction I purchased a solution to a 16x16 Sudoku puzzle for 0.10 BTC from Sean Bowe, a member of the Zcash team, as part of a demonstration performed live at Financial Cryptography 2016 in Barbados. I played my part in the transaction remotely from California.

The transfer involved two transactions:


Almost all of the engineering work behind this ZKCP implementation was done by Sean Bowe, with support from Pieter Wuille, myself, and Madars Virza.

Read more, including technical details and links to the software at
7  Bitcoin / Development & Technical Discussion / Blocksonly mode BW savings, the limits of efficient block xfer, and better relay on: February 26, 2016, 02:48:54 AM
Bitcoin Core 0.12 introduced a new blocksonly setting.  When set to blocksonly a node behaves normally but sends and receives no lose transactions; instead it handles only complete blocks. There are many applications for nodes where only confirmed transactions are interesting, and a node which still verifies and forwards blocks still contributes to network health-- less, perhaps, than one that relays transactions: but it also consumes fewer resources to begin with. An additional downside they don't get the latency advantages of signature caching since every transaction they see is totally new to them-- this isn't something miners should use.

How much less bandwidth does blocksonly use in practice?  I recently measured this using two techniques: Once by instrumenting a node to measure bandwidth used for blocks vs all other traffic, and again by repeatedly running in both modes for a day and monitoring the hosts total network usage; both modes gave effectively the same result.

How much is the savings?  Blocksonly reduced the node's bandwidth usage by 88%.

A significant implication of this is that any scheme for bandwidth reduction which works by using already relayed transactions to reduce the size of transmitted blocks can _AT MOST_ reduce the overall bandwidth usage by 12%-- assuming that differential compression achieved an "infinity-fold" improvement. (An example of Amdahl's law)

Why does relay use so much bandwidth?  The fundamental reason for this is because relay in the Bitcoin Protocol is linear in the number of transactions and the number of peers. (E.g. bandwidth is a function of Transactions * Peers).  The protocol uses an 'efficient' INV scheme --- but this scheme only reduces the bandwidth by a constant factor:  For every transaction a 36 byte long INV is sent (or received) from _every_ peer, and the full transaction is only sent once. Since the INV is a significant fraction of the transaction size (e.g. 36 bytes vs 300) this means that every 10 or so peers uses just as much bandwidth as sending the full transaction every time would take.

For the subset of nodes where blocksonly meets their needs it's available now and gives pretty much the greatest possible bandwidth savings. Full stop. For blocks the INV process is much more efficient because the ratio of INV size to a megabyte block is so large. But what can we do for all the other cases?

In the literature there are schemes for quite efficient gossip but there seems to be a trade-off between efficiency and attack robustness (for example, you could arrange nodes in a minimum spanning tree rooted at some super node(s) which would be optimal for repeated bandwidth but insanely unreliable and vulnerable to attacks). Even the simple INV scheme is vulnerable to attack: peers can delay transactions by INVing but then sending the data only after a long delay.  With ephemeral anonymous peers, attack resistance is very important to us-- but fortunately we don't need to achieve optimal efficiency.

One possible scheme I've been discussing (and working on) for a while is mempool reconciliation. My work here was originally motivated by the desire to avoid relay costs for transactions which are just going to be quickly replace or fall below the relay threshold, but this can also be applied to drastically reduce relay costs.

The idea is this, at random intervals a node will ask one of it's peers for a sketch of the top X MBytes of it's mempool. In it's request it could also signal that it's interested only in transactions whos ancestor feerate is over some minimum threshold, along with information about the size and feerate of its own mempool.  The returned sketch is an IBLT (see this reddit post for an EL5 explanation that I created for IBLT) of some size estimated by the sender based on the information sent by the requester.  Unlike the efficient block transmission IBLT proposals, this IBLT only transmits transaction IDs-- which will avoid the fragmentation overhead needed in sending whole transactions.

The requester then attempts to reconstruct the IDs using the content of it's own mempool.  If it is unable to reconstruct all of the IDs, it requests from another random peer with a different seed.

When the node has multiple IBLTs it can use the partial solutions from each of them to help solve the other ones.

As it learns new txids that it was previously unaware of it then fetches them via getdata. When it completes a reconstruction it can INV any missing transactions towards the peers that didn't have them.

(To avoid IBLT batching delays slowing transaction propagation too much, two peers can be elected to receive INVs for every transaction immediately; similar to the existing trickle-for-privacy scheme).

Most of the academic literature on IBLTs is concerned about parameter selections that minimize the asymptotic overhead (size of IBLT vs set difference) for perfect reconstruction of the set.  These parameters, however, result in quite high probabilities for complete failure.  In our case, being able to recover some of a set is fine both because mempools don't need to be completely consistent, and requests to other peers will help recover the missing entries.

A similar pattern happened (for the same reason) in the invention of fountain codes-- (one can approximately think of a IBLT as the dual of a fountain code). To achieve better odds of reconstruction, the order of nodes in the graph (isomorphic to the number of hash buckets in IBLT) should be randomly selected from a special distribution, instead of a constant. The effect is that the mixture of node orders reduces the probability that a decode gets stuck, because it is more likely that there will be be some entries that have the optimal density.  For IBLT this can be done by using the hash of the set member to pick the number of buckets for that set just like it is used to pick which buckets are used.

But right now I'm not aware of any research on what these optimal order distributions would look like for IBLTs given assumptions around capacity ratios... so I think there is a fair amount of basic science that needs to be worked out in order to really nail down a good design.  I've been trying to work on making more progress here for a couple months, but the toxic environment makes the grind a little difficult. I thought I would checkpoint my thinking here in the hope that someone would have some more thoughts to contribute.

In any case, this scheme would avoid the quadratic-like behavior of relay bandwidth. It would let nodes trade latency off vs relay overhead, and it would allow for a graceful way of handling the highest priority transactions first. -- transactions a dozen or more blocks deep in the mempool are not going to get mined any time soon, so if they have lower relay latency from reconciling them less frequently that is no great harm. I believe it could do so without substantial increases in vulnerability to attack. The same software infrastructure could also be used for bandwidth minimized block transfer for nodes that do have mempools (otherwise blocksonly gives optimal bandwidth gains), though latency minimization is much better accomplished via techniques like Matt's efficient block relay protocol-- since the most important thing to minimize latency is to minimize roundtrips.
8  Bitcoin / Development & Technical Discussion / Signature aggregation for improved scalablity. on: February 26, 2016, 12:52:05 AM
Back in 2013 an Anonymous Author wrote a brief paper on using BLS signatures to achieve a kind of non-interactive coinjoin.  I jumped on it right away, pointing out that it also could have useful anti-censorship properties. (Edit: Paper link is down, a copy is available from Andrew Poelstra.)

The core property it depends on is that fact that for a BLS signature one can take a collection of {message, pubkey, signature} tuples and merge their signatures: {[message1, message2, ...], [pubkey1, pubkey2, ...], signature}, in a way which is infeasible to unmerge them without knowing the components, and still be able to verify the result. Via a clever trick that involves signing per input and per output, the authorship of inputs and outputs can be unlinked. I have a sidechain implementation of this that I've been working on-- like coinjoin generally it composes well with CT, but I'm unsure of what to do about the performance (more below).

Another application of these aggregatable signatures which has come up from time to time in #bitcoin-wizards is using them to reduce blocksize  ("2013-09-13 21:38 < gmaxwell> sipa: I'm mildly excited about this pairing crypto aggregate signature idea. Not because of the anonymity stuff, but because it makes scalable relay fees viable. and can also reduce transaction sizes in the non-anonymous case"), since you'd save the sizes of the free standing signatures. Aggregate BLS signatures could even be made compatible with 'efficient' fraud proofs, though the cost of doing probably makes it not a win. (You would do a merkel sum tree over the interior computation, which aggregates the partial signatures as products in a 3kbit field; which would make it possible to show a block had an invalid aggregate signature with a compact proof).

The main downside of this are two fold-- the BLS signatures involve Pairing Cryptography-- less mature security assumptions than plain ECC signatures, and more importantly they're slow to verify: On hardware that can verify 70,000 secp256k1 signatures per second (or 140,000/s with batched Schnorr) they could only process about 8,000 BLS signature per second, with state of the art parameter selection and software.  So, while bandwidth would be reduced, the capacity of most nodes wouldn't be increased much because they'd be bottlenecked on signatures.  (And, right now, signature validation is extensively cached in Bitcoin-- and the aggregation would move validation back on the critical path).

[I've had some conversations about Dan Boneh about using multi-exponentiation like optimizations in computing the required product of parings here-- but it's likely that the performance improvement that this would get would only be on the order of 2x.]

Adam Back was asking me about this again today and I pointed him to the prior discussions regarding the performance limits.  Then he suggested just using Schnorr multi-signature instead of the BLS signatures. The idea is that if you have inputs with pubkeys P and P2  you can combine them and to form a P+P2 pubkey and sign with that to prove authorization with both keys.  When this had previously been discussed the fact that someone malicious could set their P2 to P3-P and then sign the aggregate with just P3 seemed to kill it.

But actually a few months ago Pieter Wuille solved this as a side effect of his key tree multi-signature work,  He suggested instead that to combine P and P2, you use P*H(P) + P2*H(P2); this doesn't inhibit signing, but it makes it impossible to create a pubkey as a linear combination of other pre-existing pubkeys. Pieter has an implementation of this as part of the Libsecp256k1 multi-signature work:

So the way we could use this to improve scalability in Bitcoin would be to add a OP_GROUPCHECKSIGVERIFY (and perhaps a multi-version of it) as part of a new segwit script typecode that takes as input sighash flags and a pubkey.  The script operation would push a the pubkey onto a transaction scoped 'signature stack'  and the sighash-masked transaction hash onto a transaction scoped hash stack.

After all inputs are processed the pubkeys on the stack would be combined to a new Pubkey as P = P1*H(H(all_Pn) || P1) + P2*H(H(all_Pn) || P2) + ..., and a root message hash computed-- then an additional group signature witness at the end of the transaction would contain a signature of the root with P.

Verification speed would be similar to individual inputs with batch schnorr (the multiplication with H() can be folded into the multi-exp in verification), so this is more of a space savings than a speedup; but the size of the transaction would be reduced by 41% asymptotically-- without any new security assumptions or performance hit. The main limit is that a transaction must have many inputs to fully realize that benefit.

Unlike other past shared signature proposals based on exploiting pubkey reuse, there is no advantage in this design in reusing public keys; so it also doesn't create a new privacy moral hazard for the network.

Critically, multiple distinct users could cooperate to securely produce a single large transaction with a two round protocol which may make it reasonable to get _many_ inputs in a single transaction-- but unlike the BLS signatures they must interact in order to merge their signatures.  Effectively this would be a CoinJoin, with a big space (and thus fee) savings for the CoinJoin users.

Even without combining multiple users this approach would make it much more economical for a single user to sweep up txout dust which would be an important improvement on its own. For block 400083 if all transactions used this (but with no increase in CoinJoin, or even behavioral changes to prefer sendmany over chains of transactions which we would expect if this actually existed; and also ignoring multisig use, which would further improve this figure) I compute this would reduce the size of the block by 172050 bytes out of 897956 or a 19% reduction which is substantial fraction of the asymptotic result. [Edit: A scan of 2016 recent blocks with accurate counting for multisig, shows the actual reduction for existing usage patterns would be 30%]

In effect this could achieve at least half the asymptotic gain while having no negative performance impact (10x-17x better performance than BLS) and no new strong cryptographic security assumptions. To me it seems like a no brainer improvement that Bitcoin is almost sure to eventually adopt once implemented.
9  Bitcoin / Hardware / Mining history... some odd avalon boards on: February 09, 2016, 07:28:02 AM
At my local hardware surplus place a bunch of gen 1 Avalon boards showed up.

Among them was a board that had none of the Avalon silk screening.

There were also a couple of  interesting mod-wires.

Due to lazyness I haven't dug up my batch 1 museum pieces to see if it looked like these.

(if anyone wants some dusty mining boards for $20/ea)
10  Bitcoin / Development & Technical Discussion / Notes on the use of midstate compression for coinbase transaction commitments on: November 19, 2015, 11:31:55 PM
I thought I'd leave a note for posterity related to the reoccurring scheme of using mid-state compression to avoid sending coinbase transactions; as it seems to get reinvented from time to time.  This has been discussed several times scattered about in various places, and I thought it useful to put the folklore understanding in one place.

In Bitcoin today various schemes like P2Pool or UTXO set commitments are suggested that require adding some mandatory data to a coinbase transaction.  To prove the mandatory data is present you must then transmit the whole coinbase transaction.  Coinbase transactions can be large, e.g. in p2pool and Eligius like pools that pay directly or for other reasons. In theory, a coinbase transaction can be upto the size of the whole block.

The reoccurring optimization is to put your mandatory data at the end of the coinbase transaction, and then send a sha256 midstate. The receiver can then complete the hashing with the mandatory data and verify agreement.

I personally think this design is inadvisable because it peels back the black box of the already fragile merkle-damgĺrd construction and may open some novel prefix-extension attacks that no one has thought to check sha2 for... but if it is done, implementations _must_ be very careful to avoid complications from serialization capture. E.g. if you require your last commitment to be a free standing output "OP_RETURN PUSH(data)" someone can instead make their final output "OP_RETURN [other stuff] PUSH__8+4+data+size", and then that output will consume the appended commitment, and then it's possible to send a misleading midstate proof, that would be parsed differently by a midstate only client and implementations that work with the whole coinbase transaction; opening up the avenue to attacks.

Another way of saying it is that due to the use of variable length fields the transaction serialization cannot be read backwards, and so no application using midstate compression-- which inherently hides the front of transaction-- can have any dependency on the serialized structure beyond "ends with these bytes"..

For commitment applications, these problems can be avoided like p2pool does by making your rule be specific to the last N bytes of the transaction (your commitment + nlocktime), regardless of what comes before them (e.g. embedded in another output is fine)... but I  still think it's best to avoid this construction entirely.
11  Bitcoin / Pools / Notice: _ALL POOLS AND SOLO MINERS MUST UPDATE FOR BIP-66_ on: June 12, 2015, 08:32:39 PM
Please see this post on bitcoin-development by Peter Todd:

We are currently at >75% hash-power with version 3 blocks, meaning BIP66 is enforced in those blocks.  If the 1001 block window reaches 95% all block with version<3 will be rejected and become orphaned.

I _strongly_ recommend upgrading to Bitcoin Core 0.10.2 immediately if you have not yet done so. If you have technical issues with the upgrade people will be glad to help, e.g. join #bitcoin-dev on freenode IRC.
12  Bitcoin / Development & Technical Discussion / Confidential Transactions, Content privacy for Bitcoin transactions on: June 09, 2015, 10:18:56 AM

I mentioned that I was using the new Borromean ringsig as a building block for a larger cryptosystem, Here it is:

Confidential Transactions

One of the most powerful new features being explored in the Elements sidechain is Confidential Transactions, a cryptographic tool to improve the privacy and security of Bitcoin. This feature keeps the amounts transferred visible only to participants in the transaction (and those they designate).

The security of the Bitcoin ledger is made possible by universal verification: each participant individually and autonomously verifies that each transaction is valid, without trusting any third party.  An unfortunate side effect is that all the transaction data must be conspicuously public so it can be verified, which is at odds with the normal expectation of privacy for traditional monetary instruments.

Insufficient financial privacy can have serious security and privacy implications for both commercial and personal transactions. Without adequate protection, thieves and scammers can focus their efforts on known high-value targets, competitors can learn business details, and negotiating positions can be undermined. Since publishing often requires spending money, lack of privacy can chill free speech.  Insufficient privacy can also result in a loss of fungibility--where some coins are treated as more acceptable than others--which would further undermine Bitcoin's utility as money.

Bitcoin partially addresses the privacy problem by using pseudonymous addresses. If someone does not know which users own which addresses, the privacy impact is reduced. But any time you transact with someone you learn at least one of their addresses. From there, you can trace out other connected addresses and estimate the values of their transactions and holdings. For example, suppose your employer pays you with Bitcoin and you later spend those coins on your rent and groceries. Your landlord and the supermarket will both learn your income, and could charge you higher prices as your income changes or target you for theft.

There are existing deployed techniques that further improve privacy in Bitcoin (such as CoinJoin, which merges the transaction history of users by making joint payments), but the utility of these techniques is reduced by the fact that it's possible to track amounts.

There have been proposed cryptographic techniques to improve privacy in Bitcoin-like systems, but so far all of them result in breaking "pruning" (section 7 of Bitcoin.pdf) and result in participants needing a perpetually growing database to verify new transactions, because these systems prevent learning which coins have been spent. Most proposed cryptographic privacy systems also have poor performance, high overhead, and/or require new and very strong (and less well understood) cryptographic assumptions.

Confidential Transactions improves the situation by making the transaction amounts private, while preserving the ability of the public network to verify that the ledger entries still add up. It does this without adding any new basic cryptographic assumptions to the Bitcoin system, and with a manageable level of overhead.

CT is possible due to the cryptographic technique of additively homomorphic commitments. As a side-effect of its design, CT also enables the additional exchange of private "memo" data (such as invoice numbers or refund addresses) without any further increase in transaction size, by reclaiming most of the overhead of the CT cryptographic proofs.

The technology behind Confidential Transactions
A high level technical primer

This work was originally proposed by Adam Back on Bitcointalk in his 2013 thread "bitcoins with homomorphic value". To build CT I had to implement several new cryptosystems which work in concert, and invented a generalization of ring signatures and several novel optimizations to make the result reasonably efficient.

The basic tool that CT is based on is a Pedersen commitment.

A commitment scheme lets you keep a piece of data secret but commit to it so that you cannot change it later. A simple commitment scheme can be constructed using a cryptographic hash:

  commitment = SHA256( blinding_factor || data )

If you tell someone only the commitment then they cannot determine what data you are committing to (given certain assumptions about the properties of the hash), but you can later reveal both the data and the blinding factor and they can run the hash and verify that the data you committed to matches. The blinding factor is present because without one, someone could try guessing at the data; if your data is small and simple, it might be easy to just guess it and compare the guess to the commitment.

A Pedersen commitment works like the above but with an additional property: commitments can be added, and the sum of a set of commitments is the same as a commitment to the sum of the data (with a blinding key set as the sum of the blinding keys):

  C(BF1, data1) + C(BF2, data2) == C(BF1 + BF2, data1 + data2) C(BF1, data1) - C(BF1, data1) == 0

In other words, the commitment preserves addition and the commutative property applies.

If data_n = {1,1,2} and BF_n = {5,10,15} then:

  C(BF1, data1) + C(BF2, data2) - C(BF3, data3) == 0

and so on.

Our specific Pedersen commitments are constructed using elliptic curve points. [The reader need not understand elliptic curve cryptography, beyond accepting the black box behaviors I describe here.]

Normally an ECC pubkey is created by multiplying a generator for the group (G) with the secret key (x):
  Pub = xG

The result is usually serialized as a 33-byte array.

ECC public keys obey the additively homomorphic property mentioned before:

   Pub1 + Pub2 = (x1 + x2 (mod n))G.

(This fact is used by the BIP32 HD wallet scheme to allow third parties to generate fresh Bitcoin addresses for people.)

The Pedersen commitment is created by picking an additional generator for the group (which we'll call H) such that no one knows the discrete log for H with respect to G (or vice versa), meaning no one knows an x such that xG = H. We can accomplish this by using the cryptographic hash of G to pick H:

    H = to_point(SHA256(ENCODE(G)))

Given our two generators we can build a commitment scheme like this:

   commitment = xG + aH

Here x is our secret blinding factor, and a is the amount that we're committing to.  You can verify just using the commutative property of addition that all the relationships given for an additively homomorphic commitment scheme hold.

The Pedersen commitments are information-theoretically private: for any commitment you see, there exists some blinding factor which would make any amount match the commitment. Even an attacker with infinite computing power could not tell what amount you committed to, if your blinding factor was truly random. They are computationally secure against fake commitment, in that you can't actually compute that arbitrary mapping; if you can it means you can find the discrete log of the generators with respect to each other, which means that the security of the group is compromised.

With this tool in hand we can go and replace the normal 8-byte integer amounts in Bitcoin transactions with 33-byte Pedersen commitments.

If the author of a transaction takes care in picking their blinding factors so that they add up correctly, then the network can still verify the transaction by checking that its commitments add up to zero:

    (In1 + In2 + In3 + plaintext_input_amount*H...) -
     (Out1 + Out2 + Out3 + ... fees*H) == 0.

This requires making the fees in a transaction explicit, but that's generally desirable.

The commitment and its checking are quite simple. Unfortunately, without additional measures this scheme is insecure.

The problem is that the group is cyclic, and addition is mod P (a 256-bit prime number that defines the order of the group). As a result, addition of large values can 'overflow' and behave like negative amounts. This means that a sums-to-zero behavior still holds when some outputs are negative, effectively allowing the creation of 5 coins from nothing:

 (1 + 1) - (-5 + 7) == 0

This would be interpreted as "someone spends two bitcoins, gets a '-5' bitcoin out that they discard out, and a 7 bitcoin output".

In order to prevent this, when there are multiple outputs we must prove that each committed output is within a range which cannot overflow (e.g. [0, 2^64)).

We could just disclose the amounts and blinding factors so that the network could check, but this would lose all of the privacy. So, instead, we need to prove that a committed amount is within the range but reveal nothing else about it: we need an additional cryptosystem to prove the range of a Pedersen commitment. We use a scheme similar to Schoenmakers’ binary decomposition but with many optimizations (including not using binary).

To build this we start with a basic EC signature. If a signature is constructed so that the 'message' is the hash of the pubkey, the signature proves that the signer knew the private key, which is the discrete log of the pubkey with respect to some generator.

For a 'pubkey' like P = xG + aH, no one knows the discrete log of P with respect to G because of the addition of H, because no one knows an x for xG = H----_unless_ a is 0. If a is zero then P = xG and the discrete log is just x; someone could sign for that pubkey.

A pedersen commitment can be proven to be a commitment to a zero by just signing a hash of the commitment with the commitment as the public key. Using the public key in the signature is required to prevent setting the signature to arbitrary values and solving for the commitment. The private key used for the signature is just the blinding factor.

Going further, let’s say I want to prove C is a commitment to 1 without telling you the blinding factor. All you do is compute

   C' = C - 1H

and ask me to provide a signature (with respect to G) with pubkey C'. If I can do that, the C must be a commitment to 1 (or else I've broken the EC discrete log security).

To avoid giving away the amount we need yet another cryptographic construct: a ring signature.  A ring signature is a signature scheme where there are two (or more) pubkeys and the signature proves that the signer knows the discrete log of at least one of the pubkeys.

So with that we can construct a scheme where I prove a commitment that C is either 0 or 1--we call this an "OR proof".

First, I give you C, and you compute C':

    C' = C - 1H

Then I provide a ring signature over {C, C'}.

If C was a commitment to 1 then I do not know its discrete log, but C' becomes a commitment to 0 and I do know its discrete log (just the blinding factor). If C was a commitment to 0 I know its discrete log, and I don't for C'.  If it was a commitment to any other amount, none of the result will be zero and I won't be able to sign.

This works for any pair of numbers, just by suitably pre-processing the amounts that are put into the ring... or even for more than two numbers.

Say I want to prove to you that C is in the range [0, 32). Now that we have an OR proof, imagine I send you a collection of commitments and OR proofs for each of them:

C1 is 0 or 1 C2 is 0 or 2 C3 is 0 or 4 C4 is 0 or 8 C5 is 0 or 16.

If I pick the blinding factors for C1..5 correctly then I can arrange it so that C1 + C2 + C3 + C4 + C5 == C.  Effectively I have built up the number in binary, and a 5-bit number can only be in the range [0,32).

Numerous optimizations are required to make this more efficient:

First, I propose a new ring signature formulation, a Borromean ring signature
Instead of expressing the amount directly, CT amounts are expressed using a decimal floating point where the digits are multiplied by a base 10 exponent.  This means that you can prove large amounts with small proofs, so long as they have few significant digits in base 10: e.g., 11.2345 and .0112345 can have the same size proof, even though one number is a thousand times larger.

There is also a non-private "minimum amount" sent, which allows a smaller proof to cover a larger range if the user doesn't mind leaking some information about the minimum amount (which might already be public for external reasons); this also allows the least significant digits to be non-zero when an exponent is used. Minimum amounts are supported by first subtracting the minimum, then proving that the result is non-negative.

The mantissa of the floating point is encoded using rings of size 4 (base 4) rather than binary, because this minimizes the number of commitments sent while not using any more signature data than base 2.

The final mantissa digit commitment can be skipped, backwards constructing it from the value being proven and the other digits, etc.

Finally, by careful use of derandomized signing in the prover, it's possible for the receiver of the coins--who shares a secret with the sender, due to ECDH key agreement with the receivers pubkey--to 'rewind' the proof and use it to extract a message sent by the sender which is 80% of the size of the proof.  We use this to signal the value and blinding factor to the receiver, but it could also be used to carry things like reference numbers or refund addresses.

The result is that a proof for a 32-bit value is 2564 bytes, and simultaneously may convey 2048 bytes of message. A 32-bit proof can cover a range of 42.94967296 BTC with 1e-8 precision, or 429.4967296 BTC with 1e-7 precision, and so on. My implementation is able to verify over 1300 32-bit range proofs per second on an i7-4770R, and there are many performance optimizations still possible.

The implementation supports proofs of any mantissa size or exponent, with the parameters controlled by the sender. Performance and size are linear in the number of mantissa bits, and odd numbers of bits are supported (by switching to radix-2 for the last digit).

In Elements, the range proofs are only required in cases where there are multiple confidential value outputs (including fees).  Transactions that merge multiple confidential amounts into a single output do not need range proofs since the fact that all the inputs were in range is sufficient.

By sharing the scanning key used to establish the shared secret used by the rewindable range proofs, this approach is completely compatible with watching wallets; users can share these keys with auditors to enable them to view their transaction amounts.

Future work may use the fact that proofs can support a minimum value to also allow skipping the range proofs when there is a single confidential output even when fees are being paid, or allow nodes to skip or delay verifying most range proofs by using fraud proofs.

The system presented here depends on no new fundamental cryptographic assumptions, only the hardness of the discrete log problem in the secp256k1 group and a random oracle assumption, just like the normal signatures in Bitcoin.

While the size of the range proofs are non-trivial, they are still an order of magnitude smaller and faster to verify than some alternatives (like Zerocoin), and most of their space can be reclaimed to communicate additional data between users, a feature which is often requested but hard to justify in a public broadcast network. Similar to signatures, the range proofs can be placed on separate tree branches in blocks to allow clients that don’t care about (e.g. historical ones) to skip receiving them.

Most importantly, this scheme is compatible with pruning and does not make the verification state for Bitcoin grow forever. It is also compatible with CoinJoin and CoinSwap, allowing for transaction graph privacy as well while simultaneously fixing the most severe limitation of these approaches to privacy (that transaction amounts compromise their privacy).

Unlike some other proposals, this system is not just speculation or pure cryptography without integration with the Bitcoin system.

Confidential Transactions is enabled in Elements and used by default by all ordinary transactions.

The source code for the just the underlying cryptosystem is also available.

13  Bitcoin / Development & Technical Discussion / Elements Project testnet sidechain alpha released on: June 09, 2015, 10:16:27 AM
I'm surprised that there isn't a thread on this yet.

I'm pleased to announce the publication of an experimental bitcoin-testnet sidechain and the software that implements it:

There is a fair amount of description and more coming at the Blockstream website:

... Including a video of a talk I prepared for the t the SF Bitcoin 'dev' meetup tonight (the video was pre-recorded, I think the presentation in the meetup went better but I work with what I've got. Smiley )

You can get the code and use the sidechain from the github repo:

The sidechain includes some powerful new features... but beyond that, I hope that this opens up new avenues for experimentation and innovation In Bitcoin, providing a way to try things out without either rebooting the network effect or unduly imposing on people who aren't interested in the benefits (or risks) of something new.
14  Bitcoin / Development & Technical Discussion / [Crypto] Borromean ringsig: Efficiently proving knowledge for monotone functions on: June 02, 2015, 07:07:36 AM
Some here may be interested in a new cryptosystem I've been working on which efficiently and privately proves the knowledge of secrets according to an policy defined by an AND/OR network:

This new ring-signature is asymptotically 2x more efficient than the one used in Monero/Bytecoin: It needs n_pubkeys+1 field elements in the signature instead of 2 * n_pubkeys. In particular, it retains this 2x efficiency gain when doing an AND of many smaller rings, because the +1 term is amortized across all of them.

The paper also describes a new way to think about ring signatures; which might be helpful to anyone who has looked at them before and found them confusing. (If we were successful, you'll think the new construction was so simple that it was completely obvious; I can assure you it was not... fortunately Andrew Poelstra came up with an especially good way to explain it.)

While the connection to Bitcoin may not be immediately obvious, I've used this as a building block in a much larger and more applicable cryptosystem which I'll be publishing, complete with implementation, shortly  (I'm trying to not flood people with too many new ideas built on new ideas all at once, and I'm still working on the description of the other constructions). I think this construction is interesting in its own right, and I'd be happy to learn if someone knows of this being previously published (though I was unable to find anything prior).
15  Bitcoin / Development & Technical Discussion / Bitcoin Core soft-fork proposal for strict DER enforcing on: January 21, 2015, 06:17:56 PM

Greetings, as suggested in the post about the recent OpenSSL incompatibility there is a proposal for strictly enforcing DER encoding for signatures as a block validity rule:

Bitcoin Core has enforced this restriction for relay/mempool/mining since 0.8 in preparation for eventually making a change like this.

I know its kind of a boring change and boring changes tend to not gather a lot of comments; but it's still important so I'd greatly appreciate it if people would review and comment even if the comment is just "I read this, makes sense, no further comments.".  Ideally you'd comment on bitcoin-development directly, but if you comment here instead I can relay views (you take my editorial judgements at your own risk: best to comment directly).
16  Alternate cryptocurrencies / Altcoin Discussion / [VXC] V.Cash (Was: [VNL] Vanillacoin), a quiet word of warning. on: January 11, 2015, 07:04:43 AM
Greeting, this evening john-connor showed up on the Bitcoin Core github with some rather aggressively ignorant minunderstandings of basic cryptographic consensus concepts:

Having no clue who he was I looked at his github account and googled a bit and found that he is the, seemingly pseudonymous, author of "Vanillacoin".

Vanillacoin was previously discussed on this forum, but he locked the threads in order to shuffle the users (victims?) off to someplace out of the light of day-- never a good sign, (nor is his BCT newbie account, for that matter).  The "vanillacoin" software has no source code available, it is binaries only (very much not a good sign, and usually severe malware concern; and an ultimate form of centralization), there are source links but they go to a basically empty github repository. There is a whitepaper, which like the comments on github show some general software development background they show no real sign of sophisticated understanding around decenteralized systems for adversarial networks or cryptocurrencies.

I don't know anything more about it, but I figure sunlight tends to be a good disinfectant; and with the threads locked it probably wasn't fair of me to say nothing while I was privately thinking "hm, that all smells pretty fishy".  Of course, the guy was a bit rude to me and also wasted my time-- so feel free to factor that bias in however you like. I'm just reporting my impression as a regular community member. You now know what I know.

[I'm the last person to play altcoin-cops... I mostly avoid this stuff except for the rare cases that are technically interesting: The drama can sink unbounded time and usually, when it comes to the more misguided altcoin cryptography, the only sane policy seems to be "If you see something,say nothing and drink to forget": there is too much crazyness and risk of being attacked for being critical of someones latest scheme. But if it shows up in my face, I can't quite stomach saying nothing at all.]


[Edit: Vanillacoin changed names to V.Cash]
17  Other / Off-topic / in which anonymint can't understand winternitz on: July 04, 2014, 02:33:39 PM
@gmaxwell, in any case I think you'd have to admit that one-time use Lamport signatures employing a cryptographic hash
I like lamport a lot— and first proposed we someday add some form of it to Bitcoin back in 2011... though straight up lamport's single use-ness is a major liability (e.g. say you need to resign an input to add fees later, maybe a couple times… oops, compromised your key).

The space overhead is really considerable— even when using Winternitz compression, to the point that it couldn't be a replacement for bread and butter transactions, but I think it would be nice to support.
18  Bitcoin / Hardware / Cointerra power stepping vs wattage (vs hashrate) on: April 05, 2014, 03:03:09 AM
Has anyone charted out the cointerra power stepping vs wattage (vs hashrate too, perhaps)?

I am precariously close to over-provisioning some breakers and short on power meters. It would be handy if someone had charted this stuff out so I can set it in a way thats more reliable than trial and error. Smiley
19  Economy / Computer hardware / WTS CoinTerra IV "2/TH" (more like 1.5-1.8TH/s) miner in Silicon valley/Bay area on: April 02, 2014, 10:31:03 PM
Greetings. I had a much delayed CoinTerra IV show up without prior notice... and while I'd love to add it to my collection— the CT units are awesome on P2Pool— I'm currently out of room.  The unit is new-in-box and located in Mountain View, California.

I'm looking to sell it immediately for USD (in person) or BTC.

It looks like these units-in-hand go for about $8k on ebay, but I'm willing to negotiate. An ideal buyer would be willing to pick it up tomorrow, in person, in mountain view and pay cash, basically minimizing my time in fussing with it.

PM on here or reach me at nick gmaxwell on freenode, or via email at greg<at>

(Also— if any in-person buyers is interested in some historic early batch 1 avalons that mine at about 80GH/s, feel free to inquire. Smiley )
20  Bitcoin / Development & Technical Discussion / Dual SNARK: How to make ZKP with trusted initilization trustless in some cases. on: March 15, 2014, 12:01:02 PM
Some of the most efficient constructions available now for proving arbitrary programs in zero-knowledge are only secure in the CRS model— in English: They require a trusted initialization step where someone has to generate some magical numbers and if they cheat (e.g. by keeping the initial randomness) they can produce false proofs.

Systems with these properties are found in some of the papers at and in, both of which are currently using systems based on the GGPR'12 cryptosystem. They have really awesome performance, though— fast enough to actually be feasible to use on the prover, and almost as verification in a couple milliseconds with a few hundred bytes for the proof regardless of the size of the program being proven or how long it takes to run. But the requirement of a trusted initialization is unacceptable for some applications.  Ultimately we'll want systems which do not depend on the trusted initialization but they're less far along in development and may never be quite as efficient.

Some things we'd like to use these proofs systems for is a more powerful and efficient replacement for Bitcoin script. Instead of every node verifying your fancy script, instead the party creating the transaction runs the script themselves and provides the network with an efficient proof that you ran it faithfully and that it accepted. This avoids network load as being a disincentive to using fancy scripts and also has privacy advantages.

For example, say I want to pay you conditional on you publishing a solution to a— say— Sudoku puzzle. A script running under an efficient proof of knowledge system could verify an encrypted solution, and because it's zero-knowledge miners couldn't come along and replace the outputs like in a plain hash-locked transaction. You could use a CRS snark here where the payer computes the trusted initialization and then the payee cannot cheat— but the fact that the payer initialized it means the payer could double-spend race the redemption and get both the solution and their coin back.

There are a couple of ways of solving this, but I wanted to mention another one which is especially general:

In a two-party trade, just have both parties compute their own trusted initiations, then require the script provide proofs under each of them. Then so long as one side isn't cheating the transaction will behave faithfully.

This can be further optimized by instead requiring Person_A_snark&&Person_B_ECDSA_sig || Person_B_snark&&Person_A_ECDSA_sig... Cross signing using ECDSA is more efficient since the snark proofs are larger (and much slower to compute) than ECDSA signature, and this equally achieves the goal of not allowing either party to take advantage of a trapdoor.  (in particular, if you use hash compression to avoid ever even bothering to publish the verifier public keys for the untaken branches).

The same approach can be extended to more than two parties though the efficiency becomes less impressive (e.g. the prover must run the proof N-1 times to handle any party possibly conspiring with them, or N/2 if you only want a honest majority security) and sadly, it doesn't really work for broadcast payment sorts of application.
Pages: [1] 2 3 4 5 »
Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!