Thanks for your reply.

Absolutely, though its worth noting that while canonical encodings like DER are known and specified, virtually nothing enforces them. Bitcoin, from day one, has used a canonical encoding (DER)-- unfortunately, the underlying implementation in OpenSSL had undocumented behavior where it also accepted invalid DER until a couple months ago. (likewise, (all?) other similar libraries, such as bouncy castle behaved likewise-- this wouldn't have been escaped by not using OpenSSL). I spent some amount of time looking for precise DER implementations and BER implementations for reference test points in libsecp256k1 and was unable to find _any_ open source software which precisely implemented (no more accepted or rejected than specified) them. Even OpenSSL's recent fix for their behavior was not to correct their parser but instead to attempt to round-trip the encoding, which is not a guarantee. (You can see this list for some of the crazy things the OpenSSL parser, as an example, would accept.).

(And FWIW, the whole Bitcoin network has filtered these encodings everywhere for several years,  even though it required quite some wallets (like Armory and MTGOX) change their behavior).

But that deals with serialization, I am aware of no evidence that anyone has observed the algebraic malleability of ECDSA prior to us, and no evidence that anyone else took any action against it (now or previously).

As far as leaving serialization elements out of protocols, great care must be taken there too as there have been many vulnerabilities created by leaving things out. For example, the original PGP fingerprints hashed the modulus and exponent (but not their potentially non-canonical serialization), but also as a result had no delimitation so someone could move some bytes of your modulus into the exponent until they found a key they could factor, and then they'd have a weak key with the same fingerprint as you.

Signatures being covered under identifying hashes is ubiquitous, e.g. x509 certificate chains work just like bitcoin in this respect; and this interacts poorly with cert blacklisting due to malleability.

In any case, I don't want to belabor this, but you were really harsh and critical where I think the reality is almost 180 degrees out:  Bitcoin suffered from undocumented behavior in third party code; the outside world while somewhat aware of problems from non-canonicality has largely not acted on it-- Bitcoin Core is a leader in responsive and responsible handling of this, and our efforts resulted in the discovery of an additional form which appears to have not been known/discussed previously), which our ecosystem (out of all the ECDSA users in the world) appears to be the only ones at all protected against.

I really don't care much what armchair jockies on the forums (not specifically referring to you) think, and so I don't usually expend a lot of resources bragging... but to have someone be really harsh on a matter where I think our response has shown considerable innovation and leadership is, at least a little, bit frustrating!

It is unclear what you mean by "would not be significant", but the matter is largely orthogonal-- 6979 or not, implementations that do not heed the lowS behavior specifically will produce non-lowS signatures at a half the time per signature, leading to most of their transactions being blocked.  No wallet that I'm aware of has ever provided a "resign" button that might break through-- and when wallets do create new transactions they usually use a new random selection of coins, so 6979 would also produce a new signature. Even if they did provide a "resign" and used a random signature, once your transaction has more than a couple inputs the probability that it would pass a lowS test becomes very very low.

RFC6979 does not really close a covert channel either, alas, it is impossible to determine if a device is using 6979 without access to its keys and a malicious device could use 6979 99.99% of the time but then sometimes change to a kleptographic method (perhaps triggered by the message itself).   The RFC does eliminate the strong need for a good RNG at signing time --  a common implementation problem-- and make review/auditing somewhat easier.


Unfortunately if the device has free choice of its nonce (which having control of low/high S implies) then it has a high bandwidth covert channel.  E.g. produce the nonce as H(ECDH(attacker_pubkey, pubkey) || message_to_be_signed). More elaborate versions of this allow the embedding of additional data, beyond just leaking the current private key. Even if something constrains the choice of nonce, a malicious device can do rejection sampling to turn get N bits of covert channel out of the nonce by doing 2^n computation-- low vs high isn't different except by being computationally cheaper to use.   So, I'm not sure I'll be able to sell it in terms of covert channel suppression, but it's worth a try. Thanks for the suggestion.

Warning about it, at least, would be productive-- as there has been a fair amount of loss from people accidentally using the wrong address (in addition to the other ways reuse causes systemic harm).  If you mean "already used" ever, then prohibiting it requires a forever growing database, which isn't all that scalable.

A middle ground would be warning (or refusing) any that the site itself had previously sent to; and I think there was general agreement to do that kind of warn on (wallet local) reuse in bitcoin core in the past.

Don't look at me: I had no idea bitcoin.org mangled the release notes. Thanks. No one working on Bitcoin core has any control over bitcoin.org, but I'll report it now that you've brought it to my attention.

I didn't contact you, you started this thread with a bunch of accusations and ranting, because -- apparently-- you weren't willing to "pay the price".

And no, just because there is a fine on something doesn't make it appropriate to do just because you're willing to pay the fine.

Please do not address anyone on this subforum in this manner. None of the developers of Bitcoin software owe you anything, and this sort of disrespect is most unwelcome.

It's not users vs developers-- it's users vs users that are in question here.

In the case of the attack you cannot respend unconfirmed coins without issues while an attack is going on, in the case of lowS enforcement with half of your transactions will randomly get stuck (and never get unstuck, holding your coins hostage). Both are bad experiences for the user. I think the latter is considerably worse: any system's abstractions leak from time to time. Bitcoin isn't a system of "account balances", and for good reason. 99% of the time wallets can simulate that it is, but sometimes the fact that it isn't matters more (and even in ordinary use, the fact that it isn't has significant impacts on fees that can be worth the user's attention).

In the case of the former, it has been a non-issue until recently because no attack of this specific type was going on-- though, of course, we knew one could begin at any time (and immediately deployed all non-disruptive fixes);  in the case of the latter software was chaining over time-- but some users will just find that their preferred software just never gets updated for it; good luck sweeping their funds into another wallet with half (or more) the transactions getting stuck. Some users will try to install fixes but end up with malware instead and have their coins stolen, etc. While there was no attacks waiting for the migration to conforming to happen naturally was an approach which was intended to mitigate total disruption (and I believe it did)-- absent amaclin's demonstration it might have gone with virtually no disruption at all.

At every moment in this process we were thinking about protecting the user's experience and the value of the Bitcoin system to them-- from anticipating an issue no one had thought of before, to implementing, testing, reviewing, deploying fix code-- nagging other developers to fix their software, etc. All of these things took considerable work.  Simply saying "ah ha, today, this new rule is enforced due to this currently hypothetical attack: deal with it!" would have been much simpler.   Sometimes there is just a genuine conflict between different harms.

What you are complaining about _is_ an orderly phase in, and is why only ~5% of transactions were affected instead of most of them.

I'm still waiting on a hyperlink to a _single_ ECDSA implementation that dealt with ecdsa inherent malleability prior to Bitcoin's. You argue that it was a standard problem with a standard solution, if so it cannot be too much effort to find an example. [I'm not just asking for didactic purposes, I'm concerned about a HSM vendor that is uninterested in changing their behavior, and if a non-bitcoin example exists it would be helpful (but I could not find one).]

Thanks! it's good to hear comments like this.

Data storage has a cost. The field you are putting bitcoin unrelated data is a public key field, it isn't Hyena's-file-sharing-field. It's used to set the rules for coins to be spent in the future. When you start cutting me a paycheck you can start dictating what my job is, but under no condition will it ever be to aid you in your quest in externalizing your data storage costs onto Bitcoin users.


Perhaps you should get control of your emotions before posting? It might help prevent you from making errors,  I think release notes are clear on this, and it was also covered in the release candidate release notes.

For the avoidance of any residual confusion, let me quote from them here:


Notable changes
===============
[...]
Minimum relay fee default increase
------------------------------------

The default for the `-minrelaytxfee` setting has been increased from `0.00001`
to `0.00005`.

This is necessitated by the current transaction flooding, causing
outrageous memory usage on nodes due to the mempool ballooning. This is a
temporary measure, bridging the time until a dynamic method for determining
this fee is merged (which will be in 0.12).

(see https://github.com/bitcoin/bitcoin/pull/6793, as well as the 0.11
release notes, in which this value was suggested)


Not only was it covered in the 0.11.1 and 0.10.3 release notes, but the 0.11.0 notes recommend users make the same change for themselves manually.

I await your apology with abated breath.

invective (noun)
1. insulting, abusive, or highly critical language

You may be suffering confusion on this point, the dust limits are a fix and don't require having any debate with you (reasonable or otherwise); economic thresholds that discourage the creation of uneconomic to spend (or unspendable) txouts-- and were it not, you would have no cause to complain. And indeed, the protection it provides is not absolute, but it doesn't need to be.  If you send hazardous materials through the post you might get caught, or you might not. But the odds are enough to provide all that is needed to keep you from harming the system. You will not be the first person to intentionally harm a shared resource to brag about how bold and brave you are for revealing issues, it's nonsense-- but also irrelevant, nodes can happily go on blocking your transactions regardless.

I suggest you take your "weakness demonstrations" to the local law enforcement office and helpfully show them how breakable their windows are... Be sure to tell them how grateful they should be for you highlighting their vulnerability. I think they are likely much more equipt to provide you with the education most suited to your current needs.

I invite you to provide us with a hyperlink to a _single_ ECDSA implementation that dealt with ecdsa inherent malleability prior to Bitcoins existance (or our implementation of the fix in Bitcoin Core).
And fixed in Bitcoin Core all this time too!


LOL.  I can assure you, there is no fear involved.

The only effect of this kind of third party malleability has on ordinary Bitcoin payment activity with competently written wallet software is a fairly mild denial of service attack (blocking payments using unconfirmed coins) and mild cosmetic issues; irritating, no doubt, but not more than that. There exists (or used to) less well written wallet software that misbehaves more severely, but there is no rescuing of that-- and it's up to each user what wallet software they run. Because of prior malleability attacks via closed off malleability vectors we're fairly experienced with what the effects of malleability are, to the extent that there has even been an academic paper written on the subject.

So-- to set the stage, we have a fix to a denial of service attack (S-flip ECDSA malleability) which was discovered by Bitcoin Core contributors and had never previously occurred on the network at any scale, but where deploying the fix would deny service to all users until they changed software. Deploying the fix, which was implemented, tested, and included in Bitcoin core for a long time requires only miners changing their software to enforce it and presumably could be done in a couple hours if there were a need. (and whom could have done so at _any_ time, regardless of what Bitcoin Core did-- even against our advice, ... and with trivial effort, since we'd already written it and integrated it but left it off.)

Bitcoin Core changed behavior to the fix compatible form and encouraged others to do so, it also implemented and tested the filtering but left it off so as to not deny service the vast majority of users. Through continual efforts to encourage parties to upgrade we reduced impacted transactions that down to around 5% before anyone performed the particular attack... and at that point we activated the fix in Bitcoin Core on the basis that 5% disrupted completely is better than 100% bothered whenever the attacker feels like it.  

At the same time, virtually every alt-coin; most developed completely after this issue was known, including many that have no relation to the Bitcoin codebase (and even at least one that specifically advertised itself as being malleability immune!)-- remain vulnerable.  

Meanwhile, what have you done to aid any of this?  As far as I can tell absolutely nothing.  You complain about leadership-- I think that's nonsense and that we've acted prudently, professionally, and intentionally... but, ignoring this disagreement, if you think it's more "leadership" is needed, why are you exhibiting a total failure to provide it?  Is it that you are "frozen with fear", enh?  

Please, if you're going to continue with this style of barely informed insults and personal attacks, I'd prefer you remove yourself from this sub-forum.   Or better, why not save some of your attacks for those people throwing similar insults at Bitcoin Core, calling it reckless for actually moving forward with the fix at this time?  One of things I find most fascinating about human behavior is that no matter how diametrically opposed people's views are their ability to unify behind being shitty towards people who _give_ them things for free seems to known no bound.


In one breath you say "mono culture is ecologically untenable", but yet you attack a strategy that is specifically constructed to be inclusive and avoid unnecessarily driving out implementations with fewer resources behind them out of the ecosystem.  You recognize that technically anyone could have fixed it, yet ignore your own completely failure to do so (or, seemingly, anything else productive in this context)-- in our case, we had a specific intention to preserve diversity in the ecosystem by not forcing updates on other implementations and their users, but you have no such excuse, arguing that it should have been done years ago. Remarkably, I find the seeming lack of logical consistency in your arguments more offensive your incivility.

Step 1. Monopolize tokens
Step 2. Totally control system.
Step 3. Profit.

The reason Bitcoin is decentralized is not insignificantly because anyone can enter or leave at any time without permission.  You can close off attacks by requiring a permission system (even soft ones like modulating the difficulty) for mining of some kind or another, but you open up other ones.

And as you note, what you suggest would kill hashrate-- you don't describe a personal incentive to secure the system but rather an incentive to hope other people do while you sit back and enjoy the benefits. ... and if they don't? oh well-- you participating wouldn't make much difference, and it would just be more cost to you.

If you are not using the Bitcoin currency except incidentally but instead abusing the system to store data there are no promises that you're going to have a good time. It's not in the users of the system's general interest to accept an eternal data storage cost on your behalf.

As mentioned above, there is no explicit "dust threshold"-- it's determined by the minimum relay fee rate.  In particular, the number "540" you were expecting was certainly never announced;-- and the change to the relay fee was absolutely release noted; and important for keeping nodes from blowing up in the short term-- yes, a somewhat lower value would have also worked but the relay fees were out of wack (adjusted predicatively down when the prices was at almost three times the current and going up) and the minimum bump might have been met immediately by the attackers (who can update their behavior much faster than people can upgrade software).

Above you write that you are creating unspendable outputs-- this is _specifically_ one abuse of the system dust threshold was intended to discourage; so it sounds like it's working as designed now.

But by all means, feel free to not upgrade your software or change its behavior... it's not the action of your own software that will inhibit you-- all it does it tell you quickly and cleanly so your transactions don't get stuck, it's the action of everyone else that inhibits the abuse. And it is quite effective, or else I suspect we wouldn't be enduring your vicious invective.

If someone is looking for a safe dust approximation without any relayfee intelligence, the original level of 5460 base units is probably a better expectation.

1. Plagiarize the work, shared freely, of Adam Back, Shen, Denis, myself, and others (and in my case even implemented in a high performance implementation).

2. Ask for twenty grand in donations.

3. Profit.

Here's a hint to someone who might think of funding this stuff, "just in case":   Non-contributiors being paid for what is primarily your work is incredibly demoralizing-- doubly so when they don't even add anything to it (not even a good implementation); if you want to kill science and engineering in this space go ahead and fund more vaporware scams.

Science needs to happen in the open. I'm also very supportive of people being paid for their work, but they need to actually do work, not just sell snakeoil to others.  This community often does a much better job at funding scams than people who reliably contribute.

I don't agree, not even in the most lax of terms.

It's a standardizes rule,  and already on the order of 95% of transactions were already conforming.  If a wallet produces a non-conforming transaction anyone in the world can mutate it to the conforming form.  Even absent the auto-mutation, when it doesn't work-- it fails safe: you can still receive transactions, but your sends may not work; and you can either update your software or find someone/something to mutate them for you. It's likely that due to 'helpful' mutation and non-upgraded nodes and miners the remainder will continue to get confirmed (with delays) for some time... and that 5% remaining should drop as electrum and armory get updates out.

I would have preferred to continue to wait to activate the filtering, which has been implemented and waiting in Bitcoin core for years now, until even more users were upgraded... but the ongoing attacks made that a poor trade-off:  There is no reason that the creators of 95% of the transactions should suffer attacks because we were worried about inconveniencing the remaining 5%.   I also called in this thread multiple times for help driving that 5% down to nothing and no one else cares, so-- how much can the negative impact matter when basically none of all the noisy people in this thread cared to lift a finger to help mitigate it?

All the other avenues for "nuisance mutability" that we're aware of were closed long ago, some of those also required getting wallets to update, but far fewer wallets were broken with respect to those rules. Fortunately, until recently, no one who wanted to attack had bothered figuring out how to perform this particular attack (as it's ever so slightly tricker-- can't just be done by stuffing an extra byte in a transaction); unfortunately that didn't last, and so we got to lose huge amounts of effort dealing with this and creating a small amount of additional collateral disruption instead of working on other things; the result will likely also delay the deployment of CLTV some due to difficult in getting miners to update twice in rapid succession... but thats life.

I suggest reading this thread.

Seriously. such an embarrassment.  So much grunting and flinging of poo and people don't bother to read messages of actual substance.

You can operate a full node* with pruning, what you're going on to describe there is UTXO commitments which are totally unneeded for pruning and are of unclear cost, unclear value, and unclear implications.


Unclear cost: rehashing the utxo set for every block, even if maintained in a high overhead efficient data structure is a multiplicative IO cost on running a full node.  Making the initial cost cheaper is essential, but at what runtime penalty is it acceptable?  is 20x acceptable?  If the utxo set is constantly changing, how do people synchronize it from you?  do you have to hold old versions? does a peer have to fetch it only from a single other peer?

Unclear value:  starting off a utxo commitment means you have only SPV security. If you were happy with SPV security; why not run SPV in the first place and dispense with the intermediate step. The answer is that it's only SPV of the history, but where is the dividing line? When someone today says how do you know that the system's creator didn't secretly mint himself a trillion coins-- the answer is because your own node verified it wasn't so.

Unclear implications: with no incentive to not run a committed bootstrapped node, is there any reason to believe the old history would even be _available_ to someone who wanted to do a full security initialization?

(*And in Git master Bitcoin Core relays blocks and transactions, and gives full wallet functionality, minus rescan when pruned.)

AFAIK franky1 has nothing to do with the development of Bitcoin Core (yet).  We'd love to have more features (and actually there are many very advanced features that are not available most other places... there just are not as many "middle tier" features), but people need to step up and contribute. Most of us are completely saturated just keeping the with the increasing demands on the network.


You can use Bitcoin Core offline-- I do and have for years. It's just not super user friendly to do that.

Because s_nakamoto@1a98c847-1fd6-4fd8-948a-caf3550aa51b  was never an account on github anyone could add that as one of their email address to claim these commits-- just stupidity on github's part.

This test is not quite precise, so implementers of bitcoin software shouldn't implement the test this way. The threshold is not a power of two, so no tidy hex digit sniffing can be precise.

x is private by virtue of being the conveyed by an ECDH key negotiation. No external communication is required.  (E.g. go build elements alpha, and give me an address from it and I'll send you some coins!).

They're required for outputs only, and technically only when there are multiple outputs. Inputs are already known to be in range by virtue of having been created as outputs.

I declare all software that doesn't transfer all your coins to me out of bounds! I declare all software that doesn't assign me a trillion bitcoins out of bounds! Hurrah! This is fun.

... Come on.  Going and dictating to people what software they could run would not be well aligned with the principals that make Bitcoin (or the internet) interesting in the first place..., particular because taking that approach to it's logical conclusion we'd rapidly reach a state where Bitcoin Core was the only or nearly only software people could use with the network (as it has an order of magnitude more development effort going into it).

I have considerable experience in the IETF, and I have to say your comment drew a bit of a chuckle from me... The IETF has no authority to control what software or behavior other parties have. You're drawing exactly the wrong comparison there.

In Bitcoin Core we discovered, designed solutions, implemented them, adjusted the behavior of our own software, specified the needed behavior, and recommended these changes to others. We've even gone out and reviewed, audited, advised, and contributed patches to other implementations. I'd say we'd done everything short of written invitations, but if you count emails-- I've sent many an invitation (many of which have been warmly received and resulted in improvements in wallet implementations).

And basically all of this was done by people who weren't being paid a cent for it, by you or anyone else-- I might add. And when there was a simple, not very technical, part of the effort that I thought wider help would be important for, I plead for it here-- to seemingly no effect. Apparently its not as big a deal as some are making it out to be.

... so in the meantime, while all of that was going on ... precisely what have you done except insult the few people who've done anything productive at all about this subject?

Kind of you to confirm that you're responding in this thread without having read it (or my response to you in the other thread on the same subject.)

Specific forms of malleability are useful and intentional design features of the protocol. Additional forms allowing unintended malleability arose primarily out of undocumented behavior in underlying cryptographic libraries which have also resulted in CVEed vulnerabilities in unrelated applications-- indeed those later kind were a design error-- but most of them (save the one whos fix was inhibited by wallet behavior were largely closed off ages ago). Meanwhile, in spite of these issues having been well understood for years virtually every altcoin is also vulnerable to them including many 'from scratch ones' and ones created years after the issues were known in Bitcoin and fixed in Bitcoin Core (and even, in at least one case-- one that specifically promotes itself as being 'immune').  While it is unfortunate the the system was created with merely superhuman foresight and not perfect foresight, if you're looking to complain about _incompetent_ you need to go look to those other places, not Bitcoin Core which immediately implemented solutions for every known unintended malleability vector once they were first highlighted as an issue and discovered several new ones along the way.

The network itself is working fine and is completely uninterrupted by this stuff, whats impacted are some wallets in some applications. See also http://fc15.ifca.ai/preproceedings/bitcoin/paper_9.pdf

For example, anyonecanpay sighash flag allows arbitrary parties to add funds to a transaction. It's what makes lighthouse possible, but every time someone updates the transaction the txid changes.

Because they are widely used (or had been historically-- we're getting to the point where .this is less true). Just blocking the transactions for non-trivial amounts of users does not yield a good experience, to say the least. Forcing people to constantly rev their software reduces decentralization-- and who precisely has the authority to go decide what is "old" or "actively maintained"?--if people are happy happy with what they're running, I am not eager to disrupt that.  I am also not eager to try to dictate how often authors of wallet software must revise their software (again, something that would reduce decentralization-- by pushing out development teams with less resources).  As to why you should care and go help move them along: it's cheap to do, and the failure to do so holds the ecosystem back.--- the same reason I've done so.

If you note the patch I linked to, my change was only a few characters--- why? because the code was already written a long time ago... but not activated due to waiting for the ecosystem to catch up; we're ready.

There are a dozen different malleability vectors in the protocol as originally designed; some are quite useful and important intentional features-- others are not.  Though the harm from malleability is very moderate-- and because of the intentional features and the potential for ordinary double spends, wallets must have basically sane handling for it--, unwanted third party malleability is a nuisance. In Bitcoin Core's wallet the nuisance can be greatly mitigated by setting spendzeroconfchange=0.

Because of it being a nuisance all of vectors for malleability except for one were blocked as non-standard transactions in Bitcoin Core years ago.  The remaining one could not be simply blocked because it requires transactions to confine their signatures to a particular form-- low-S-- and all software was violating before the issue was known.  Because of this applying that final constraint would have blocked almost all transactions on the network-- something not justified for a nuisance level attack. Bitcoin Core changed constrain its own transactions to this form in 2013 but it has taken a long time for other software to update themselves. Fortunately, the final remaining type of malleability was ever so slightly trickier to exploit, so people haven't been doing so at scale. In the meantime a proposal was made, as part of BIP62, for a v3 transaction type where parties creating transactions could opt into the protective behavior if they were recent enough to support it. Unfortunately BIP62 is fairly complex and no one outside of a small group of contributors to Bitcoin Core have cared at all about advancing it.  So we've been breaking up parts of them and applying them to the consensus incrementally (e.g. BIP66).

Current git master Bitcoin Core enforces the requirement for all transactions it relays or mines, once this is in a release and widely deployed it will end this irritation; but it will also block most transactions from small portion of the network on software which is out of date or hasn't been updated to produces anti-malleability-friendly low-S signatures (on the order of 5% of all transactions now; due to ongoing efforts to harass parties to fix their wallet software).

I've called for assistance several times in identifying the origin of a list of lowS violating transactions in order to help speed deployment of this, but it seems that the Bitcoin community is a lot more interested in whining and throwing blame then stepping up and doing a little bit of the non-development work needed to get this deployed.

It is saying that if this case weren't rejected then CHECKLOCKTIMEVERIFY could be bypassed by the signer, via making all their txins final.  To prevent this, _this_ txin is required to be non-final.  From a modularity/clarity perspective it's preferable if a script has only local effects (only looks at it's own input) unless more is needed.