Soft-forks shouldn't be and aren't tied to Bitcoin core major versions.  There will be 0.11.2 and 0.10. releases out with this in the next week or so.

If you run git master (to be 0.12 later) instead of 0.11 you will also relay new blocks as they are found while pruning.

Doing that invalidates the security provided by running it in the first place. (also I make no promises that someone can't elevate giving you database files into full remote code execution).

If you're willing to trust a source of the database why not just dump out the utxo database and just give people a feed of it?

So import a million, performance is acceptable with very large numbers of keys.

This sounds like you're asking about something for yourself to run.

Mostly as a precaution as it hadn't been adequately tested.  It's enabled in master now and has been for a long time; considering the crazy things you were considering (concurrent access to the wallet database?!?) running master should be a no brainer.

If it would be useful to you, supporting rescan that goes back as far as the non-pruned data shouldn't be a big deal-- mostly an interface question of changing the rescan argument to take a depth and permit the import w/ rescan when the depth is compatible with the current level of pruning.  For what I think you're trying to do you really only need to rescan back a block or two, no?

Great, may be a bug in salvagwallet where it's willing to add empty keys. I'll look into that...  potentially a result of the same reading code that accepts them at runtime.

Will do.  I'll let you know shortly. (Sent PM with a link to a binary)

Correct, in versions before 0.10.0 (I think) we only checked the first key on decrypt. Later versions check all of them once per execution, as a result of PR#4670.

Right your wallet has an encrypted key object in it with an empty pubkey.  It's not clear how this could have happened.  Has the wallet ever become corrupted, have you ever used any special wallet utilities on it?

In any case, the entry is meaningless and should be ignored.


Right. The code path for non-encrypted keys checks the pubkey already; so I couldn't see how this was possible unless it was encrypted.

Can you try https://github.com/bitcoin/bitcoin/pull/6906 and see if it rejects this? If you're not compiling it for yourself, please tell me what OS you're running and I'll get a test binary for you.
 
Right, when it decrypts it checks that the private key and public keys agree, finds that they don't and then rejects the corrupted wallet.

Is this wallet encrypted?

Would you be willing and/or able to test a patch?

Services providing services to other users should almost certainly not be using SPV.

Canonicalization is not sufficient to eliminate malleability; and some forms of malleability are very useful. intentional, features-- like the ability to construct anyone-can-pay transactions.

When all the actors are known the problem you're trying to solve is called "distributed consensus" and there are many fine algorithms for it.

The mechanism would be that you've transported it over a secure transport in the first place, e.g. HTTPS or encrypted email. No different than a Bitcoin address or plain payment URI.

I wasn't even speaking about bitcoin. I was speaking about traditional toxic waste and your yard.   Uranium hexafluoride in alumnium cans, if we must name something specific... My point being that your argument was generic, "Using Nicolas Dorier's lawn for other purposes is not abuse if you are willing to pay the price".  

In any case, going back to Bitcoin, no one is obligated to relay or mine any particular transaction; and might not do so if they think they're harmful by whatever criteria they're using. One need not define anything.

But to your question of "arbitrary", do you intend to insult me?   I do not think it is arbitrary to say that Bitcoin, which was created, operated, maintained, and adopted with the express stated purpose of being an P2Pecash system ... has an intended purpose of being a P2P ecash system, and that use of that that is using the system outside of its stated purpose. By comparison there are alternative networks based on the Bitcoin code base which have stated purposes which explicitly include other things; and, as a point of history, when someone first proposed storing name registrations in the Bitcoin system the system's creator vigorously opposed this and recommended doing so in an alternative network.

Uses which are not related to Bitcoin ecash and instead just ride atop it as a communication/storage channel (or worse, try to explicitly compete with the Bitcoin currency and replace it in the market while using Bitcoin's own network) are a pretty good start.   Case in point, we're in a thread where someone was upset because nodes were blocking transactions of theirs which were creating purposefully unspendable txouts (e.g. adding non-Bitcoin bloat to the UTXO set) in order to run a commercial service called "crypto graffiti" which does what it says on the tin. I think that this is indisputable abusive: It benefits nothing but it's operator/users to the cost of every current and future user of Bitcoin, it uses the system not as an ecash system (which is what virtually all of its users signed up for), but as a messaging and free perpetual data storage layer,  this (data storage) usage potentially risks subjecting node operators to nuisances (e.g. DMCA notices), and even the name acknowledges its nature. ... but on the scale of abusive things it's not very interesting: something can be abusive without being worth worrying about, as the overall behavior of the system already confines the amount of damage to uninteresting levels.  It's only worth mentioning that it's abusive by way of explanation as to why I do not expect this kind of use to be reliable.

So, if I start accepting fees for it... people can leave toxic waste in your yard?  After all... they paid a fee.

Things are not so simple as you make them out to be. One user (drawn from a tiny subset of mostly a dozen people, currently) ... collects a fee then everyone else for all time in the future takes a bandwidth and storage cost.

Fees are a useful mechanism and, at times, an effective tool, but they do not themselves create moral authority or guarantee system survival because they do not and cannot pay the enormous externalities that dominate the system costs and they do not reflect a consensual interaction with all people they might impact.

It appears (from the writeup) that you are unaware of sybil attacks?

The graph of transactions in bitcoin already functions like what you describe (except 'accounts' are single use txouts); Bram Cohen likes to call Bitcoin without the blockchain "bitpeso".  In Bitcoin the blockchain is overlaid on top of "bitpeso" to resolve "complex forks" (using your language) in a manner that allows someone to join the network and know which resolution is authoritative in a way which is both robust to sybil attacks and does not require membership (because a membership process would create control over the system).

In your description you appears to liberally conflate forks and double-spending.  In Bitcoin, double spending a traditional txout requires malicious behavior, just as you describe.  Blockchain forking in the absence of double spending is benign and no different to the "multiple resolution rounds" you mention from your resolution protocol but fail to describe detail sufficient to permit any analysis.

I invented it in 2011: https://bitcointalk.org/index.php?topic=19137.msg239768#msg239768  as a response to people reusing addresses because they didn't want to have private keys online. It was later standardized as BIP32.

You may not currently care, and its your right to screw over your future self (which you may well be doing)-- but reuse also harms _other_ users in Bitcoin, who do care about privacy, and this degrades the fungibility of coins; since coins which are linked to this or that party are clearly different, and differently valuable than coins which are linked to other parties or are not linked.

Pedantically, the reuse of address harms third parties... and the decision to send coins is the sole business of the sender. I don't think you can answer this questions with simplistic reductions to arguments about free choice: there are multiple people involved in the question, and their free choices may conflict.

New addresses can be generated without generating new private keys.

Monero, for example, has no address reuse at all in the blockchain-- it's required for the prevention of double spending there. It seems to do okay with it.  The original bitcoin software never addresses w/ pay-to-ip; and even with addresses in use it the practice of reusing is somewhat inexplicable from a technology standpoint: it _really_ screws up your privacy along with that of people you transact with, and you can't reliably tell which of the payments you had outstanding were confirmed.... I think if it had been realized that people would behave the way they do, it likely would have been prohibited in the Bitcoin system from the start.

[I make these points as points of correctness, not to further argue it-- I think warning is more prudent, and will also have the effect of educating on this matter).

It can be done using OpenSSL,

The code from Bitcoin core for this-- back when it used OpenSSL for signing:

        BN_CTX *ctx = BN_CTX_new();
        BN_CTX_start(ctx);
        const EC_GROUP *group = EC_KEY_get0_group(pkey);
        BIGNUM *order = BN_CTX_get(ctx);
        BIGNUM *halforder = BN_CTX_get(ctx);
        EC_GROUP_get_order(group, order, ctx);
        BN_rshift1(halforder, order);
        if (BN_cmp(sig->s, halforder) > 0) {
            // enforce low S values, by negating the value (modulo the order) if above order/2.
             BN_sub(sig->s, order, sig->s);
         }
        BN_CTX_end(ctx);
        BN_CTX_free(ctx);


(You might also want to look to using libsecp256k1 for signing in the future, not only does it handle this for you, it is sidechannel attack resistant and OpenSSL is not for this curve, it's also likely much better tested code than OpenSSL for this particular application.)

Thanks for your reply.

Absolutely, though its worth noting that while canonical encodings like DER are known and specified, virtually nothing enforces them. Bitcoin, from day one, has used a canonical encoding (DER)-- unfortunately, the underlying implementation in OpenSSL had undocumented behavior where it also accepted invalid DER until a couple months ago. (likewise, (all?) other similar libraries, such as bouncy castle behaved likewise-- this wouldn't have been escaped by not using OpenSSL). I spent some amount of time looking for precise DER implementations and BER implementations for reference test points in libsecp256k1 and was unable to find _any_ open source software which precisely implemented (no more accepted or rejected than specified) them. Even OpenSSL's recent fix for their behavior was not to correct their parser but instead to attempt to round-trip the encoding, which is not a guarantee. (You can see this list for some of the crazy things the OpenSSL parser, as an example, would accept.).

(And FWIW, the whole Bitcoin network has filtered these encodings everywhere for several years,  even though it required quite some wallets (like Armory and MTGOX) change their behavior).

But that deals with serialization, I am aware of no evidence that anyone has observed the algebraic malleability of ECDSA prior to us, and no evidence that anyone else took any action against it (now or previously).

As far as leaving serialization elements out of protocols, great care must be taken there too as there have been many vulnerabilities created by leaving things out. For example, the original PGP fingerprints hashed the modulus and exponent (but not their potentially non-canonical serialization), but also as a result had no delimitation so someone could move some bytes of your modulus into the exponent until they found a key they could factor, and then they'd have a weak key with the same fingerprint as you.

Signatures being covered under identifying hashes is ubiquitous, e.g. x509 certificate chains work just like bitcoin in this respect; and this interacts poorly with cert blacklisting due to malleability.

In any case, I don't want to belabor this, but you were really harsh and critical where I think the reality is almost 180 degrees out:  Bitcoin suffered from undocumented behavior in third party code; the outside world while somewhat aware of problems from non-canonicality has largely not acted on it-- Bitcoin Core is a leader in responsive and responsible handling of this, and our efforts resulted in the discovery of an additional form which appears to have not been known/discussed previously), which our ecosystem (out of all the ECDSA users in the world) appears to be the only ones at all protected against.

I really don't care much what armchair jockies on the forums (not specifically referring to you) think, and so I don't usually expend a lot of resources bragging... but to have someone be really harsh on a matter where I think our response has shown considerable innovation and leadership is, at least a little, bit frustrating!

It is unclear what you mean by "would not be significant", but the matter is largely orthogonal-- 6979 or not, implementations that do not heed the lowS behavior specifically will produce non-lowS signatures at a half the time per signature, leading to most of their transactions being blocked.  No wallet that I'm aware of has ever provided a "resign" button that might break through-- and when wallets do create new transactions they usually use a new random selection of coins, so 6979 would also produce a new signature. Even if they did provide a "resign" and used a random signature, once your transaction has more than a couple inputs the probability that it would pass a lowS test becomes very very low.

RFC6979 does not really close a covert channel either, alas, it is impossible to determine if a device is using 6979 without access to its keys and a malicious device could use 6979 99.99% of the time but then sometimes change to a kleptographic method (perhaps triggered by the message itself).   The RFC does eliminate the strong need for a good RNG at signing time --  a common implementation problem-- and make review/auditing somewhat easier.


Unfortunately if the device has free choice of its nonce (which having control of low/high S implies) then it has a high bandwidth covert channel.  E.g. produce the nonce as H(ECDH(attacker_pubkey, pubkey) || message_to_be_signed). More elaborate versions of this allow the embedding of additional data, beyond just leaking the current private key. Even if something constrains the choice of nonce, a malicious device can do rejection sampling to turn get N bits of covert channel out of the nonce by doing 2^n computation-- low vs high isn't different except by being computationally cheaper to use.   So, I'm not sure I'll be able to sell it in terms of covert channel suppression, but it's worth a try. Thanks for the suggestion.

Warning about it, at least, would be productive-- as there has been a fair amount of loss from people accidentally using the wrong address (in addition to the other ways reuse causes systemic harm).  If you mean "already used" ever, then prohibiting it requires a forever growing database, which isn't all that scalable.

A middle ground would be warning (or refusing) any that the site itself had previously sent to; and I think there was general agreement to do that kind of warn on (wallet local) reuse in bitcoin core in the past.

Don't look at me: I had no idea bitcoin.org mangled the release notes. Thanks. No one working on Bitcoin core has any control over bitcoin.org, but I'll report it now that you've brought it to my attention.

I didn't contact you, you started this thread with a bunch of accusations and ranting, because -- apparently-- you weren't willing to "pay the price".

And no, just because there is a fine on something doesn't make it appropriate to do just because you're willing to pay the fine.

Please do not address anyone on this subforum in this manner. None of the developers of Bitcoin software owe you anything, and this sort of disrespect is most unwelcome.