Miners: Time to deprioritise/filter address reuse!

[Strophon]

If no other pool ever adopts a similar set of rules then at most the impact is 15% slower confirmation times.

What about non-standard transactions? Isn't eligius the only game in town for that?

[AtlasONo]

If it's this important shouldn't Eligius start forcing new addresses for every withdrawal? Think the miners would go along with that?

It's easy to force your changes when the users of the pool can just passively go along with it. It's a lot tougher when you force them to actively make a decision on the issue.

[Luke-Jr]

If it's this important shouldn't Eligius start forcing new addresses for every withdrawal?

Yes, migrating to HD/recurring invoice ids has been on the todo list for Eligius for a long time.
Unfortunately, wallet software has not matured enough for that yet.

[Cryddit]

This is a good start.  I think you've hit a real sweet spot with this and I hope all the miners take it up.

Without shutting anyone down, It puts just a bit of pressure on everyone to find ways to make it easier and simpler to use new addresses every time. 

And the pressure is adjustable; as it gets easier to do the right thing, it's easy to "dial up" the number of blocks between reuses making the wrong thing ever just a little bit less tenable. 

One problem though is 'perverse incentives' for miners in the form of transaction fees.  In est, the more miners deploy this patch, the greater the additional revenue enjoyed by the miner who does *not* deploy this patch.  This is because the miner who does not deploy the patch will be picking up tx fees from tx left "on the ground" by miners who have deployed it. 

As yet, I have never reused an address.  I intend not to.  I wish my client made it easier to be sure.

[gmaxwell]

Unfortunately, wallet software has not matured enough for that yet.

Well, you don't have to wait for wallet software yet.

Here is an example:

Grab pycoin:

https://github.com/richardkiss/pycoin


$ python pycoin/scripts/genwallet.py -u -g > private_key
$ cat private_key
xprv9s21ZrQH143K4SHxWhmMDebDpB85Jo9wgwxnbHwXeGs5nnfCdwohgHv3bMnVGZMtgsVkNp1FsuA jSFu7caFh6qMrQ4ognrs5MUkoQChT1QM


Thats a private key, put it someplace safe. Now:


$ python pycoin/scripts/genwallet.py -f private_key -s 0.pub
xpub69Nh7f4uTEF6N1CdXpEp44AevN31pt1KdS8S4mqoLk2X4CS684gY2enwoJdmeaemB2atbU5gF1s waCwpaP4aAhgsFUzRMyHckSyoi8rbsbi


Thats a public key, it's what you'd give to eligius.

Eligius would generate addresses for you:

$ python pycoin/scripts/genwallet.py -k xpub69Nh7f4uTEF6N1CdXpEp44AevN31pt1KdS8S4mqoLk2X4CS684gY2enwoJdmeaemB2atbU5gF1s waCwpaP4aAhgsFUzRMyHckSyoi8rbsbi -a -s 0
1B7Y1zpPqcNoXDeB9NqRXg97gHCTYuQzNv
$ python pycoin/scripts/genwallet.py -k xpub69Nh7f4uTEF6N1CdXpEp44AevN31pt1KdS8S4mqoLk2X4CS684gY2enwoJdmeaemB2atbU5gF1s waCwpaP4aAhgsFUzRMyHckSyoi8rbsbi -a -s 1
1Q53tMUTzKjxYjnSFC58MtCQs6tEszUu7M
$ python pycoin/scripts/genwallet.py -k xpub69Nh7f4uTEF6N1CdXpEp44AevN31pt1KdS8S4mqoLk2X4CS684gY2enwoJdmeaemB2atbU5gF1s waCwpaP4aAhgsFUzRMyHckSyoi8rbsbi -a -s 2
1LyNrtpHuEjHMTmg9PWRxeP7jCpZuKiBsr
$ python pycoin/scripts/genwallet.py -k xpub69Nh7f4uTEF6N1CdXpEp44AevN31pt1KdS8S4mqoLk2X4CS684gY2enwoJdmeaemB2atbU5gF1s waCwpaP4aAhgsFUzRMyHckSyoi8rbsbi -a -s 3
16QPNHmkac7JurbJLwmBvkPjrFgNR13yvt


and you can generate the matching private keys in WIF format:


$ python pycoin/scripts/genwallet.py -f private_key -s 0/0 -w
L4DobQyRtccLvA3Hna5DDpUSfKdWx7wLMbTA9DXgUfq61vypeckb
$ python pycoin/scripts/genwallet.py -f private_key -s 0/1 -w
L4tytdhG8p6oJXQpFmCrZ4GWRVfHJe8FthxDTGn7HHYc6Yx7ZtuT
$ python pycoin/scripts/genwallet.py -f private_key -s 0/2 -w
L5iwTNQYuwC6VG4SnsFKSULB6mvCjW6om1L9AifAYC2FPGi3ZGwV
$ python pycoin/scripts/genwallet.py -f private_key -s 0/3 -w
L4mAzvyZvRdtLa2u2BhT8dxLFDELgWR5p8wipSrJ6iLXhXzNDFaq


No reason to delay supporting this on eligius, as at least some users could be using it today.

[BurtW]

Cool.  I can talk to some other pool owners I know.

[gmaxwell]

Cool.  I can talk to some other pool owners I know.

I think the message would be to educate them on this— that they can have users provide a single public key and then generate unique sequential addresses without any further user interaction where the user will have the private key... and find out what they need to make it a reality.

Wallet support for being able to easily just ask for a xpub address is in the pipeline (at least from armory, electrum, and bitcoin-qt) but not there yet— so the only people who could use it now would be people doing it manual style like I just described, but it would be good to get the pipeline primed. E.g. are there OSS tools that they're using that need the support added to them?

[giszmo]

I don't agree at all that this solves anything and thus was forced to opt for one of the personal assault options (why not word it less personal? yeah, tonal bitcoins edit war haha).

blacklisting would not work on an address basis but on an output basis and all of this output's descendants would be flagged, so it makes no change to incentivize using more addresses.

At first I was afraid your patch was about some security issue from private keys being revealed but now I feel like this move is simply bad for mobile clients that typically use just one address, for vanity addresses, for charities, satoshi dice (your biggest friend?), etc.

Is at least only the transaction from that address throttled or is it also the transaction to known addresses?

[mikegogulski]

I can very well understand Luke's motivation for publishing and implementing this patch as a counterweight to CoinInvalidation and similar awfulness. Admittedly, my poor brain hasn't had time to sort through all the pros and cons here. But:

Globally visible public records in finance are completely unheard-of. They are undesirable and arguably intolerable.

Er, not so. If you're arguing from the standpoint of a profit-driven business in a competitive landscape or from the standpoint of an individual burdened by increasingly pervasive financial surveillance, you're right, but those are not the only use cases for a currency.

There are lots of people around the world continuously pushing for more and more transparency in government finance, the logical endpoint of which is real-time, globally-visible transactions records for state treasury accounts. Most inputs (taxes, fines, fees, etc.) blinded for privacy reasons, and most outputs (salaries, goods, services, etc., but not, for example, welfare payments) required to be explicitly linked to an identity, for transparency and accountability purposes. Want to work for the state or one of its corporate appendages/owners? Be transparent and accountable.

The non-profit sector, and donors to it, would also benefit from such radical transparency. Charities that can prove that they're not blowing donor funds on exorbitant executive pay, corporate jets, fancy hotels and catering, for example, and are actually delivering real benefits to people in need (or at least working diligently toward what they've promised to donors), globally visible (at least up to the fiat interface or the real-world-goods/services interface), supplant and, er, out-compete, those which waste so much on overheads. For them and for donors, globally visible public records are a desirable feature, at least for the ones that survive the transition to a cryptocoin economy.

As another example, I operate a Bitcoin legal defense fund for the accused Dread Pirate Roberts (link in my sig). The transaction record to the fund's sole address shows that I've disbursed nothing of what's been given, and it needs to stay that way until I can properly justify an expenditure.

All of these models, and others, can benefit from reusing addresses tied to individual and/or institutional identity, and address re-use carries a massive benefit in simplicity ("donate here" QR codes on paper flyers? yes we can!), despite the known drawbacks. Deprioritizing or (especially!) filtering transactions seems to raise the costs for implementing such models, significantly.

(At another level, all of this sums down to mining pool centralization as systemic risk, but that's another discussion entirely.)

[Luckybit]

One very common scenario is to use a bitcoin address as the identification. This may be used in faucets, mining pool, gambling sites, and maybe future block-chain based exchanges. If address reusing is discouraged, any alternative is suggested? To introduce another identity-> bitcoin address translation layer?

Maybe you mean outgoing only? An bitcoin can accept many incoming transactions, but it can send out coins only once? That's a little bit better, but still in many scenarios people are expecting coins are sent from the same address. Mastercoin is an example.

If people want to hide their trace, they can choose not to reuse addresses. If people don't care, why have to increase the difficulty for them? Why not just let the users to make the decision?
 

https://en.bitcoin.it/wiki/Identity_protocol_v1 ?

[Peter Todd]

I can very well understand Luke's motivation for publishing and implementing this patch as a counterweight to CoinInvalidation and similar awfulness. Admittedly, my poor brain hasn't had time to sort through all the pros and cons here. But:

Globally visible public records in finance are completely unheard-of. They are undesirable and arguably intolerable.

Er, not so. If you're arguing from the standpoint of a profit-driven business in a competitive landscape or from the standpoint of an individual burdened by increasingly pervasive financial surveillance, you're right, but those are not the only use cases for a currency.

There are lots of people around the world continuously pushing for more and more transparency in government finance, the logical endpoint of which is real-time, globally-visible transactions records for state treasury accounts. Most inputs (taxes, fines, fees, etc.) blinded for privacy reasons, and most outputs (salaries, goods, services, etc., but not, for example, welfare payments) required to be explicitly linked to an identity, for transparency and accountability purposes. Want to work for the state or one of its corporate appendages/owners? Be transparent and accountable.

The non-profit sector, and donors to it, would also benefit from such radical transparency. Charities that can prove that they're not blowing donor funds on exorbitant executive pay, corporate jets, fancy hotels and catering, for example, and are actually delivering real benefits to people in need (or at least working diligently toward what they've promised to donors), globally visible (at least up to the fiat interface or the real-world-goods/services interface), supplant and, er, out-compete, those which waste so much on overheads. For them and for donors, globally visible public records are a desirable feature, at least for the ones that survive the transition to a cryptocoin economy.

As another example, I operate a Bitcoin legal defense fund for the accused Dread Pirate Roberts (link in my sig). The transaction record to the fund's sole address shows that I've disbursed nothing of what's been given, and it needs to stay that way until I can properly justify an expenditure.

All of these models, and others, can benefit from reusing addresses tied to individual and/or institutional identity, and address re-use carries a massive benefit in simplicity ("donate here" QR codes on paper flyers? yes we can!), despite the known drawbacks. Deprioritizing or (especially!) filtering transactions seems to raise the costs for implementing such models, significantly.

(At another level, all of this sums down to mining pool centralization as systemic risk, but that's another discussion entirely.)

I what's important to remember is what Luke is doing with Eligius is a: just a one block delay, and b: can be easily changed later. Right now we have a serious problem with address re-use - doing what we can to stop it makes a lot of sense even if what we do isn't necessarily the perfect solution.

Even if reusing addresses was 100% banned and it was impossible to re-use an address ever, with appropriate tools all those examples of transparency that you cite would still be possible using BIP32: rather than achieve transparency by publishing the address, publish a BIP32 pubkey seed and derive all the addresses where funds are stored from it. In many cases this is an even better solution, because you now have the option to be selective about who you are transparent too.

On the other hand if you truly are trying to be globally transparent re-using a single fixed address has an advantage over the BIP32 option: it lets other people know that your transactions are not anonymous. If I receive funds in a transaction where the change address is the same as the address the funds were originally at, I know the funds I've received are not as anonymous as they could be, and I might want to take steps in response. But again, in this case a minor one-block delay does no real harm.

tl;dr: I fully support what Luke is doing in the here and now, and we can and should monitor the situation to determine if the strategy needs to be changed in the future.

[trout]

Luke, can you describe more clearly what the patch does?
in particular:
- one reuse of any given address per block, or one address reuse per block?
- does it concern only  inputs, outputs, or both?  for example, an address
that has several inputs and tries to spend them all is deprioritized or not ?

right now the  [rationale/code] and  [reasoning/action] ratios are really too low.

[justusranvier]

The non-profit sector, and donors to it, would also benefit from such radical transparency. Charities that can prove that they're not blowing donor funds on exorbitant executive pay, corporate jets, fancy hotels and catering, for example, and are actually delivering real benefits to people in need (or at least working diligently toward what they've promised to donors), globally visible (at least up to the fiat interface or the real-world-goods/services interface), supplant and, er, out-compete, those which waste so much on overheads. For them and for donors, globally visible public records are a desirable feature, at least for the ones that survive the transition to a cryptocoin economy.

As another example, I operate a Bitcoin legal defense fund for the accused Dread Pirate Roberts (link in my sig). The transaction record to the fund's sole address shows that I've disbursed nothing of what's been given, and it needs to stay that way until I can properly justify an expenditure.

All of these models, and others, can benefit from reusing addresses tied to individual and/or institutional identity, and address re-use carries a massive benefit in simplicity ("donate here" QR codes on paper flyers? yes we can!), despite the known drawbacks. Deprioritizing or (especially!) filtering transactions seems to raise the costs for implementing such models, significantly.

The right way for an organization that desires this level of transparency is to use BIP32 and publish their extended public key.

The single address model for simplicity is like trying to use a screwdriver as a hammer.

[mikegogulski]

The right way for an organization that desires this level of transparency is to use BIP32 and publish their extended public key.

The single address model for simplicity is like trying to use a screwdriver as a hammer.

My overall point is that this kind of solution tends to raise costs for certain use cases.

BIP32 is all well and good, but requires significant tooling for a donation-accepting entity to implement, requires much broader dissemination among wallet clients, and is largely incompatible with the widely-deployed single-address model already in use by many entities.

I'm not here to scream bloody murder about Luke's initiative, but rather to encourage looking at this from as many sides as possible. CoInvalidation etc. present real threats to Bitcoin as a privacy-enhancing system, but, as I think I demonstrated above, privacy is not the sole or even primary concern for every use case.

[Strophon]

The right way for an organization that desires this level of transparency is to use BIP32 and publish their extended public key.

This would be all well and good if every wallet client (or even most of them) actually, you know, implemented BIP32! As it is, the best you can do is point me to a python tool? I'm capable of using it, but that just seems like a massive pain in the ass to go to. (Oh look, someone used another one of my addresses! Better boot to linux, generate the private key, and import it into my wallet!) And that's for me, a software engineer. What about the less tech-savvy users?

Moreover, most Bitcoin-related websites don't have BIP32 support either! They take one address for withdrawals. Some allow you to give multiple addresses, if they're a bit fancier, but even then, I'm supposed to post 50 addresses and log in to switch the address every time I get a payout?

And then there's Blockchain.info. Did you know Blockchain.info wallets reuse addresses by default unless you create new ones?

I appreciate the intent behind this proposal, but I don't think we're ready for this change yet. I do agree that we need to do this at some point in the future, but shouldn't we, you know, roll out the infrastructure necessary to make it actually workable first?

[Mike Hearn]

Hmm, I only scan read this thread, but as bitcoinj based wallets are one of the primary guilty parties in address re-use I'll just pop in to give my view:

People already have the right incentives to not re-use coins (privacy). SPV wallets suck at it today because I chose backup safety over privacy, but BIP32 let's us have our cake and eat it in that regard and it's now being implemented. So at some point SPV wallets will update and addresses will start automatically changing, users won't have to do anything. I'm figuring out ways existing wallets can be upgraded in place.

Punishing address re-use NOW, when many users who are doing it don't have any good way to stop, is just attempting to pressure volunteer wallet developers through their userbase. That's not cool. I already spend a lot of my spare time at evenings and weekends fixing bugs and keeping up with the Bitcoin protocol (20% only gets you so far and can't always be taken). Being "forced" to do it by punishing my users would be quite upsetting, especially as it's coming anyway and Bitcoin-Qt itself doesn't even implement BIP32 properly yet (AFAIK only Electrum has done this?).

[justusranvier]

This would be all well and good if every wallet client (or even most of them) actually, you know, implemented BIP32! As it is, the best you can do is point me to a python tool? I'm capable of using it, but that just seems like a massive pain in the ass to go to. (Oh look, someone used another one of my addresses! Better boot to linux, generate the private key, and import it into my wallet!) And that's for me, a software engineer. What about the less tech-savvy users?

Moreover, most Bitcoin-related websites don't have BIP32 support either! They take one address for withdrawals. Some allow you to give multiple addresses, if they're a bit fancier, but even then, I'm supposed to post 50 addresses and log in to switch the address every time I get a payout?

And then there's Blockchain.info. Did you know Blockchain.info wallets reuse addresses by default unless you create new ones?

I appreciate the intent behind this proposal, but I don't think we're ready for this change yet. I do agree that we need to do this at some point in the future, but shouldn't we, you know, roll out the infrastructure necessary to make it actually workable first?

I was not commenting on the advisability of the patch in the OP, just pointing out that just because people are using single address wallets does not mean that it's the right thing to keep doing.

That behaviour was known broken ever since the beginning, but since Satoshi didn't worry about the ability of users to backup their keys when he wrote the client (otherwise he would have gone with deterministic key generation), client developers responded by making wallets with a single key. Now a bunch of new users have been trained with bad practises and it will take some now to reverse.

[mikegogulski]

Hmm, I only scan read this thread, but as bitcoinj based wallets are one of the primary guilty parties in address re-use I'll just pop in to give my view:

"Guilty" is rather too strong a word, especially given that a library like bitcoinj sits fairly distant from the wallet UIs, which is where the flags and cautions need to be shown to the user.

What would be very nice to see as at least a minor mitigation here would be wallet clients presenting users with an additional challenge when attempting to send to an address the client has already spent to. That only takes a chink out of the systemic problem under discussion, but has a nice side effect in reducing the risk of a mis-click someplace leading to an unintended irrevocable spend to an incorrect address (which, er, I've done more than once).

People already have the right incentives to not re-use coins (privacy). SPV wallets suck at it today because I chose backup safety over privacy, but BIP32 let's us have our cake and eat it in that regard and it's now being implemented. So at some point SPV wallets will update and addresses will start automatically changing, users won't have to do anything. I'm figuring out ways existing wallets can be upgraded in place.

Cool. I'd be thrilled if wallet clients used the BIP32 scheme as the default. It's a win for the whole ecosystem -- provided that, in light of the use cases above and others -- the choice to re-use addresses is not phased out.

Punishing address re-use NOW, when many users who are doing it don't have any good way to stop, is just attempting to pressure volunteer wallet developers through their userbase. That's not cool. I already spend a lot of my spare time at evenings and weekends fixing bugs and keeping up with the Bitcoin protocol (20% only gets you so far and can't always be taken). Being "forced" to do it by punishing my users would be quite upsetting, especially as it's coming anyway and Bitcoin-Qt itself doesn't even implement BIP32 properly yet (AFAIK only Electrum has done this?).

Another very good perspective on this.

[Cryddit]

People already have the right incentives to not re-use coins (privacy).

This is not the case.  My privacy is not an incentive for someone who wants to pay me coin, nor for someone whom I want to pay coin to.  It is only an incentive for me, and I need the cooperation of these other parties to make it work.  In the absence of some further incentive that works for all the parties involved, privacy simply has not happened and evidently will not happen.

It's nice to think it's "coming" in these newer clients, but my privacy is more secure, AFAICT, if when it does so people have some substantial reason to *USE* and *PREFER* these features of the newer clients. Otherwise I wind up out on the short end of the stick dealing with people who don't want the new client for some reason and who aren't sufficiently inconvenienced by my lack of privacy to care.

[DeathAndTaxes]

This would be all well and good if every wallet client (or even most of them) actually, you know, implemented BIP32! As it is, the best you can do is point me to a python tool? I'm capable of using it, but that just seems like a massive pain in the ass to go to. (Oh look, someone used another one of my addresses! Better boot to linux, generate the private key, and import it into my wallet!) And that's for me, a software engineer. What about the less tech-savvy users?

Really as a tech savy person you can't think of a better workaround?  How about generate and import 1,000 private keys.  Then when you need to change the address simply use the python tool to find the next one in the sequence.

Moreover, most Bitcoin-related websites don't have BIP32 support either! They take one address for withdrawals. Some allow you to give multiple addresses, if they're a bit fancier, but even then, I'm supposed to post 50 addresses and log in to switch the address every time I get a payout?

Well they won't until there is a reason to do so and the patch gives them a reason.  In the interim if a site allows posting 50 addresses why would you need to manually change them.  It would be a trivial amount of coding for a site to use the next one automatically and notify you when the queue is low or out.  Still BIP32 is a better longer term solution.  However if sites don't implement them you can still use funds with a small disincentive.  That would encourage people to use sites which do promote single use addresses.

And then there's Blockchain.info. Did you know Blockchain.info wallets reuse addresses by default unless you create new ones?

Then encourage blockchain.info to change their default behavior.  The issue of reuse has been known since a YEAR PRIOR to the genesis block.  It is unlikely without an incentive anything will change.  Lastly remember this doesn't prohibit reuse, it doesn't make coins sent to a used address lost, it doesn't even mean higher mandatory fees.  It is merely delaying secondary spends from the same address.

I appreciate the intent behind this proposal, but I don't think we're ready for this change yet. I do agree that we need to do this at some point in the future, but shouldn't we, you know, roll out the infrastructure necessary to make it actually workable first?

Chicken or the egg.  Without the incentive will the infrastructure ever exist.  Blockchain.info could easily (within minutes) change their code.  Will they?  Probably not.  Not until users complain.  Will users complain unless their is an incentive?  Once again probably not.  It isn't like Bitcoin just launched.  There has been years to do things right and the lazy, easy approach has been used in most cases.