Bitcoin Forum
October 24, 2020, 04:02:04 AM
 News: Latest Bitcoin Core release: 0.20.0 [Torrent]
 Home Help Search Login Register More
 Pages: [1] 2  All
 Author Topic: FRAGMENTED BACKUPS VULNERABILITY!! IF YOU USE THEM, READ THIS!!  (Read 18357 times)
goatpig
Moderator
Legendary

Offline

Activity: 2674
Merit: 1177

Armory Developer

 September 22, 2017, 05:03:40 PMLast edit: September 23, 2017, 03:11:22 AM by goatpigMerited by malevolent (5)

TLDR:

Fragmented backups in Armory use a broken implementation of Shamir's Secret Sharing (SSS). All users of fragmented backups need to treat the wallets backed up in this fashion compromised and migrate all funds to a fresh wallet.

1) The story

A couple days ago I was warned by Gregory Maxwell that the implementation for Shamir's Secret Sharing Armory uses is broken. After reviewing the code in question, I've concluded the implementation of SSS introduces a vulnerability in fragmented backups that requires not only an immediate fix but also a community wide warning to alert all fragmented backup users.

These users need to sweep all funds from these wallets. All version starting 0.96.3 will have the vulnerability fix.

2) A high level look at SSS

SSS is a scheme that takes a secret and outputs a set of N fragments. M of these fragments are sufficient to reconstruct the secret.

In order to do so, you first constructs a polynomial f of degree M over a finite field, in which M-1 of the coefficients are random values, and the last one is the secret itself. You then compute N points of f. Each point, which is a pair (x, f(x)), constitutes a fragment.

To reveal the secret, you need to reconstruct the polynomial, which you do by interpolating M points together.

I will not go into the details of the interpolation, but one element here is crucial: The effect of a partial interpolation is to narrow down the possible candidates for valid coefficients. In other words, the more points on the curve you interpolate, the clearer a picture of the curve you get. The curve is constructed around your secret, a more refined interpolation results in an ever increasing leak of information, to the point where you can brute force the secret.

This property of the interpolation is counteracted by the use of a finite field. Since all points generated with the polynomial belong to the underlying finite field, the cyclic nature of finite fields widens the range of possible solutions that would yield your particular polynomial from a partial interpolation (by making them infinite), in such a way that you simply cannot brute force your way out of the interpolating at least M points.

A simple way to demonstrate that property is as follows:

Consider the equation:

Code:
8 = 5 + x

It's obvious that x is 3, and that this is the only solution. Now add this variation to the equation:

Code:
8 = (5 + x) mod 11

Suddenly x can be 3, 14, 25, 36... and so on. There is now an infinite amount of solutions for x, due to the introduction of the modulo operation. This is essentially the effect the finite field has on the interpolation equations.

3) The vulnerability

The whole point of the previous section was to demonstrate how SSS is constructed to prevent partial information leaking. One of the requirements to insure that property is that all of the polynomial coefficients but the secret are chosen at random.

This is precisely what Armory's implementation breaks, by rolling deterministic and chained coefficients, where the first coefficient is deterministically derived from the secret, and the subsequent ones are derived successively from one another.

If you look at it in the context of the previous paragraph, where SSS was crafted to deny any alley to gather information about coefficients short of M fragments, the newly introduced deterministic relationship between all coefficents provides a path that potentially dumbs down the security of SSS to that of the hash function used to generate the coefficients.

This can ultimately lead to a subset of fragments leaking information about coefficients where none should be.

4) In the code

Code wise, there were 2 instances of this implementation, both of which are faulty.

a) The first version

It did 2 things wrong, one more aggravating than the other:

The coefficients were not picked at random, instead they were derived as hashes of the secret, in a fashion that boils down to this:

Code:
coefficient_0 = hash(secret)
coefficient_(n+1) = hash(coefficient_n)

The fragments themselves were constructed as the following points on the curve:

Code:
fragment_n = (coefficient_n, f(coefficient_n))

There are 2 important issues with this setup. First of all, the coefficients are not selected at random, therefor it breaks SSS security assumptions.

Next, and most aggravating, the coefficients were provided as is on the fragments, since you need to provide the pair (x, f(x)), and x in this case were the actual coefficients.

Note that since the coefficients were derived from each other, the second mistake is twice as aggravating, as anyone with fragment n would have coefficient n and could derive all coefficients n+1.

The second mistake was caught and fixed eventually. I was not involved with anything regarding cryptography and security in Armory at the time, therefor I have no recollection of the event. I expect there was a write up of the issue and people were told to cycle wallets, but I can't remember any of it.

You can look at the code in its original form here:
https://github.com/etotheipi/BitcoinArmory/commit/80e373a#diff-27fe88d2c6032fecb93912a17d72081bR1615

b) The second version

The second version only made sure points were generated using the [1...N] sequence for x instead of coefficient themselves. At this point fragmented backups where no longer as broken but the code still did not implement SSS correctly.

Notably the assumption that no amount of fragments less than M can leak any data about the secret is not true with that faulty implementation.

One way to look at it is that this implementation introduces a deterministic relationship between the coefficients in a way that it can effectively reduce the security of the system to that of a single pass of HMAC512 provided an amount L of fragments, with L < M.

Here is the commit for the second version:

https://github.com/etotheipi/BitcoinArmory/commit/0824b632600116bd6395cec939fa6fd398efeb19#diff-27fe88d2c6032fecb93912a17d72081bR1915

5) Affected versions

Fragmented backups were introduced in version 0.88 (04/18/2013) and the first fix was deployed in version 0.90 (11/26/2013).

The final fix was introduced in v0.96.3 (9/21/2017)

6) The fix

The coefficients were made deterministic in order to present deterministic fragments to the user when fragmenting a wallet over a given scheme. In other words, fragmenting wallet W over a M-of-N scheme would always yield the save fragment values for the same fragment index.

This introduces scenarios where any amount of fragments can be recomputed from the private wallet root without invalidating fragments still in the wild.

The fix was to randomize the coefficients at the cost of the deterministic characteristic of the fragments.  The choice was fairly simple:

a) The deterministic attribute gained by bastardizing SSS is worthless in the face of how it damages its security properties.

b) Even if a change to SSS is designed so that it does not so obviously erode its security properties, this is still an act of rolling a custom cryptographic function, which commands a level of review and security analysis that will most likely not be performed at the adequate level.

c) There are no scenarios I can think of in which the feature of determinism in fragments is actually necessary and central to this type of backups. Introducing it at the cost of security is therefor doubly unacceptable. Backups are supposed to be forever afterall, lacing solid crypto with any kind of bootlegged algorithm does not stand to reason.

Therefor, it is without hesitation that the faulty feature was undone, and the faulty code removed from the repo so as to prevent unaware users from copying it into their own projects.

The changes can be seen here:

https://github.com/goatpig/BitcoinArmory/commit/94d2a7556d25cf788da639d81a7162694982f6b7
https://github.com/goatpig/BitcoinArmory/commit/7bd9887891ac88e2e49954ef034bedef88f23eaf

7) GUI changes

Since the fragments are not deterministic anymore, they are now generated with a unique set ID which is reflected on the backup strings and printed backups. Fragments are only useful within a their own set. Another way to put it is that you cannot mix and match fragments from different sets. This is the only difference between the pre and post fix implementation.

The fixed version is compatible with fragments generated from the deterministic version. You will still be able to restore from these with version 0.96.3+

8 ) Recommendations

It is hard to say exactly how effectively this custom take on SSS breaks security at the fragment level. How fewer fragments would it take to reproduce a secret than intended? Honestly, I don't know, but while the first implementation was effectively breaking all security assumptions of SSS, the second version is SSS at least theoretically.

I don't expect an attacker can snatch a single fragment and reveal the secret the next minute with minimal code. This vulnerability reduces the overall complexity of the problem that of a hash function, it doesn't outright bypass all complexity.

Since we're talking 32 byte integers, breaking the scheme isn't trivial, but it has certainly been weakened to a state that is difficult to precisely assess. Therefor, to remain on the conservative side, my recommendation is as follows:

If you created a fragment backup of your wallet, consider that wallet compromised. Create a new wallet and sweep the funds from the compromised wallet to the new one. You can redo your fragmented backup scheme on the new wallet provided you use Armory 0.96.3 and newer.

If you do not use fragmented backups, you have nothing to do.

9) Notes

Special thanks to Gregory Maxwell for finding the vulnerability and helping with the review of the fix.

Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1603512124
Hero Member

Offline

Posts: 1603512124

Ignore
 1603512124

1603512124
 Report to moderator
1603512124
Hero Member

Offline

Posts: 1603512124

Ignore
 1603512124

1603512124
 Report to moderator
1603512124
Hero Member

Offline

Posts: 1603512124

Ignore
 1603512124

1603512124
 Report to moderator
gangtraet
Full Member

Offline

Activity: 159
Merit: 100

 September 25, 2017, 09:11:54 AM

There are two things should be random in SSS.  The coefficients of the polynomial.  And the x-values where the polynomial is evaluated.

Non-random x values is probably relatively benign, since all information about the polynomial is in the f(x) values.  Wikipedia does not even mention that the x values should be random.

Pseudo-random coefficients is certainly a bad idea.  And printing the coefficients of the polynomial by using them as x-values is a horrendous mistake, I can hardly believe that it has been done!

Is there any way I can see on my 3-of-5 backup if it was made before or after that update was made?  I am of course going to move my funds, but doing so before a new 3-of-5 backup is in place is also a risk.

EDIT: Could not spell 'pseudo'
gangtraet
Full Member

Offline

Activity: 159
Merit: 100

 September 25, 2017, 01:35:09 PM

And printing the coefficients of the polynomial by using them as x-values is a horrendous mistake, I can hardly believe that it has been done!

This is even worse than I first thought.  In an N-of-M backup, each of the M sheets encodes a linear equation (linear in the unknown coefficients, not in x).  Together, they can be solved to find the N coefficients of a polynomial.  It requires N equations to find the N unknowns, this is where the most of the security comes from.  In addition, you want to avoid leaking information that could potentially reduce the search space for the unknowns, hence the random x values.

But if each sheet reveals a coefficient, then there are TWO equations on each sheet!  (y = c_0 + c_1 * x + c_2 * x^2 + ...  and c_i = x)

Thus you only need N/2 sheets to recover the key !!!.  The seed can be recovered from a single sheet of what was supposed to be a 2-of-M backup, or from two sheets of a 3-of-M or 4-of-M backup, etc etc.

At least only the oldest Armory backups have this awful flaw.  The newer ones potentially leak some information that makes finding the key a little less impossible if you already have N-1 sheets.

NB: This is of course assuming that I have not misunderstood goatpig's undoubtedly slightly simplified explanation of the flawed algorithm.

goatpig
Moderator
Legendary

Offline

Activity: 2674
Merit: 1177

Armory Developer

 September 25, 2017, 02:50:05 PM

Non-random x values is probably relatively benign, since all information about the polynomial is in the f(x) values.

You can choose any x as long as the coefficients are random. I don't quite see how randomizing x achieves anything, as you have to provide entire points (x, f(x)) as fragments, therefor x is public information. Maybe you were trying to say that the x values should be chosen at random, instead of say, as part of a sequence? I don't think that's relevant at all for SSS, since all operation are performed on a finite field.

Quote
Is there any way I can see on my 3-of-5 backup if it was made before or after that update was made?  I am of course going to move my funds, but doing so before a new 3-of-5 backup is in place is also a risk.

The coefficients are 32bytes, therefor your backup would have 5 lines (1 header line + 4 * (16bytes of data + 2bytes of checksum) lines) per fragment. If the fragment only has 1 header and 2 data lines, it's using implicit [1,...,N] for x.

Quote
Thus you only need N/2 sheets to recover the key !!!.  The seed can be recovered from a single sheet of what was supposed to be a 2-of-M backup, or from two sheets of a 3-of-M or 4-of-M backup, etc etc.   Shocked

It's far worse than that. Look at how the coefficients are constructed, they're hashes of the previous coefficient. Whatever fragment the attacker gets access to, he will be able to compute all following coefficients just by hashing x once.

Read this again and look at the code again. In the first scheme, anyone with access to a fragment that carries the first coefficient can compute *ALL* all of the coefficients. The single point he has with the fragment is then enough to reveal the secret.

This is slightly worsened by the fact that since there are more fragments than coefficients, certain coefficients reoccur on several fragments. Now people being people, when they distribute fragments they tend keep the first few for themselves, by happenstance. If the coefficients were distributed in order with fragments, this very human yet entirely irrelevant quirk would actually save these users from total disaster. Again, the order of coefficients matter since they are derived consecutively from one another

gangtraet
Full Member

Offline

Activity: 159
Merit: 100

 September 25, 2017, 03:43:03 PM

You can choose any x as long as the coefficients are random. I don't quite see how randomizing x achieves anything, as you have to provide entire points (x, f(x)) as fragments, therefor x is public information. Maybe you were trying to say that the x values should be chosen at random, instead of say, as part of a sequence? I don't think that's relevant at all for SSS, since all operation are performed on a finite field.

Quote
The coefficients are 32bytes, therefor your backup would have 5 lines (1 header line + 4 * (16bytes of data + 2bytes of checksum) lines) per fragment. If the fragment only has 1 header and 2 data lines, it's using implicit [1,...,N] for x.
Mine has a header line, and two data lines.  I am not completely pwned then, but only has a slightly reduced security.  Ah, well.

Quote
It's far worse than that. Look at how the coefficients are constructed, they're hashes of the previous coefficient. Whatever fragment the attacker gets access to, he will be able to compute all following coefficients just by hashing x once.

Oh my god!  It must be one of the worst security f*ckups in a bitcoin wallet!

andytoshi
Full Member

Offline

Activity: 174
Merit: 136

-

 September 25, 2017, 04:53:00 PM

There are two things should be random in SSS.  The coefficients of the polynomial.  And the x-values where the polynomial is evaluated.

This is false. Where did this idea come from?

Quote
Non-random x values is probably relatively benign, since all information about the polynomial is in the f(x) values.  Wikipedia does not even mention that the x values should be random.

What does "probably relatively benign" mean in cryptographic terms?

Quote
Pseudo-random coefficients is certainly a bad idea.

Why is this?
BitUsher
Legendary

Offline

Activity: 994
Merit: 1027

 September 25, 2017, 10:25:21 PM

Does this effect SSS with mycelium entropy?
goatpig
Moderator
Legendary

Offline

Activity: 2674
Merit: 1177

Armory Developer

 September 25, 2017, 10:29:07 PM

Does this effect SSS with mycelium entropy?

I'm don't know what you are referring to.

BitUsher
Legendary

Offline

Activity: 994
Merit: 1027

 September 25, 2017, 11:07:49 PM

Does this effect SSS with mycelium entropy?

I'm don't know what you are referring to.

Entropy , used by many to create SSS fragments for cold storage uses a similar scheme -

https://github.com/mycelium-com/entropy

Specification -

https://github.com/cetuscetus/btctool/blob/bip/bip-xxxx.mediawiki
achow101
Staff
Legendary

Offline

Activity: 2282
Merit: 3513

Just writing some code

 September 26, 2017, 12:58:01 AM

Entropy , used by many to create SSS fragments for cold storage uses a similar scheme -

https://github.com/mycelium-com/entropy

Specification -

https://github.com/cetuscetus/btctool/blob/bip/bip-xxxx.mediawiki
The SSS implementation in Armory is completely unrelated to that. It also predates that by a few years.

BitUsher
Legendary

Offline

Activity: 994
Merit: 1027

 September 26, 2017, 04:05:46 AM

The SSS implementation in Armory is completely unrelated to that. It also predates that by a few years.

Since Greg Has concerns about every SSS version he has come across -

I think we should be highly skeptical about entropy wallets until more peer review is done as well.
goatpig
Moderator
Legendary

Offline

Activity: 2674
Merit: 1177

Armory Developer

 September 26, 2017, 05:08:12 AM

The SSS implementation in Armory is completely unrelated to that. It also predates that by a few years.

Since Greg Has concerns about every SSS version he has come across -

I think we should be highly skeptical about entropy wallets until more peer review is done as well.

I think you're reading too much into gmax's words. His general concern with SSS implementations is that they are basically all done over prime fields, which the easier implementation, but introduces side channel attacks cause it relies on bignum operations.

A GF(256) implementation would be in constant time, but that's harder to pull and therefor brings in question whether it's worth the effort at all, considering multisig covers a lot of what SSS does. I argued that SSS still has an edge over multisig scripts (plausible deniability), therefor I'll consider implementing SSS over GF(256) for the new wallets.

bitpop
Legendary

Offline

Activity: 2660
Merit: 1048

https://keybase.io/bitpop

 September 26, 2017, 05:29:05 AM

Is it fine if I don't care 1 sheet can now recover my Bitcoin? And just keep them as a paper backup? There's no actual leak right?

Reputation  |  PGP  |  Ethereum Classic
Bitcoin: 3DSh6AnmvBpDJFUz2mnLirMLmTMcFs9nDm
goatpig
Moderator
Legendary

Offline

Activity: 2674
Merit: 1177

Armory Developer

 September 26, 2017, 05:36:40 AM

Is it fine if I don't care 1 sheet can now recover my Bitcoin? And just keep them as a paper backup? There's no actual leak right?

I'd rather you don't confuse the 2 for the sake of convenience, but that's a way to look at it.

bitpop
Legendary

Offline

Activity: 2660
Merit: 1048

https://keybase.io/bitpop

 September 26, 2017, 05:38:03 AM

Is it fine if I don't care 1 sheet can now recover my Bitcoin? And just keep them as a paper backup? There's no actual leak right?

I'd rather you don't confuse the 2 for the sake of convenience, but that's a way to look at it.

I'll consider it 1 of 1 for now. Thanks.

Reputation  |  PGP  |  Ethereum Classic
Bitcoin: 3DSh6AnmvBpDJFUz2mnLirMLmTMcFs9nDm
goatpig
Moderator
Legendary

Offline

Activity: 2674
Merit: 1177

Armory Developer

 September 26, 2017, 05:40:46 AM

Is it fine if I don't care 1 sheet can now recover my Bitcoin? And just keep them as a paper backup? There's no actual leak right?

I'd rather you don't confuse the 2 for the sake of convenience, but that's a way to look at it.

I'll consider it 1 of 1 for now. Thanks.

That's a way to look at the security of the fragment. There is no code allowing to go from 1 fragment to your secret atm, and doing this off of the second version of the SSS scheme is non trivial, just so we are clear.

goatpig
Moderator
Legendary

Offline

Activity: 2674
Merit: 1177

Armory Developer

 September 26, 2017, 05:48:03 PM

In case I am 100% sure that the fragments have not been compromised (sealed envelopes, under the custody of people whom I trust), would it be possible to :
- install Armory 0.96.3 on a fresh machine,
- restore the current wallet here (originally generated with Armory 0.93.3)
- print new "secure" fragments of the same wallet,
- destroy the old fragments and the machine ?

You should cycle the wallets if you're going to bother printing new fragments. Better safe that sorry. Who knows where sha256 will be 30 years from now.

Quote
Would Armory 0.93.3 be able to restore a newly printed wallet in case of need ?

The restoration code is the same, but the fragment IDs are not deterministic anymore, so it's possible the GUI will refuse to proceed further with newly generated fragments until you comment out that check from the Python code. In other words, it can work if you're willing to deal with a little Python.

BitUsher
Legendary

Offline

Activity: 994
Merit: 1027

 September 26, 2017, 08:04:32 PM

The SSS implementation in Armory is completely unrelated to that. It also predates that by a few years.

Since Greg Has concerns about every SSS version he has come across -

I think we should be highly skeptical about entropy wallets until more peer review is done as well.

I think you're reading too much into gmax's words. His general concern with SSS implementations is that they are basically all done over prime fields, which the easier implementation, but introduces side channel attacks cause it relies on bignum operations.

A GF(256) implementation would be in constant time, but that's harder to pull and therefor brings in question whether it's worth the effort at all, considering multisig covers a lot of what SSS does. I argued that SSS still has an edge over multisig scripts (plausible deniability), therefor I'll consider implementing SSS over GF(256) for the new wallets.

Well said . couldn't hurt for more peer review regardless.
gangtraet
Full Member

Offline

Activity: 159
Merit: 100

 September 28, 2017, 07:25:16 AM

I think you're reading too much into gmax's words. His general concern with SSS implementations is that they are basically all done over prime fields, which the easier implementation, but introduces side channel attacks cause it relies on bignum operations.

I doubt that side-channel attacks are at all relevant for paper backups.  That normally requires statistical data on the timing or the amount of power it takes to do the operation.  It might be a worry that a compromized computer can gain information from a hardware wallet through side channels, but not when restoring a wallet.

My guess is that the main problem with SSS is that it is one of the few cryptographic operations that are simple enough that people implement it themselves, instead of getting it from a peer-reviewed and well-debugged library.

Quote
A GF(256) implementation would be in constant time, but that's harder to pull and therefor brings in question whether it's worth the effort at all, considering multisig covers a lot of what SSS does. I argued that SSS still has an edge over multisig scripts (plausible deniability), therefor I'll consider implementing SSS over GF(256) for the new wallets.

SSS does not just have a plausible deniability edge over multisig scripts (I am sure it does, although I cannot see how).  It solves a different problem: that of safely keeping backups.  Multisig of course gives the same security to your backup, but it also makes spending from the wallet on a day-to-day basis significantly more cumbersome.  And in times with high pressure on the blockchain, it also adds to the fee.

goatpig
Moderator
Legendary

Offline

Activity: 2674
Merit: 1177

Armory Developer

 September 28, 2017, 08:19:05 AM

I doubt that side-channel attacks are at all relevant for paper backups.  That normally requires statistical data on the timing or the amount of power it takes to do the operation.  It might be a worry that a compromized computer can gain information from a hardware wallet through side channels, but not when restoring a wallet.

My guess is that the main problem with SSS is that it is one of the few cryptographic operations that are simple enough that people implement it themselves, instead of getting it from a peer-reviewed and well-debugged library.

If you are using constant time AES and ECDSA, the only time to perform a timing attack is at restore time, which only applies to SSS in this case (assuming it's not in constant time).

Quote
SSS does not just have a plausible deniability edge over multisig scripts (I am sure it does, although I cannot see how).

1) You have assume any skilled/well equipped attacker will always get access to your watching only wallet (think law enforcement raiding your home).

2) This means an attacker will know not only of all your coins on chain, but also have access to all the script hash preimage. Therefor, if you used multisig, they will know of this as well.

3) On the other hand, SSS allows you to split your backups without reflecting that on chain, i.e. you could have all your coins in single sig scripts and there is no way for the attacked to know this just with your WO wallet.

 Pages: [1] 2  All