Bitcoin Forum
September 25, 2024, 06:31:40 PM *
News: Latest Bitcoin Core release: 27.1 [Torrent]
 
  Home Help Search Login Register More  
  Show Posts
Pages: « 1 ... 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 [63] 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 ... 233 »
1241  Bitcoin / Armory / Re: Armory 0.96.3 fragment ID during test backup does not match on: September 25, 2017, 09:27:35 PM
From the changelog: https://github.com/goatpig/BitcoinArmory/blob/master/changelog.txt#L17

Quote
   - Fragment sets are now generated randomly, therefor an unique ID has been added to each set to identify them. You cannot mix
     and match sets.

The IDs are different because the fragments are different. You cannot mix and match fragments from within different sets. The randomized ID is by design, so that users don't end up mixing them.
1242  Bitcoin / Armory / Re: Using Armory on the BCH chain on: September 25, 2017, 05:27:06 PM
Granted, the only thing you are actually after is the blk file with the first post fork block. If that one is taken out, Armory can compute the chain past the fork point and you should be good.
since i have done some tx's of BTC post fork, i have to use your trick of this truncated blockchain from what i understand.  will this block file range work?

Only if you have not moved coins on the prefork chain past that point.

The logic is as follows: your last set of valid utxos pre fork is still valid on the Bcash chain (up until the point you move your bcash). If you roll back the BTC blockchain to a point in time where it stands past your last valid utxo set state and before you moved coins post fork on the BTC chain, these utxos are also valid on the Bcash chain.

Graphically, it would look like this:

Code:
]--------V------------F-------M--------[

- F is the fork point. Anything past F is the Bitcoin side of the fork on this graphic.
- V is the last block you transacted in pre fork. This is your last valid utxo set state pre fork.
- M is the first block you transacted in post fork on the Bitcoin side.

- Transacted means both sending and receiving coins.

Any chain tip between V and M has the utxo set state you need to move coins on the Bcash side of the fork (as there is no divergence in utxo sets yet in between the 2 sides of the fork).

So what you want to do is to feed Armory a copy of the chain with the tip belonging to [V, M[
1243  Bitcoin / Armory / Re: Using Armory on the BCH chain on: September 25, 2017, 02:53:07 PM
rev files shouldn't matter at all, they're only relevant for Core's reorgs. What you need to nuke as well is your ArmoryDB (if you're not going to use a fresh folder).

Also, there's no guarantee file 949 will do it for you. Core does not download files sequentially, there's a chance post fork blocks are embedded somewhere deeper down your blk files. This file # is only relevant if your node was running at the time of the fork. If you caught up for more than a day of blocks after the fork, chances are you have post fork blocks all over the place. Granted, the only thing you are actually after is the blk file with the first post fork block. If that one is taken out, Armory can't compute the chain past the fork point and you should be good.
1244  Bitcoin / Armory / Re: FRAGMENTED BACKUPS VULNERABILITY!! IF YOU USE THEM, READ THIS!! on: September 25, 2017, 02:50:05 PM
Non-random x values is probably relatively benign, since all information about the polynomial is in the f(x) values.  

You can choose any x as long as the coefficients are random. I don't quite see how randomizing x achieves anything, as you have to provide entire points (x, f(x)) as fragments, therefor x is public information. Maybe you were trying to say that the x values should be chosen at random, instead of say, as part of a sequence? I don't think that's relevant at all for SSS, since all operation are performed on a finite field.


Quote
Is there any way I can see on my 3-of-5 backup if it was made before or after that update was made?  I am of course going to move my funds, but doing so before a new 3-of-5 backup is in place is also a risk.

The coefficients are 32bytes, therefor your backup would have 5 lines (1 header line + 4 * (16bytes of data + 2bytes of checksum) lines) per fragment. If the fragment only has 1 header and 2 data lines, it's using implicit [1,...,N] for x.

Quote
Thus you only need N/2 sheets to recover the key !!!.  The seed can be recovered from a single sheet of what was supposed to be a 2-of-M backup, or from two sheets of a 3-of-M or 4-of-M backup, etc etc.   Shocked

It's far worse than that. Look at how the coefficients are constructed, they're hashes of the previous coefficient. Whatever fragment the attacker gets access to, he will be able to compute all following coefficients just by hashing x once.

Read this again and look at the code again. In the first scheme, anyone with access to a fragment that carries the first coefficient can compute *ALL* all of the coefficients. The single point he has with the fragment is then enough to reveal the secret.

This is slightly worsened by the fact that since there are more fragments than coefficients, certain coefficients reoccur on several fragments. Now people being people, when they distribute fragments they tend keep the first few for themselves, by happenstance. If the coefficients were distributed in order with fragments, this very human yet entirely irrelevant quirk would actually save these users from total disaster. Again, the order of coefficients matter since they are derived consecutively from one another
1245  Bitcoin / Armory / Re: ArmoryDB exits silently upon synchronization on: September 24, 2017, 06:05:26 PM
It's scanning two address sets independently for whatever reason. Do a rebuild & rescan.
1246  Bitcoin / Armory / Re: Using Armory on the BCH chain on: September 24, 2017, 05:11:14 PM
Read the "Tricks" section
1247  Bitcoin / Armory / Re: Armory 0.96.3 released on: September 24, 2017, 04:08:55 PM
I managed to nail it down to calling with --ram-usage=1. When I don't use the parameter, it scans tx history just fine. Please have an attempt to repro this.

Hmm that sounds unlikely but I will definitely test it out.

Quote
Several hours later, I am at this point able to tell that removing --ram-usage fixes all the issues and building completes with success. Looking at the memory usage, I'm not even noticing any differences(stays at around 700-900MB). Is that parameter still valid even or is it just legacy at this point?

This parameter controls the write queue depth for scan threads. It's only one element that controls total RAM footprint for scans, which would be broken down to these items:

- size of batches (hardcoded here: https://github.com/goatpig/BitcoinArmory/blob/master/cppForSwig/BlockchainScanner.h#L24)
- batch queue depth
- constant overhead from the container objects
- speed at which your system goes through the batches

After a substantial rework of the scanning queu mechanism, this parameter doesn't have much effect on full node scans as they're leaner, nor is it really necessary anymore. It is very significant for supernode however.
1248  Bitcoin / Armory / Re: ArmoryDB exits silently upon synchronization on: September 23, 2017, 10:14:30 PM
Use pastebin, my browser blocks that domain.
1249  Bitcoin / Armory / Re: Armory 0.96.3 released on: September 23, 2017, 10:13:15 PM
Something is off with how you pass your custom arguments. The cookie arg is not mandatory, if you start an instance of the DB and give that arg to your client you should be fine. Are you using something else than 127.0.0.1? Is localhost resolved to some other IP? You need to list out all parts of you setup that deviate from default settings and isolate them one after another.
1250  Bitcoin / Armory / Re: Armory 0.96.3 released on: September 23, 2017, 08:13:27 PM
ArmoryDB is running on localhost with no exotic IP:port setup. But it is running independently. What commandline should Qt use to connect?

It should find the DB instance by default.
1251  Bitcoin / Armory / Re: Armory 0.96.3 released on: September 23, 2017, 07:35:06 PM
ArmoryQt will first look for a DB on the given IP:port. If it can't find one, it will spawn a DB. If you are running a DB but ArmoryQt can't find it, you aren't giving it the proper IP:port to look at.
1252  Bitcoin / Armory / Re: Armory 0.96.3 released on: September 23, 2017, 07:05:25 PM
since when is DB managed by Qt???

Which Qt? BitcoinQt or ArmoryQt?

Quote
-INFO  - 19:25:25.437: (..\BlockchainScanner.cpp:52) no history to scan

That means the DB has seen a new block but it has not address data to scan against it.
1253  Bitcoin / Armory / Re: Armory 0.96.3 released on: September 23, 2017, 04:42:43 PM
There was no issue with the block files. Something in your setup is preventing the DB from progressing forward. You're going to have to get me some debug data.

You can setup a Ubuntu VM and run the DB in there, or make in a different version of Windows, while your client remains on the host, or you need to whip out MSVC, build a debug version of the DB and set some key breakpoints to figure out what's going on. Pick your poison. I'm out of idea about how to do this easily.
1254  Bitcoin / Armory / Re: Armory 0.96.3 released on: September 23, 2017, 03:14:54 PM
Hashes are pointless, that data is not downloaded sequentially, no one has the same hash for this stuff.

I have no idea why your setup does not work. You're the only person with this issue so I have no clue to go with. You could give me the full log of a fresh DB bootstrap one last time, but if that doesn't yield anything, at some point I'm going to need to see some backtraces or look at your block files. Later is easier to deliver.
1255  Bitcoin / Armory / Re: Armory 0.96.3 released on: September 23, 2017, 02:53:46 PM
Sounds like your blockchain data is corrupt. Probably somewhere in the first 5 or so files. Could you upload these somewhere so that I can take a look?
1256  Bitcoin / Armory / Armory 0.96.3 released on: September 22, 2017, 05:04:05 PM
Notes

This is a point release to fix a vulnerability with fragmented backups. If you are using fragmented backups on for your wallets, read the dedicated topic:

https://bitcointalk.org/index.php?topic=2199659.0

Binaries

https://github.com/goatpig/BitcoinArmory/releases/tag/v0.96.3

Changelog

Quote
== Vulnerability Fix ==
   - Fragmented backups were using a faulty implementation of Shamir's Secret Sharing (SSS).
     One of the requirement of SSS security parameters is that the coefficients of the curve are chozen randomly. The implementation
     up to this point was deriving these coefficients deterministically.

   - While it is hard to determine how far the deterministic coefficient generation erodes the security of SSS, and how exploitable
     the vulnerability is, the recommendation for users of fragmented backups is to treat the wallets backed up in this fashion as
     compromised and to migrate all funds to a new wallet.

   - The fragmented backup code now properly randomizes the SSS coefficients. Fragmented backups created with version 0.96.3 and later
     are safe to use.

   - The result of this change is that fragmented backups will no longer be deterministic. The previous behavior guaranteed a given
     wallet will always return the same set of fragments for a given M-of-N scheme. Since it deteriorates SSS security properties,
     the behavior has to be rolled back.
   - Fragment sets are now generated randomly, therefor an unique ID has been added to each set to identify them. You cannot mix
     and match sets.
   - While Armory can no longer generate deterministic fragments, it can still restore wallets from deterministic fragments.

   - Many thanks to Gregory Maxwell (greg@xiph.org) for identifying and reporting the vulnerability as well as reviewing the fix.

== Fixed ==
   - Fixed faulty version packet deserialization revealed by Core 0.15.0.1
1257  Bitcoin / Armory / FRAGMENTED BACKUPS VULNERABILITY!! IF YOU USE THEM, READ THIS!! on: September 22, 2017, 05:03:40 PM
TLDR:

Fragmented backups in Armory use a broken implementation of Shamir's Secret Sharing (SSS). All users of fragmented backups need to treat the wallets backed up in this fashion compromised and migrate all funds to a fresh wallet.

1) The story

A couple days ago I was warned by Gregory Maxwell that the implementation for Shamir's Secret Sharing Armory uses is broken. After reviewing the code in question, I've concluded the implementation of SSS introduces a vulnerability in fragmented backups that requires not only an immediate fix but also a community wide warning to alert all fragmented backup users.

These users need to sweep all funds from these wallets. All version starting 0.96.3 will have the vulnerability fix.

2) A high level look at SSS

SSS is a scheme that takes a secret and outputs a set of N fragments. M of these fragments are sufficient to reconstruct the secret.

In order to do so, you first constructs a polynomial f of degree M over a finite field, in which M-1 of the coefficients are random values, and the last one is the secret itself. You then compute N points of f. Each point, which is a pair (x, f(x)), constitutes a fragment.

To reveal the secret, you need to reconstruct the polynomial, which you do by interpolating M points together.

I will not go into the details of the interpolation, but one element here is crucial: The effect of a partial interpolation is to narrow down the possible candidates for valid coefficients. In other words, the more points on the curve you interpolate, the clearer a picture of the curve you get. The curve is constructed around your secret, a more refined interpolation results in an ever increasing leak of information, to the point where you can brute force the secret.

This property of the interpolation is counteracted by the use of a finite field. Since all points generated with the polynomial belong to the underlying finite field, the cyclic nature of finite fields widens the range of possible solutions that would yield your particular polynomial from a partial interpolation (by making them infinite), in such a way that you simply cannot brute force your way out of the interpolating at least M points.

A simple way to demonstrate that property is as follows:

Consider the equation:

Code:
8 = 5 + x

It's obvious that x is 3, and that this is the only solution. Now add this variation to the equation:

Code:
8 = (5 + x) mod 11

Suddenly x can be 3, 14, 25, 36... and so on. There is now an infinite amount of solutions for x, due to the introduction of the modulo operation. This is essentially the effect the finite field has on the interpolation equations.

3) The vulnerability

The whole point of the previous section was to demonstrate how SSS is constructed to prevent partial information leaking. One of the requirements to insure that property is that all of the polynomial coefficients but the secret are chosen at random.

This is precisely what Armory's implementation breaks, by rolling deterministic and chained coefficients, where the first coefficient is deterministically derived from the secret, and the subsequent ones are derived successively from one another.

If you look at it in the context of the previous paragraph, where SSS was crafted to deny any alley to gather information about coefficients short of M fragments, the newly introduced deterministic relationship between all coefficents provides a path that potentially dumbs down the security of SSS to that of the hash function used to generate the coefficients.

This can ultimately lead to a subset of fragments leaking information about coefficients where none should be.

4) In the code

Code wise, there were 2 instances of this implementation, both of which are faulty.

a) The first version

It did 2 things wrong, one more aggravating than the other:

The coefficients were not picked at random, instead they were derived as hashes of the secret, in a fashion that boils down to this:

Code:
coefficient_0 = hash(secret) 
coefficient_(n+1) = hash(coefficient_n)

The fragments themselves were constructed as the following points on the curve:

Code:
fragment_n = (coefficient_n, f(coefficient_n))

There are 2 important issues with this setup. First of all, the coefficients are not selected at random, therefor it breaks SSS security assumptions.

Next, and most aggravating, the coefficients were provided as is on the fragments, since you need to provide the pair (x, f(x)), and x in this case were the actual coefficients.

Note that since the coefficients were derived from each other, the second mistake is twice as aggravating, as anyone with fragment n would have coefficient n and could derive all coefficients n+1.

The second mistake was caught and fixed eventually. I was not involved with anything regarding cryptography and security in Armory at the time, therefor I have no recollection of the event. I expect there was a write up of the issue and people were told to cycle wallets, but I can't remember any of it.

You can look at the code in its original form here:
https://github.com/etotheipi/BitcoinArmory/commit/80e373a#diff-27fe88d2c6032fecb93912a17d72081bR1615

b) The second version

The second version only made sure points were generated using the [1...N] sequence for x instead of coefficient themselves. At this point fragmented backups where no longer as broken but the code still did not implement SSS correctly.

Notably the assumption that no amount of fragments less than M can leak any data about the secret is not true with that faulty implementation.

One way to look at it is that this implementation introduces a deterministic relationship between the coefficients in a way that it can effectively reduce the security of the system to that of a single pass of HMAC512 provided an amount L of fragments, with L < M.

Here is the commit for the second version:

https://github.com/etotheipi/BitcoinArmory/commit/0824b632600116bd6395cec939fa6fd398efeb19#diff-27fe88d2c6032fecb93912a17d72081bR1915

5) Affected versions

Fragmented backups were introduced in version 0.88 (04/18/2013) and the first fix was deployed in version 0.90 (11/26/2013).

The final fix was introduced in v0.96.3 (9/21/2017)

6) The fix

The coefficients were made deterministic in order to present deterministic fragments to the user when fragmenting a wallet over a given scheme. In other words, fragmenting wallet W over a M-of-N scheme would always yield the save fragment values for the same fragment index.

This introduces scenarios where any amount of fragments can be recomputed from the private wallet root without invalidating fragments still in the wild.

The fix was to randomize the coefficients at the cost of the deterministic characteristic of the fragments.  The choice was fairly simple:

a) The deterministic attribute gained by bastardizing SSS is worthless in the face of how it damages its security properties.

b) Even if a change to SSS is designed so that it does not so obviously erode its security properties, this is still an act of rolling a custom cryptographic function, which commands a level of review and security analysis that will most likely not be performed at the adequate level.

c) There are no scenarios I can think of in which the feature of determinism in fragments is actually necessary and central to this type of backups. Introducing it at the cost of security is therefor doubly unacceptable. Backups are supposed to be forever afterall, lacing solid crypto with any kind of bootlegged algorithm does not stand to reason.

Therefor, it is without hesitation that the faulty feature was undone, and the faulty code removed from the repo so as to prevent unaware users from copying it into their own projects.

The changes can be seen here:

https://github.com/goatpig/BitcoinArmory/commit/94d2a7556d25cf788da639d81a7162694982f6b7
https://github.com/goatpig/BitcoinArmory/commit/7bd9887891ac88e2e49954ef034bedef88f23eaf

7) GUI changes

Since the fragments are not deterministic anymore, they are now generated with a unique set ID which is reflected on the backup strings and printed backups. Fragments are only useful within a their own set. Another way to put it is that you cannot mix and match fragments from different sets. This is the only difference between the pre and post fix implementation.

The fixed version is compatible with fragments generated from the deterministic version. You will still be able to restore from these with version 0.96.3+

8 ) Recommendations

It is hard to say exactly how effectively this custom take on SSS breaks security at the fragment level. How fewer fragments would it take to reproduce a secret than intended? Honestly, I don't know, but while the first implementation was effectively breaking all security assumptions of SSS, the second version is SSS at least theoretically.

I don't expect an attacker can snatch a single fragment and reveal the secret the next minute with minimal code. This vulnerability reduces the overall complexity of the problem that of a hash function, it doesn't outright bypass all complexity.

Since we're talking 32 byte integers, breaking the scheme isn't trivial, but it has certainly been weakened to a state that is difficult to precisely assess. Therefor, to remain on the conservative side, my recommendation is as follows:

If you created a fragment backup of your wallet, consider that wallet compromised. Create a new wallet and sweep the funds from the compromised wallet to the new one. You can redo your fragmented backup scheme on the new wallet provided you use Armory 0.96.3 and newer.

If you do not use fragmented backups, you have nothing to do.

9) Notes

Special thanks to Gregory Maxwell for finding the vulnerability and helping with the review of the fix.

1258  Bitcoin / Armory / Re: Armory 0.96.2 is out (SegWit enabled) on: September 22, 2017, 11:38:37 AM
Try without your node running.
1259  Bitcoin / Armory / Re: ArmoryDB exits silently upon synchronization on: September 21, 2017, 04:00:24 PM
1) Don't use Core 0.15.0.1 until the fix is out.

2) Delete your logs, start Armory, post the fresh logs.
1260  Bitcoin / Armory / Re: ArmoryDB exits silently upon synchronization on: September 21, 2017, 02:54:23 PM
Post your logs.
Pages: « 1 ... 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 [63] 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 ... 233 »
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!