Bitcoin Forum
July 05, 2024, 02:01:41 AM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: « 1 [2]  All
  Print  
Author Topic: Can we validate all the blocks created till date with the first Satoshi client?  (Read 1507 times)
gmaxwell
Moderator
Legendary
*
expert
Offline Offline

Activity: 4200
Merit: 8442



View Profile WWW
September 04, 2022, 02:58:21 PM
Merited by pooya87 (2)
 #21

It's probably better to run with an old openssl... god knows what other consensus relevant changes they've made.
PowerGlove
Hero Member
*****
hacker
Offline Offline

Activity: 534
Merit: 4358



View Profile
September 04, 2022, 10:32:09 PM
Merited by vapourminer (2)
 #22

It's probably better to run with an old openssl...
Yep, that might have been easier than patching the source. It's not a bad approach, however, if you have the time and interest. I imagine that manually patching each encountered issue will be pretty educational.

[...] god knows what other consensus relevant changes they've made.
Can anyone elaborate on this? I find the wording a wee bit alarming. If someone as knowledgeable as gmaxwell is hazy on exactly what consensus changes might be caused by running with different versions of a dependency, then isn't there something wrong in how the software is put together?

The only sensible design choices are (I would think) to either roll your own stuff or to only call third-party libraries through carefully designed wrappers to prevent (minimize, more likely) unwanted consensus changes from sneaking in. Or is this a matter of older versions getting this wrong and newer versions getting it right?
gmaxwell
Moderator
Legendary
*
expert
Offline Offline

Activity: 4200
Merit: 8442



View Profile WWW
September 05, 2022, 01:30:50 AM
Merited by pooya87 (4), PowerGlove (4), vapourminer (3), n0nce (2)
 #23

Can anyone elaborate on this? I find the wording a wee bit alarming. If someone as knowledgeable as gmaxwell is hazy on exactly what consensus changes might be caused by running with different versions of a dependency, then isn't there something wrong in how the software is put together?

The only sensible design choices are (I would think) to either roll your own stuff or to only call third-party libraries through carefully designed wrappers to prevent (minimize, more likely) unwanted consensus changes from sneaking in. Or is this a matter of older versions getting this wrong and newer versions getting it right?

The latter: recent versions don't use OpenSSL at all (or didn't last I checked), and it hasn't been used for consensus since 0.12 in 2016.

Back when Bitcoin was written OpenSSL had been very stable for a looong time and had a good track record of preserving legacy interfaces and behaviors when it changed.  Satoshi could be forgiven for thinking it was sufficiently stable, particularly because it was far from clear how tricky consensus consistency was.  Your use of the word "minimize" makes it clear you know, but it wasn't always quite so obvious how truly difficult it was.

In 2015/2016-ish a number of concerning vulnerabilities were found in OpenSSL and a lot of attention was put on the project massively reworking things, making incompatible interfaces changes, and putting out giant updates (I seem to recall one of their diffs was like 400k lines...).  They managed to make changes in signature validation that would have been utterly consensus breaking for Bitcoin, and did so without even release note-ing the change.

Fortunately by then Bitcoin had mostly completed the move off of it and had deployed a soft-fork to restrict the set of accepted signatures going forward enough that their consensus break wouldn't matter for newly created blocks, as a result of an openssl consensus consistency vulnerability we'd discovered.  Though there is a window of openssl versions + bitcoin versions where it will compile and run without issue then reject some older blocks due to openssl's changes, this footgun is mostly avoided due to the fact that openssl broke their api a number of times around the same time.

It could have been quite a mess, had the openssl behavior changed before bitcoin devs realized it was a potential source of issues. Today AFAIK bitcoin uses no external dependencies in consensus beyond the programming language itself. Maybe some small amount of boost usage still exists in consensus, though there was a long term effort to eliminate it, simply because the authors of external code can't be expected to manage the very special requirements of consensus consistency.

The patches people use to use modern openssl with old bitcoin code paper over the major known consensus change in openssl, but there isn't any guarantee that there haven't been other ones or that its a complete fix (since even at the time openssl made those changes the 'fix' was to upgrade to bitcoin that didn't use it for consensus).  Not that I think it's too likely that there will be extra issues, but if the exercise is to tell if old versions would work, you should reconize that if you use an inauthentic openssl you're only getting an approximate answer.  

Because post-BIP66 bitcoin is very exacting in the signature encodings it will accept, I think it's unlikely that any new signatures have been accepted that really old openssl would reject-- though not impossible: we did also find bugs in openssl's bignums, just none that we could find a way to exploit for a consensus split.  It's more likely that if there were issues it would be of the form of more modern openssl being even more picky and rejecting the chain when old openssl wouldn't have.
PowerGlove
Hero Member
*****
hacker
Offline Offline

Activity: 534
Merit: 4358



View Profile
September 05, 2022, 05:46:50 AM
 #24

@gmaxwell: Thanks, man. What a great answer! I always enjoy reading your posts. You're an asset to this forum and if 1 out of every 100 members had half as much knowledge as you do, I'd be logged in 24/7 Smiley
ABCbits
Legendary
*
Offline Offline

Activity: 2926
Merit: 7626


Crypto Swap Exchange


View Profile
September 05, 2022, 12:21:58 PM
Merited by vapourminer (3)
 #25

In 2015/2016-ish a number of concerning vulnerabilities were found in OpenSSL and a lot of attention was put on the project massively reworking things, making incompatible interfaces changes, and putting out giant updates (I seem to recall one of their diffs was like 400k lines...).

I spend few minutes to find that commit. While i didn't one with 400k lines change, i found one with 292K addition and 293K deletion[1]. The commit message says it's just reformatting source code, although how many people verify it. But submitting single big commit seems to be accepted by OpenSSL. Here's my finding,
  • 420 commit with 1000-9999 line addition
  • 20 commit with 10000-99999 line addition
  • 3 commit with >=100K line addition[2], although first appearance is migration to Git or GitHub

Anyway, i'll leave command/regex i used to get and filter commit history.

Code:
git clone https://github.com/openssl/openssl
cd openssl
git config --global diff.renameLimit 1000000
git log --format=format:"%H" --shortstat > commit.log
 [0-9]{6,} ins
 [0-9]{6,} del

[1] https://github.com/openssl/openssl/commit/0f113f3ee4d629ef9a4a30911b22b224772085e5
[2] https://github.com/openssl/openssl/commit/e613b1eff40f21cd99240f9884cd3396b0ab50f1
[3] https://github.com/openssl/openssl/commit/d02b48c63a58ea4367a0e905979f140b7d090f86

█▀▀▀











█▄▄▄
▀▀▀▀▀▀▀▀▀▀▀
e
▄▄▄▄▄▄▄▄▄▄▄
█████████████
████████████▄███
██▐███████▄█████▀
█████████▄████▀
███▐████▄███▀
████▐██████▀
█████▀█████
███████████▄
████████████▄
██▄█████▀█████▄
▄█████████▀█████▀
███████████▀██▀
████▀█████████
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
c.h.
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▀▀▀█











▄▄▄█
▄██████▄▄▄
█████████████▄▄
███████████████
███████████████
███████████████
███████████████
███░░█████████
███▌▐█████████
█████████████
███████████▀
██████████▀
████████▀
▀██▀▀
mukmaster
Newbie
*
Offline Offline

Activity: 2
Merit: 3


View Profile
September 06, 2022, 09:37:38 AM
 #26

It's probably better to run with an old openssl...
Yep, that might have been easier than patching the source. It's not a bad approach, however, if you have the time and interest. I imagine that manually patching each encountered issue will be pretty educational.

This was quite valuable for my own learning. Just my 2 sats.
Pages: « 1 [2]  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!