why is core development and hosting centralized?

[got_eth]

The latest fiasco with 0.16.3 has got me thinking.

The sha256sum asc keys for the binaries are hosted on bitcoin.org
The binaries are hosted on bitcoin.org.
The torrent for the binaries are hosted on bitcoin.org.

--->How can I positively confirm I'm getting an untainted binary given all the data is hosted on centralized bitcoin.org?

I've never looked deeply at the bitcoin core code until this fiasco. But yesterday for the first time i went thru the github commit log...Why are there 1000+ commits for a consensus issue? I see there were tons of changes related to improving code quality. Feels a bit rushed, I would have expected this release to be only a bug fix but it seems a lot more was done. Who is deciding what goes into the latest releases?

Am I insane for thinking I should my own private fork of bitcoin core so that I can validate what changes are pushed to my client? It's way too much work and beyond my abilities, but this experience has left a sour taste.

[1715140117]

1715140117

[1715140117]

1715140117

[1715140117]

1715140117

[achow101]

The latest fiasco with 0.16.3 has got me thinking.

The sha256sum asc keys for the binaries are hosted on bitcoin.org
The binaries are hosted on bitcoin.org.
The torrent for the binaries are hosted on bitcoin.org.

They are hosted on both bitcoin.org and bitcoincore.org. Bitcoincore.org is the Bitcoin Core project's website.

Also, sha256sum.asc is not a key, it is a signed text file.

--->How can I positively confirm I'm getting an untainted binary given all the data is hosted on centralized bitcoin.org?

First of all, the sha256sums.asc file is signed by Wladimir's release signing key. This key is held by Wladimir and him only. Presumably it is held safely. You can check that the binaries you downloaded have the hash that is listed in the sha256sums.asc file, then you can verify the GPG signature on that file. If the file is valid, it will be signed by Wladimir's release signing key. If Bitcoin.org or bitcoincore.org were compromised and the uploaded binaries changed, the sha256sums.asc file would not have a valid signature from Wladimir unless it was Wladimir himself who was the attacker.

Furthermore, Bitcoin Core uses a deterministic build process. This means that anyone can build the same code again using that build process and get exactly the same binaries. Before releases are officially announced and published, multiple people perform these deterministic builds and the hashes of their results are checked to make sure that everyone has built the same binary. These hashes, and GPG signatures over the hashes, are all available here: https://github.com/bitcoin-core/gitian.sigs. If you don't trust the uploaded binaries, you can check it's hashes against those from the other deterministic builders and the GPG signatures of people who did the deterministic build. Lastly, you can do the build yourself and check that what you built matches what was published.

I've never looked deeply at the bitcoin core code until this fiasco. But yesterday for the first time i went thru the github commit log...Why are there 1000+ commits for a consensus issue?

Where do you see this? You are likely looking at the wrong thing or are confused about some aspect of GitHub's UI. If a pull request has several thousand commits (or even more than a few tens of commits), it probably won't be merged as it is too large.

I see there were tons of changes related to improving code quality. Feels a bit rushed, I would have expected this release to be only a bug fix but it seems a lot more was done.

It was just a bug fix.

If you are looking at the difference between 0.17.0rc4 and 0.16.2, of course there will be lots of changes. 0.17.0 is a major release that introduces new features and other bug fixes. Bitcoin Core is not just bug fixes, there's new stuff constantly being introduced.

Who is deciding what goes into the latest releases?

The maintainers do.

Am I insane for thinking I should my own private fork of bitcoin core so that I can validate what changes are pushed to my client? It's way too much work and beyond my abilities, but this experience has left a sour taste.

You don't need to fork. You can watch the Bitcoin Core repository yourself and review pull requests and the things that are being merged. You can also just not upgrade if you think that things are going too quickly, but be aware that there are many bug fixes with every release that you may not have.

[Carlton Banks]

--->How can I positively confirm I'm getting an untainted binary given all the data is hosted on centralized bitcoin.org?

do a gitian build (a reproducible build system), then check that the hash is the same as the binaries on https://bitcoin.org

yesterday for the first time i went thru the github commit log...Why are there 1000+ commits for a consensus issue?

You're looking at the wrong branch, 0.16.3 only changes 1 value, a literal boolean in a condition statement from "false" to "true".


Before:

Code:

        if (!CheckTransaction(*tx, state, false))

       
After:

Code:

        if (!CheckTransaction(*tx, state, true))

[vit05]

The latest fiasco with 0.16.3 has got me thinking.

Please avoid spreading FUD on such a complicated subject. 0.16.3 was not a fiasco. Talking like this may cause people to waste time researching a possible new problem, which until today does not exist.

0.16.3 was a bug fix.

The failure lasted from 0.14.0 to 0.16.2. And it has never been exploited.

[Kakmakr]

Do we have some kind of redundancy or backup plan, if Wladimir's release signing key is compromised? I know some other people had commit access back when Gavin's access was revoked, so I presume that some of the other Core developers have the same "rights" / keys to sign releases, if Wladimir's key is compromised or if he turns rogue.  

We know some governments can go to extremes to push out the competition or to maintain their power. <Putin is a good example>  

Huge responsibility and risky position to be in, so I know it is a sensitive subject. 

[achow101]

Do we have some kind of redundancy or backup plan, if Wladimir's release signing key is compromised? I know some other people had commit access back when Gavin's access was revoked, so I presume that some of the other Core developers have the same "rights" / keys to sign releases, if Wladimir's key is compromised or if he turns rogue.  

The release signing key and commit access are two unrelated things.

If the release signing key is compromised, then a new one will be created and the original one revoked. Then people will just need to know that there is a new release signing key (it would be announced in many different places, including here) and to verify releases with the new key instead of the old one. The release signing key has changed before, first when Wladimir took over releases from Gavin, and later when Wladimir created a release specific key instead of his own GPG key.

If Wladimir went rogue or he stopped doing releases, then someone else would begin doing releases and they would create a new release signing key that would be announced and used just like of the current key were compromised.

[gmaxwell]

Similarly if the repo on github went rogue we'd just move to another one and announce the new location. We already even have a backup on github too ( https://github.com/bitcoin-core/ ), though if github itself went rogue we'd need to put one elsewhere.

[cellard]

Most core devs have accounts here, and have also twitters, why not post the signed stuff, checksums etc in there too? the more diverse points of failure the better, I would feel safer that way than sticking to one or two websites.

Also will the older versions be patched from the bug and released (I mean 0.14.X up to 0.16.2) or people will be forced to upgrade to 0.16.3+?

[achow101]

Most core devs have accounts here, and have also twitters, why not post the signed stuff, checksums etc in there too? the more diverse points of failure the better, I would feel safer that way than sticking to one or two websites.

I'll start adding the contents of the sha256sums.asc files to the bottom of my release posts.

Also will the older versions be patched from the bug and released (I mean 0.14.X up to 0.16.2) or people will be forced to upgrade to 0.16.3+?

Yes. 0.14.3 and 0.15.2 (patched versions for 0.14 and 0.15) will be released soon (should be a day or two). There are enough gitian builds for them, Wladimir was just traveling recently and did not have access to his release signing key.

[gmaxwell]

or people will be forced to upgrade to 0.16.3+?

As noted above, they're not.  But you really should as there are other issues fixed in newer versions that aren't fixed in older versions.

Is there any reason you wouldn't want to upgrade from 0.14.x or 0.15.x to 0.16.3?

[Kprawn]

Similarly if the repo on github went rogue we'd just move to another one and announce the new location. We already even have a backup on github too ( https://github.com/bitcoin-core/ ), though if github itself went rogue we'd need to put one elsewhere.

What is the backup plan for Github after the rumours that Microsoft were planning to buy the site? (I know a lot of people were

worried about that) Can governments restrict or mandate that certain software like Bitcoin source code, may not be hosted or

stored on these sites? Why are we depending on "external" sources for redundancy and not purely on our own independent

sites? Will the lead maintainer be the announcer if the repo has changed?

[TryNinja]

~

We can always just move to a different website, like Bitbucket, GitLab or SourceForge.

[bob123]

Most core devs have accounts here, and have also twitters, why not post the signed stuff, checksums etc in there too? the more diverse points of failure the better, I would feel safer that way than sticking to one or two websites.

You might feel safer, but in reality you wouldn't actually gain any safety/security.

As long as the checksum file is signed by the correct key AND the binaries you have downloaded match the checksum, you can be sure of having a non-compromised version.
Spreading the same file between 10 more sites doesn't add any security and/or safety. It just adds more hassle when a key has to be revoked (since there are more places to announce the revocation).

[jnano]

How come Gitian verifiers sometimes do the different build variants (win-signed, win-unsigned...) days apart?

What is the backup plan for Github after the rumours that Microsoft were planning to buy the site?

Not rumors, it's a done deal. The process just isn't fully complete yet.

[achow101]

How come Gitian verifiers sometimes do the different build variants (win-signed, win-unsigned...) days apart?

There are two stages to the gitian builds. First is to build linux, win-unsigned, and osx-unsigned. These resulting binaries are not code signed, i.e. there is no signature over the binary from a Microsoft or Apple blessed key (linux does not need code signing). In order to do the code signing, the unsigned binaries must exist first. So once there are sufficient gitian builds of the non-code-signed binaries, the people who hold the code signing keys produce detached signatures for Windows and MacOS using their respective code-signing keys. These detached signatures are then combined into the non-code-signed binaries produced earlier to create the code-signed binaries. This step uses the win-signed and osx-signed targets.

Since there are two phases to gitian builds and because a sufficient number of people must build before moving from the first to the second phase, there is usually a day or two of delay between the unsigned targets and the signed targets.

[Cøbra]

Having ~2 sites where the binaries are distributed is ideal, but no more. Having distribution centralized helps spot any compromised binaries quicker than if they were spread out between 10 different sites (it would take longer for anyone to detect malicious binaries on one site because users and eyeballs are spread out between sites). Probably even if you had 10 sites, most users would converge on 2 anyway.

I generally always recommend bitcoin.org as the best place to get the binaries. The number of users and businesses running scripts to fetch the binaries from there and check the integrity is considerable. For most users who don't check the signatures, it's the least risky place to get them.