Bitcoin Forum
June 17, 2021, 09:38:35 PM *
News: Latest Bitcoin Core release: 0.21.1 [Torrent]
 
   Home   Help Search Login Register More  
Pages: 1 2 [All]
  Print  
Author Topic: [RFC] Trusted build process  (Read 4245 times)
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1085


Chief Scientist


View Profile WWW
January 22, 2011, 04:50:33 PM
 #1

It is time to build and test a new version of bitcoin.

In the past, Satoshi built the Windows and Linux releases, Laszlo built the Mac releases, and we trusted them not to put malware in them (or we compiled ourself from source).

Satoshi is busy, and even if he wasn't he shouldn't be spending his time doing a job that a lot of the rest of us are capable of doing.  So we need a new process.

Ideally, that process should be open and verifiably trustworthy.  So I'd like to propose that we do the following:

1. For each platform, somebody creates a pristine, reproducible build environment, preferably as a virtual machine image that anybody can download, inspect, clone, run, etc.

Anybody should be able to reproduce the build environment by running or following a script (e.g. "Install Ubuntu X.Y.Z.  apt-get the following versions of the following packages... etc").

2. A copy of that virtual machine is used to build/package the release.

3. Anybody can audit the process by re-creating the build environment and ensuring that they end up with "identical" executables.  (where "identical" means compare the code in the executables, ignoring timestamps or other meta-info linkers put into executables -- are there already tools to do that, or do we need to roll our own?).

Feedback?  Suggestions for improvement, or are there better ways of creating 'trusted builds' ?

How often do you get the chance to work on a potentially world-changing project?
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1623965915
Hero Member
*
Offline Offline

Posts: 1623965915

View Profile Personal Message (Offline)

Ignore
1623965915
Reply with quote  #2

1623965915
Report to moderator
MoneyTree
Newbie
*
Offline Offline

Activity: 14
Merit: 0



View Profile
January 22, 2011, 06:10:29 PM
 #2

Hello Gavin,

totally agree. There should be a validated process to release new builds without being dependent on a single (or small group of) persons. This is also good for the continuity of the bitcoin project as a whole.

For me personally: The sooner the nolisten=1 config option is compiled into the windows binary the better!  Grin

LZ
Legendary
*
Offline Offline

Activity: 1722
Merit: 1035


P2P Cryptocurrency


View Profile
January 22, 2011, 09:50:02 PM
 #3

Good idea. Besides, I prefer that this image can be mounted, so I can just chroot to it. Smiley

My OpenPGP fingerprint: 5099EB8C0F2E68C63B4ECBB9A9D0993E04143362
devrandom
Newbie
*
Offline Offline

Activity: 26
Merit: 0



View Profile WWW
January 25, 2011, 03:47:06 AM
 #4

I started an initiative ( https://gitian.org/  ) for exactly this purpose.  My motivational article is here: http://hyper.to/blog/link/attack-scenarios-software-distribution/

What is needed is a deterministic build process.  I would love to collaborate on tools to achieve this.
Nefario
Hero Member
*****
Offline Offline

Activity: 602
Merit: 502


GLBSE Support support@glbse.com


View Profile WWW
January 25, 2011, 04:30:57 AM
 #5

This is only a little related to the above thread. Can we make building bitcoin a little easier? After finding the binary doesn't work on my server (amd64 ubuntu 8.10) I went to build and found out about all the libraries it needs (that are not available as apt-get for this distribution).

Probably not too important.

PGP key id at pgp.mit.edu 0xA68F4B7C

To get help and support for GLBSE please email support@glbse.com
ribuck
Donator
Hero Member
*
Offline Offline

Activity: 826
Merit: 1008


View Profile
January 25, 2011, 11:45:59 AM
 #6

...that process should be open and verifiably trustworthy ... a pristine, reproducible build environment, preferably as a virtual machine image ...

I think a virtual machine image is a bad idea, because a malicious person could include a modified tool within this virtual image. The C++ compiler, for example, could be modified for malicious purposes, and this might not be detected.

Surely it's safer to specify the build environment (i.e. version numbers and configuration of each tool) and leave others to assemble the build environment from those specifications.

Ken Thompson's very readable 1984 presentation describes this form of attack:

"Reflections on Trusting Trust"
http://cm.bell-labs.com/who/ken/trust.html
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1085


Chief Scientist


View Profile WWW
January 25, 2011, 03:58:14 PM
 #7

ribuck:  what do you suggest instead?

I've read Ken Thompson's paper; that is what motivated me to propose a reproducible virtual machine image as the best we can do IN PRACTICE.

Yes, it is theoretically possible somebody might sneak bitcoin-stealing code into the gcc compiler.  Code that detects that bitcoin is being compiled and injects instructions to send coins to the sneaky gcc hacker.

In reality, we trust that the gcc maintainers are trustworthy and careful and that they care about their reputations.   If you don't trust them, what is the alternative?

Nefario: can "we" make building bitcoin easier?  I've been asking people to submit patches to the bitcoin linux build process, but so far none have been forthcoming...

devrandom:  Nice!

How often do you get the chance to work on a potentially world-changing project?
ribuck
Donator
Hero Member
*
Offline Offline

Activity: 826
Merit: 1008


View Profile
January 25, 2011, 05:07:20 PM
 #8

Gavin, I think you misunderstood what I'm trying to say.

I'm not worried about the developers of gcc. I'm happy to trust them. I'm worried about the person who prepares the virtual image, and who could include their own patched malicious version of gcc.

What I suggest instead is simply to specify the build environment (tools, version numbers and configuration options) and allow others to set up that environment with software from standard repositories.

A virtual image of the build environment is not a big risk, but it seems to me it makes things slightly worse rather than slightly better.
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1085


Chief Scientist


View Profile WWW
January 25, 2011, 06:00:48 PM
Last edit: January 25, 2011, 07:16:35 PM by gavinandresen
 #9

ribuck:  I did misunderstand, sorry.

And maybe I should have been clearer:  the idea behind a reproducible virtual image is exactly to capture tools, version numbers, etc etc.   Auditors that create their own virtual images with the same CPU, OS, tools, etc. will be relied upon to confirm that yes, the bitcoin executables on the website were indeed built using standard compilers/linkers/etc from a particular source revision.


How often do you get the chance to work on a potentially world-changing project?
gigitrix
Hero Member
*****
Offline Offline

Activity: 697
Merit: 500



View Profile
January 25, 2011, 07:11:15 PM
 #10

This all sounds good, but for the ultraparanoid, get 2/3/n people to build on a given platform with an agreement on compiler and just compare the checksums, perhaps? Or am I missing a potential attack here?
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1596
Merit: 1015


View Profile
January 25, 2011, 07:25:13 PM
 #11

With modern packaging, checksumming is quite easy -- provided that prelink is disabled (prelink rewrites binaries).  So gavin's proposal seems reasonable, and reproducible.

Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
devrandom
Newbie
*
Offline Offline

Activity: 26
Merit: 0



View Profile WWW
January 25, 2011, 08:29:31 PM
 #12

The build environment can be specified as a VM build script.  Such as script will be relatively simple and easy to audit.

On Ubuntu/Debian, this could be done with a vmbuilder setup.  This would include vmbuilder config files and execscripts.  It would still be good to capture the set of packages installed, maybe with aptproxy or possibly just the version list and sha sums, so that the build is fully reproducible.

This of course assumes that the distributions are reliable, as we would depend on the distribution signature to certify that the distribution wasn't tampered with and is free of trojans.  In the future, it would be good to build an entire distribution from scratch and then use that as a trusted base.
ribuck
Donator
Hero Member
*
Offline Offline

Activity: 826
Merit: 1008


View Profile
January 25, 2011, 08:43:30 PM
 #13

The build environment can be specified as a VM build script.

That sounds like a good solution.

In the future, it would be good to build an entire distribution from scratch and then use that as a trusted base.

As the bitcoin economy grows, there may come a time when that makes sense, but I think it's a while in the future still.
devrandom
Newbie
*
Offline Offline

Activity: 26
Merit: 0



View Profile WWW
January 25, 2011, 08:54:25 PM
 #14

Just to list potential sources of non-determinism that I know of when building on linux:

  • zip/gzip (timestamps)
  • tar (timestamps)
  • dependencies (libraries, headers)
  • compilers
  • things I am forgetting right now Wink

For other platforms there might be additional sources.  For things like zip/gzip/tar, it would be best to have a fixup tool that sets the timestamps to an agreed value.  For dependencies and compilers, we need a reference build platoform, such as a VM build script.

In the future, it would be good to build an entire distribution from scratch and then use that as a trusted base.
As the bitcoin economy grows, there may come a time when that makes sense, but I think it's a while in the future still.

My initiative (mentioned above) is for software distribution as a whole.  For that, it would be good to based all software distribution on multiple code signers.  For bitcoin itself, I agree that it's not yet appropriate, since there are easier attack vectors at this point.
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1596
Merit: 1015


View Profile
January 25, 2011, 09:47:08 PM
 #15

Just to list potential sources of non-determinism that I know of when building on linux:

  • zip/gzip (timestamps)
  • tar (timestamps)
  • dependencies (libraries, headers)
  • compilers
  • things I am forgetting right now Wink


We should not be building anything except bitcoin itself.  Choose a distro with checksum'd packages, and you wipe out almost everything on your list.

Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
ptd
Member
**
Offline Offline

Activity: 114
Merit: 10


View Profile
January 26, 2011, 09:34:25 AM
 #16

Just to list potential sources of non-determinism that I know of when building on linux:

  • zip/gzip (timestamps)
  • tar (timestamps)
  • dependencies (libraries, headers)
  • compilers
  • things I am forgetting right now Wink


We should not be building anything except bitcoin itself.  Choose a distro with checksum'd packages, and you wipe out almost everything on your list.

Distro's use cryptographically signed packages rather than checksummed ones.
bitcoinex
Sr. Member
****
Offline Offline

Activity: 350
Merit: 250


probiwon.com


View Profile WWW
January 26, 2011, 01:25:16 PM
Last edit: January 26, 2011, 04:35:50 PM by bitcoinex
 #17

The Linux distributions have a maintainers who are preparing to build sources. If you 'aptitude source package-name' a source package and compile it after checking src signature by yourself it's safe enough.

There is, however, the probability of a "third level" trojan , which can not be detected if has not been investigated source of bitcoin and source of compiler, but it is too high level matter and is can not resolved at this level of abstraction.

Also, I urge DO NOT use digital signatures in the packages because it do not give the opportunity to build two identical package, because signature are uses random numbers when creating. Instead, repository must be signed.

We should achieve a sustainable build of packages for most Linux distributions ubiquitous and discontinue to include binaries into tar archives since it is insecure.

What do users of windows and mac I do not know. I am feel sorry for them. Smiley Most likely Gavin's proposal it is for them. (about what it needed to establish some virtual environment and that everyone can verify that it works correctly)

Another idea: this is a kind of philosophical problem can be published on the various IT sites, cause a discussion that may help and also thus attract new bitcoin users.

New bitcoin lottery: probiwon.com
- Moжeт, ты eщё и в Heвидимyю Pyкy Pынкa вepyeшь? - Зaчeм жe вepoвaть в тo, чтo мoжнo нaблюдaть нeпocpeдcтвeннo?
devrandom
Newbie
*
Offline Offline

Activity: 26
Merit: 0



View Profile WWW
January 26, 2011, 05:49:39 PM
 #18

The Linux distributions have a maintainers who are preparing to build sources. If you 'aptitude source package-name' a source package and compile it after checking src signature by yourself it's safe enough.

There is, however, the probability of a "third level" trojan , which can not be detected if has not been investigated source of bitcoin and source of compiler, but it is too high level matter and is can not resolved at this level of abstraction.

The threat model that I think we could target here is a compromise of the build process, either by the builder or by a third party attacking their build environment.  An assertion from multiple builders that [source checksum, build script, build environment checksum] produces [executable checksum] gives high assurance that no such compromise occurred.

I'm going to spend some time on this for the linux environment in the next few days.
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1596
Merit: 1015


View Profile
January 26, 2011, 05:54:09 PM
 #19

Distro's use cryptographically signed packages rather than checksummed ones.

Don't be so literal Smiley  "Checksum" is convenient shorthand.  You see this with the "sum" prefix in binaries named "md5sum", "sha1sum", etc.

Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
devrandom
Newbie
*
Offline Offline

Activity: 26
Merit: 0



View Profile WWW
January 30, 2011, 11:57:03 PM
Last edit: January 31, 2011, 05:03:13 AM by devrandom
 #20

I have an initial implementation of the VM based build process.

The code is here: https://github.com/devrandom/gitian-builder

You will also need a build descriptor file, which is here: https://gist.github.com/803438 .  The file wxWidgets-2.9.1.tar.bz2 goes in the ./inputs directory.

This builds a 64 bit executable with an sha256sum of 53c586ca76d6548a722c11ab3e51936d2bc185d60cd7234314896aa0c1ae5d1e.  Let me know if you get anything different.  My host is Ubuntu Maverick, but this should not matter to the result.  The environment report is captured in the result directory.

Still to do:

* Generate a tar.gz file instead of just an executable.  This will require fixing the timestamps and compiling for 32 and 64 bits.
* Pin the deb package versions so that future upgrades to the target OS don't cause changes to the result.
bitcoinex
Sr. Member
****
Offline Offline

Activity: 350
Merit: 250


probiwon.com


View Profile WWW
January 31, 2011, 07:22:09 AM
 #21

The Linux distributions have a maintainers who are preparing to build sources. If you 'aptitude source package-name' a source package and compile it after checking src signature by yourself it's safe enough.

There is, however, the probability of a "third level" trojan , which can not be detected if has not been investigated source of bitcoin and source of compiler, but it is too high level matter and is can not resolved at this level of abstraction.

The threat model that I think we could target here is a compromise of the build process, either by the builder or by a third party attacking their build environment.  An assertion from multiple builders that [source checksum, build script, build environment checksum] produces [executable checksum] gives high assurance that no such compromise occurred.

I'm going to spend some time on this for the linux environment in the next few days.


As far as I know, linux distributions are builds a packages on multiple machines in parallel, and the identity of resulting packets is checked afterwards. It is necessary to avoid errors of the hardware.

That is why:
Quote
Also, I urge DO NOT use digital signatures in the packages because it do not give the opportunity to build two identical package, because signature are uses random numbers when creating. Instead, repository must be signed.

New bitcoin lottery: probiwon.com
- Moжeт, ты eщё и в Heвидимyю Pyкy Pынкa вepyeшь? - Зaчeм жe вepoвaть в тo, чтo мoжнo нaблюдaть нeпocpeдcтвeннo?
bitcoinex
Sr. Member
****
Offline Offline

Activity: 350
Merit: 250


probiwon.com


View Profile WWW
January 31, 2011, 07:42:34 AM
Last edit: January 31, 2011, 08:01:50 AM by bitcoinex
 #22

Code:
$ git tag -l
help

Why help?
We must try to use signatures in the git commits. Gavin could do this right now by command like:

Code:
$ git tag -s v0.3.20-rc -m 'please help to test this version!'

Tags can be freely changed, so then after testing this tag can be replaced by 'v0.3.20'

Also all gurus who viewed the source of this commit also can place a signed tag like:

v0.3.20-rc-alex
v0.3.20-rc-31337-haxor
newest-viewed-by-denis
etc...

Then users can check signatures of these people and if they trust them they can be calm.

New bitcoin lottery: probiwon.com
- Moжeт, ты eщё и в Heвидимyю Pyкy Pынкa вepyeшь? - Зaчeм жe вepoвaть в тo, чтo мoжнo нaблюдaть нeпocpeдcтвeннo?
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1085


Chief Scientist


View Profile WWW
January 31, 2011, 02:24:04 PM
 #23

I have an initial implementation of the VM based build process.

The code is here: https://github.com/devrandom/gitian-builder

You will also need a build descriptor file, which is here: https://gist.github.com/803438 .  The file wxWidgets-2.9.1.tar.bz2 goes in the ./inputs directory.

Very nice!

Is there a standard spot to put the build descriptor file in the source tree, or a standard name for it?  I'd like to commit the build descriptor file.

bitcoinex:  I'll tag the tree when I think we have a release candidate.  There are still a few loose ends I hope to tie up today:

1. New -testnet genesis block.
2. New block chain lock-in point.  I'm thinking block 105,000 is a good candidate for the lock-in point.
3. Compile/run/sanity test on Windows.  I am planning on spinning up an Amazon EC2 Windows instance to create a build/testing environment (although probably a VMWare image would be better-- can anybody help with this?  I normally don't do windows).

Am I missing anything else stopping a 0.3.20 release candidate?

How often do you get the chance to work on a potentially world-changing project?
devrandom
Newbie
*
Offline Offline

Activity: 26
Merit: 0



View Profile WWW
January 31, 2011, 06:34:14 PM
 #24

I have an initial implementation of the VM based build process.

The code is here: https://github.com/devrandom/gitian-builder

You will also need a build descriptor file, which is here: https://gist.github.com/803438 .  The file wxWidgets-2.9.1.tar.bz2 goes in the ./inputs directory.

Very nice!

Is there a standard spot to put the build descriptor file in the source tree, or a standard name for it?  I'd like to commit the build descriptor file.


Since you guys would be the first real users of the VM build process, you can influence the standard. Smiley  Maybe it can look in multiple locations, such as SRC/bitcoin.gitian-desc, SRC/gitian/bitcoin-desc.yml.  Do you have a preference?

The build descriptor will need to change slightly.  The commit hash has to be removed if you are committing it into git.  Also, need a bit more added to it to support creation of the whole .tar.gz file.  The latter requires that the build process be run twice, once with a 32 bit VM and once with a 64 bit one.  I tried compiling for 32 on a 64 bit VM, but there's no 32 bit dev packaging for gtk et al.

In any case, I should be able to tie up the loose ends in the next couple of days.
tuxsoul
Newbie
*
Offline Offline

Activity: 40
Merit: 0



View Profile WWW
February 14, 2011, 06:34:30 AM
 #25

I went to build and found out about all the libraries it needs (that are not available as apt-get for this distribution).

@nefario: you can use my sources and ubuntu backports maybe, or only the debian directory and check the debian/control file, only is necesary you change maybe the build-depends line, after that, only do debuild or git-buildpackage.

I think a virtual machine image is a bad idea, because a malicious person could include a modified tool within this virtual image. The C++ compiler, for example, could be modified for malicious purposes, and this might not be detected.

+1

Tags can be freely changed, so then after testing this tag can be replaced by 'v0.3.20'
Also all gurus who viewed the source of this commit also can place a signed tag like:

v0.3.20-rc-alex
v0.3.20-rc-31337-haxor
newest-viewed-by-denis
etc...

Then users can check signatures of these people and if they trust them they can be calm.

+1

Is not necesary delete tags, i like the tags to know which code is used in the build process.

I can help you for debian package:

1.- You need a debian system, in this case maybe lenny or squeeze.
2.- Follow this guide: https://wiki.ubuntu.com/PbuilderHowto
     ¿Sure?, yes, this work for debian and ubuntu.
3.- Use cowbuilder with pbuilder.
4.- Create the sources (.orig.tar.gz, .dsc, .debian.tar.gz), using "dpkg-source -b".
5.- And build the packages with cowbuilder.
6.- Test your packages.

For example, in my slow machine, bitcoin-cpuminer is compiled for five debian versions, using a virtualbox install, the first thing is the source code, after that, is necesary ~2 hour to compile.

Prepare the source code (ubuntu and debian)(~2 hour) + Compile (only debian, only i386)(~2 hours) = ~4 hours.
Sometimes is very less than 4 hours, and other times is more that 4 hours.

Greetings.
genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1039


View Profile
February 14, 2011, 12:24:42 PM
 #26

why not just record your desktop while you build the package from scratch, sign the package with a key you display in the video & upload pkg. put video on youtube. extra points for 2nd webcam video of monitor.
bitcoinex
Sr. Member
****
Offline Offline

Activity: 350
Merit: 250


probiwon.com


View Profile WWW
February 14, 2011, 10:23:29 PM
 #27

tuxsoul, you know that in the Debian unstable came with bitcon-cli package? Maintainer Jonas Smedegaard <dr@jones.dk>.

New bitcoin lottery: probiwon.com
- Moжeт, ты eщё и в Heвидимyю Pyкy Pынкa вepyeшь? - Зaчeм жe вepoвaть в тo, чтo мoжнo нaблюдaть нeпocpeдcтвeннo?
devrandom
Newbie
*
Offline Offline

Activity: 26
Merit: 0



View Profile WWW
April 24, 2011, 05:01:18 PM
 #28


In any case, I should be able to tie up the loose ends in the next couple of days.


Gavin,

BlueMatt was able to replicate the build process and got an identical hash.  I've also created tools to collect signatures and verify distributions, gsign and gverify here:

https://github.com/devrandom/gitian-builder/tree/master/bin

An experimental git repository for collecting signatures produced by gsign:

https://github.com/devrandom/bitcoin-release/tree/master/sample-release/c1.devrandom@niftybox.net

It seems we are far enough along to start using this process for releases and maybe even nightlies.  What do you need from me to make this a reality?
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1085


Chief Scientist


View Profile WWW
April 24, 2011, 11:40:42 PM
 #29

It seems we are far enough along to start using this process for releases and maybe even nightlies.  What do you need from me to make this a reality?

I dunno, you tell me-- the idea is anybody can use gitian-builder to create trusted releases, right?  Working with BlueMatt to make the nightlies use it seems like the right place to start.

Mucking with the Linux build process isn't high on my own personal TODO list, I have my hands busy wrestling with the Windows build (can gitian build windows mingw bitcoin binaries?) and setup.nsi...

How often do you get the chance to work on a potentially world-changing project?
devrandom
Newbie
*
Offline Offline

Activity: 26
Merit: 0



View Profile WWW
April 25, 2011, 12:08:28 AM
 #30

It seems we are far enough along to start using this process for releases and maybe even nightlies.  What do you need from me to make this a reality?

I dunno, you tell me-- the idea is anybody can use gitian-builder to create trusted releases, right?  Working with BlueMatt to make the nightlies use it seems like the right place to start.

Mucking with the Linux build process isn't high on my own personal TODO list, I have my hands busy wrestling with the Windows build (can gitian build windows mingw bitcoin binaries?) and setup.nsi...


Sounds good.

In other news - BlueMatt got cross compiling to work with mingw, so we could get gitian to do those too.
Matt Corallo
Hero Member
*****
expert
Offline Offline

Activity: 755
Merit: 501


View Profile
April 25, 2011, 10:05:59 AM
 #31

I dunno, you tell me-- the idea is anybody can use gitian-builder to create trusted releases, right?  Working with BlueMatt to make the nightlies use it seems like the right place to start.
I would, but currently nighties are built on a VM, and as gitian uses a VM to build deterministically, I can't run it (VM within a VM is verboten). 
In any case, I will be building releases with gitian starting with 0.3.21 (and putting them on the nightlies server) if anyone wants to join and sign them as well. 

Bitcoin Core, rust-lightning, http://bitcoinfibre.org etc.
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
Pages: 1 2 [All]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!