[RFC] Trusted build process

[bitcoinex]

The Linux distributions have a maintainers who are preparing to build sources. If you 'aptitude source package-name' a source package and compile it after checking src signature by yourself it's safe enough.

There is, however, the probability of a "third level" trojan , which can not be detected if has not been investigated source of bitcoin and source of compiler, but it is too high level matter and is can not resolved at this level of abstraction.

The threat model that I think we could target here is a compromise of the build process, either by the builder or by a third party attacking their build environment.  An assertion from multiple builders that [source checksum, build script, build environment checksum] produces [executable checksum] gives high assurance that no such compromise occurred.

I'm going to spend some time on this for the linux environment in the next few days.

As far as I know, linux distributions are builds a packages on multiple machines in parallel, and the identity of resulting packets is checked afterwards. It is necessary to avoid errors of the hardware.

That is why:

Also, I urge DO NOT use digital signatures in the packages because it do not give the opportunity to build two identical package, because signature are uses random numbers when creating. Instead, repository must be signed.

[1714122445]

1714122445

[1714122445]

1714122445

[1714122445]

1714122445

[1714122445]

1714122445

[bitcoinex]

Code:

$ git tag -l
help

Why help?
We must try to use signatures in the git commits. Gavin could do this right now by command like:

Code:

$ git tag -s v0.3.20-rc -m 'please help to test this version!'

Tags can be freely changed, so then after testing this tag can be replaced by 'v0.3.20'

Also all gurus who viewed the source of this commit also can place a signed tag like:

v0.3.20-rc-alex
v0.3.20-rc-31337-haxor
newest-viewed-by-denis
etc...

Then users can check signatures of these people and if they trust them they can be calm.

[Gavin Andresen]

I have an initial implementation of the VM based build process.

The code is here: https://github.com/devrandom/gitian-builder

You will also need a build descriptor file, which is here: https://gist.github.com/803438 .  The file wxWidgets-2.9.1.tar.bz2 goes in the ./inputs directory.

Very nice!

Is there a standard spot to put the build descriptor file in the source tree, or a standard name for it?  I'd like to commit the build descriptor file.

bitcoinex:  I'll tag the tree when I think we have a release candidate.  There are still a few loose ends I hope to tie up today:

1. New -testnet genesis block.
2. New block chain lock-in point.  I'm thinking block 105,000 is a good candidate for the lock-in point.
3. Compile/run/sanity test on Windows.  I am planning on spinning up an Amazon EC2 Windows instance to create a build/testing environment (although probably a VMWare image would be better-- can anybody help with this?  I normally don't do windows).

Am I missing anything else stopping a 0.3.20 release candidate?

[devrandom]

I have an initial implementation of the VM based build process.

The code is here: https://github.com/devrandom/gitian-builder

You will also need a build descriptor file, which is here: https://gist.github.com/803438 .  The file wxWidgets-2.9.1.tar.bz2 goes in the ./inputs directory.

Very nice!

Is there a standard spot to put the build descriptor file in the source tree, or a standard name for it?  I'd like to commit the build descriptor file.

Since you guys would be the first real users of the VM build process, you can influence the standard.   Maybe it can look in multiple locations, such as SRC/bitcoin.gitian-desc, SRC/gitian/bitcoin-desc.yml.  Do you have a preference?

The build descriptor will need to change slightly.  The commit hash has to be removed if you are committing it into git.  Also, need a bit more added to it to support creation of the whole .tar.gz file.  The latter requires that the build process be run twice, once with a 32 bit VM and once with a 64 bit one.  I tried compiling for 32 on a 64 bit VM, but there's no 32 bit dev packaging for gtk et al.

In any case, I should be able to tie up the loose ends in the next couple of days.

[tuxsoul]

I went to build and found out about all the libraries it needs (that are not available as apt-get for this distribution).

@nefario: you can use my sources and ubuntu backports maybe, or only the debian directory and check the debian/control file, only is necesary you change maybe the build-depends line, after that, only do debuild or git-buildpackage.

I think a virtual machine image is a bad idea, because a malicious person could include a modified tool within this virtual image. The C++ compiler, for example, could be modified for malicious purposes, and this might not be detected.

+1

Tags can be freely changed, so then after testing this tag can be replaced by 'v0.3.20'
Also all gurus who viewed the source of this commit also can place a signed tag like:

v0.3.20-rc-alex
v0.3.20-rc-31337-haxor
newest-viewed-by-denis
etc...

Then users can check signatures of these people and if they trust them they can be calm.

+1

Is not necesary delete tags, i like the tags to know which code is used in the build process.

I can help you for debian package:

1.- You need a debian system, in this case maybe lenny or squeeze.
2.- Follow this guide: https://wiki.ubuntu.com/PbuilderHowto
     ¿Sure?, yes, this work for debian and ubuntu.
3.- Use cowbuilder with pbuilder.
4.- Create the sources (.orig.tar.gz, .dsc, .debian.tar.gz), using "dpkg-source -b".
5.- And build the packages with cowbuilder.
6.- Test your packages.

For example, in my slow machine, bitcoin-cpuminer is compiled for five debian versions, using a virtualbox install, the first thing is the source code, after that, is necesary ~2 hour to compile.

Prepare the source code (ubuntu and debian)(~2 hour) + Compile (only debian, only i386)(~2 hours) = ~4 hours.
Sometimes is very less than 4 hours, and other times is more that 4 hours.

Greetings.

[genjix]

why not just record your desktop while you build the package from scratch, sign the package with a key you display in the video & upload pkg. put video on youtube. extra points for 2nd webcam video of monitor.

[bitcoinex]

tuxsoul, you know that in the Debian unstable came with bitcon-cli package? Maintainer Jonas Smedegaard <dr@jones.dk>.

[devrandom]

In any case, I should be able to tie up the loose ends in the next couple of days.

Gavin,

BlueMatt was able to replicate the build process and got an identical hash.  I've also created tools to collect signatures and verify distributions, gsign and gverify here:

https://github.com/devrandom/gitian-builder/tree/master/bin

An experimental git repository for collecting signatures produced by gsign:

https://github.com/devrandom/bitcoin-release/tree/master/sample-release/c1.devrandom@niftybox.net

It seems we are far enough along to start using this process for releases and maybe even nightlies.  What do you need from me to make this a reality?

[Gavin Andresen]

It seems we are far enough along to start using this process for releases and maybe even nightlies.  What do you need from me to make this a reality?

I dunno, you tell me-- the idea is anybody can use gitian-builder to create trusted releases, right?  Working with BlueMatt to make the nightlies use it seems like the right place to start.

Mucking with the Linux build process isn't high on my own personal TODO list, I have my hands busy wrestling with the Windows build (can gitian build windows mingw bitcoin binaries?) and setup.nsi...

[devrandom]

It seems we are far enough along to start using this process for releases and maybe even nightlies.  What do you need from me to make this a reality?

I dunno, you tell me-- the idea is anybody can use gitian-builder to create trusted releases, right?  Working with BlueMatt to make the nightlies use it seems like the right place to start.

Mucking with the Linux build process isn't high on my own personal TODO list, I have my hands busy wrestling with the Windows build (can gitian build windows mingw bitcoin binaries?) and setup.nsi...

Sounds good.

In other news - BlueMatt got cross compiling to work with mingw, so we could get gitian to do those too.

[Matt Corallo]

I dunno, you tell me-- the idea is anybody can use gitian-builder to create trusted releases, right?  Working with BlueMatt to make the nightlies use it seems like the right place to start.

I would, but currently nighties are built on a VM, and as gitian uses a VM to build deterministically, I can't run it (VM within a VM is verboten). 
In any case, I will be building releases with gitian starting with 0.3.21 (and putting them on the nightlies server) if anyone wants to join and sign them as well.