Title: Nightly Builds Post by: Matt Corallo on March 16, 2011, 08:49:07 PM I just finished setting up a nightly build server. It currently only builds i686 for linux, but I'm hoping to add Windows soon. It also includes an up to date copy of the blockchain with each nightly.
It can be found at Edit: URL is now bitcoin.bluematt.me/bitcoin-nightly (http://bitcoin.bluematt.me/bitcoin-nightly). It is now DNS load balanced. Thanks to validus and vrs for bandwidth. Title: Re: Nightly Builds Post by: Matt Corallo on March 16, 2011, 09:02:03 PM At the request of tcatm and jgarzik, I think it would be nice to have a version that includes various patches. It might be updated nightly or just on new releases. I'm thinking patches like getblock and other similar patches which aren't of use to enough people to make it into mainline, but are sensible useful patches.
Please list any patches you want to see included. Title: Re: Nightly Builds Post by: tcatm on March 16, 2011, 09:04:17 PM * xlisttransactions http://bitcointalk.org/index.php?topic=611.0
* getblock https://www.bitcoin.org/smf/index.php?topic=724.0 Title: Re: Nightly Builds Post by: jgarzik on March 16, 2011, 09:09:46 PM my patches
--------------------------------------------- xlisttransactions RPC: http://yyz.us/bitcoin/patch.bitcoin-listtransactions getblockbycount RPC: http://yyz.us/bitcoin/patch.bitcoin-getblock dumpblock RPC: http://yyz.us/bitcoin/patch.bitcoin-dumpblock settxfee RPC: http://yyz.us/bitcoin/patch.bitcoin-settxfee print POW failures: http://yyz.us/bitcoin/patch.bitcoin-pow-fail other peoples' hackery ----------------------------------------------------- - gavin's monitor* stuff - gavin's portoption branch - UPNP things to avoid ---------------------------------------------------- Breaking changes, like Luke JR's RPC versioning changes. administrivia --------------------------------------------------- IMO require people to submit (or write yourself) a summary of each patch, so that people know exactly what's in there. Title: Re: Nightly Builds Post by: Matt Corallo on March 16, 2011, 09:32:55 PM things to avoid I'll try, but for something like this I can't make any guarantees.---------------------------------------------------- Breaking changes, like Luke JR's RPC versioning changes. administrivia Good idea, care to submit a summary of yours?--------------------------------------------------- IMO require people to submit (or write yourself) a summary of each patch, so that people know exactly what's in there. Title: Re: Nightly Builds Post by: jgarzik on March 16, 2011, 09:39:29 PM Associated patch summaries...
xlisttransactions RPC: http://yyz.us/bitcoin/patch.bitcoin-listtransactions Adds "xlisttransactions" RPC call. This lists all wallet transactions, using logic similar to the GUI's display code. Quote getblockbycount RPC: http://yyz.us/bitcoin/patch.bitcoin-getblock Adds "getblockbycount" RPC call. This dumps the block on the main chain at the specified height, in its entirety. All data fields, including all transactions, are dumped in a single JSON response. Quote dumpblock RPC: http://yyz.us/bitcoin/patch.bitcoin-dumpblock A clone of "getblockbycount" RPC call. Same behavior as "getblockbycount" -- dumps block at specified height. Adds a "_fulldump" boolean to the output, indicating whether or not the entire block is present on disk (preparing for lightweight block chain support), hoping to address some objections of satoshi. Quote settxfee RPC: http://yyz.us/bitcoin/patch.bitcoin-settxfee Add "settxfee" RPC call. This permits changing of the 'paytxfee' variable at runtime. Quote print POW failures: http://yyz.us/bitcoin/patch.bitcoin-pow-fail Print a response for every work solution submitted to bitcoind. Since we already logged successful proof-of-work solutions, the only thing remaining was to log when a proof-of-work check failed, which this patch does. Title: Re: Nightly Builds Post by: genjix on March 16, 2011, 10:07:58 PM RPC methods return strings instead of floats:
https://github.com/genjix/bitcoin/tree/strrpc Immediate initialisation of RPC + isinitialized in getinfo: https://github.com/genjix/bitcoin/tree/isinitialized Name resolution for addresses (genjix@foo.org -> 1kjdskjds3232dsjds4343): https://github.com/genjix/bitcoin/tree/bitnom Maybe you could setup a way to generate custom builds for distribution. Title: Re: Nightly Builds Post by: jgarzik on March 16, 2011, 10:21:53 PM I was thinking this would be mostly about non-breaking changes, such as those patches people are already applying manually and running in production. RPC methods return strings instead of floats: https://github.com/genjix/bitcoin/tree/strrpc disagree - breaking change Quote Immediate initialisation of RPC + isinitialized in getinfo: https://github.com/genjix/bitcoin/tree/isinitialized disagree - breaking change (people must rewrite their code to expect the RPC server to appear early) Quote Name resolution for addresses (genjix@foo.org -> 1kjdskjds3232dsjds4343): https://github.com/genjix/bitcoin/tree/bitnom That's more like it :) I think this sort of change -- presuming it doesn't break any existing APIs, which I don't think it does -- would be perfect for these "useful patches" nightly builds. Quote Maybe you could setup a Bitcoin cloud build service? I login... add my branch and click generate builds. Sounds like a great bitcoin business or website for somebody... Title: Re: Nightly Builds Post by: Matt Corallo on March 16, 2011, 10:43:52 PM I agree with jgarzik, I want this version to be stable and compatible, but add new features which might be considered "beta" or might just not have any use in mainline.
The current version can be see here (https://github.com/TheBlueMatt/bitcoin/tree/patched). Maybe you could setup a way to generate custom builds for distribution. Great idea. It wouldn't be hard to do with simple bash scripts. The problem is currently I compile the new version on my server and upload it to a VPS provided by validus because I don't have the bandwidth to provide good service. Also, the compilation might take a while so it would need powerful servers if it was to serve more than one simultaneous connection. If anyone knows of a place where I can get a free, powerful server with plenty of bandwidth I'd set it up (or would be willing to pay for one, donation address is in my sig ;)). Title: Re: Nightly Builds Post by: genjix on March 16, 2011, 10:46:47 PM I'm not going to re-ignite an argument over those 2 first patches, but if this is for stable nightly builds, then please don't include my third branch yet- it's not complete.
There's 2 issues I need to conceptually solve. (privacy concern + a security scheme) Title: Re: Nightly Builds Post by: Matt Corallo on March 17, 2011, 08:07:55 AM I'm not going to re-ignite an argument over those 2 first patches, but if this is for stable nightly builds, then please don't include my third branch yet- it's not complete. My goal is to have two branches. One nightly, stable build from the current git version of bitcoin. One which is either nightly or in sync with bitcoin releases (haven't made up my mind yet). The second one will be patched with all kinds of "beta" patches but which don't break existing functionality. There's 2 issues I need to conceptually solve. (privacy concern + a security scheme) Title: Re: Nightly Builds Post by: Matt Corallo on March 17, 2011, 12:04:33 PM First release of the patched branch has been uploaded. I will try to get the bitnom included later today, but it currently conflicts with sendmany so it needs a bit of work to get it to merge properly. Included patches are listed in the README.md file. I also need to add Gavin's monitor stuff.
Title: Re: Nightly Builds Post by: devrandom on March 19, 2011, 06:57:20 PM Would be cool if the builds used the gitian process:
http://bitcointalk.org/index.php?topic=2926.20 This would allow people to verify that the builds are not compromised. Title: Re: Nightly Builds Post by: Matt Corallo on March 19, 2011, 09:37:48 PM Would be cool if the builds used the gitian process: Sadly that is not possible as I cannot run VMs within the existing build VMs. However, I am looking into building stable releases (of -patched, and potentially of mainline) with gitian. http://bitcointalk.org/index.php?topic=2926.20 This would allow people to verify that the builds are not compromised. Title: Re: Nightly Builds Post by: devrandom on March 19, 2011, 11:08:44 PM Would be cool if the builds used the gitian process: Sadly that is not possible as I cannot run VMs within the existing build VMs. However, I am looking into building stable releases (of -patched, and potentially of mainline) with gitian. http://bitcointalk.org/index.php?topic=2926.20 This would allow people to verify that the builds are not compromised. Ah, yes, running VM within VM is a problem. I hear that's possible in some configurations, but haven't looked into it. I'm available if you run into any issues with building and to compare hashes. Title: Re: Nightly Builds Post by: Matt Corallo on March 19, 2011, 11:22:29 PM Would be cool if the builds used the gitian process: Sadly that is not possible as I cannot run VMs within the existing build VMs. However, I am looking into building stable releases (of -patched, and potentially of mainline) with gitian. http://bitcointalk.org/index.php?topic=2926.20 This would allow people to verify that the builds are not compromised. Ah, yes, running VM within VM is a problem. I hear that's possible in some configurations, but haven't looked into it. I'm available if you run into any issues with building and to compare hashes. Title: Re: Nightly Builds Post by: devrandom on March 19, 2011, 11:26:00 PM After building the VMs, attempting to run the build script results in apt errors fetching from my host name. ie "Failed to fetch from http://HostHostName:...". My host name will not resolve via DNS which is apparently the problem here. Looking into this. Title: Re: Nightly Builds Post by: devrandom on March 19, 2011, 11:48:12 PM After building the VMs, attempting to run the build script results in apt errors fetching from my host name. ie "Failed to fetch from http://HostHostName:...". My host name will not resolve via DNS which is apparently the problem here. Try doing: Code: export MIRROR_HOST=10.0.2.2 I think I'll make that the default. Title: Re: Nightly Builds Post by: Matt Corallo on March 20, 2011, 12:44:27 AM After building the VMs, attempting to run the build script results in apt errors fetching from my host name. ie "Failed to fetch from http://HostHostName:...". My host name will not resolve via DNS which is apparently the problem here. Try doing: Code: export MIRROR_HOST=10.0.2.2 I think I'll make that the default. Title: Re: Nightly Builds Post by: devrandom on March 20, 2011, 01:11:58 AM After building the VMs, attempting to run the build script results in apt errors fetching from my host name. ie "Failed to fetch from http://HostHostName:...". My host name will not resolve via DNS which is apparently the problem here. Try doing: Code: export MIRROR_HOST=10.0.2.2 I think I'll make that the default. Strange. Could you try: * While make-base-vm is running, run Code: ps auxww | grep vmbuilder * Make sure that you didn't specify -i or --skip-image * I just realized that the --quiet flag to gbuild actually does --skip-image due to a bug . If you had this flag on, it didn't pick up the new base vm image. I've just pushed a new version that fixed the latter and makes 10.0.2.2 the default. I'm on irc if you want to chat instead of messaging here. Title: Re: Nightly Builds Post by: devrandom on March 21, 2011, 12:53:04 AM New version pushed just now.
The default is now to use 127.0.0.1 while bootstrapping (on the host) and 10.0.2.2 in the VM. This will always hit the host. Let me know if it works for you. Title: Re: Nightly Builds Post by: jgarzik on March 21, 2011, 01:54:31 AM It would be nice if directories such as http://bitcoin.bluematt.me/bitcoin-nightly/ included a README that described the contents of the directory / purpose of the project / etc.
Title: Re: Nightly Builds Post by: Luke-Jr on April 21, 2011, 03:22:08 AM things to avoid More trolling, I see.---------------------------------------------------- Breaking changes, like Luke JR's RPC versioning changes. For anyone interested, my RPCv1 changes do NOT break anything, and are completely backward compatible. Title: Re: Nightly Builds Post by: Garrett Burgwardt on April 21, 2011, 03:24:59 AM things to avoid More trolling, I see.---------------------------------------------------- Breaking changes, like Luke JR's RPC versioning changes. For anyone interested, my RPCv1 changes do NOT break anything, and are completely backward compatible. Weren't those the ones that didn't actually do anything? Title: Re: Nightly Builds Post by: Luke-Jr on April 21, 2011, 03:29:37 AM things to avoid More trolling, I see.---------------------------------------------------- Breaking changes, like Luke JR's RPC versioning changes. For anyone interested, my RPCv1 changes do NOT break anything, and are completely backward compatible. Weren't those the ones that didn't actually do anything? More importantly, both the oldest and newest Win32 nightlies on the site crash when stopped. Title: Re: Nightly Builds Post by: Matt Corallo on April 21, 2011, 09:16:50 AM More importantly, both the oldest and newest Win32 nightlies on the site crash when stopped. I can't reproduce, could you provide more information (ie Windows version, etc)Title: Re: Nightly Builds Post by: Luke-Jr on April 21, 2011, 03:11:36 PM More importantly, both the oldest and newest Win32 nightlies on the site crash when stopped. I can't reproduce, could you provide more information (ie Windows version, etc)1. Run start /B bitcoind -rpcpassword=1 (wait for it to finish startup) 2. Run bitcoind -rpcpassword=1 stop 3. Windows says it crashed (and db error visible) Title: Re: Nightly Builds Post by: Matt Corallo on April 24, 2011, 09:59:14 PM Windows 7, vanilla fresh install. Ok, so I wasn't able to fix that on Windows Server 2003. I'd assume it would work on XP but I gave up on windows. Now the win32 nightlies are compiled on Ubuntu via https://github.com/TheBlueMatt/bitcoin/tree/crosscompile (https://github.com/TheBlueMatt/bitcoin/tree/crosscompile). They appear to work 100% now. If you find any additional issues, please notify me.1. Run start /B bitcoind -rpcpassword=1 (wait for it to finish startup) 2. Run bitcoind -rpcpassword=1 stop 3. Windows says it crashed (and db error visible) Title: Re: Nightly Builds Post by: khal on April 24, 2011, 10:16:44 PM On linux, bitcoind, 64bits :
./bitcoind => bitcoind doesn't display "bitcoin server starting" and doesn't return to the next line of the terminal. But, from another terminal, i can send rpc commands and all seams to work. I can launch "./bitcoind stop" and bitcoind stops in the first terminal. The problem doesn't appear with the -daemon option. This problem didn't exist in the 0.3.20.2 release. Title: Re: Nightly Builds Post by: Matt Corallo on April 24, 2011, 10:24:37 PM On linux, bitcoind, 64bits : That isn't a problem with the nightlies. That was a conscious change made and will be in the 0.3.21 release (I believe its in the 0.3.21 RC now). ./bitcoind => bitcoind doesn't display "bitcoin server starting" and doesn't return to the next line of the terminal. But, from another terminal, i can send rpc commands and all seams to work. I can launch "./bitcoind stop" and bitcoind stops in the first terminal. The problem doesn't appear with the -daemon option. This problem didn't exist in the 0.3.20.2 release. Title: Re: Nightly Builds Post by: LZ on August 21, 2011, 11:22:32 PM Thanks for your builds. :)
Title: Re: Nightly Builds Post by: Venkatesh Srinivas on August 22, 2011, 02:22:26 AM Would you be interested in nightly builds for DragonFly BSD and OpenBSD?
Title: Re: Nightly Builds Post by: ctoon6 on August 22, 2011, 02:35:10 AM if it is possible, could you do a 64 bit windows build? just to see how much faster it is.
Title: Re: Nightly Builds Post by: Matt Corallo on August 22, 2011, 05:43:12 PM Would you be interested in nightly builds for DragonFly BSD and OpenBSD? Currently I do all the building on one Ubuntu VM, so adding two more VMs is quite a bit more effort. If you want to do them though, setting up a cronjob to build is quite easy.if it is possible, could you do a 64 bit windows build? just to see how much faster it is. The Linux 32/64 difference is unnoticeable, the only reason a Linux 64-bit binary is provided is the 32-bit one will fail to load due to 32-bit BDB on 64-bit Ubuntu attempting to load 64-bit plugins. Because they are both compiled with gcc, the difference for Windows should be virtually identical to that on Linux. That and setting up a 64-bit build for windows would be substantially more effort.Sorry to those who have noticed the builds have not been happening over the past couple days, Im hoping to get them back running today or tomorrow (Im in the process of switching build servers). Title: Re: Nightly Builds Post by: Gavin Andresen on August 22, 2011, 06:12:54 PM Would you be interested in nightly builds for DragonFly BSD and OpenBSD? Yes, would you be interested in building?Title: Re: Nightly Builds Post by: Venkatesh Srinivas on August 23, 2011, 12:28:24 AM Yes; was asking to see if they could be hosted at the same place or would have to be elsewhere.
|