testnet4 in a container

[mocacinno]

Hi guys, tought i would share my work (in progress)  

I was part of this discussion: https://bitcointalk.org/index.php?topic=5496329.0
It was about a potential switch to testnet4...

I quickly pointed out how to merge the pull request for testnet4, but last night i started to think: i want to do this in a reproducible way... So i created a docker image you could easily run yourself...

Code:

FROM ubuntu:22.04 AS builder
#start.sh sets proxy for apt, needed for my env...
#COPY start.sh /usr/local/bin/
#RUN chmod +x /usr/local/bin/start.sh
#RUN /usr/local/bin/start.sh

#install all prereqs
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y tzdata && apt-get -y install git autoconf pkg-config libtool build-essential bsdmainutils libevent-dev  libdb-dev libdb++-dev clang python3 libssl-dev  libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev libminiupnpc-dev libzmq3-dev libqt5gui5 libqt5core5a libqt5dbus5 qttools5-dev qttools5-dev-tools libprotobuf-dev protobuf-compiler libsqlite3-dev ccache

#pull pr
RUN git clone https://github.com/bitcoin/bitcoin.git /bitcoin
WORKDIR /bitcoin
RUN git fetch origin pull/29775/head:pr-29775 && git checkout pr-29775

#compile
RUN ./autogen.sh
RUN ./configure --with-incompatible-bdb CC=clang CXX=clang++
RUN make -j "$(($(nproc) + 1))"
WORKDIR /bitcoin/src
RUN strip bitcoin-util && strip bitcoind && strip bitcoin-cli && strip bitcoin-tx && strip qt/bitcoin-qt

#multistage
FROM ubuntu:22.04
RUN apt-get update && apt-get install -y libdb5.3++-dev libminiupnpc-dev libevent-dev libzmq3-dev libsqlite3-dev
COPY --from=builder /bitcoin/src/bitcoin-util /usr/local/bin
COPY --from=builder /bitcoin/src/bitcoin-cli /usr/local/bin
COPY --from=builder /bitcoin/src/bitcoin-tx /usr/local/bin
COPY --from=builder /bitcoin/src/bitcoind /usr/local/bin
COPY --from=builder /bitcoin/src/qt/bitcoin-qt /usr/local/bin

you can either build the image yourself (should be reproducible), or you can use the image i built: https://hub.docker.com/r/mocacinno/btc_testnet4

Afterwards, create a simple docker-compose.yml

Code:

version: '3'
services:
  bitcoind:
    image: mocacinno/btc_testnet4
    privileged: true
    container_name: bitcoind
    volumes:
      - /root/project/run_btc_testnet4/data:/root/.bitcoin/
    command: ["bitcoind", "-testnet4", "-server", "-rpcuser=demo", "-rpcpassword=demo", "-rpcallowip=127.0.0.1"]
    ports:
      - "8333:8333"
      - "48332:48332"

Ofcourse, change the volume path to a path that exists on your host, and change the rpc user and pass

Then it's just a matter of "docker-compose up -d" to pull the image and start bitcoind.

Connect to the running container and create a wallet:

Code:

docker exec -it bitcoind /bin/bash
bitcoin-cli -testnet4 -rpcuser=demo -rpcpassword=demo createwallet mywally

a (testnet4) tipping address: tb1qumlhr8tn9gsdyujy464jkk4c5r488u8kxteyx5  (only send testnet4 tBTC!!!)

[mocacinno]

I have tried to put a "mining" image together aswell:

Code:

FROM ubuntu:22.04 AS miner
#start.sh sets proxy for apt, needed for my env
#COPY start.sh /usr/local/bin/
#RUN chmod +x /usr/local/bin/start.sh
#RUN /usr/local/bin/start.sh
RUN apt-get update \
  && apt-get install -y \
    build-essential \
    libssl-dev \
    libgmp-dev \
    libcurl4-openssl-dev \
    libjansson-dev \
    automake \ 
	git \
	zlib1g-dev \
    && rm -rf /var/lib/apt/lists/*
RUN git clone https://github.com/JayDDee/cpuminer-opt /cpuminer
WORKDIR /cpuminer
RUN ./build.sh


FROM ubuntu:22.04 AS builder
#start.sh sets proxy for apt, needed for my env
#COPY start.sh /usr/local/bin/
#RUN chmod +x /usr/local/bin/start.sh
#RUN /usr/local/bin/start.sh

#install all prereqs
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y tzdata && apt-get -y install git autoconf pkg-config libtool build-essential bsdmainutils libevent-dev  libdb-dev libdb++-dev clang python3 libssl-dev  libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev libminiupnpc-dev libzmq3-dev libqt5gui5 libqt5core5a libqt5dbus5 qttools5-dev qttools5-dev-tools libprotobuf-dev protobuf-compiler libsqlite3-dev ccache

#pull pr
RUN git clone https://github.com/bitcoin/bitcoin.git /bitcoin
WORKDIR /bitcoin
RUN git fetch origin pull/29775/head:pr-29775 && git checkout pr-29775

#compile
RUN ./autogen.sh
RUN ./configure --with-incompatible-bdb CC=clang CXX=clang++
RUN make -j "$(($(nproc) + 1))"
WORKDIR /bitcoin/src
RUN strip bitcoin-util && strip bitcoind && strip bitcoin-cli && strip bitcoin-tx && strip qt/bitcoin-qt

#multistage
FROM ubuntu:22.04
RUN apt-get update && apt-get install -y libdb5.3++-dev libminiupnpc-dev libevent-dev libzmq3-dev libsqlite3-dev \ 
	libjansson-dev  libcurl4-openssl-dev
	
COPY --from=builder /bitcoin/src/bitcoin-util /usr/local/bin
COPY --from=builder /bitcoin/src/bitcoin-cli /usr/local/bin
COPY --from=builder /bitcoin/src/bitcoin-tx /usr/local/bin
COPY --from=builder /bitcoin/src/bitcoind /usr/local/bin
COPY --from=builder /bitcoin/src/qt/bitcoin-qt /usr/local/bin
COPY --from=miner /cpuminer/cpuminer /usr/local/bin

you can build it yourself, or get if from dockerhub mocacinno/btc_testnet4:cpuminer

if you use this docker-compose.yml to start:

Code:

version: '3'
services:
  bitcoind:
    image: mocacinno/btc_testnet4:cpuminer
    privileged: true
    container_name: bitcoind
    volumes:
      - /root/project/run_btc_testnet4/data:/root/.bitcoin/
    command: ["bitcoind", "-testnet4", "-server", "-rpcuser=demo", "-rpcpassword=demo", "-rpcallowip=127.0.0.1", "-rpcport=5000"]
    ports:
      - "8333:8333"
      - "48332:48332"

you can afterwards log in to the running container like this:

Code:

docker exec -it bitcoind /bin/bash

and start cpu mining like this (replace my address by yours offcourse)

Code:

cpuminer -a sha256d -o http://127.0.0.1:5000 -O demo:demo --coinbase-addr=tb1qumlhr8tn9gsdyujy464jkk4c5r488u8kxteyx5

the problem is the only vps i currently have is slooooooooooooooow (it's a small vps from hostnamaste and they seem to have overloaded their hosts. Logging in takes about 2 minutes, let alone running a cpu miner)... I barely get 20 Mhashes per second (really, not kidding......). I have no idear if the mining actually works, since at this hashrate i will not find anything.


update it seems like somebody is running an asic on the testnet4... hashrate is 425 Th, so my 22Mh is < 1/19.000.000 of the current network hashrate... On average, i'd hit one block every >300 years. I just hope this asic-miner turns off his asic from time to time to give the cpuminers a chance...
Maybe i have to dust off my old geccoscience compaq... If i remember correctly it hashed somewhere at the 50 Gh range??? At least i'd have 1/8500 of the total network hashrate, and i would be able to mine a block every couple of months... ~sigh~

[DaveF]

the problem is the only vps i currently have is slooooooooooooooow (it's a small vps from hostnamaste and they seem to have overloaded their hosts. Logging in takes about 2 minutes, let alone running a cpu miner)... I barely get 20 Mhashes per second (really, not kidding......). I have no idear if the mining actually works, since at this hashrate i will not find anything.


update it seems like somebody is running an asic on the testnet4... hashrate is 425 Th, so my 22Mh is < 1/19.000.000 of the current network hashrate... On average, i'd hit one block every >300 years. I just hope this asic-miner turns off his asic from time to time to give the cpuminers a chance...
Maybe i have to dust off my old geccoscience compaq... If i remember correctly it hashed somewhere at the 50 Gh range??? At least i'd have 1/8500 of the total network hashrate, and i would be able to mine a block every couple of months... ~sigh~

That will continue to be an issue forever with anything that is 'low use' and has a mining algo that has asics for it.

There are so many 2 or 3 gen out miners that are just sitting there that it's not a big deal to mine something worthless with it.

The difference in loss for the cost of power between something that has $0 value and the potential profit of mining BTC (or LTC or whatever) that has value but a difficulty that is that high is just about nothing.

Back to this, has anyone been able to compile this for windows? I tried last night and got a bunch of errors. Was exhausted and doing it remotely and just went to bed. Didn't even record the errors.

-Dave 

[seoincorporation]

Great job mocacinno, you automate the full process. Now people can start working with Testnet 4 in an easy way.

And it would be if user could mine with CPU, but as you say, 1 modern iner would kill that option for all. It would be fun to include a rule on the node setting a limit for miners. But i know that's not possible because that would not only affect the big engines, but would affect the pools too.  This gives me the idea of opening a mining pool for tBTC 4, that way even if users do not have the hardware, they could get a small piece of the cake with their CPU. But tBTC4 is still green, let's give it some time to mature.

And again, thanks for the apport.

[mocacinno]

I built (yet another) image that is including c-lighting + a simple walktrouh

I've updated the info on dockerhub:
https://hub.docker.com/repository/docker/mocacinno/btc_testnet4/general

If anybody is interested, i'm running two bitcoin nodes on testnet4, and two testnet4 lightning nodes on top of my bitcoin testnet4 node... I'll be AFK during the weekend, but when i get back on monday, i'll see if there are people interested in connecting to my testnet4 lightning node, creating some funded channels and creating some invoices

the Dockerfile itself is as follows:

Code:

FROM ubuntu:22.04 AS lightning
RUN apt-get update && \
	apt-get install -y  git build-essential python3 sqlite3 libsqlite3-dev autoconf libtool python3-dev python3-mako gettext  python3-pip jq
RUN pip install protobuf grpcio-tools
RUN git clone https://github.com/ElementsProject/lightning /lightning
WORKDIR /lightning
RUN git fetch --all --tags
RUN git checkout tags/v24.02.2 -b v24.02.2
RUN sed -i '1311,1313d' /lightning/lightningd/chaintopology.c
RUN ./configure
#RUN apt-get install -y python3-grpc-tools
RUN make -j "$(($(nproc) + 1))"

FROM ubuntu:22.04 AS miner
#start.sh sets proxy for apt, needed for my env
COPY start.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/start.sh
RUN /usr/local/bin/start.sh
RUN apt-get update \
  && apt-get install -y \
    build-essential \
    libssl-dev \
    libgmp-dev \
    libcurl4-openssl-dev \
    libjansson-dev \
    automake \
	git \
	zlib1g-dev \
    && rm -rf /var/lib/apt/lists/*
RUN git clone https://github.com/JayDDee/cpuminer-opt /cpuminer
WORKDIR /cpuminer
RUN git fetch --all --tags
RUN git checkout tags/v24.2 -b v24.2
RUN ./build.sh


FROM ubuntu:22.04 AS builder
#start.sh sets proxy for apt, needed for my env
COPY start.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/start.sh
RUN /usr/local/bin/start.sh

#install all prereqs
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y tzdata && apt-get -y install git autoconf pkg-config libtool build-essential bsdmainutils libevent-dev  libdb-dev libdb++-dev clang python3 libssl-dev  libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev libminiupnpc-dev libzmq3-dev libqt5gui5 libqt5core5a libqt5dbus5 qttools5-dev qttools5-dev-tools libprotobuf-dev protobuf-compiler libsqlite3-dev ccache

#pull pr
RUN git clone https://github.com/bitcoin/bitcoin.git /bitcoin
WORKDIR /bitcoin
RUN git fetch origin pull/29775/head:pr-29775 && git checkout pr-29775

#compile
RUN ./autogen.sh
RUN ./configure --with-incompatible-bdb --with-gui=no  CC=clang CXX=clang++
RUN make -j "$(($(nproc) + 1))"
WORKDIR /bitcoin/src
RUN strip bitcoin-util && strip bitcoind && strip bitcoin-cli && strip bitcoin-tx

#multistage
FROM ubuntu:22.04
RUN apt-get update && apt-get install -y libdb5.3++-dev libminiupnpc-dev libevent-dev libzmq3-dev libsqlite3-dev \
	libjansson-dev  libcurl4-openssl-dev python3
	
COPY --from=builder /bitcoin/src/bitcoin-util /usr/local/bin
COPY --from=builder /bitcoin/src/bitcoin-cli /usr/local/bin
COPY --from=builder /bitcoin/src/bitcoin-tx /usr/local/bin
COPY --from=builder /bitcoin/src/bitcoind /usr/local/bin
COPY --from=miner /cpuminer/cpuminer /usr/local/bin
COPY --from=lightning /lightning/cli/lightning-cli /usr/local/bin
COPY --from=lightning /lightning/tools/reckless /usr/local/bin
COPY --from=lightning /lightning/lightningd/lightningd /usr/local/bin
COPY --from=lightning /lightning/lightningd/lightning_channeld /usr/local/libexec/c-lightning/
COPY --from=lightning /lightning/lightningd/lightning_closingd /usr/local/libexec/c-lightning/
COPY --from=lightning /lightning/lightningd/lightning_connectd /usr/local/libexec/c-lightning/
COPY --from=lightning /lightning/lightningd/lightning_gossipd /usr/local/libexec/c-lightning/
COPY --from=lightning /lightning/lightningd/lightning_hsmd /usr/local/libexec/c-lightning/
COPY --from=lightning /lightning/lightningd/lightning_onchaind /usr/local/libexec/c-lightning/
COPY --from=lightning /lightning/lightningd/lightning_openingd /usr/local/libexec/c-lightning/
COPY --from=lightning /lightning/plugins /opt/lightningd/plugins

[baryonlee]

Thank you very much. I am launching the testnet4 docker container in live.
Great work!

🔴 [Live] The Note block explorer is currently syncing with the BTC Testnet4. We've throttled the speed to one block per second so you can observe this process in real-time.
Check it out here:  https://testnet4.noteprotocol.org

And For the main network https://explorer.noteprotocol.org