Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: nullama on September 29, 2022, 10:31:36 AM



Title: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: nullama on September 29, 2022, 10:31:36 AM
If you need some testnet bitcoins, you can easily get them by solo mining. Some other guides I've found in the web are outdated, so I made this one to work with the latest software/hardware.

You'll need:
  • Latest version of Bitcoin Core(v23.0 at the time of writing)
  • Special build of cgminer that allows solo mining, available from https://github.com/cmmodtools/cgminer with a small update from this guide.
  • Ideally an ASIC miner. You can use any miner that works with cgminer, even the USB miner from GekkoScience, the Compac F:
    https://i.imgur.com/oCzSCZN.jpg

To make things simple, for this guide I am going to assume you have two computers, one running a Compac F miner and another one running Bitcoin core. It shouldn't matter if you're running both on the same computer or if you have a different miner.

Step 1: Install and run Bitcoin Core

We're going to use the folder ~/bitcoin_testnet to save everything so that it's easy to remove once you're done and to keep things simple. You can of course use a different path if you want.

Inside ~/bitcoin_testnet create a folder called data:

Code:
mkdir -p ~/bitcoin_testnet/data

Download the Bitcoin Core (https://bitcoincore.org/en/download/) for your platform into ~/bitcoin_testnet/ and extract it there.

Now let's prepare the configuration file. Create a text file named bitcoin_testnet.conf in ~/bitcoin_testnet/ and put these content in it:

Code:
testnet=1
txindex=1
server=1
[test]
rpcport=5000
rpcallowip=YOUR_MINER_IP
rpcuser=YOUR_RPC_USER
rpcpassword=YOUR_RPC_PASSWORD
rpcbind=0.0.0.0

The thing that you need to understand here is that you're setting up the Bitcoin node to run in testnet, and you're defining an RPC port(5000 in this case, can be anything), user and password, and whitelisting a specific IP to connect to your node(YOUR_MINER_IP). You'll need these details and your Bitcoin node IP later when connecting from cgminer.

You can now start running your Bitcoin node by doing the following(make sure to change user to your actual user:

Code:
/home/user/bitcoin_testnet/bitcoin-23.0/bin/bitcoind -conf=/home/user/bitcoin_testnet/bitcoin_testnet.conf -datadir=/home/user/bitcoin_testnet/data

If you want, you can keep this running in the background with screen:

Code:
screen -dm -S bitcoin_testnet /home/user/bitcoin_testnet/bitcoin-23.0/bin/bitcoind -conf=/home/user/bitcoin_testnet/bitcoin_testnet.conf -datadir=/home/user/bitcoin_testnet/data

You can then see what's happening with:

Code:
screen -r bitcoin_testnet

To detach the screen, simply press Ctrl-A and then Ctrl-D. You'll be back in the console, and the command will continue running in the background.

Step 2: Install and run cgminer

We're going to use an updated repo of cgminer, this one has the latest changes from kano so that we can use the Compac F, plus some fixes that allow solo mining. Of course you might want to change the options to match your environment or miner.

Code:
cd ~/bitcoin_testnet/
git clone https://github.com/cmmodtools/cgminer
cd cgminer
./autogen.sh
CFLAGS="-O2 -Wall -march=native -fcommon" ./configure --enable-gecko
make

Now, in theory we should be ready, but we need one extra change. Open the file cgminer.c with your favorite text editor and change this line:

Code:
if (opt_btc_address[0] != '1') {

To this:

Code:
if (opt_btc_address[0] != '1' && opt_btc_address[0] != 'm' && opt_btc_address[0] != 'n'  ) {

Basically we need to also consider the first symbol of the testnet address, which is either n or m. OK, now run make again:
Code:
make

And now cgminer is compiled and ready. Let's create the configuration file:

Create a text file named ~/bitcoin_testnet/cgminer_testnet.conf with these contents:

Code:
{
"pools" : [
        {
                "url" : "http://YOUR_BITCOIN_CORE_IP:5000",
                "user" : "YOUR_RPC_USER",
                "pass" : "YOUR_RPC_PASSWORD"
        }
],

"btc-address": "YOUR_TESTNET_BTC_ADDRESS",

"gekko-compacf-freq" : "400",
"gekko-compacf-detect" : true,
"gekko-start-freq" : "300",
"gekko-mine2" : true,
"gekko-tune2" : "60",

"suggest-diff" : "442",
"failover-only" : true,
}

If you don't have a testnet btc address yet, note that Bitcoin Core now doesn't generate a wallet by default any more so you will need to create one. You can read about it in createwallet (https://developer.bitcoin.org/reference/rpc/createwallet.html). Then you should be able to getnewaddress (https://developer.bitcoin.org/reference/rpc/getnewaddress.html). Make sure to use the legacy format as that's what cgminer is expecting. You address should start with n or m.

Alternatively, you can simply use the GUI. To open the GUI, first stop the bitcoind process (Ctrl-C) and then run bitcoin-qt instead. There you'll be able to easily create wallets and generate addresses by clicking around.

And now you can simply run cgminer like this:

Code:
sudo /home/user/bitcoin_testnet/cgminer/cgminer -c /home/user/bitcoin_testnet/mine_testnet.conf

You can of course also run it in the background with screen, or run this at startup, etc. It will depend on your particular setup. If you're on a pi for example, you could create a bash shell executable (~/start_mining.sh) with that code and then simply add this to /etc/rc.local:

Code:
su - pi -c "screen -dm -S cgminer ~/start_mining.sh"

That's just one example, you can choose to run it however you prefer.

There it is, it should now say:

Code:
Solo mining to valid address: YOUR_TESTNET_BTC_ADDRESS

If you leave it overnight you should hit a few blocks with the Compac F USB miner.

Happy testnet solo mining!  ;D


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: NotATether on September 29, 2022, 11:30:07 AM
A few blocks overnight? That means the testnet difficulty is extremely low. Exactly what was the testnet difficulty when you wrote this guide?

I'm sure this will help a bunch of people who are setting up testnet bitcoin mining farms.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: nullama on September 29, 2022, 11:41:42 AM
A few blocks overnight? That means the testnet difficulty is extremely low. Exactly what was the testnet difficulty when you wrote this guide?

I'm sure this will help a bunch of people who are setting up testnet bitcoin mining farms.

The difficulty was in the millions:

Code:
Network diff set to 76.6M

But apparently testnet3 resets diff to 1 at some blocks, as those are the ones I hit:

Code:
Accepted Diff 1/1 GSF 0


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: d3bt3 on September 29, 2022, 11:46:15 AM
Nicely detailed guide!


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: ABCbits on September 29, 2022, 12:21:44 PM
Thanks for writing this guide. Although people who currently don't have ASIC better use Signet or Regtest, unless they also want to try their luck with solo mining on Bitcoin mainnet.

--snip--

But apparently testnet3 resets diff to 1 at some blocks, as those are the ones I hit:

Code:
Accepted Diff 1/1 GSF 0

Not at some blocks, but after 20 minutes since last block mined. That might be why you mine more block than @NotATether expected.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: n0nce on September 29, 2022, 04:39:52 PM
--snip--

But apparently testnet3 resets diff to 1 at some blocks, as those are the ones I hit:

Code:
Accepted Diff 1/1 GSF 0
Not at some blocks, but after 20 minutes since last block mined. That might be why you mine more block than @NotATether expected.
That's great to know! I wasn't aware of this rule. Will spin a Compac F up to mine some testnet BTC, then.. :D
I've never hit a solo block, so this should be a pretty cool experience.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: nullama on September 29, 2022, 11:40:24 PM
~snip~
That's great to know! I wasn't aware of this rule. Will spin a Compac F up to mine some testnet BTC, then.. :D
I've never hit a solo block, so this should be a pretty cool experience.

That was the motivation for me to do this in the first place  ;D

A couple of things you might want to do:

Keep the log of your first block:

Code:
sudo /home/user/bitcoin_testnet/cgminer/cgminer -c /home/user/bitcoin_testnet/mine_testnet.conf 2>> /home/user/bitcoin_testnet/my_first_block_log.txt

And you can of course write something into the coinbase of the block you found, just add this option to the config file:

Code:
--btc-sig <arg>     Set signature to add to coinbase when solo mining (optional)


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: nullama on September 30, 2022, 04:17:17 PM
And you can of course write something into the coinbase of the block you found, just add this option to the config file:

Code:
--btc-sig <arg>     Set signature to add to coinbase when solo mining (optional)

Do you know format of <arg> (such as HEX or plain text) or have example of working <arg>? I tried using search feature, but GitHub doesn't enable such feature on forked repository.

It's a simple text argument, the same as the btc-address argument you have to give in the config file. Just add another line with that.

Just try it out, that's the whole point of the testnet  ;D


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: nullama on October 04, 2022, 05:47:25 AM
~snip~
I'd do it if i have SHA-256 ASIC or testnet difficulty is low enough where i can try it with CPU or GPU ;D. That's also why i sometimes use Signet network to try/test something.

I just posted a new guide that doesn't require an ASIC (https://bitcointalk.org/index.php?topic=5415861.0). It only uses the CPU and/or the GPU of a normal computer, so you should be able to test it.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: n0nce on December 17, 2022, 01:55:53 PM
A few recommendations and fixes:

(1) Just add datadir=/home/user/bitcoin_testnet/data to the conf file, as well. Then you won't have to specify it through CLI anymore.
Now let's prepare the configuration file. Create a text file named bitcoin_testnet.conf in ~/bitcoin_testnet/ and put these content in it:

Code:
testnet=1
txindex=1
server=1
[test]
rpcport=5000
rpcallowip=YOUR_MINER_IP
rpcuser=YOUR_RPC_USER
rpcpassword=YOUR_RPC_PASSWORD
rpcbind=0.0.0.0

(2) There is a typo here. It should be --enable-gekko; otherwise it will compile just fine, but without Gekko support and you won't even notice. ;)
Code:
cd ~/bitcoin_testnet/
git clone https://github.com/cmmodtools/cgminer
cd cgminer
./autogen.sh
CFLAGS="-O2 -Wall -march=native -fcommon" ./configure --enable-gecko
make

(3) Cgminer complains about the comma at the end of the last line ("failover-only" : true,); it has to be removed to work.
Code:
{
"pools" : [
        {
                "url" : "http://YOUR_BITCOIN_CORE_IP:5000",
                "user" : "YOUR_RPC_USER",
                "pass" : "YOUR_RPC_PASSWORD"
        }
],

"btc-address": "YOUR_TESTNET_BTC_ADDRESS",

"gekko-compacf-freq" : "400",
"gekko-compacf-detect" : true,
"gekko-start-freq" : "300",
"gekko-mine2" : true,
"gekko-tune2" : "60",

"suggest-diff" : "442",
"failover-only" : true,
}

(4) For anyone unsure how to do this:
Code:
$ ./bitcoin-24.0.1/bin/bitcoin-cli -conf=/home/user/bitcoin_testnet/bitcoin_testnet.conf getnewaddress mining legacy
Then you should be able to getnewaddress (https://developer.bitcoin.org/reference/rpc/getnewaddress.html). Make sure to use the legacy format as that's what cgminer is expecting. You address should start with n or m.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: n0nce on December 18, 2022, 07:10:46 PM
Nullama, your guide worked great (besides the few points I raised)! Amazing; I hadn't expected to mine multiple blocks per day.
In fact, I managed to grab almost 30 blocks in 24h. :o Sitting at roughly 0.9tBTC right now.
Even consecutive blocks a few times... I must have very good network latency or something. 0:) Only a single Compac F hashing at just under 300GH/s.

It was an awesome experience to solo-mine again (and hit blocks!) and I'll definitely do it again if I need testnet coins and / or for validating a miner / miner driver software.

PS: Check last 24h of blocks for hidden message! ;)


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: BitcoinSoloMiner on December 18, 2022, 11:17:53 PM
What needs to be changed for this to work on main net? Remove testnet=1 ?


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: n0nce on December 19, 2022, 10:17:08 AM
In fact, I managed to grab almost 30 blocks in 24h. :o Sitting at roughly 0.9tBTC right now.
Even consecutive blocks a few times... I must have very good network latency or something. 0:) Only a single Compac F hashing at just under 300GH/s.
The result is much better compared with when i mined with GPU. It looks like getting tBTC isn't that hard for those who already own ASIC.
Interesting; I would have expected a CPU / GPU setup to work fine, too then. At difficulty 1, you need a fast network latency in my opinion, but also a block candidate immediately. In case the ASIC is 100x faster, that could play a bigger role than what I anticipated.
Again; I got consecutive blocks a few times, so I guess that a couple hundred GH suffice for consistent blocks rewards.

PS: Check last 24h of blocks for hidden message! ;)
Using blockchair help[1], did you send message "cgminer42 solo on Compac F"?
Yes ;) It is also displayed by mempool.space (https://mempool.space/testnet/address/mnPiocoteQkqzcy2MKsZPsViKnT3yMwhTv).
Oddly enough; someone just sent me ‎0.00017893 tBTC, which was no coinbase transaction. Typo? Never happened on mainnet to me, in all these years. :P



I mined:
Block 2411861 (https://mempool.space/testnet/block/0000000087038c83ae232843252f7a840e2a152f502392dbe64cfcd898bb5507), Block 2411862 (https://mempool.space/testnet/block/00000000cead31e68cd161220ba76abf7725ed988b00465f9220f5fd3c7be8f7) and Block 2411863 (https://mempool.space/testnet/block/0000000029fb0480cae528256732433c49fe3cbdb7b5811c8555a672643a1f90) in a row.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: nullama on December 20, 2022, 12:31:53 AM
Glad to see people now solo mining testnet bitcoins with this guide  ;D

https://i.imgur.com/DzjFg8j.png (https://mempool.space/testnet/block/00000000f584c0047a068d5d1c27fdf9344632fae4bd0b27a6ab2a48e94b2bb7)

https://i.imgur.com/3MbxbD1.png (https://mempool.space/testnet/block/000000009b14fce135d3f998ad662dcdce97042237c79831fb3df2a304690b46)

https://i.imgur.com/WozcMaM.png (https://mempool.space/testnet/block/0000000013207660b3bb450f3d553b39997461b8a4f4142f9902c7b93afa02d8)


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: LoyceMobile on January 27, 2023, 04:23:21 PM
In fact, I managed to grab almost 30 blocks in 24h. :o Sitting at roughly 0.9tBTC right now.
Even consecutive blocks a few times... I must have very good network latency or something. 0:) Only a single Compac F hashing at just under 300GH/s.
That's 50,000 times faster than my CPU mining! Would you be interested to try? I still have a synced testnet blockchain running, and could give you access. At low difficulty, I wonder if I made a mistake for not getting anything in days.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: LoyceMobile on January 28, 2023, 06:16:38 PM
How old/slow is your CPU? My old GPU manage to mine 4 blocks within 4 days, although 2 of them become stale block.
It's a quad core Xeon E3-1270 V2 @ 3.50GHz. That's why I'm wondering if I did something wrong somewhere.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: DaveF on January 29, 2023, 12:24:09 PM
Stale has less to do with mining and more to do with propagation. If you have all the cores screaming running the SHA did you have enough CPU power left over to do the rest and get it out to the world. It's not a lot of computing power, but if you don't have any to spare it could cause issues. The other half of stale is networking. How many nodes is your node talking to? If there is a lag there and more nodes see the other block then you loose.

Also, it could be other miners deliberately causing reorgs. I don't see the point on testnet, but if I have enough power I can just ignore some blocks figuring I will get 2 or 3 while the rest of the network gets 1. That would just be a waste, but I can see people messing around to see if they can do it.

-Dave


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: n0nce on January 29, 2023, 08:37:58 PM
Glad to see people now solo mining testnet bitcoins with this guide  ;D

https://i.imgur.com/DzjFg8j.png (https://mempool.space/testnet/block/00000000f584c0047a068d5d1c27fdf9344632fae4bd0b27a6ab2a48e94b2bb7)

https://i.imgur.com/3MbxbD1.png (https://mempool.space/testnet/block/000000009b14fce135d3f998ad662dcdce97042237c79831fb3df2a304690b46)

https://i.imgur.com/WozcMaM.png (https://mempool.space/testnet/block/0000000013207660b3bb450f3d553b39997461b8a4f4142f9902c7b93afa02d8)
You found my messages! Nice! :D I've got to plant some more (on a new address) then.. :P

In fact, I managed to grab almost 30 blocks in 24h. :o Sitting at roughly 0.9tBTC right now.
Even consecutive blocks a few times... I must have very good network latency or something. 0:) Only a single Compac F hashing at just under 300GH/s.
That's 50,000 times faster than my CPU mining! Would you be interested to try? I still have a synced testnet blockchain running, and could give you access. At low difficulty, I wonder if I made a mistake for not getting anything in days.
Sure, let's do it. Sorry, only just revisited this thread again.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: pooya87 on January 30, 2023, 03:28:24 AM
If you have all the cores screaming running the SHA did you have enough CPU power left over to do the rest and get it out to the world.
Usually in parallel programming (multi-thread) when one thread finds the answer it breaks out of it and sends a "signal" to other threads to break too. Which means after finding the new block at least part of CPU has to be freed to be used for the broadcasting process.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: LoyceV on January 30, 2023, 09:25:14 AM
If you use bitcoind or CLI environment, did you check whether you actually mine block but become stale?
As far as I could see, I never mined a block.

Quote
Did you follow this guide (which is meant for ASIC) or other one (which use bfgminer)?
I followed Solo mine testnet bitcoins with bfgminer, Bitcoin Core, and a CPU (https://bitcointalk.org/index.php?topic=5415861.0) (but somehow ended up posting in this thread).

If you have all the cores screaming running the SHA did you have enough CPU power left over to do the rest and get it out to the world.
I assume that's not a problem, Linux multitasking should be able to handle this. And I used nice for the miner.

Quote
The other half of stale is networking. How many nodes is your node talking to? If there is a lag there and more nodes see the other block then you loose.
If that would be the problem, I would have expected to see orphaned blocks, right?

Sure, let's do it. Sorry, only just revisited this thread again.
Thanks! I'll PM you the details (and let's switch back to the correct thread (https://bitcointalk.org/index.php?topic=5415861.0)).


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: SilverDesert on February 14, 2023, 01:19:14 PM
Hi,

I cannot install the cgminer on my Ubuntu 20.04.5 LTS. I keep on getting this error message:

make[2]: *** [Makefile:999: cgminer-cgminer.o] Error 1
make[2]: Leaving directory '/home/user/bitcoin_testnet/cgminer_phaenomenon'
make[1]: *** [Makefile:1894: all-recursive] Error 1
make[1]: Leaving directory '/home/user/bitcoin_testnet/cgminer_phaenomenon'
make: *** [Makefile:808: all] Error 2

I have installed and updated all the libraries and tried to do the whole process over and over but I get the same end result.

Any thoughts on this issue?


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: n0nce on February 15, 2023, 02:43:10 AM
Any thoughts on this issue?
The part of the log output showing the actual issue is missing. You can post the whole log in a code block here or if you feel unsure if there is any PII in it, send it to me via DM.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: nullama on February 15, 2023, 05:58:23 AM
Hi,

I cannot install the cgminer on my Ubuntu 20.04.5 LTS. I keep on getting this error message:

make[2]: *** [Makefile:999: cgminer-cgminer.o] Error 1
make[2]: Leaving directory '/home/user/bitcoin_testnet/cgminer_phaenomenon'
make[1]: *** [Makefile:1894: all-recursive] Error 1
make[1]: Leaving directory '/home/user/bitcoin_testnet/cgminer_phaenomenon'
make: *** [Makefile:808: all] Error 2

I have installed and updated all the libraries and tried to do the whole process over and over but I get the same end result.

Any thoughts on this issue?

As mentioned earlier, you're not sharing here the full log of errors, so it's hard to help with what you posted only.

And based on what you did post, it looks like you're not following the instructions exactly, as it seems like you're using a different repository (cgminer_phaenomenon instead of cmmodtools).

So, first, try following the guide correctly(instead of doing it wrong multiple times), and then if that doesn't work for you, post detailed logs.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: alexeyneu on February 15, 2023, 09:25:56 AM
Hi,

I cannot install the cgminer on my Ubuntu 20.04.5 LTS. I keep on getting this error message:

make[2]: *** [Makefile:999: cgminer-cgminer.o] Error 1
make[2]: Leaving directory '/home/user/bitcoin_testnet/cgminer_phaenomenon'
make[1]: *** [Makefile:1894: all-recursive] Error 1
make[1]: Leaving directory '/home/user/bitcoin_testnet/cgminer_phaenomenon'
make: *** [Makefile:808: all] Error 2

I have installed and updated all the libraries and tried to do the whole process over and over but I get the same end result.

Any thoughts on this issue?

As mentioned earlier, you're not sharing here the full log of errors, so it's hard to help with what you posted only.

And based on what you did post, it looks like you're not following the instructions exactly, as it seems like you're using a different repository (cgminer_phaenomenon instead of cmmodtools).

So, first, try following the guide correctly(instead of doing it wrong multiple times), and then if that doesn't work for you, post detailed logs.
i think you'd overcomplicated it really. it could be 400 lines cpp single file( with 50 lines go to simd switch/case) where you broadcast tx later yourself with a wallet.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: SilverDesert on February 15, 2023, 05:58:01 PM
Thanks all for your kind help and support.
I tried cmmodtools and got the same errors, then I tried phaenomenon. Same errors. Then finally I tried vthoang and it compiled perfectly. I couldn't figure what was causing the error.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: n0nce on February 15, 2023, 06:20:24 PM
i think you'd overcomplicated it really. it could be 400 lines cpp single file( with 50 lines go to simd switch/case) where you broadcast tx later yourself with a wallet.
What transaction? You want to write a miner which doesn't automatically submit a new block to the blockchain, but requires you to do it manually? :D That would be hilarious. You would mine so many stale blocks. I mean, a correct block at difficulty 1 will be stale in a matter of seconds, if not less. My ASIC alone, generates around 100 valid blocks per second as soon as the difficulty drops.

Furthermore, while you can submit raw transactions with some wallets, you cannot submit blocks without running Bitcoin Core.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: alexeyneu on February 15, 2023, 07:11:20 PM
so you have a block with only coinbase transaction. what can make it stalled after 5 seconds? and what means stalled? yeah for educational purpose it would be not bad to make it splitted from broadcast unit. It can be done like in openssl ( openssl.exe ) where you can make stand-alone utils instead


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: paid2 on February 15, 2023, 07:49:51 PM
so you have a block with only coinbase transaction. what can make it stalled after 5 seconds? and what means stalled? yeah for educational purpose it would be not bad to make it splitted from broadcast unit. It can be done like in openssl ( openssl.exe ) where you can make stand-alone utils instead

When a stale block is mined, it is not included in the longest version of the blockchain, and is therefore not valid.

Basically this can be due to network problems, by the time you distribute your block and it is spread among enough nodes, it may have been overtaken by another block or become invalid in the meantime. That's why when you mine solo it's very important to have the lowest possible network latency (we often say less than 100ms with the solo pool to be ok)

Please note that even if you see a reward for a stale block, you will never be able to spend it because the coins will be invalid. If I'm not mistaken, the coins are returned to the mempool
Fortunately most of the pools have a low rate of orphan and stale blocks, another problem is that the big pools accept to mine empty blocks without transaction, which is anything but good for the BTC network...


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: n0nce on September 18, 2023, 10:22:53 AM
Today I noticed something weird on the Testnet blockchain. We are getting blocks about every 2 minutes; difficulty is way too low. According to https://mempool.space/testnet, the difficulty was already raised in the last difficulty adjustment and it will be raised again today by 300%.

Was there a huge dip of hashpower a few weeks or months ago that tanked the difficulty or did someone point an enormous amount of miners to Testnet recently?


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: LoyceV on September 18, 2023, 10:26:44 AM
We are getting blocks about every 2 minutes; difficulty is way too low.
It came from several blocks per second, see Interesting behaivor of TESTNET difficulty (https://bitcointalk.org/index.php?topic=5466740.0).

This caused it:
The issue here is that the block of difficulty 1 after a 20 minute period was the last block in difficulty epoch.

And this made it last:
There is cap on how much it can rise (or fall) in each adjustment because if the time is smaller (or bigger) than a quarter of (or bigger than 4x) 2 weeks in seconds, it will fall down to default 302,400 (or 4,838,400) sec.
In other words in this case, even though it takes them minutes to mine 2016 blocks, the protocol assumes it takes 84 hours and increases the target based on that.



Note that there's not much left to mine on testnet: the block reward is now 0.024, about 300 times lower than Bitcoin mainnet.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: n0nce on September 18, 2023, 10:35:39 AM
This caused it:
The issue here is that the block of difficulty 1 after a 20 minute period was the last block in difficulty epoch.
Oh wow, that makes total sense! I never thought about that. Very interesting. It doesn't sound intended; more like a bug that should be fixed. On the other hand, I feel like nobody really cares too much about Testnet.

Edit: Since this happened before (https://bitcointalk.org/index.php?topic=5466740.msg62863773#msg62863773), I guess there will be no changes made!

And this made it last:
There is cap on how much it can rise (or fall) in each adjustment because if the time is smaller (or bigger) than a quarter of (or bigger than 4x) 2 weeks in seconds, it will fall down to default 302,400 (or 4,838,400) sec.
In other words in this case, even though it takes them minutes to mine 2016 blocks, the protocol assumes it takes 84 hours and increases the target based on that.
I'm aware of this, yes. There were even discussions about it here on Bitcointalk a while back.. ;)



Unfortunately, for now this means we won't have 20-minute block times in a while, i.e. no solo-mined tBTC. As the base difficulty has already exceeded 16M a few days ago. I expect it to take about 2-3 more epochs to reach normal levels again.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: vjudeu on September 18, 2023, 12:39:42 PM
Quote
Note that there's not much left to mine on testnet: the block reward is now 0.024, about 300 times lower than Bitcoin mainnet.
For that reason, it is very unlikely that testnet3 will ever be killed. Because now, it is the only network that is quite close to mainnet, and where you can test, what will happen when the basic block reward will be very small. It is also probably the first network, that will reach zero block reward, and then a lot of people will carefully observe, what will happen next.

By the way, testnet coins are worthless by definition. So, if nothing else will happen, then testnet3 will finally reach a state, where the basic block reward will truly represent the real value of those coins. In general, testnet3 could have zero supply, but some coins are needed as a spam protection.

Quote
On the other hand, I feel like nobody really cares too much about Testnet.
Well, if you use other test networks, then you can easily get much more coins. But as far as I know, testnet3 is the only truly decentralized test network, that we currently have. Signet is centralized, because of signet challenge. Regtest is centralized, because you can create a blockstorm on CPU, so it cannot be used online. So, for better or worse, we are left with testnet3, because only in this network you can test everything, including mining. But of course, it could be improved, and some improvements were proposed, so we will see, how it will evolve.

Quote
Unfortunately, for now this means we won't have 20-minute block times in a while, i.e. no solo-mined tBTC.
You can always have some 20-minute block with the minimal difficulty. After 20 minutes, there are two options:
1. Some block with difficulty one will be accepted.
2. Some block with the real difficulty will be accepted.
And even if you cannot produce some strong block, there is always a chance to produce some weak block. It can be reorged or not, it depends if someone will try to attack, and if you will be well-connected or not. Because when it comes to weak blocks, everyone can produce them, by using CPU, GPU, FPGA, ASIC or whatever. So, in that case, the winner is the one who will spread it faster to the rest of the network, and who will not be reorged. In those cases, CPUs are more important than other hardware, because they are used to send blocks, so it is a competition between nodes, and not between miners.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: LoyceV on September 18, 2023, 02:03:21 PM
Quote
Note that there's not much left to mine on testnet: the block reward is now 0.024, about 300 times lower than Bitcoin mainnet.
For that reason, it is very unlikely that testnet3 will ever be killed. Because now, it is the only network that is quite close to mainnet, and where you can test, what will happen when the basic block reward will be very small. It is also probably the first network, that will reach zero block reward, and then a lot of people will carefully observe, what will happen next.

By the way, testnet coins are worthless by definition. So, if nothing else will happen, then testnet3 will finally reach a state, where the basic block reward will truly represent the real value of those coins. In general, testnet3 could have zero supply, but some coins are needed as a spam protection.
Despite being worthless, people are still mining testnet. So I expect at least some of them to continue mining when there's no block reward left. That also means total fees can be under the dust limit. And even if they don't, with low difficulty just one miner is enough to continue the chain. We may see 1000 block 99% attacks once in a while.

Testnet coins already seem to be quite scarce, my guess is many are lost. After all, there's not much of a reason to be careful with worthless coins. That means testing in the future will have to happen with smaller and smaller amounts.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: n0nce on September 18, 2023, 02:41:30 PM
Quote
Unfortunately, for now this means we won't have 20-minute block times in a while, i.e. no solo-mined tBTC.
You can always have some 20-minute block with the minimal difficulty.
Not at the current difficulty. People are mining blocks every 2 minutes; there is realistically no chance for a 20 min time period without blocks.

After 20 minutes, there are two options:
1. Some block with difficulty one will be accepted.
2. Some block with the real difficulty will be accepted.
I know, that's what this whole thread is about and I've been scooping up tons of 1-difficulty blocks months ago, myself! :D

When it comes to weak blocks, everyone can produce them, by using CPU, GPU, FPGA, ASIC or whatever.
That is not true; LoyceV has shown it in an experiment and we've calculated it on paper, too. ASICs solve these blocks in a fraction of a second, meanwhile a (regular / not highest-end) CPU can need minutes or more; no advantage in internet speed will be enough to offset that.

In those cases, CPUs are more important than other hardware, because they are used to send blocks, so it is a competition between nodes, and not between miners.
That's incorrect.

This is where I was wrong:
The highest possible target (difficulty 1) is defined as 0x1d00ffff

The average time to find a block can be approximated by calculating:

Code:
time = difficulty * 2**32 / hashrate
The expected number of hashes we need to calculate to find a block with difficulty D is therefore

D * 2**256 / (0xffff * 2**208)
Actually, if we use this formula, it appears that with difficulty=1, you need on average 4295032833 or ~4.3GH to find a valid one. So on a 7MH/s machine, that's 613 seconds or 10 minutes. Impossible.. ;D

Code:
>>> 1 * 2**32 / 7000000
613.5667565714285

[...]



Code:
>>> 4.3e9 / 300e9 # hashes needed on average / hashrate of compac f
0.014333333333333333
With just 300GH/s and difficulty=1, it only takes it 0.0143s to go through those 7 million hashes.
This would also mean that the formula expecting @LoyceV's CPU to take on average 10 minutes, is correct.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: n0nce on September 19, 2023, 09:24:15 AM
There we go! Another block found, using the R909 actually. I like how it can quickly be set up without worrying about external fans and fine tuning any power settings. [Review coming... soonTM?]

Sadly the block reward is quickly dwindling in Testnet already.. +0.044tBTC this time.

Edit: Another one, an hour later! +‎0.02996310 tBTC
Edit 2: I got the immediately following block, too, at full difficulty. :o

With my previous calculations, that is very unlikely. I have 1.7TH/s, so:
Code:
Difficulty * 2^32 / Hashrate
67108864 * 2**32 / 1700e9
= 169547 seconds
= 47h
= almost 2 days

I'm basically expecting to need 2 days to mine a block at this difficulty, but did it in 3 minutes. Pretty neat.

Edit 3: I found another difficulty=1 block, as well as another at 67108864. How is that possible? I need to double-check my calculations...


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: vjudeu on September 20, 2023, 05:25:27 AM
Quote
no advantage in internet speed will be enough to offset that
Well, your gains will be always proportional to your power, in the long-term perspective. Which means, CPU-based miners will lose that battle very, very often, but once per N blocks, they could win. Or rather, to make it realistic: a pool of CPU miners could win.

Also, for that reason I left mining testnet3, and tried merged-mining with mainnet difficulty, but in a model, where each miner has its own difficulty, and is always rewarded every 10 minutes. Because one of the problems with testnet3 is that so much power is "wasted" into mining worthless coins. In that case, merged-mining fits better, because you can get coins that are almost worthless, but not quite. And you can easily test that kind of things inside LN, where normally you would have to pay, but if you are a miner, you can use your computing power as your currency. Also, because normally nodes don't want to pay you, what you can focus on is cheaper transactions: a node can give you some discount, or process some transactions for free, if you show your shares.

Quote
How is that possible?
It is simple: if you have any mining power, then mathematically, you can always hit a block. Your blocks are as good as produced by any huge miner. So, instead of hitting some block every 10 minutes, you can hit it every day, every week, or even every month. But it also means, once per year you can hit three blocks in a row, or something like that. But then, you should not fall into gambler's fallacy, and think that you will hit the next block in the same day, if you were extremely lucky for a while.

Quote
I need to double-check my calculations...
It is not something you can fix. Your calculations are correct. But the very low probability, close to zero, does not mean that something will not happen. Even if you reach zero in your calculations, you could still encounter such situation in the real world.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: LoyceV on September 20, 2023, 07:48:11 AM
Quote
no advantage in internet speed will be enough to offset that
Well, your gains will be always proportional to your power, in the long-term perspective. Which means, CPU-based miners will lose that battle very, very often, but once per N blocks, they could win.
There's no point in trying, if it's once in a million (or maybe billion). That means it will probably never happen, and that's what makes CPU mining futile.

Quote
Or rather, to make it realistic: a pool of CPU miners could win.
At difficulty 1, getting the data from your pool may already take more time than an ASIC miner needs to find the next block.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: n0nce on September 21, 2023, 04:34:41 PM
Quote
How is that possible?
It is simple: if you have any mining power, then mathematically, you can always hit a block. Your blocks are as good as produced by any huge miner. So, instead of hitting some block every 10 minutes, you can hit it every day, every week, or even every month. But it also means, once per year you can hit three blocks in a row, or something like that. But then, you should not fall into gambler's fallacy, and think that you will hit the next block in the same day, if you were extremely lucky for a while.
Actually, I got maybe 6 or 7 'regular difficulty' (67.108.864 - only 30% lower than what it should probably be) blocks in just a couple of days now. I have to check the numbers, but it seems very unlikely to be this lucky this often.

Quote
Or rather, to make it realistic: a pool of CPU miners could win.
At difficulty 1, getting the data from your pool may already take more time than than an ASIC miner needs to find the next block.
I believe so, too. Even a CompacF takes just a fraction of a second.
Actually, one reason that might explain how I'm able to get almost every single 1-difficulty block is that 'regular' miners connect to a pool, meanwhile with nullama's method, we connect to our own full nodes. This should surely result in a smaller delay.

I'll run some numbers as soon as I can; likelihood of mining a 67M block (normal distribution), as well as ratio of 1-diff blocks I was able to scoop up vs. the total number in a given period of time.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: Nexus9090 on May 28, 2024, 01:42:43 PM
What needs to be changed for this to work on main net? Remove testnet=1 ?


I know I'm late to the party with this one but, for mainnet you have to change the [test] to [main] as well as removing testnet=1

Code:
txindex=1
server=1
[main]
rpcport=5000
rpcallowip=YOUR_MINER_IP
rpcuser=YOUR_RPC_USER
rpcpassword=YOUR_RPC_PASSWORD
rpcbind=0.0.0.0

Anyone's guess as to if it'll actually mine a valid block, but it does seem to be running.

Code:
[2024-05-28 14:41:03.526] Solo mining to valid address: 3MbtxahpbkC1fGhvcWR7Ja38u8cwN9wzY8

With 500GH/s pointed at it, I guess I'll be waiting a while to find out.  ;D


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: ABCbits on May 29, 2024, 09:55:08 AM
What needs to be changed for this to work on main net? Remove testnet=1 ?
I know I'm late to the party with this one but, for mainnet you have to change the [test] to [main] as well as removing testnet=1

--snip--

With 500GH/s pointed at it, I guess I'll be waiting a while to find out.  ;D

Aside from low hashrate, you probably better use solo mining pool (such as ckpool or kano) which supposed to offer better block propagation speed.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: Nexus9090 on May 29, 2024, 11:22:05 AM
Aside from low hashrate, you probably better use solo mining pool (such as ckpool or kano) which supposed to offer better block propagation speed.

I know, alledgedly pools are supposed to have lower latency.

TBH I'm not so sure given I have a low latency GBit fibre connection to the internet I cant see why it'd be any less effective than a server in a data centre somewhere that only has a 400Mbit capped link tied to it. Unless the other bitcoin nodes are located in the same data centre, the latency between nodes would be similar.

Anyway, this was purely a fun experiment and technical exercise to see if I could get it to work.

I merged the changes from https://github.com/cmmodtools/cgminer into the last maintained version of cgminer 4.12.1 https://github.com/kanoi/cgminer/ and getting it to build, that was technical exercise 1.

Then the challenge was getting bitcoind running and accepting RPC connections for testnet, that worked ok then moving over to mainnet.

Once I'd figured that the named block in the config file was [main] not [MainNet] or [mainnet] it all started to work as it did for testnet, although it did take nearly 3 days to sync the blockchain.

Its been running overnight and statistically its performing similar to that of a pool looking at the volume of data exchanged and the current best share.

There's nothing presently that makes me think solo mining to a pool has any advantage, of course I dont know how things would perform should it hit a block and if the block would successfully propagate. Like I said it could be a very long time before I found out.


Never the less with that said, I guess it's easier to rely on someone else's technical expertise in setting up the bitcoind and maintaining the back-end of things.



Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: ABCbits on May 30, 2024, 09:09:14 AM
Aside from low hashrate, you probably better use solo mining pool (such as ckpool or kano) which supposed to offer better block propagation speed.
I know, alledgedly pools are supposed to have lower latency.

TBH I'm not so sure given I have a low latency GBit fibre connection to the internet I cant see why it'd be any less effective than a server in a data centre somewhere that only has a 400Mbit capped link tied to it. Unless the other bitcoin nodes are located in the same data centre, the latency between nodes would be similar.

--snip--

Never the less with that said, I guess it's easier to rely on someone else's technical expertise in setting up the bitcoind and maintaining the back-end of things.

Aside from latency, i also hear claim that node run by mining pool connected to other node which belong to different mining pool and node run by major company. Anyway, i agree the biggest reason to use them is we don't have to setup and maintain stuff.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: Nexus9090 on May 30, 2024, 09:58:54 AM
Aside from latency, i also hear claim that node run by mining pool connected to other node which belong to different mining pool and node run by major company. Anyway, i agree the biggest reason to use them is we don't have to setup and maintain stuff.

I don't think the nodes connected to other nodes is any different to any other machine connecting to a server elsewhere on the net, it still has to traverse the same gateways, routers, switches and other network infrastructure.

Bitcoind looks like it handles all of the node to node connections automatically anyway. In that respect it looks no different from any other DNS lookup or TCP/IP communication.

Presently I just can't see anything to support the claim that mining solo to a pool is any better than mining directly to bitcoind beyond the fact that someone else has taken the time to set-up and maintain the pool and all the backend side of it reducing the initial technical hassles.

Once I have a couple more mining boards built, I'll point them at my bitcoind server and let them run indefinately, if by chance (and it is a remote chance) they hit a block I'll celebrate that here!

Cheers

G.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: BitcoinSoloMiner on July 01, 2024, 03:12:36 PM
What needs to be changed for this to work on main net? Remove testnet=1 ?


I know I'm late to the party with this one but, for mainnet you have to change the [test] to [main] as well as removing testnet=1

Code:
txindex=1
server=1
[main]
rpcport=5000
rpcallowip=YOUR_MINER_IP
rpcuser=YOUR_RPC_USER
rpcpassword=YOUR_RPC_PASSWORD
rpcbind=0.0.0.0

Anyone's guess as to if it'll actually mine a valid block, but it does seem to be running.

Code:
[2024-05-28 14:41:03.526] Solo mining to valid address: 3MbtxahpbkC1fGhvcWR7Ja38u8cwN9wzY8

With 500GH/s pointed at it, I guess I'll be waiting a while to find out.  ;D

Thanks for the reply, I am going to get a bitaxe supra I think and do this method. What is giving you 500G/H? How much power?

Goodluck!


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: Nexus9090 on July 01, 2024, 04:55:04 PM
-SNIP-

Thanks for the reply, I am going to get a bitaxe supra I think and do this method. What is giving you 500G/H? How much power?

Goodluck!

I have something of my own creation a somewhat overclocked (725Mhz) BM1397 that averages 480GH/s, its using about 18Watts at the DC port including the power for the fan and the RaspberryPI Zero 2W thats controlling it.

Presently its pointing at CK pool, but I had it running on my local node for about a week without any issue.

Updating Kano's CGMiner is a bit of a chore but its achievable provided you follow the differences carefully.

Good luck!


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: vjudeu on July 09, 2024, 07:28:17 AM
A bash script to mine coins on derived addresses, to improve privacy:
Code:
$ cat mining4.sh 
nonce=0
while [ 1 ]
do
  numberofblocks=$(./bitcoin-cli -testnet4 getblockcount)
  address=$(./bitcoin-cli -testnet4 deriveaddresses "wpkh([badc0ded/84h/1h/0h]tpub.../0/*)#00000000" "[$numberofblocks,$numberofblocks]" | grep "tb1q")
  echo ./bitcoin-cli -testnet4 generatetoaddress 1 $address 100000000 > address.sh
  ./address.sh
  echo nonce: $nonce
  ((nonce=nonce+1))
done
Of course, instead of bitcoin-cli, the same trick can be used to mine on mining pools. However, note that many pools require producing at least N shares before paying anything, so make sure you produce enough, before you switch to some other address. But: as far as I understand it, that method of switching addresses seems to be good enough, right?

I also thought about mining on silent payment addresses, but I have to think about deriving it properly, because existing clients don't understand "sp1q" addresses.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: nullama on July 09, 2024, 12:29:46 PM
~snip~
I merged the changes from https://github.com/cmmodtools/cgminer into the last maintained version of cgminer 4.12.1 https://github.com/kanoi/cgminer/ and getting it to build

Nice, I wonder if kano (https://bitcointalk.org/index.php?action=profile;u=36044) would be OK bringing those changes upstream.

At least for me, @kano branch of cgminer is the one that I use whenever I build cgminer after CKolivas stopped development.

Hopefully there is only one branch with all the required changes.


Title: Re: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F
Post by: Nexus9090 on July 09, 2024, 01:57:00 PM
~snip~
I merged the changes from https://github.com/cmmodtools/cgminer into the last maintained version of cgminer 4.12.1 https://github.com/kanoi/cgminer/ and getting it to build

Nice, I wonder if kano (https://bitcointalk.org/index.php?action=profile;u=36044) would be OK bringing those changes upstream.

At least for me, @kano branch of cgminer is the one that I use whenever I build cgminer after CKolivas stopped development.

Hopefully there is only one branch with all the required changes.

The changes aren't that difficult to do if you're careful patching them into the correct locations. If I recall most of it is in cgminer.c and there's only a handfull of functions that are affected.

I'm no expert on driving GIT otherwise I'd create my own branch with the adaptations already done.

I don't know if Kano is interested in porting in the GBT mods from cmmodtools, they released these updates quite a long time ago and its never made it back into the master branch.