Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: bc1plainview on June 07, 2024, 02:30:00 AM



Title: Requesting Testnet4 tBTC
Post by: bc1plainview on June 07, 2024, 02:30:00 AM
I am looking to set up testnet4 to begin testing my current project without needing to index all the testnet3 blocks. However, I haven't been able to find a testnet4 faucet yet. Would anyone be willing to share some tBTC with me at the following address?

tb1preru33xeap6v33g0jaypn7x2dsuheyny0n0hh48wany8h8kz98us8l9rv9

Thanks in advance!


Title: Re: Requesting Testnet4 tBTC
Post by: vjudeu on June 07, 2024, 05:13:31 AM
Quote
However, I haven't been able to find a testnet4 faucet yet.
Why would anyone run a faucet for the network, which is not yet finalized, and can be resetted at any time?

Quote
Would anyone be willing to share some tBTC with me at the following address?
Why you need a faucet, if you can mine a block on your CPU? For example, note how many blocks were mined by wiz: https://mempool.space/testnet4/mining/pool/wiz

One of the latest blocks: https://mempool.space/testnet4/block/0000000012bfe95b1e2fcdccf0f855792a051e2412287869c758491a5cacdbf7

See? Those blocks have just the minimal difficulty. You need only CPU to mine it.


Title: Re: Requesting Testnet4 tBTC
Post by: punk.zink on June 08, 2024, 02:53:32 AM
I am looking to set up testnet4 to begin testing my current project without needing to index all the testnet3 blocks. However, I haven't been able to find a testnet4 faucet yet. Would anyone be willing to share some tBTC with me at the following address?

It seems that you are not serious enough to find out about the testnet4 faucet site. If you really need testnet4 tBTC, you can get some tBTC from the sites

https://coinfaucet.eu/en/btc-testnet4/
https://mempool.space/testnet4/faucet
https://testnet4.anyone.eu.org/

As explained above, the testnet4 network https://mempool.space/testnet4 is not final, and it may not be this network that will be used later.


Title: Re: Requesting Testnet4 tBTC
Post by: ercewubam on June 09, 2024, 02:53:36 PM
Test coins are weird: some of them are free (https://mempool.space/testnet4/tx/8f4cf55da2abc21ac4f5d176bd5aefcfdc91852343dbfe91215d7c8afa966baf), some of them are burned (https://mempool.space/testnet4/tx/c3750456a633ddc6493b26fd5ef288b339e84f231f31128fd6d84276b8871abf). But it seems you got (https://mempool.space/testnet4/tx/ffa3f9949b1c6cafc1b8aa8780eb0d6ed92c101c577fb3cc4b3babc45053dd0b) what you wanted anyway.


Title: Re: Requesting Testnet4 tBTC
Post by: bc1plainview on June 10, 2024, 05:45:31 AM
Test coins are weird: some of them are free (https://mempool.space/testnet4/tx/8f4cf55da2abc21ac4f5d176bd5aefcfdc91852343dbfe91215d7c8afa966baf), some of them are burned (https://mempool.space/testnet4/tx/c3750456a633ddc6493b26fd5ef288b339e84f231f31128fd6d84276b8871abf). But it seems you got (https://mempool.space/testnet4/tx/ffa3f9949b1c6cafc1b8aa8780eb0d6ed92c101c577fb3cc4b3babc45053dd0b) what you wanted anyway.

to whoever did this, thanks a ton!


Title: Re: Requesting Testnet4 tBTC
Post by: mocacinno on June 10, 2024, 06:02:42 AM

Why you need a faucet, if you can mine a block on your CPU? For example, note how many blocks were mined by wiz: https://mempool.space/testnet4/mining/pool/wiz


Just for your information: no, you can't... The diff is already sky-high (well, not sky-high compared to the main net, but sky-high compared to a cpu-mining setup)... I did the math a couple of weeks ago, and at that time, on average, it would take several hundreds of years to mine a block using your cpu on testnet 4. I have no idear why people would run ASIC's on a testnet, but apparently, they do...

EDIT: here are the current stats from my testnet4 node... A lot lower than a couple weeks ago, but still waaaaaaaaay to high for cpu mining:

Code:
{
  "blocks": 29322,
  "difficulty": 32874715.22081029,
  "networkhashps": 71876496349832.81,
  "pooledtx": 272,
  "chain": "testnet4",
  "warnings": [
    "This is a pre-release test build - use at your own risk - do not use for mining or merchant applications"
  ]
}



Title: Re: Requesting Testnet4 tBTC
Post by: vjudeu on June 10, 2024, 06:45:59 AM
Quote
Just for your information: no, you can't...
Yes, I can, and I did. It is always a lottery, but I can put something in the next coinbase, if you want. There is always a chance, that it will take some time, because of frequent chain reorganizations, but it is possible.

Quote
The diff is already sky-high
The diff for CPU mining is always one. Obviously, I won't try mining any more difficult blocks on CPUs.

Also see: https://bitcointalk.org/index.php?topic=5468925
Connected topic: https://blog.lopp.net/griefing-bitcoin-testnet/

And note, that on testnet4, some tricks are easier to achieve, than on testnet3 (as long as the basic block reward is still quite high).


Title: Re: Requesting Testnet4 tBTC
Post by: mocacinno on June 10, 2024, 07:23:46 AM
I have mined on testnet3 in the past, i'm well aware of the fact that the difficulty can reset to 1. But still, when you cpu mine at a couple dozen Mh/s, you're still vastly outnumbered against a multi Terrahash/second asic that's also mining at the same difficulty.

When i created my testnet4 node container, i tested out the cpuminer that i included into my image for 2 weeks to see if it was stable, and i did not solve a single block... Sure, it might not be impossible, but your odds are really low.

EDIT: truth be told, based on the info i got from my node, it does seem like a lot of people did turn off their ASIC's since i last tested my container.... A couple weeks ago, my node estimated the total network hashrate to be >400Th/s, whilst now it's only >70 Th/s, and i do indeed see blocks being mined at difficulty 1 (which was not the case a couple weeks ago). So yes, maybe at the moment it might be possible to mine a couple testnet4 blocks with your cpu if you're really lucky.


Title: Re: Requesting Testnet4 tBTC
Post by: vjudeu on June 10, 2024, 10:09:12 AM
Quote
But still, when you cpu mine at a couple dozen Mh/s, you're still vastly outnumbered against a multi Terrahash/second asic that's also mining at the same difficulty.
It is obvious, that you have to prepare some blocks in advance (for example in testnet3, you have everything filled, up to 2 hours in the future). And then, it is more related to network connections, than to the computing power. You just prepare a block, and you have your block with difficulty one, vs some ASIC block, also with difficulty one. Then, it is not about "who will mine it faster", but rather about "who will propagate it faster".

And of course, ASIC miners rule that world of test chains, so you have to mine your blocks around theirs. But still, it is possible to propagate some block faster, than some ASIC will do, because both players will prepare them in advance, and then the competition is related only to network connections.

Another thing is that ASIC miners potentially could always reorg CPU-mined blocks, but for some reason they don't. It is more profitable to mine a strong, regular block on top of them, because then, you can pick any block time you want. In case of CPU block times, it is more restricted.

Quote
i tested out the cpuminer that i included into my image for 2 weeks to see if it was stable, and i did not solve a single block...
It should mine at least some blocks, but they probably were reorged. If that's the case, then you have to improve the code for your node, not for your mining equipment. For example, to mine testnet blocks easily, I slighttly modified Bitcoin Core, and since then, it works on my CPU.

Quote
So yes, maybe at the moment it might be possible to mine a couple testnet4 blocks with your cpu if you're really lucky.
I repeated the same thing on testnet3, just to be sure. Now, I can mine on CPU in both networks, but testnet4 is more stable. However, mining in testnet3 is still quite good idea, if you want to test block rewards, based on transaction fees.


Title: Re: Requesting Testnet4 tBTC
Post by: garlonicon on June 10, 2024, 04:41:23 PM
Quote
Just for your information: no, you can't...
Sure, but my node, and some block explorers think otherwise:

https://mempool.space/testnet/address/tb1qvy67actlsslmvwpjzk9hnx6jf04g5y26sz5crx
https://mempool.space/testnet4/address/tb1qvy67actlsslmvwpjzk9hnx6jf04g5y26sz5crx

Public key: 0261c2c04c8133b863d9df2ef4082c21074f8849ef86ddf54b13e07d6c828faac6
Legacy address: mpNxT11hJpGH2YhUgQKZm2S1rBnnieWbHr
Segwit address: tb1qvy67actlsslmvwpjzk9hnx6jf04g5y26sz5crx

Code:
verifymessage "mpNxT11hJpGH2YhUgQKZm2S1rBnnieWbHr" "IORhFfwQl/VK8NU6sQrRxQ6pqoGjHTElUirU+H0lrsyPKE+cOYUZ3TNx3JF8WM69TvhfwZ9iafDk2f6WK1tZZMg=" "June 10, 2024, 04:13:45 PM: I am Garlo Nicon, and I confirm in topic 5499150 on bitcointalk, that I can mine new blocks in testnet3 and testnet4, by using address tb1qvy67actlsslmvwpjzk9hnx6jf04g5y26sz5crx. All of those blocks have minimal difficulty, equal to one, and were mined on my own CPU."
true


Title: Re: Requesting Testnet4 tBTC
Post by: detmm244 on June 10, 2024, 06:32:50 PM
Quote
Just for your information: no, you can't...
Sure, but my node, and some block explorers think otherwise:

https://mempool.space/testnet/address/tb1qvy67actlsslmvwpjzk9hnx6jf04g5y26sz5crx
https://mempool.space/testnet4/address/tb1qvy67actlsslmvwpjzk9hnx6jf04g5y26sz5crx

Public key: 0261c2c04c8133b863d9df2ef4082c21074f8849ef86ddf54b13e07d6c828faac6
Legacy address: mpNxT11hJpGH2YhUgQKZm2S1rBnnieWbHr
Segwit address: tb1qvy67actlsslmvwpjzk9hnx6jf04g5y26sz5crx

Code:
verifymessage "mpNxT11hJpGH2YhUgQKZm2S1rBnnieWbHr" "IORhFfwQl/VK8NU6sQrRxQ6pqoGjHTElUirU+H0lrsyPKE+cOYUZ3TNx3JF8WM69TvhfwZ9iafDk2f6WK1tZZMg=" "June 10, 2024, 04:13:45 PM: I am Garlo Nicon, and I confirm in topic 5499150 on bitcointalk, that I can mine new blocks in testnet3 and testnet4, by using address tb1qvy67actlsslmvwpjzk9hnx6jf04g5y26sz5crx. All of those blocks have minimal difficulty, equal to one, and were mined on my own CPU."
true

how to mine testnet4 useing cpu
please guide me step by step
how to cerate testnet4 address
i am using to create testnet address
https://www.bitaddress.org/bitaddress.org-v3.3.0-SHA256-dec17c07685e1870960903d8f58090475b25af946fe95a734f88408cef4aa194.html?testnet=true

example
Bitcoin testnet Address
mk4Lmwd1g787twjQYbswdotpYDz9XVD3hH
Private Key
cPLfGZGNgVsYvGfUTP62JdPBzHhL3jjqQ82RobkamETm7WzdgcGE

this testnet mk4Lmwd1g787twjQYbswdotpYDz9XVD3hH adderess also use to testnet4 ??


Title: Re: Requesting Testnet4 tBTC
Post by: vjudeu on June 10, 2024, 08:05:18 PM
Quote
how to mine testnet4 useing cpu
Code:
$ cat mining4.sh
nonce=0
while [ 1 ]
do
  ./bitcoin-cli --testnet4 generatetoaddress 1 mk4Lmwd1g787twjQYbswdotpYDz9XVD3hH 100000000
  echo nonce: $nonce
  ((nonce=nonce+1))
done

Quote
please guide me step by step
1. Download, build and run Bitcoin Core, made by fjahr, from branch 2024-04-testnet-4-fix: https://github.com/fjahr/bitcoin/tree/2024-04-testnet-4-fix
2. Generate some new address (or import it into Bitcoin Core).
3. Start mining. Most likely, your blocks will be ignored, but you will see them in your GUI, if you use Bitcoin Core as a wallet.

Quote
how to cerate testnet4 address
Code:
getnewaddress
mk4Lmwd1g787twjQYbswdotpYDz9XVD3hH
dumpprivkey mk4Lmwd1g787twjQYbswdotpYDz9XVD3hH
cPLfGZGNgVsYvGfUTP62JdPBzHhL3jjqQ82RobkamETm7WzdgcGE

Quote
this testnet mk4Lmwd1g787twjQYbswdotpYDz9XVD3hH adderess also use to testnet4 ??
Yes: https://mempool.space/testnet4/address/mk4Lmwd1g787twjQYbswdotpYDz9XVD3hH


Title: Re: Requesting Testnet4 tBTC
Post by: garlonicon on August 05, 2024, 05:28:26 AM
BIP for Testnet4 is now officially merged: https://github.com/bitcoin/bips/blob/master/bip-0094.mediawiki

Edit:
Quote
I did the math a couple of weeks ago, and at that time, on average, it would take several hundreds of years to mine a block using your cpu on testnet 4.
This sentence didn't age well, because now I can produce something around 10% of testnet4 blocks, and all of them are CPU-mined.


Title: Re: Requesting Testnet4 tBTC
Post by: LoyceV on August 05, 2024, 11:42:06 AM
This sentence didn't age well, because now I can produce something around 10% of testnet4 blocks, and all of them are CPU-mined.
Can you get me an Eli5 for this? I tried for testnet3 in the past, but didn't mine anything after a week and gave up.


Title: Re: Requesting Testnet4 tBTC
Post by: garlonicon on August 05, 2024, 04:53:39 PM
My comment in testnet4 pull request: https://github.com/bitcoin/bitcoin/pull/29775#issuecomment-2218329966
My code changes: https://bitcointalk.org/index.php?topic=5496494.msg64205870#msg64205870

How to reproduce:

1. Go to: https://github.com/fjahr/bitcoin/tree/2024-04-testnet-4-fix
2. Execute "git clone" on this repo, or get source code in any other way.
3. Edit "src/chain.h" and "src/node/miner.cpp" exactly as I presented on forum.
4. Build Bitcoin Core from source code.
5. Apply vjudeu's scripts from above in your terminal (they are for Bash on Linux, but you can use "Git Bash for Windows", or modify it slightly, to get it working under Windows).
6. See increasing nonces in your terminal, and observe mined blocks.

Quote
but didn't mine anything after a week and gave up
If you use my method, and your block will have the minimal difficulty, and will be valid, then it is guaranteed to always see it in your Bitcoin Core client, as long as your private keys are properly imported. It may be stale, it may be not accepted by other nodes, but it will be always visible.

Also, if you want to see any block, then you can use a simple shortcut:

1. Open Bitcoin Core.
2. Type "generatetoaddress 1 <address> 100000000".
3. Repeat it several times manually (or use bitcoin-cli). As long as your difficulty is set into the minimum, you should get a block after around 4 billion hashes (or 40 nonces, if you use that line above inside some loop).

Long time ago, when I started, I simply used "generatetoaddress" command manually, and modified my system clock, to set it 20 minutes after the last block. But: if you think about a stable environment, then modifying the source code is easier, because then your node will report the correct time, and you will not see a lot of warnings saying that "please check your system clock, because it may be set incorrectly". Also, if some blocks will already be in the future, you won't be forced to mine blocks with the real difficulty (by the way, once per 2016 blocks, the real difficulty is used anyway; you can of course run your CPUs 24/7, but I usually turn off my miners, and wait for ASICs to do that job).

Also, a good starting point is regtest, or signet with signetchallenge set to OP_TRUE. Then, you will always see some generated blocks.

So, to sum up: you can always mine blocks, on testnet, mainnet, signet, regtest, just everywhere. Command line and "generatetoaddress" is your friend. The hardest part is just setting your system clock 20 minutes after the last block. But if you can cover that, or if your Bitcoin Core can give you future timestamps in your block templates, then you are good to go.


Title: Re: Requesting Testnet4 tBTC
Post by: vjudeu on August 05, 2024, 05:27:41 PM
Quote
Can you get me an Eli5 for this?
Baby steps: run mainnet in offline mode, without any connections, start from the Genesis Block, and mine a single block, on top of that, by using Bitcoin Core. For example:
Code:
$ ./bitcoin-qt -noconnect
generatetodescriptor 1 "pk(0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798)#gn28ywm7" 1000000000
[
  "0000000085c70827e0b4e5590d820d66dce32ef4cdbab06968c6372a45d51fa6"
]
getblock 0000000085c70827e0b4e5590d820d66dce32ef4cdbab06968c6372a45d51fa6
{
  "hash": "0000000085c70827e0b4e5590d820d66dce32ef4cdbab06968c6372a45d51fa6",
  "confirmations": 1,
  "height": 1,
  "version": 536870912,
  "versionHex": "20000000",
  "merkleroot": "20bb379fd28f090730ac2d207b92c6e95c69dfa48a8118577edb20e068c9a65d",
  "time": 1722878378,
  "mediantime": 1722878378,
  "nonce": 3526334891,
  "bits": "1d00ffff",
  "difficulty": 1,
  "chainwork": "0000000000000000000000000000000000000000000000000000000200020002",
  "nTx": 1,
  "previousblockhash": "000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f",
  "strippedsize": 225,
  "size": 225,
  "weight": 900,
  "tx": [
    "20bb379fd28f090730ac2d207b92c6e95c69dfa48a8118577edb20e068c9a65d"
  ]
}
decoderawtransaction 02000000010000000000000000000000000000000000000000000000000000000000000000ffffffff025100ffffffff0200f2052a0100000023210279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798ac0000000000000000266a24aa21a9ede2f61c3f71d1defd3fa999dfa36953755c690689799962b48bebd836974e8cf900000000
{
  "txid": "20bb379fd28f090730ac2d207b92c6e95c69dfa48a8118577edb20e068c9a65d",
  "hash": "20bb379fd28f090730ac2d207b92c6e95c69dfa48a8118577edb20e068c9a65d",
  "version": 2,
  "size": 144,
  "vsize": 144,
  "weight": 576,
  "locktime": 0,
  "vin": [
    {
      "coinbase": "5100",
      "sequence": 4294967295
    }
  ],
  "vout": [
    {
      "value": 50.00000000,
      "n": 0,
      "scriptPubKey": {
        "asm": "0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798 OP_CHECKSIG",
        "desc": "pk(0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798)#gn28ywm7",
        "hex": "210279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798ac",
        "type": "pubkey"
      }
    },
    {
      "value": 0.00000000,
      "n": 1,
      "scriptPubKey": {
        "asm": "OP_RETURN aa21a9ede2f61c3f71d1defd3fa999dfa36953755c690689799962b48bebd836974e8cf9",
        "desc": "raw(6a24aa21a9ede2f61c3f71d1defd3fa999dfa36953755c690689799962b48bebd836974e8cf9)#cav96mf3",
        "hex": "6a24aa21a9ede2f61c3f71d1defd3fa999dfa36953755c690689799962b48bebd836974e8cf9",
        "type": "nulldata"
      }
    }
  ]
}
If you can get it right, then you can mine blocks with the minimal difficulty, and we can go further. This simple exercise is what I started with, long time ago, when I was curious, how to mine blocks with Bitcoin Core.


Title: Re: Requesting Testnet4 tBTC
Post by: pooya87 on August 06, 2024, 03:46:34 AM
If you can get it right, then you can mine blocks with the minimal difficulty, and we can go further. This simple exercise is what I started with, long time ago, when I was curious, how to mine blocks with Bitcoin Core.
Using RegTest is far better since the fixed difficulty there is a lot lower than the minimum difficulty in both MainNet and TestNet. This means you don't want to waste time waiting for your computer to compute millions of hashes to find the next block at difficulty one. On RegTest you'd generate blocks almost immediately.


Title: Re: Requesting Testnet4 tBTC
Post by: vjudeu on August 06, 2024, 05:45:43 AM
Quote
This means you don't want to waste time waiting for your computer to compute millions of hashes to find the next block at difficulty one.
But that's the whole point! You want to do that, to make sure, that you are ready for testnet3 or testnet4 mining. For example, you want to abandon the idea of using unoptimized Python code for testnet mining. Or: a similar idea of using "sha256sum", running in a bash loop (and creating and destroying the whole ELF stack for computing every single hash).

1. If you mine with regtest difficulty, then you don't know, if you hit correct blocks by accident or not. Then, you can even manually increase nonce, and use "submitblock", to mine anything there.
2. In regtest, you have no difficulty adjustments, so you don't know, how many testnet blocks you could mine, with your current setup. You have to soft-fork that feature, or switch to signet, to learn it.
3. If your tests require more than 15k coins, then you won't do that on regtest. All other networks have 21 million coins. You need to cycle 15k coins, 1400 times, to reach similar outcome.
4. You cannot measure the actual effort of mining blocks with regtest. You can mine with maximum speed of 43,200 blocks per two hours, and increasing your computing power won't change it.

Also, I wonder, if I should create a pull request, which will bring CPU-only mining into Bitcoin Core, as a feature. Then, it could be similar to grinding blocks from bitcoin-util: it will always give you a block, matching the minimal difficulty, on all networks, including mainnet. Because in general, that's how testnet should probably work: you have a mainnet chain, up to the block 789,012, and then you say "I want to mine a block 789,013, with those transactions, and test that scenario". Because if testnet blocks are not stale, then they always have a chance to get some value, and be traded on centralized exchanges.


Title: Re: Requesting Testnet4 tBTC
Post by: garlonicon on August 29, 2024, 07:12:03 AM
Quote
but it's easier to just wait for testnet3 to be shredded and replaced by the new version by default
New version released: https://groups.google.com/g/bitcoindev/c/EmAOO-Nbmzw

It is release candidate, but it contains fully working testnet4 implementation.

Of course, for CPU-based mining, some modifications are needed, but I think grinding coins with bitcoin-util should still be possible. I will think about some kind of such implementation.


Title: Re: Requesting Testnet4 tBTC
Post by: LoyceV on August 29, 2024, 07:53:11 AM
Test coins are weird: some of them are free (https://mempool.space/testnet4/tx/8f4cf55da2abc21ac4f5d176bd5aefcfdc91852343dbfe91215d7c8afa966baf)
Can you explain this one?



Title: Re: Requesting Testnet4 tBTC
Post by: BlackHatCoiner on August 29, 2024, 08:14:59 AM
Test coins are weird: some of them are free (https://mempool.space/testnet4/tx/8f4cf55da2abc21ac4f5d176bd5aefcfdc91852343dbfe91215d7c8afa966baf), some of them are burned (https://mempool.space/testnet4/tx/c3750456a633ddc6493b26fd5ef288b339e84f231f31128fd6d84276b8871abf).
Both the "free" and the "burned" link to the same transaction. I get that the coinbase reward is burned, because it is sent to OP_RETURN, but where are the free coins?

Of course, for CPU-based mining, some modifications are needed, but I think grinding coins with bitcoin-util should still be possible. I will think about some kind of such implementation.
What's the recommended way to mine testnet4 with CPU? Is it possible directly from Core with generatetoaddress? And, is there mining software for testnet that is configurable with GPUs?


Title: Re: Requesting Testnet4 tBTC
Post by: LoyceV on August 29, 2024, 08:30:17 AM
Test coins are weird: some of them are free (https://mempool.space/testnet4/tx/8f4cf55da2abc21ac4f5d176bd5aefcfdc91852343dbfe91215d7c8afa966baf), some of them are burned (https://mempool.space/testnet4/tx/c3750456a633ddc6493b26fd5ef288b339e84f231f31128fd6d84276b8871abf).
Both the "free" and the "burned" link to the same transaction.
They have different txids:
"free": 8f4cf55da2abc21ac4f5d176bd5aefcfdc91852343dbfe91215d7c8afa966baf
"burned": c3750456a633ddc6493b26fd5ef288b339e84f231f31128fd6d84276b8871abf


Title: Re: Requesting Testnet4 tBTC
Post by: vjudeu on August 29, 2024, 08:35:52 AM
Quote
Can you explain this one?
Those coins were sent to OP_TRUE. Which means, that anyone could sweep them, as it was done here: https://mempool.space/testnet4/tx/09096e1e6fb31f33f3de40b9a1d76908e565e9646e5aed4e0813e0a2a799fc4c

As you can see, no signatures were provided.

Similar case was in https://mempool.space/testnet4/tx/3a72c025a8995fe94af9ec00d61046f55fe18133ebb98f290f81aa68f3bc8fc1
The previous script was simply OP_TRUE, but that one was P2WSH, with OP_TRUE, which made it standard.

Quote
Both the "free" and the "burned" link to the same transaction.
Huh?

First link: https://mempool.space/testnet4/tx/8f4cf55da2abc21ac4f5d176bd5aefcfdc91852343dbfe91215d7c8afa966baf
Second link: https://mempool.space/testnet4/tx/c3750456a633ddc6493b26fd5ef288b339e84f231f31128fd6d84276b8871abf

They are different. But yes, those free coins are now taken, so you have to find something else.

Quote
What's the recommended way to mine testnet4 with CPU?
The easiest way is to make two small changes in the source code (https://bitcointalk.org/index.php?topic=5496494.msg64205870#msg64205870), and just recompile it. But there are also other ways.

Quote
Is it possible directly from Core with generatetoaddress?
Yes, if your Bitcoin Core implementation will always give you the block template, moved forward by 20 minutes.

Quote
And, is there mining software for testnet that is configurable with GPUs?
You have regular Bitcoin Core node. You can connect CPUs, GPUs, FPGAs, ASICs, whatever. The most difficult part is to get the block header, which has the minimal difficulty. For example, if you call "getblocktemplate", you should see this:
Code:
$ ./bitcoin-cli -testnet4 getblocktemplate '{"rules": ["segwit"]}'
{
  "capabilities": [
    "proposal"
  ],
  "version": 536870912,
  "rules": [
    "csv",
    "!segwit",
    "taproot"
  ],
  "vbavailable": {
  },
  "vbrequired": 0,
  "previousblockhash": "00000000000000308df706a8736dfe78eee7cea416adc1dc5bfcf4bc11202f6c",
  "transactions": [
    {
      "data": "020000000001047ce0d3213f28576ae83c57b920f4e20538d5e6d4e07ca488385aa0f1931e64110000000000fdffffff248aa47fdced15c751d2d300f1cb31ffef9d2f3a49215f6532104a1c78038f410100000000fdffffffb40af33649e7adf8e7efd2b1f01a2bee16ba1586b1a4e572b3c18f47d98f598e0000000000fdffffff2aad614089f83012a5975c807eb903fdb3ee83c843a6612186de6290c79520ce0100000000fdffffff02534d09000000000017a9145902315224c25891835b6821cd1cdc551de83238870e800a00000000001976a91498bde795388b03245d24953077dab58b24a7293488ac024730440220492f752d2b5ed7e0f0315ae1a6bd801488d88ffe340dfe9601b8918463bdfb0f022038a58690c81d4b071ac723936087af0317b2f226483e76b08c99ed7191ddef6d012102685699ed77f94e735d50f82eb4c11d40b864becb9743de95abf7ef8f29445d6c0247304402203704695217a419dce8c47cfa1bf206cf17f73a9c61b86b364f5b4c0c4371ddcb02207a2730ac55198033d4559a1ddf937f842c77e74026423d39a8441199fd8af57d012103be9522eaa49f32cd3eb559ea3c3187b3559043a720ceda1a215ac3405b40cce00247304402200b4e0ba529ad8c4568144fb081d3cb737b11af330fe03951d48787beef37761602201323dea016586094b45002a78dcceb4ad25eb48cf0341fd99727230648dc00da012103780699159f1a6f5242348bc8d61f58d3bd7b00c78a0d40fd7f2e57a178b379690247304402205fb3f7174e98b469d707246e3d62913066ae23bd17fc5429978500dd9508d009022049d2abcd23e3c78244b6cc9a8563ca783fda57d080b81ac4bd0af9359650418b012102d9e9ddd6c2ea0dc6c057a04e50e8d0915e9cd01c0e96b9502b04da269dd43dcca9a20000",
      "txid": "ff162dc6fcf24922a43662e445f7a0ddaec61dece249f4e01364696802bd9aee",
      "hash": "ecaba0fa0b919a167fbeee188ae394822e8e2e26d779df4c4132093fc7642797",
      "depends": [
      ],
      "fee": 605,
      "sigops": 8,
      "weight": 1390
    },
    {
      "data": "0200000000010295698b94fc9021b4b2a62dbb11e23acde77acf48ce8ce027332e81df15d4f7810000000000fdffffffecd68f83a2c9b4c76f6ec81990423ecad0bf5a513f58bb6e6f69e6872e6da2ee0200000000fdffffff020e800a00000000001976a914ff47820227d99093460b37f9616091a7ddb3e56688ac534d09000000000017a914ca2943e61732f9efd9cdf561c23fde303da03791870247304402205906443248e791515dad9bd35faccd3f07c88dc85a87c7eddc7b949419baea1902203beec98f1ff929b03641f45f1ee360e23e5ad20f9a0d7ef33fccb7e9918509da0121032a9c67647bf7d3b1ccf1965af9fd177c1abab097c0cbd5aaa59c195c7f07736302473044022034a08ba1de8e51bd59391682052c003a1360f10cd6e8b7353f10315afa27227202201c52da6a6e9813633153ffb711a95241b04c794e35029d1e26ef1553e6029aa50121032566f685bf194f85fa25617adbbaa174d2ef463d1ff9500d6704523243ada098a9a20000",
      "txid": "fb743188fee53fb54e4a839a176e37d74209a5eaf0df20513e16005147a7c5a3",
      "hash": "80e7071bac1b5ac08a8ddfe4c13dc29e0c1d7063a884de9703e51b28fb71ad9f",
      "depends": [
      ],
      "fee": 255,
      "sigops": 6,
      "weight": 848
    },
    {
      "data": "02000000000101d4c733a3d78574af8a5f8242f036f0529b8cdf049504f532f3cb4376a53876950100000000fdffffff03f70100000000000022512010397b2d4a2d4a67d55ccaee88c0e8e92c7f38aeebf4ca0e40499962586309a9ec030000000000002251205ae432a8aa5e7aa98d47c74a28390db89edec262d4e2ca1f6b41704495c01d4bb28a070000000000225120d2912b91d0802aa584f4c8ff364f9bb2d5af103368fef4c61584b34f1f081f8b01407848bd0612823ee11f6153ac354f91238cf9c9db66a60a234fbf1f43d3749228e1a1475eae46f66cf5606f262be605383268fe608ab25cb7c0458684cba6149c00000000",
      "txid": "9e5fd8b8a6d206db9117852e9fecb54a1792be229c09c004912e9f2d455d49b1",
      "hash": "464504217e5bcd3ddce9d6db8adc1ac3f65af434cfc6e1094ae03e6420b9ced5",
      "depends": [
      ],
      "fee": 197,
      "sigops": 0,
      "weight": 788
    },
    {
      "data": "02000000000101b1495d452d9f2e9104c0099c22be92174ab5ec9f2e851791db06d2a6b8d85f9e0000000000fdffffff014a01000000000000225120d2912b91d0802aa584f4c8ff364f9bb2d5af103368fef4c61584b34f1f081f8b03406bbe431da4a0931ee616f95c5f00d4ac4ed3828abd2b46c029a11ea5b2cc0ab1eae3bab0ee977409deaa1b598f5fdb5e4a4ec740b7cacc7960cf77fe139d6e0bab206a11672055d3ad2b612dc3372a7101f402816920842ce10c1aeeff3de093c72fac0063036f7264010118746578742f706c61696e3b636861727365743d7574662d38004c647b2270223a22736e73222c226f70223a22757064617465222c226e616d65223a2268657265222c22706572736f6e616c5f6e616d65223a2279697a68656e66656e67222c22706572736f6e616c5f64657363223a2231323334203837363520313638227d6821c16a11672055d3ad2b612dc3372a7101f402816920842ce10c1aeeff3de093c72f00000000",
      "txid": "7f551414f35e270415de3199396c93a097fd65264e805b2d0213c08c6bcfaa85",
      "hash": "bb8c41b018f6c4539102be484c5ad1326adf1ab152a58e6d123c78f0f896943d",
      "depends": [
        3
      ],
      "fee": 173,
      "sigops": 0,
      "weight": 650
    },
    {
      "data": "0200000002df830a452b60b91173357cd5bdb326f880f88b385ab97b950f26e8bc5a2a941e000000006a4730440220096855154d1fcdfc6dab47daac98579924f334bb444c735d0bfa75b009511872022074de7516dc8618c50173be6d15030774d18a6cbfcf508f1f2c9893a1980ffda70121026261f23b0f4f8009fd41437b547a3e50f4fddf34e8eeeb9524151d79e0f8c4ccfdffffff2efe099bcb6a8240891fd0f1ac2b4b79d00d20a36490901f7fd4d02f533bba17010000006a47304402202fec5222ad4bb68992f5edd0185bbfce61aadf03b12875993335adc461596f73022038bfe361f226548743bb7f56f827012221e7317796ace2199835795d89a828b60121030b0b57615595a0fcd04d466ab85313cd6369f07e9028c76b6f54d9fac1a8bbc4fdffffff03e71b050000000000160014386a025c6618718f6d12f605f169f4da380269842ee607000000000017a914f52cb69f975a79e9ad6be43b4f2ad5a3c8d7d56187534d090000000000160014134e35380abc71a8f678fcc49ccb5fd95e1bc7cba9a20000",
      "txid": "1a739e3ece13dd70b9e2d7184e31609e158ba0f8962a5eea76bf6c1d5579ae7b",
      "hash": "1a739e3ece13dd70b9e2d7184e31609e158ba0f8962a5eea76bf6c1d5579ae7b",
      "depends": [
      ],
      "fee": 398,
      "sigops": 0,
      "weight": 1592
    }
  ],
  "coinbaseaux": {
  },
  "coinbasevalue": 5000001628,
  "longpollid": "00000000000000308df706a8736dfe78eee7cea416adc1dc5bfcf4bc11202f6c324",
  "target": "00000000ffff0000000000000000000000000000000000000000000000000000",
  "mintime": 1724919762,
  "mutable": [
    "time",
    "transactions",
    "prevblock"
  ],
  "noncerange": "00000000ffffffff",
  "sigoplimit": 80000,
  "sizelimit": 4000000,
  "weightlimit": 4000000,
  "curtime": 1724920984,
  "bits": "1d00ffff",
  "height": 41642,
  "default_witness_commitment": "6a24aa21a9ed9392f1fe90c4f596eda2d286bba00dc8e2d66c0ec049783f9893f11043575063"
}
As you can see, "bits" field is set into "1d00ffff", and the time of the block is pushed forward. If you can see the real difficulty instead, then your CPU miner will try to mine a block with that difficulty, and will probably fail to do that on time.

In general, I can see two options:

1. Modifying Bitcoin Core.
2. Having unmodified Bitcoin Core, getting original block template, and modifying it in your mining software, or just before passing it to your mining software.

I think it is easier to recompile Bitcoin Core, but it is possible to get for example some 80-byte block header, and then write some code, to replace block time and difficulty, and then pass it into some mining software. Also, mining with Bitcoin Core is easier, because then you don't have to worry about dumping the whole block, making a proper Segwit commitment, and so on.


Title: Re: Requesting Testnet4 tBTC
Post by: BlackHatCoiner on August 29, 2024, 08:43:56 AM
I could swear that both these links sent me to the same transaction. SMF bug? ... Anyway, thanks, I'll try to mine a block and let you know.


Title: Re: Requesting Testnet4 tBTC
Post by: LoyceV on August 29, 2024, 11:19:45 AM
If you can get it right, then you can mine blocks with the minimal difficulty, and we can go further. This simple exercise is what I started with, long time ago, when I was curious, how to mine blocks with Bitcoin Core.
This worked:
Code:
{
  "hash": "0000000085550338d113b883f211c87c7aa3e045ba6e03bae855ea884d1a1be7",
  "confirmations": 1,
  "height": 1,
  "version": 536870912,
  "versionHex": "20000000",
  "merkleroot": "20bb379fd28f090730ac2d207b92c6e95c69dfa48a8118577edb20e068c9a65d",
  "time": 1724929784,
  "mediantime": 1724929784,
  "nonce": 157273468,
  "bits": "1d00ffff",
  "difficulty": 1,
  "chainwork": "0000000000000000000000000000000000000000000000000000000200020002",
  "nTx": 1,
  "previousblockhash": "000000000933ea01ad0ee984209779baaec3ced90fa3f408719526f8d77f4943",
  "strippedsize": 225,
  "size": 225,
  "weight": 900,
  "tx": [
    "20bb379fd28f090730ac2d207b92c6e95c69dfa48a8118577edb20e068c9a65d"
  ]
}

3. Edit "src/chain.h" and "src/node/miner.cpp" exactly as I presented on forum.
If this is ELI5, I may need ELI4....


Title: Re: Requesting Testnet4 tBTC
Post by: vjudeu on August 29, 2024, 12:01:57 PM
Quote
If this is ELI5, I may need ELI4....
If you can get a single block, after the Genesis Block, then it means, that you have enough computing power, to mine on testnet4.

Quote
Edit "src/chain.h" and "src/node/miner.cpp" exactly as I presented on forum.
Those code changes are simple:

1. Go to "src/chain.h".
2. CTRL+F "static constexpr int64_t MAX_FUTURE_BLOCK_TIME = 2 * 60 * 60;"
3. Replace it with: "static constexpr int64_t MAX_FUTURE_BLOCK_TIME = 20 * 60 * 60;"
4. Save changes.

5. Go to "src/node/miner.cpp".
6. CTRL+F "int64_t nNewTime{std::max<int64_t>(pindexPrev->GetMedianTimePast() + 1, TicksSinceEpoch<std::chrono::seconds>(NodeClock::now()))};"
7. Replace it with: "int64_t nNewTime{std::max<int64_t>(pindexPrev->GetMedianTimePast() + 1, pindexPrev->GetBlockTime() + consensusParams.nPowTargetSpacing*2 + 1)};".
8. Save changes.

As I said: you can do that, and recompile Bitcoin Core, then all "generateXYZ" commands will give you easier blocks. The alternative is to not touch Bitcoin Core at all, but then, you have to extract the whole block, save it somewhere, replace time and difficulty, and then pass your 80-byte header to your mining software. But for me, recompiling Bitcoin Core is easier.

Quote
Long time ago, when I started, I simply used "generatetoaddress" command manually, and modified my system clock, to set it 20 minutes after the last block.
This is another way, if you don't want to touch your source code. You can simply set your system clock 20 minutes into the future. But then:

1. You will get some warnings, that "your clock is different, than in other nodes".
2. It will work only for a single block. When you mine it, then you won't mine a second block on top of that, because you will need to move your time again, 20 more minutes forward, or you will be mining with the real difficulty.
3. If any other CPU miner will produce a block, then you will mine again with the real difficulty.
4. Your browser will tell you, that HTTPS connection is insecure, because your system time will be set differently.
5. Other programs, or even your Operating System, may try to update your time, and set it automatically, if you don't disable it.

So, by manually tweaking your system clock, and using unmodified Bitcoin Core, you can mine a block or two, but it quickly becomes tedious.

But of course, if you have nothing else, then you can start with that. If you change your system clock manually, and then restart Bitcoin Core, you should be able to mine a block on your CPU, if your block template will give you "1d00ffff".


Title: Re: Requesting Testnet4 tBTC
Post by: LoyceV on August 29, 2024, 03:55:27 PM
Quote
If this is ELI5, I may need ELI4....
If you can get a single block, after the Genesis Block, then it means, that you have enough computing power, to mine on testnet4.

Quote
Edit "src/chain.h" and "src/node/miner.cpp" exactly as I presented on forum.
Those code changes are simple:

1. Go to "src/chain.h".
2. CTRL+F "static constexpr int64_t MAX_FUTURE_BLOCK_TIME = 2 * 60 * 60;"
3. Replace it with: "static constexpr int64_t MAX_FUTURE_BLOCK_TIME = 20 * 60 * 60;"
4. Save changes.

5. Go to "src/node/miner.cpp".
6. CTRL+F "int64_t nNewTime{std::max<int64_t>(pindexPrev->GetMedianTimePast() + 1, TicksSinceEpoch<std::chrono::seconds>(NodeClock::now()))};"
7. Replace it with: "int64_t nNewTime{std::max<int64_t>(pindexPrev->GetMedianTimePast() + 1, pindexPrev->GetBlockTime() + consensusParams.nPowTargetSpacing*2 + 1)};".
8. Save changes.
Thanks, I got this far. But after compiling, it doesn't seem to know "testnet4":
Code:
./bitcoind --testnet4 -prune=25000
Error: Error parsing command line arguments: Invalid parameter --testnet4
This was after I used https://github.com/fjahr/bitcoin/tree/2024-04-testnet-4-fix so not what I expected.


Title: Re: Requesting Testnet4 tBTC
Post by: garlonicon on August 29, 2024, 05:23:09 PM
Maybe the author changed that branch in the meantime. Now, you can just clone the official code from https://github.com/bitcoin/bitcoin/ from the master branch, because things are already merged.

Also:
Quote
This branch is 360 commits behind bitcoin/bitcoin:master.
So, I guess using official code is better now. At the time of writing, it was not yet merged, so there was no other way.

Or: you can also use https://github.com/bitcoin/bitcoin/tree/28.x because this is what you can get, if you download release candidate for v28. But usually I just stick with the master.

Edit:
Quote
Code:
./bitcoind --testnet4 -prune=25000
A single dash in -testnet4, not --testnet4.


Title: Re: Requesting Testnet4 tBTC
Post by: LoyceV on August 29, 2024, 07:13:07 PM
Or: you can also use https://github.com/bitcoin/bitcoin/tree/28.x because this is what you can get, if you download release candidate for v28. But usually I just stick with the master.
I'll try this next, thanks.

Quote
Edit:
Quote
Code:
./bitcoind --testnet4 -prune=25000
A single dash in -testnet4, not --testnet4.
Both work. Bitcoin Core isn't so picky. That's why I got sloppy with the dashes.


Title: Re: Requesting Testnet4 tBTC
Post by: vjudeu on August 30, 2024, 05:35:45 AM
Quote
Invalid parameter --testnet4
Which version? Both master and v28.0 should have it.
Code:
$ ./bitcoind --version
Bitcoin Core version v28.0.0rc1
Copyright (C) 2009-2024 The Bitcoin Core developers

Please contribute if you find Bitcoin Core useful. Visit
<https://bitcoincore.org/> for further information about the software.
The source code is available from <https://github.com/bitcoin/bitcoin>.

This is experimental software.
Distributed under the MIT software license, see the accompanying file COPYING
or <https://opensource.org/licenses/MIT>


Title: Re: Requesting Testnet4 tBTC
Post by: LoyceV on August 30, 2024, 06:55:45 AM
Quote
Invalid parameter --testnet4
Which version? Both master and v28.0 should have it.
I'm not even sure anymore. I now used https://github.com/bitcoin/bitcoin/tree/28.x and that works. I have enough testnet4 coins to give them away until the end of days :D

One more question: how do I get one of those cute little Miner tags "LoyceV was here" (just like Portland.HODL (https://mempool.space/testnet4/block/000000000000001f403a7b9a65736686ca90c6ffc051e0b6f98d5a2db141368f))? That would be a cool thing to have before I turn this off again :) I tried Googling it, but couldn't find it.


Title: Re: Requesting Testnet4 tBTC
Post by: garlonicon on August 30, 2024, 07:18:50 AM
Quote
how do I get one of those cute little Miner tags "LoyceV was here"
Do you want to get only some message, or something special? Usually, I just change the code, for example:
Code:
// Create coinbase transaction.
CMutableTransaction coinbaseTx;
coinbaseTx.vin.resize(1);
coinbaseTx.vin[0].prevout.SetNull();
CAmount nFeesWithSubsidy = nFees + GetBlockSubsidy(nHeight, chainparams.GetConsensus());
if(nFeesWithSubsidy<=10*1000*COIN)
{
    coinbaseTx.vout.resize(1);
    coinbaseTx.vout[0].scriptPubKey = scriptPubKeyIn;
    coinbaseTx.vout[0].nValue = nFeesWithSubsidy;
}
else
{
    coinbaseTx.vout.resize(1);
    coinbaseTx.vout[0].scriptPubKey = scriptPubKeyIn;
    coinbaseTx.vout[0].nValue = 10*1000*COIN;
}
coinbaseTx.vin[0].scriptSig = CScript() << nHeight << ParseHex("4c6f79636556207761732068657265");
After "nHeight", you can add any Script in "src/node/miner.cpp", recompile it, and then restart your node. But of course, if you don't want to touch the client, then there are other options. Usually, some mining client has something called "coinbaseaux" or similar. For "LoyceV was here", it would be "4c6f79636556207761732068657265" (sometimes you have to specify it in hex, sometimes not, probably you should try it in regtest first, to be sure).

Edit: Let's see: https://mempool.space/testnet/tx/5e739e2a69fe22f8b3f7536c35cbb710ddb725e5660f3996e902ee6eeb36c21a


Title: Re: Requesting Testnet4 tBTC
Post by: LoyceV on August 30, 2024, 08:32:55 AM
Code:
coinbaseTx.vin[0].scriptSig = CScript() << nHeight << ParseHex("4c6f79636556207761732068657265");
After "nHeight", you can add any Script in "src/node/miner.cpp", recompile it, and then restart your node. But of course, if you don't want to touch the client, then there are other options. Usually, some mining client has something called "coinbaseaux" or similar. For "LoyceV was here", it would be "4c6f79636556207761732068657265" (sometimes you have to specify it in hex, sometimes not, probably you should try it in regtest first, to be sure).
Thanks, that (kinda) worked. It still shows the Miner as unknown (https://mempool.space/testnet4/mining/pool/unknown), but it has a Coinbase tag:
https://loyce.club/other/testnet4.png
I don't know why there's ] and | in front.


Title: Re: Requesting Testnet4 tBTC
Post by: garlonicon on August 30, 2024, 08:37:40 AM
Quote
I don't know why there's ] and | in front.
Because of block height, which is mandatory, since BIP-34 (https://github.com/bitcoin/bips/blob/master/bip-0034.mediawiki). And because mempool.space decodes block height as ASCII, it is what it is.

Also note, that there are more mining pools, marked as "Unknown". The list of tags is probably centrally managed by mempool.space, so you can ask them about it. Or try to format it in the same way as "wiz" did.


Title: Re: Requesting Testnet4 tBTC
Post by: BlackHatCoiner on August 31, 2024, 12:43:55 PM
Is it possible to run bitcoin-cli generatetoaddress in multiple threads? I wrote a C program which calls 12 forks, each having its own thread, but it doesn't get all the CPU power. It's strange; it only takes 33% of the CPU. (My total threads are 12.)

Code:
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>

void* runner(void* param);

int main(int argc, char** argv)
{
int i, num_threads = sysconf(_SC_NPROCESSORS_ONLN);
printf("Num of available threads: %d\n", num_threads);

pthread_t* threads = malloc(num_threads * sizeof(pthread_t));
int* thread_ids = malloc(num_threads * sizeof(int));

for(i = 0; i < num_threads; i++){
thread_ids[i] = i;
if(pthread_create(&threads[i], NULL, runner, &thread_ids[i]) != 0){
perror("pthread_create failed");
free(threads);
free(thread_ids);
}


}

for(i = 0; i < num_threads; i++){
pthread_join(threads[i], NULL);
}

free(threads);
free(thread_ids);

printf("All threads have finished execution\n");

return 0;
}

void* runner(void* param)
{

printf("thread %d is running...\n", *(int*)param);
pid_t pid = fork();
    if (pid == 0) {
        // In child process: execute the command
        execl("./bitcoin-cli", "bitcoin-cli", "--testnet4", "generatetoaddress", "1", "<ADDRESS>", "10000000", NULL);
        // If execl fails
        perror("execl failed");
        exit(1);
    } else if (pid > 0) {
        // In parent process: wait for the child process to finish
        int status;
        waitpid(pid, &status, 0);
    } else {
        perror("fork failed");
    }

return NULL;
}

Everything seems to be executing properly:
Code:
$ ./exec2
Num of available threads: 12
thread 0 is running...
thread 1 is running...
thread 2 is running...
thread 3 is running...
thread 4 is running...
thread 5 is running...
thread 6 is running...
thread 7 is running...
thread 9 is running...
thread 8 is running...
thread 11 is running...
thread 10 is running...
[
]
[
]
[
]
[
]
[
]
[
]
[
]
[
]
[
]
[
]
[
]
[
]
All threads have finished execution

I know that there is already some CPU mining software out there for multiple-thread mining, but I want to write my own. It's also made for testing purposes, since we're in testnet.


Title: Re: Requesting Testnet4 tBTC
Post by: LoyceV on August 31, 2024, 02:12:08 PM
Is it possible to run bitcoin-cli generatetoaddress in multiple threads? I wrote a C program which calls 12 forks, each having its own thread, but it doesn't get all the CPU power. It's strange; it only takes 33% of the CPU. (My total threads are 12.)
Here, it tops off at 400.0% CPU (so 4 cores). I'm not sure how useful it is though: sometimes different threads find the same block, and sometimes they find different blocks.


Title: Re: Requesting Testnet4 tBTC
Post by: BlackHatCoiner on August 31, 2024, 02:32:52 PM
Here, it tops off at 400.0% CPU (so 4 cores).
In Linux Mint, CPU usage goes up to 100%. If you have 12 cores, and the program uses 8.3% of the CPU, it means it uses 1 core. If it uses 33%, it means it uses 4 cores. So, it perhaps depends on the architecture of the OS. (I guess that the fact that it's using the same number of cores in both of us, is a good sign.)

I'm not sure how useful it is though: sometimes different threads find the same block, and sometimes they find different blocks.
That's a problem, indeed. You can avoid it by using different Bitcoin address in each thread, but it makes everything more complicated for no reason. An extra nonce field in generatetoaddress would solve this problem.

Edit: Just solved the same block four times :D

Code:
$ ./exec2
Num of available threads: 12
thread 0 is running...
thread 1 is running...
thread 2 is running...
thread 3 is running...
thread 4 is running...
thread 5 is running...
thread 6 is running...
thread 7 is running...
thread 8 is running...
thread 9 is running...
thread 10 is running...
thread 11 is running...
[
  "00000000f095255ea490763a99f02b2adfae1f2ca4df708db02fa5a1777b4217"
]
[
  "00000000f095255ea490763a99f02b2adfae1f2ca4df708db02fa5a1777b4217"
]
[
  "00000000f095255ea490763a99f02b2adfae1f2ca4df708db02fa5a1777b4217"
]
[
  "00000000f095255ea490763a99f02b2adfae1f2ca4df708db02fa5a1777b4217"
]


Title: Re: Requesting Testnet4 tBTC
Post by: LoyceV on August 31, 2024, 03:01:05 PM
You can avoid it by using different Bitcoin address in each thread, but it makes everything more complicated for no reason.
That's easy to fix:
Code:
address=$(shuf -n1 addresses.txt)
Even better if you get a new address each time after finding a block.


Title: Re: Requesting Testnet4 tBTC
Post by: BlackHatCoiner on August 31, 2024, 03:59:01 PM
Added --debug=http, and here's what I get:
Code:
2024-08-31T15:50:25Z [http] Received a POST request for / from 127.0.0.1:37146
2024-08-31T15:50:25Z [http] Received a POST request for / from 127.0.0.1:37174
2024-08-31T15:50:25Z [http] Received a POST request for / from 127.0.0.1:37162
2024-08-31T15:50:25Z [http] Received a POST request for / from 127.0.0.1:37180
2024-08-31T15:50:25Z CreateNewBlock(): block weight: 896 txs: 0 fees: 0 sigops 400
2024-08-31T15:50:25Z [http] Received a POST request for / from 127.0.0.1:37192
2024-08-31T15:50:25Z [http] Received a POST request for / from 127.0.0.1:37208
2024-08-31T15:50:25Z CreateNewBlock(): block weight: 896 txs: 0 fees: 0 sigops 400
2024-08-31T15:50:25Z CreateNewBlock(): block weight: 896 txs: 0 fees: 0 sigops 400
2024-08-31T15:50:25Z [http] Received a POST request for / from 127.0.0.1:37216
2024-08-31T15:50:25Z [http] Received a POST request for / from 127.0.0.1:37218
2024-08-31T15:50:25Z CreateNewBlock(): block weight: 896 txs: 0 fees: 0 sigops 400
2024-08-31T15:50:25Z [http] Received a POST request for / from 127.0.0.1:37224
2024-08-31T15:50:25Z [http] Received a POST request for / from 127.0.0.1:37238
2024-08-31T15:50:25Z [http] Received a POST request for / from 127.0.0.1:37254
2024-08-31T15:50:25Z [http] Received a POST request for / from 127.0.0.1:37260

It uses only 4 cores, because it calls CreateNewBlock() only 4 times, whereas I'm calling it 12. (It does receive all 12 requests though.) There has to be an internal function in Bitcoin Core which blocks me from running CreateNewBlock more than 4 times in parallel.

BTW, another problem is mempool. For some reason, it's empty to me. (And that's why I mine empty blocks.)


Title: Re: Requesting Testnet4 tBTC
Post by: vjudeu on September 02, 2024, 04:37:27 AM
Quote
Is it possible to run bitcoin-cli generatetoaddress in multiple threads?
Yes, sure, bitcoin-util has the "grind" command, which does exactly that. The hardest part is getting 80-byte block header, and passing it correctly. But if you can:

1. Dump the whole block.
2. Extract the block header.
3. Pass that header to bitcoin-util.
4. Submit mined block to the network.

Then it can work. But currently, I had some cases, where I had six future blocks in a row, and then needed to slow down my miners, so I am not planning to get more power out of it, because after passing two hours limit, the risk of getting your block reorged is increased (because if you don't re-submit your block properly, then it won't be broadcasted automatically, if you mine it too fast).

Quote
You can avoid it by using different Bitcoin address in each thread, but it makes everything more complicated for no reason.
It can be simpler than that. See bitcoin-util implementation (or just use it, by passing your block headers there). If you have nonces from 1 to 100, and you have two threads, then the first thread can check 1,3,5,7, and the second one can check 2,4,6,8, at the same time. Then, you have "i+=2" in both cases, and you use "i=startRange+threadId" as a starting point.

Quote
Just solved the same block four times
Yes, for that reason, I use a different address in each generatetoaddress call, or assign different range of nonces for each call.


Title: Re: Requesting Testnet4 tBTC
Post by: LoyceV on September 02, 2024, 07:07:05 AM
Then it can work. But currently, I had some cases, where I had six future blocks in a row, and then needed to slow down my miners, so I am not planning to get more power out of it, because after passing two hours limit, the risk of getting your block reorged is increased (because if you don't re-submit your block properly, then it won't be broadcasted automatically, if you mine it too fast).
Someone figured out how to break Something broke garlonicon's code changes (https://bitcointalk.org/index.php?topic=5496494.msg64205870#msg64205870). My node is stuck on block 42335:
Code:
2024-09-02T05:25:52Z UpdateTip: new best=000000000000004ac75332dec3b2ed28f2e94a2a2a978b9373b9dd3e9e1dfa11 height=42334 version=0x27ba0000 log2_work=70.828749 tx=781363 date='2024-09-01T21:29:08Z' progress=0.999425 cache=0.3MiB(16txo)
2024-09-02T05:25:52Z 0 block-relay-only anchors will be tried for connections.
2024-09-02T05:25:52Z init message: Starting network threadsâ¦
2024-09-02T05:25:52Z dnsseed thread start
2024-09-02T05:25:52Z Waiting 300 seconds before querying DNS seeds.
2024-09-02T05:25:52Z net thread start
2024-09-02T05:25:52Z addcon thread start
2024-09-02T05:25:52Z opencon thread start
2024-09-02T05:25:52Z msghand thread start
2024-09-02T05:25:52Z UpdateTip: new best=00000000ccf137e3823d03c9f23c1623ae3ce25fe29048e31674cfe4d3e47a98 height=42335 version=0x20000000 log2_work=70.828749 tx=781367 date='2024-09-01T21:49:09
This was after I restarted it.

Checking older logfiles, this happened last night:
Code:
2024-09-02T02:06:09Z CreateNewBlock(): block weight: 23257 txs: 22 fees: 6573 sigops 465
2024-09-02T02:06:09Z CreateNewBlock(): block weight: 23257 txs: 22 fees: 6573 sigops 465
2024-09-02T02:07:04Z CreateNewBlock(): block weight: 23257 txs: 22 fees: 6573 sigops 465
2024-09-02T02:07:04Z CreateNewBlock(): block weight: 23257 txs: 22 fees: 6573 sigops 465
2024-09-02T02:07:04Z CreateNewBlock(): block weight: 23257 txs: 22 fees: 6573 sigops 465
2024-09-02T02:07:04Z CreateNewBlock(): block weight: 23257 txs: 22 fees: 6573 sigops 465
2024-09-02T02:07:59Z CreateNewBlock(): block weight: 23257 txs: 22 fees: 6573 sigops 465
2024-09-02T02:07:59Z CreateNewBlock(): block weight: 23257 txs: 22 fees: 6573 sigops 465
2024-09-02T02:07:59Z CreateNewBlock(): block weight: 23257 txs: 22 fees: 6573 sigops 465
2024-09-02T02:07:59Z CreateNewBlock(): block weight: 23257 txs: 22 fees: 6573 sigops 465
2024-09-02T02:08:54Z CreateNewBlock(): block weight: 23257 txs: 22 fees: 6573 sigops 465
2024-09-02T02:08:54Z CreateNewBlock(): block weight: 23257 txs: 22 fees: 6573 sigops 465
2024-09-02T02:08:54Z CreateNewBlock(): block weight: 23257 txs: 22 fees: 6573 sigops 465
2024-09-02T02:08:54Z CreateNewBlock(): block weight: 23257 txs: 22 fees: 6573 sigops 465
2024-09-02T02:12:10Z Potential stale tip detected, will try using extra outbound peer (last tip update: 16722 seconds ago)
2024-09-02T02:22:40Z Potential stale tip detected, will try using extra outbound peer (last tip update: 17352 seconds ago)
2024-09-02T02:33:10Z Potential stale tip detected, will try using extra outbound peer (last tip update: 17982 seconds ago)
2024-09-02T02:43:40Z Potential stale tip detected, will try using extra outbound peer (last tip update: 18612 seconds ago)
2024-09-02T02:54:10Z Potential stale tip detected, will try using extra outbound peer (last tip update: 19242 seconds ago)
2024-09-02T03:02:24Z Flushed fee estimates to fee_estimates.dat.
2024-09-02T03:04:40Z Potential stale tip detected, will try using extra outbound peer (last tip update: 19872 seconds ago)
2024-09-02T03:15:10Z Potential stale tip detected, will try using extra outbound peer (last tip update: 20502 seconds ago)
2024-09-02T03:25:40Z Potential stale tip detected, will try using extra outbound peer (last tip update: 21132 seconds ago)
2024-09-02T03:36:10Z Potential stale tip detected, will try using extra outbound peer (last tip update: 21762 seconds ago)
2024-09-02T03:46:40Z Potential stale tip detected, will try using extra outbound peer (last tip update: 22392 seconds ago)
2024-09-02T03:57:10Z Potential stale tip detected, will try using extra outbound peer (last tip update: 23022 seconds ago)
2024-09-02T04:02:24Z Flushed fee estimates to fee_estimates.dat.
2024-09-02T04:07:40Z Potential stale tip detected, will try using extra outbound peer (last tip update: 23652 seconds ago)
2024-09-02T04:18:10Z Potential stale tip detected, will try using extra outbound peer (last tip update: 24282 seconds ago)
2024-09-02T04:28:40Z Potential stale tip detected, will try using extra outbound peer (last tip update: 24912 seconds ago)
2024-09-02T04:39:10Z Potential stale tip detected, will try using extra outbound peer (last tip update: 25542 seconds ago)
2024-09-02T04:49:40Z Potential stale tip detected, will try using extra outbound peer (last tip update: 26172 seconds ago)
2024-09-02T05:00:10Z Potential stale tip detected, will try using extra outbound peer (last tip update: 26802 seconds ago)
2024-09-02T05:02:24Z Flushed fee estimates to fee_estimates.dat.
2024-09-02T05:10:40Z Potential stale tip detected, will try using extra outbound peer (last tip update: 27432 seconds ago)
2024-09-02T05:21:10Z Potential stale tip detected, will try using extra outbound peer (last tip update: 28062 seconds ago)
2024-09-02T05:24:59Z tor: Thread interrupt
2024-09-02T05:24:59Z Shutdown: In progress...
2024-09-02T05:24:59Z addcon thread exit
2024-09-02T05:24:59Z torcontrol thread exit
2024-09-02T05:24:59Z msghand thread exit
2024-09-02T05:24:59Z net thread exit
2024-09-02T05:25:02Z opencon thread exit
2024-09-02T05:25:02Z DumpAnchors: Flush 0 outbound block-relay-only peer addresses to anchors.dat started
2024-09-02T05:25:02Z DumpAnchors: Flush 0 outbound block-relay-only peer addresses to anchors.dat completed (0.00s)
2024-09-02T05:25:02Z scheduler thread exit
2024-09-02T05:25:02Z Writing 39 mempool transactions to file...
2024-09-02T05:25:02Z Writing 0 unbroadcast transactions to file.
2024-09-02T05:25:02Z Dumped mempool: 0.000s to copy, 0.029s to dump, 13558 bytes dumped to file
2024-09-02T05:25:02Z Flushed fee estimates to fee_estimates.dat.
2024-09-02T05:25:06Z Shutdown: done

As far as I understand the code changes, it changes the "window" from 2 to 20 hours. And other nodes reject blocks with a time more than 2 hours in the future, so the chain forks locally?

I wiped the testnet4 blockchain and restarted it, with the same result:
Code:
2024-09-02T07:07:59Z UpdateTip: new best=00000000a5b18671cb77a85ed98cf64a3a9f97d3c5a3f661465f6bbe1e97a250 height=42327 version=0x20000000 log2_work=70.827931 tx=781321 date='2024-09-01T21:08:13Z' progress=0.999277 cache=75.7MiB(589834txo)
2024-09-02T07:07:59Z UpdateTip: new best=000000000000000e2daf4ee59372be354ac640a9e6686794f6ac68254b8d451c height=42328 version=0x2f296000 log2_work=70.828095 tx=781328 date='2024-09-01T20:51:02Z' progress=0.999257 cache=75.7MiB(589837txo)
2024-09-02T07:07:59Z UpdateTip: new best=00000000e853d1dd75ac2a1c424d6c90730897038074d85a0c374a73ac2322e9 height=42329 version=0x20000000 log2_work=70.828095 tx=781332 date='2024-09-01T21:11:03Z' progress=0.999281 cache=75.7MiB(589835txo)
2024-09-02T07:07:59Z UpdateTip: new best=000000000000004cd3ea3324d8bdfdf64eda9ef324e27103cfa98afcbd9bb65e height=42330 version=0x323a8000 log2_work=70.828258 tx=781342 date='2024-09-01T21:06:38Z' progress=0.999275 cache=75.7MiB(589834txo)
2024-09-02T07:07:59Z UpdateTip: new best=000000000000004c3233c6a02aed84e01aef09fc0c1bd74615d0bbd19e32eb92 height=42331 version=0x20000000 log2_work=70.828422 tx=781348 date='2024-09-01T21:11:50Z' progress=0.999282 cache=75.7MiB(589832txo)
2024-09-02T07:07:59Z UpdateTip: new best=0000000000000000b7059720cb0394af824c1eff5139c7d7396c51145ea5692c height=42332 version=0x2d752000 log2_work=70.828585 tx=781355 date='2024-09-01T21:21:38Z' progress=0.999293 cache=75.7MiB(589837txo)
2024-09-02T07:07:59Z UpdateTip: new best=00000000b5108be963a280da486ee21860bfdc931af42564c6a165d8dcfb387a height=42333 version=0x20000000 log2_work=70.828585 tx=781359 date='2024-09-01T21:41:39Z' progress=0.999318 cache=75.7MiB(589836txo)
2024-09-02T07:07:59Z UpdateTip: new best=000000000000004ac75332dec3b2ed28f2e94a2a2a978b9373b9dd3e9e1dfa11 height=42334 version=0x27ba0000 log2_work=70.828749 tx=781363 date='2024-09-01T21:29:08Z' progress=0.999302 cache=75.7MiB(589839txo)
2024-09-02T07:07:59Z UpdateTip: new best=00000000542792e54a720567ba66157d48cdae7bfd01c1b678d0f07a2ed56e99 height=42335 version=0x20000000 log2_work=70.828749 tx=781367 date='2024-09-01T21:49:09Z' progress=0.999327 cache=75.7MiB(589843txo)
2024-09-02T07:08:17Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42391, peer=19
Mempool.space (https://mempool.space/testnet4) is at block 42391 (https://mempool.space/testnet4/block/0000000019a48a5fce02c3c5004c53f6cfcc03f9d3aa49bbadd583bdfd7bc19b) already.

What happened here?


Title: Re: Requesting Testnet4 tBTC
Post by: garlonicon on September 02, 2024, 07:39:29 AM
Quote
And other nodes reject blocks with a time more than 2 hours in the future, so the chain forks locally?
Yes, it should be probably reverted back into 2 hours, if your goal is to have a stable environment, running 24/7. I needed this change mainly for testnet3, because it was normal, to have six future blocks, which made it impossible to CPU-mine any block on top of that. However, if you will change it into "2 hours 20 minutes" or "3 hours", then it may work better. I guess "20 hours" is definitely too much. But it was fine for me, because I didn't put enough mining power, to really reach those limits.

Also, if you mine any block, which is moved forward beyond those two hours, then when it will fall back into two hours window (because time is naturally moving forward), then you probably have to re-submit such block, because it won't be submitted automatically.

So yes, in general, if you are further than two hours in the future, then there is a huge risk of mining a chain, which will be reorged anyway. But: if you want to measure your mining power, then you need some way of doing that, and by having a secret, longer chain, you can measure, how many of your CPU-mined blocks will be reorged, when a single ASIC block will appear. And also, if you have a GUI version, then you will see all of those blocks as "stale" or "generated, not confirmed", or something like that.

Quote
Mempool.space is at block 42391 already.
The funny thing is that if you have a full P2P node, then you should also notice, that mempool.space is not some centralized source of truth anymore. If you have only CPU blocks in conflicting chains, then you should probably see, on top of which chain, you have more chainwork, and more ASIC blocks. Because for example, I thought that "mempool.space confirmed? Then safe", but I stopped thinking in that way, when I successfully reorged mempool.space, and when ASIC miners started to build new blocks on top of my history, and not theirs. If you are a node runner, then you are really P2P peer, and you should focus on chainwork, and not "just some centralized block explorer".


Title: Re: Requesting Testnet4 tBTC
Post by: LoyceV on September 02, 2024, 08:00:40 AM
Quote
And other nodes reject blocks with a time more than 2 hours in the future, so the chain forks locally?
Yes, it should be probably reverted back into 2 hours, if your goal is to have a stable environment, running 24/7. I needed this change mainly for testnet3, because it was normal, to have six future blocks, which made it impossible to CPU-mine any block on top of that. However, if you will change it into "2 hours 20 minutes" or "3 hours", then it may work better. I guess "20 hours" is definitely too much. But it was fine for me, because I didn't put enough mining power, to really reach those limits.
I didn't think my mining power was causing it (but I can't be sure). I'm recompiled an unedited Bitcoin Core:
Code:
2024-09-02T07:52:57Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=0
2024-09-02T07:52:58Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=1
2024-09-02T07:53:04Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=2
2024-09-02T07:53:05Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42335, peer=3
2024-09-02T07:53:05Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42335, peer=4
2024-09-02T07:53:13Z P2P peers available. Skipped DNS seeding.
2024-09-02T07:53:13Z dnsseed thread exit
2024-09-02T07:53:23Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=5
2024-09-02T07:53:23Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=6
2024-09-02T07:53:24Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=7
2024-09-02T07:53:24Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=8
2024-09-02T07:53:25Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=9
2024-09-02T07:53:26Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=10
2024-09-02T07:53:27Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=11
2024-09-02T07:53:28Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=12
2024-09-02T07:53:34Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=13
2024-09-02T07:53:35Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=14
2024-09-02T07:53:35Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=15
2024-09-02T07:53:36Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=16
2024-09-02T07:53:37Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=17
2024-09-02T07:53:43Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=18
2024-09-02T07:53:43Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=19
2024-09-02T07:53:44Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=21
2024-09-02T07:53:45Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=23
2024-09-02T07:53:46Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=26
2024-09-02T07:54:05Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=28
2024-09-02T07:54:05Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=29
2024-09-02T07:54:16Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=31
2024-09-02T07:54:23Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=33
2024-09-02T07:54:24Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=34
2024-09-02T07:54:25Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=35
2024-09-02T07:54:25Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=36
2024-09-02T07:54:31Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42335, peer=37
2024-09-02T07:54:43Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=38
2024-09-02T07:54:49Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=39
2024-09-02T07:54:49Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=40
2024-09-02T07:54:51Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=41
2024-09-02T07:54:57Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=42
2024-09-02T07:54:58Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42335, peer=43
2024-09-02T07:54:59Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42397, peer=44
Some nodes are stuck at 42335, most are at 42397. Shouldn't it reorg on it's own to follow the longest chain?

Quote
Also, if you mine any block, which is moved forward beyond those two hours, then when it will fall back into two hours window (because time is naturally moving forward), then you probably have to re-submit such block, because it won't be submitted automatically.
So you'll kinda need to "manually" keep track and upload them?

Quote
So yes, in general, if you are further than two hours in the future, then there is a huge risk of mining a chain, which will be reorged anyway. But: if you want to measure your mining power, then you need some way of doing that, and by having a secret, longer chain, you can measure, how many of your CPU-mined blocks will be reorged, when a single ASIC block will appear. And also, if you have a GUI version, then you will see all of those blocks as "stale" or "generated, not confirmed", or something like that.
I don't want to measure mining power, I just wanted to feel the fun of mining blocks that I missed out of in 2009 ;)

Quote
Mempool.space is at block 42391 already.
The funny thing is that if you have a full P2P node, then you should also notice, that mempool.space is not some centralized source of truth anymore. If you have only CPU blocks in conflicting chains, then you should probably see, on top of which chain, you have more chainwork, and more ASIC blocks. Because for example, I thought that "mempool.space confirmed? Then safe", but I stopped thinking in that way, when I successfully reorged mempool.space, and when ASIC miners started to build new blocks on top of my history, and not theirs. If you are a node runner, then you are really P2P peer, and you should focus on chainwork, and not "just some centralized block explorer".[/quote]
I used mempool.space as an example, because I always assumed Bitcoin Core would follow the longest chain automatically. I'm still surprised it got stuck that many blocks in the past.
I've wiped my .bitcoin directory again, to see if the unpatchedk Bitcoin Core now goes for the longest testnet4 chain.
Update: it doesn't:
Code:
2024-09-02T08:04:19Z UpdateTip: new best=0000000000000021760623041a7a1b78f92d80beec504f43a46edf27a8993787 height=42326 version=0x2c94e000 log2_work=70.827931 tx=781320 date='2024-09-01T20:48:12Z' progress=0.999185 cache=75.7MiB(589833txo)
2024-09-02T08:04:19Z UpdateTip: new best=00000000a5b18671cb77a85ed98cf64a3a9f97d3c5a3f661465f6bbe1e97a250 height=42327 version=0x20000000 log2_work=70.827931 tx=781321 date='2024-09-01T21:08:13Z' progress=0.999209 cache=75.7MiB(589834txo)
2024-09-02T08:04:19Z UpdateTip: new best=000000000000000e2daf4ee59372be354ac640a9e6686794f6ac68254b8d451c height=42328 version=0x2f296000 log2_work=70.828095 tx=781328 date='2024-09-01T20:51:02Z' progress=0.999189 cache=75.7MiB(589837txo)
2024-09-02T08:04:19Z UpdateTip: new best=00000000e853d1dd75ac2a1c424d6c90730897038074d85a0c374a73ac2322e9 height=42329 version=0x20000000 log2_work=70.828095 tx=781332 date='2024-09-01T21:11:03Z' progress=0.999213 cache=75.7MiB(589835txo)
2024-09-02T08:04:19Z UpdateTip: new best=000000000000004cd3ea3324d8bdfdf64eda9ef324e27103cfa98afcbd9bb65e height=42330 version=0x323a8000 log2_work=70.828258 tx=781342 date='2024-09-01T21:06:38Z' progress=0.999208 cache=75.7MiB(589834txo)
2024-09-02T08:04:19Z UpdateTip: new best=000000000000004c3233c6a02aed84e01aef09fc0c1bd74615d0bbd19e32eb92 height=42331 version=0x20000000 log2_work=70.828422 tx=781348 date='2024-09-01T21:11:50Z' progress=0.999214 cache=75.7MiB(589832txo)
2024-09-02T08:04:19Z UpdateTip: new best=0000000000000000b7059720cb0394af824c1eff5139c7d7396c51145ea5692c height=42332 version=0x2d752000 log2_work=70.828585 tx=781355 date='2024-09-01T21:21:38Z' progress=0.999226 cache=75.7MiB(589837txo)
2024-09-02T08:04:19Z UpdateTip: new best=00000000b5108be963a280da486ee21860bfdc931af42564c6a165d8dcfb387a height=42333 version=0x20000000 log2_work=70.828585 tx=781359 date='2024-09-01T21:41:39Z' progress=0.999250 cache=75.7MiB(589836txo)
2024-09-02T08:04:19Z UpdateTip: new best=000000000000004ac75332dec3b2ed28f2e94a2a2a978b9373b9dd3e9e1dfa11 height=42334 version=0x27ba0000 log2_work=70.828749 tx=781363 date='2024-09-01T21:29:08Z' progress=0.999235 cache=75.7MiB(589839txo)
2024-09-02T08:04:19Z UpdateTip: new best=00000000542792e54a720567ba66157d48cdae7bfd01c1b678d0f07a2ed56e99 height=42335 version=0x20000000 log2_work=70.828749 tx=781367 date='2024-09-01T21:49:09Z' progress=0.999259 cache=75.7MiB(589843txo)
2024-09-02T08:04:23Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42401, peer=26
2024-09-02T08:05:10Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42401, peer=30
2024-09-02T08:06:49Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42401, peer=37
2024-09-02T08:13:06Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42401, peer=55
2024-09-02T08:13:35Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42401, peer=56
2024-09-02T08:13:46Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42335, peer=57
2024-09-02T08:15:07Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42401, peer=59
2024-09-02T08:17:07Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42401, peer=62
2024-09-02T08:19:30Z New outbound-full-relay v2 peer connected: version: 70016, blocks=42401, peer=66
The weird part is: the generatetoaddress thing also doesn't find block 43336. I would have expected that to continue, but clearly I'm missing something.

I needed this change mainly for testnet3, because it was normal, to have six future blocks, which made it impossible to CPU-mine any block on top of that.
At the moment, testnet3 doesn't have 6 blocks in the future, it has someone who mines a block 2 hours ahead all the time:
https://loyce.club/other/testnet3.png
(source (https://mempool.space/testnet/mining/pool/unknown))
The ones with the long Coinbase tag look like ASIC miners. The ones with the short Coinbase tag have a Timestamp about 2 hours in the future. It looks like the ASIC Timestamp is also ahead, but less.
It looks like this 2 hour ahead thing blocks your code changes.


Title: Re: Requesting Testnet4 tBTC
Post by: vjudeu on September 02, 2024, 08:28:18 AM
Quote
Shouldn't it reorg on it's own to follow the longest chain?
Maybe some developers are testing "7200 seconds rule" or "600 seconds rule"? If CPU miners push the time too much into the future, then after Median Time Past (MTP), miners have no chance to adjust the time backwards, and put the current time in their blocks. Which can cause some additional inflation, if it happens around difficulty adjustment.

Quote
So you'll kinda need to "manually" keep track and upload them?
Only those, which are above two hours. And only if you use different settings, than most of the network. Because if 99% nodes would use 20 hours rule, then it would be "de facto standard", and if the heaviest chainwork would be there, then 2 hours rule would simply be lifted. Because there is no centralized source of time: if you have a brand new node, and it tries to download the whole chain history, it doesn't know, if the chain with 2^70 chainwork contains many years of history, or are there just two 2^69 blocks, with today's timestamp (and a lot of smaller ones, to adjust the difficulty).

Quote
I just wanted to feel the fun of mining blocks that I missed out of in 2009
After reaching two hours limit, you have those options:

1. Stick to two hours rule, and stop mining, waiting for some ASIC block, to put the current time.
2. Stick to two hours rule, and still try to mine something, but with ASIC difficulty on CPU, which will likely just burn that power for nothing.
3. Produce a secret, longer chain, and do your testing, knowing that after a while, a new ASIC block will come, and destroy your chain (but in the meantime, you will have a fully working playground, to test non-standard transactions, mining power, or anything else).

Also note, that when there was almost no competition, and we were mining CPU blocks alone, or with a few friends, or when it was "my first node vs my second node" or even "garlonicon's node vs vjudeu's node" then we didn't need to think about all of those consequences, and implications of those rules. But now, when more bitcointalk users will try CPU mining, we will probably encounter more issues, than we found, when we were alone, or with some small, closed and well-coordinated group of friends. We simply didn't put enough CPU power, to test some corner cases, like "more than 51% of blocks, mined by CPUs".

To sum up: it is nice to see some competition: if more bitcointalk users will start mining testnet coins, then maybe some people will get some knowledge, for example "how hard is to CPU-mine things at difficulty one".


Title: Re: Requesting Testnet4 tBTC
Post by: garlonicon on September 02, 2024, 08:47:35 AM
Quote
it has someone who mines a block 2 hours ahead all the time
Yes, I put "20 minutes and 1 second" for a reason: in this case, you can mine six CPU blocks for two hours. However, if you put instead "1 hour 59 minutes 59 seconds", or something like that, then your single CPU block will knock every CPU miner out, for the next 20 minutes. But: there is a catch: if someone will mine more CPU blocks in a row, then that single block will be reorged. So, if some CPU miner will send one block with "20 minutes 1 second", and second block with "40 minutes 2 seconds", then those two blocks will knock out that "1 hour 59 minutes 59 seconds" block.

So, it is a game of CPU-mined chain reorgs. But: winning the game, and picking an optimal strategy, is a big challenge. Because the default implementation is definitely not the best one, and can be beaten by someone, who will write a better code, specifically for testnets.

Edit: One more thing: any CPU miners can be easily stopped by ASIC miners, if they will simply change their strategy, to fully fill the queue of easy blocks, and only then mine their ASIC blocks on top of that. The fact, that CPU miners still have some chances, is a signal, that ASIC miners are just sticking with unmodified Bitcoin Core, and are not trying to get the maximum number of blocks, out of it. Or: maybe they don't have enough knowledge, to write a better code, and they don't want to risk mining invalid ASIC blocks.


Title: Re: Requesting Testnet4 tBTC
Post by: LoyceV on September 02, 2024, 09:26:50 AM
I started my default bitcoin-qt on testnet4:
Code:
Number of blocks left: Unknown. Syncying Headers (42335, 99.8%)...
I know it's testnet, but doesn't this mean it's broken? Could it be the code changes that removed the "block storm" created something unexpected, that makes standard nodes unable to keep syncing?

Quote
Shouldn't it reorg on it's own to follow the longest chain?
Maybe some developers are testing "7200 seconds rule" or "600 seconds rule"? If CPU miners push the time too much into the future, then after Median Time Past (MTP), miners have no chance to adjust the time backwards, and put the current time in their blocks. Which can cause some additional inflation, if it happens around difficulty adjustment.
I've been wondering for a while what happens if most Bitcoin miners adjust their clock to the future.

Quote
Only those, which are above two hours. And only if you use different settings, than most of the network. Because if 99% nodes would use 20 hours rule, then it would be "de facto standard", and if the heaviest chainwork would be there, then 2 hours rule would simply be lifted. Because there is no centralized source of time: if you have a brand new node, and it tries to download the whole chain history, it doesn't know, if the chain with 2^70 chainwork contains many years of history, or are there just two 2^69 blocks, with today's timestamp (and a lot of smaller ones, to adjust the difficulty).
This goes right over my head. It's amazing Bitcoin has been working without major problems all those years.

Quote
After reaching two hours limit, you have those options:

1. Stick to two hours rule, and stop mining, waiting for some ASIC block, to put the current time.
2. Stick to two hours rule, and still try to mine something, but with ASIC difficulty on CPU, which will likely just burn that power for nothing.
3. Produce a secret, longer chain, and do your testing, knowing that after a while, a new ASIC block will come, and destroy your chain (but in the meantime, you will have a fully working playground, to test non-standard transactions, mining power, or anything else).
I think the 20 * 60 * 60 code changes did #3 by default, right? But then why doesn't it find new blocks anymore after block 42335?

Quote
Also note, that when there was almost no competition, and we were mining CPU blocks alone, or with a few friends, or when it was "my first node vs my second node" or even "garlonicon's node vs vjudeu's node" then we didn't need to think about all of those consequences, and implications of those rules. But now, when more bitcointalk users will try CPU mining, we will probably encounter more issues, than we found, when we were alone, or with some small, closed and well-coordinated group of friends. We simply didn't put enough CPU power, to test some corner cases, like "more than 51% of blocks, mined by CPUs".
It's interesting to see though. I'm still flabbergasted my standard testnet4 node is stuck because of what looks like miners running non-standard code.

Quote
To sum up: it is nice to see some competition: if more bitcointalk users will start mining testnet coins, then maybe some people will get some knowledge, for example "how hard is to CPU-mine things at difficulty one".
Considering how long it still takes to mine a block at difficulty 4 on a 5 years old Xeon CPU, this must have been much longer in 2009.

Quote
it has someone who mines a block 2 hours ahead all the time
Yes, I put "20 minutes and 1 second" for a reason: in this case, you can mine six CPU blocks for two hours. However, if you put instead "1 hour 59 minutes 59 seconds", or something like that, then your single CPU block will knock every CPU miner out, for the next 20 minutes. But: there is a catch: if someone will mine more CPU blocks in a row, then that single block will be reorged.
Doesn't that mean you'd have to adjust your code to mine 2 blocks starting from the previous block, instead of mining on top of the 1:59:59-ahead block? So it requires more code changes, right?

Quote
So, it is a game of CPU-mined chain reorgs. But: winning the game, and picking an optimal strategy, is a big challenge. Because the default implementation is definitely not the best one, and can be beaten by someone, who will write a better code, specifically for testnets.
Do I get it right that this only works on testnet because of the difficulty drop after 20 minutes? Wouldn't it be better to change the default rules, let's say the difficulty drops only after 3 hours, so normal mining isn't affected? Or reduce the time difference between miners to no more than 10 minutes instead of 2 hours?

Quote
Edit: One more thing: any CPU miners can be easily stopped by ASIC miners, if they will simply change their strategy, to fully fill the queue of easy blocks, and only then mine their ASIC blocks on top of that. The fact, that CPU miners still have some chances, is a signal, that ASIC miners are just sticking with unmodified Bitcoin Core, and are not trying to get the maximum number of blocks, out of it. Or: maybe they don't have enough knowledge, to write a better code, and they don't want to risk mining invalid ASIC blocks.
Thanks for the explanation.  I'm still amazed people have been burning ASIC power on testnet for years.

@garlonicon's signature: I still can't wrap my head around dealing with private keys in descriptor wallets, so brainwallets aren't really an option at the moment. I'll give it a try when Electrum supports testnet4.



It turns out I did mine block 42335 (https://mempool.space/testnet4/block/00000000542792e54a720567ba66157d48cdae7bfd01c1b678d0f07a2ed56e99). How come multiple nodes can't get passed this block?


Title: Re: Requesting Testnet4 tBTC
Post by: vjudeu on September 02, 2024, 10:10:19 AM
Quote
I think the 20 * 60 * 60 code changes did #3 by default, right?
It starts with #3, and then, it rejects your own blocks, which is the same, what happens, if you run out of two hours. By changing from 2 to 20 hours, you just accept more blocks, but in general, if they are CPU-mined, then there is a huge risk, that they will be reorged.

Quote
But then why doesn't it find new blocks anymore after block 42335?
Because the difference between block 42335 and block 42336 is 11 minutes and 6 seconds. And if you enforce 600 seconds rule, then this ASIC block doesn't meet it. Which means, that if you enable BIP 94, then you land on a shorter chain.

So: do you want to disable 600 seconds rule?
Quote
Code:
if (consensusParams.enforce_BIP94) {
    // Height of block to be mined.
    const int height{pindexPrev->nHeight + 1};
    if (height % consensusParams.DifficultyAdjustmentInterval() == 0) {
        nNewTime = std::max<int64_t>(nNewTime, pindexPrev->GetBlockTime() - MAX_TIMEWARP);
    }
}
I really wonder, if ASICs will follow that kind of timewarp fixes. Because there is a risk, that people may ignore 600 seconds rule, or you may have some nodes, with MAX_TIMEWARP of 600 seconds, and some with 7200 seconds, and then there will be some competing chains.

For now, I simply use some version without those timewarp protections. I don't know, what will happen after block 42335: maybe the current chain will be reorged, or maybe it will be grandfathered in, and the activation of all BIP 94 rules will come later. Who knows, time will tell.

So, if you want to switch to a longer chain, then upgrade your master, switch MAX_TIMEWARP from 600 to 7200, or use some version, where it is entirely disabled, like probably mempool.space is doing.


Title: Re: Requesting Testnet4 tBTC
Post by: LoyceV on September 02, 2024, 11:09:42 AM
Because the difference between block 42335 and block 42336 is 11 minutes and 6 seconds. And if you enforce 600 seconds rule, then this ASIC block doesn't meet it. Which means, that if you enable BIP 94, then you land on a shorter chain.
I take it BIP 94 is enabled already by default. I read bip-0094.mediawiki (https://github.com/bitcoin/bips/blob/master/bip-0094.mediawiki):
Quote
To protect against the time warp attack, the following rule proposed as part of The Great Consensus Cleanup[5] is enforced: "The nTime field of each block whose height, mod 2016, is 0 must be greater than or equal to the nTime field of the immediately prior block minus 600. For the avoidance of doubt, such blocks must still comply with existing Median-Time-Past nTime restrictions."
I won't claim to understand it entirely, but it looks like the problem arose because 21*2016=42336. And that block gets rejected.
But, if that's the default, how come most nodes continue nonetheless? I guess this does explain why mining 20 minutes in the future is no longer possible.

Quote
So, if you want to switch to a longer chain, then upgrade your master, switch MAX_TIMEWARP from 600 to 7200, or use some version, where it is entirely disabled, like probably mempool.space is doing.
I guess this is the path to go. I don't want it to be stuck until someone changes something.

Code:
vi test/functional/mining_basic.py
MAX_TIMEWARP = 1200

src/consensus/consensus.h
static constexpr int64_t MAX_TIMEWARP = 1200;
Update: my testnet4 node is now processing blocks again. Let's see if it mines blocks again :)


Title: Re: Requesting Testnet4 tBTC
Post by: sjors on September 02, 2024, 12:11:27 PM
The timewarp attack mitigation for testnet4 requires the first block of a new difficulty adjustment period to have a timestamp later than that of the last block of the previous period.

Initially there was a 7200 second (2 hour) grace period. This was changed to 600 seconds (10 minutes) two weeks ago: https://github.com/bitcoin/bitcoin/pull/30647 (matching the original Great Consensus Cleanup proposal).

This change is effectively a soft fork on testnet4, but with no activation mechanism.

Anyone who runs the v28.0rc1 release candidate, or a very recent master, will enforce it. Anyone who runs an older version won't enforce it.

My guess is that most miners on testnet4 haven't updated yet. Even if they did that today, the node won't automatically go back to validate old blocks.

If enough miners switch to v28.0(rc...) and call `invalidateblock 00000000000000263393ce5f648afd53676f13d360cc9f264b89351623bf1242 ` followed by `reconsiderblock 00000000000000263393ce5f648afd53676f13d360cc9f264b89351623bf1242 ` they will reject the block at height 42336. This will lead to a one day reorg, or more if it's done later.

An alternative approach would be to change Bitcoin Core to enforce the modified timewarp rule from a certain flag height in the future, but that seems like a lot of complexity for a test network that hasn't even been released yet.

As an aside: the SegWit soft fork did have a mechanism for handling this situation. If a user upgrade their pre-segwit node to a segwit node sometime after activation, it would roll back the chain and check everything again.


Title: Re: Requesting Testnet4 tBTC
Post by: sjors on September 02, 2024, 12:17:25 PM
By the way, I think what happened here is that someone tried to CPU mine coins on testnet by setting their clock > 20 minutes into the future.

Such a block would be accepted by all nodes (upgrade or not). But v28 nodes then require the next block to be >10 minutes in the future, whereas before they would allow 1 hour and 50 minutes in the past (which includes the present).

v28.0rc1 contains a fix for the Bitcoin Core miner code (getblocktemplate, etc) that ensures timestamps are adjusted: https://github.com/bitcoin/bitcoin/issues/30614

---

Update: the above invalidateblock / reconsiderblock sequence won't work (https://github.com/bitcoin/bitcoin/issues/30786#issuecomment-2324677976).

The only way to get an upgraded node to re-check the timewarp rules is:

Code:
bitcoind -tesnet4 -reindex


Title: Re: Requesting Testnet4 tBTC
Post by: LoyceV on September 04, 2024, 06:50:31 AM
I'm not sure what's the best thread, so I'll just post it here: testnet3 (https://mempool.space/testnet)'s mempool is currently empty. For the past months, mempool was full and fees where high. It looks like the "data spam" suddenly stopped, while ASIC miners and "20 minute miners" have halved the block time.
Amazingly, that means testnet3 can now actually be used again for testing :)


Title: Re: Requesting Testnet4 tBTC
Post by: vjudeu on September 04, 2024, 07:47:34 AM
Quote
testnet3's mempool is currently empty
It's nothing new. There are waves of transactions: sometimes mempools are full, sometimes they are empty. And the cost to fill up the whole 300 MB is quite low.

And note that even if mempool.space shows you a fully filled mempool, then it doesn't mean, that you have to pay that much. For example: if some CPU mining node is just starting up, then it could have an empty mempool, and fetching all of those transactions in P2P way is quite slow process, and if you can see a low block health, then it doesn't always mean censorship: it could mean as well, that this particular miner just turned on its node, and it was turned off recently, for whatever reason. And during that time, you can get your transactions confirmed cheaper, if you send it to that particular node. Which means that again: the default fee estimation is far from being optimal, because it doesn't consider such details.

Quote
that means testnet3 can now actually be used again for testing
You can always mine future blocks, and prioritize your own transactions. And you can pay zero fee in your own transactions, if you are a miner. And even if there is some opportunity cost, then statistically, sometimes blocks are not fully filled, and in a general case, it makes a lot of sense to send and mine your own transactions with a low feerate (also because if you use less than 1000 sat/kvB, then you can "cancel" that easier, in case of mistakes).

Also note, that if you want to mine some coins, then you can do that on signet, too: https://delvingbitcoin.org/t/proof-of-work-based-signet-faucet/937


Title: Re: Requesting Testnet4 tBTC
Post by: ercewubam on September 04, 2024, 10:33:38 AM
Quote
Amazingly, that means testnet3 can now actually be used again for testing
Not only that. It also means, that mining testnet3 is now less profitable, and switching into testnet4 is a better idea. But a short while before that, it was the opposite, when it was not certain, if testnet4 will be reorged or not. Now we know, that those, who consider 00000000000000263393ce5f648afd53676f13d360cc9f264b89351623bf1242 as a valid block, are simply mining coins, which will vanish, after upgraded ASICs will mine an alternative chain, on top of 0000000000000016be66ab39af9f97e1294080c94d0a34cc304e391794a481bd.

In general, we are about to see at least 360 blocks reorg, at the time of writing. I guess it could be 500 blocks reorg, or even 1000 blocks reorg, depending on how many people will mine blocks, when using some older version.


Title: Re: Requesting Testnet4 tBTC
Post by: ABCbits on September 04, 2024, 10:59:32 AM
I'm not sure what's the best thread, so I'll just post it here: testnet3 (https://mempool.space/testnet)'s mempool is currently empty. For the past months, mempool was full and fees where high. It looks like the "data spam" suddenly stopped, while ASIC miners and "20 minute miners" have halved the block time.

Weird, since yesterday testnet3 was busy where mempool.space recommend over 500 sat/vB for high priority TX. I wonder if most ordinal/rune created on testnet3 no longer have value.

Amazingly, that means testnet3 can now actually be used again for testing :)

Although i wonder how easy is it to obtain testnet3 coins now (aside from mining). Tesnet faucet was abused on many occurrence.


Title: Re: Requesting Testnet4 tBTC
Post by: LoyceV on September 04, 2024, 11:03:08 AM
Weird, since yesterday testnet3 was busy where mempool.space recommend over 500 sat/vB for high priority TX. I wonder if most ordinal/rune created on testnet3 no longer have value.
Just like I've speculated about it on real Bitcoin, I think here the ordinal spam also comes from only one person. That explains why all transactions are in the same style, and start and stop at the same time too.