Academic question about mining on the testnet

[McStone]

Hi there,

For academic insights and just for fun I've written a python script for mining, with support for my Nvidia TX 1080 ti.
Using python Numba and CUDA, I've coded a double sha256 CUDA kernel, and it works all fine with a (misserable) hashrate of 54MH/s (but for my GPU not so bad).

Don't get me wrong, I'm not after real BTC mining, just wanted to understand all the bits and bytes of BTC mining (what I guess I'm coming close now).

As the final step for my script would be to submit the new mined block (the only thing I could not test yet), I tried to use the testnet3 to do so.
But as the difficulty on the testnet3 is already quite big, it would take in average 43 years for me to get a hit.

But on testnet3, after 20min of no block submit, the difficulty drops down to 1, and I see blocks beeing submitted with only 4 leading 00 bytes.
This is within the reach of my GPU script, so I thought I should be able to submit my own block after 20min block idle.

Now here comes my problem:

When I loop through RPC call getmininginfo, I see that the difficulty drops down to 1, but when I after that drop get the getblocktemplate answer,
I still see the difficulty bits beeing the same as before. I was expecting that this difficulty bits target drops too, but this doesn't seem to be the case...

Also when I dig out the blocks with only 4 leading 00 bytes in the testnet block explorer, I see that the bits there are as high as it would require 7 leasing 00 bytes
that this block would be accepted.

I also tried to just submit my new block with 4 leading zeros after the difficulty drops to 1, but still it gets refused with "AcceptBlock FAILED (high-hash, proof of work failed)"

Is this just a timing issue? I mean somebody else is faster submitting a 4 leading zero block before me, and the difficulty has already been higher again when I submit my block?

Or what else I'm missing here?

Thanks for any anwer to my quite academic question.

Cheers McStone

[1714769274]

1714769274

[1714769274]

1714769274

[NotFuzzyWarm]

A better place to ask would be in the Bitcoin Technical Support area where the nutz&boltz of code is discussed. I've asked the mods to move this there.

While outside my area of expertise my guess would be that yes, someone else is finding the block before you because a GPU is so frickin' slow. What you see is the POW algo in action. Perhaps pick up one of Sidehack's USB miner sticks to experiment with? Rather inexpensive and even the slowest runs >20GHs giving you a much better shot.

[McStone]

thanks for your answer and suggestion to move my post.

Also your hint about the USB miner sticks is interesting.

But to explain: my GPU is easily fast enough to work out a 4 leading zero solution in the 20min that there are before difficulty is set down to 1.
So I was thinking I can wait until this drop happend, and then submit my solution block.

But for some reason it seams not to be accepted. Could be another miner has also "prepared" a 4 leading zero block and is just faster with submitting when the difficulty drop comes... so a USB miner could hardly help in this case...

My main problem of understanding is how the difficulty can drop from a few hundret thousends to 1, but the bits from a freshly made getblocktemplate call (after the drop in difficulty is validated with getmininginfo) is still the same.

[mikeywith]

my GPU is easily fast enough to work out a 4 leading zero solution in the 20min that there are before difficulty is set down to 1.
So I was thinking I can wait until this drop happend, and then submit my solution block.

IIRC you need to include the last block (the one that has 20+ mins timestamp) in your mined block, so you can't pre-mine a block, you can start trying when difficulty is 1 but not prior to that, the drop will only happen if a block isn't found in 20 mins, the difficulty will drop to 1 for 2016 blocks and I suppose that happens quite too often because miners could be mining blocks in the future by adding 20 mins to their node timestamp just to cause a difficulty drop, but there are only 2016 blocks is only 1 block to be mined and some people abuse the testnet with S9s, so there is no guarantee that you will manage to hit a block before the difficulty gets adjusted again, but if your hashrate is good enough to solve a block with 1 diff in a few mins, then I can't see why not!

My main problem of understanding is how the difficulty can drop from a few hundret thousends to 1, but the bits from a freshly made getblocktemplate call (after the drop in difficulty is validated with getmininginfo) is still the same.

It's probably because the latter gets you the difficulty of the last block submitted while the former returns the "current/next block" difficulty.

[Coding Enthusiast]

It is not easy to guess the problem you are having without seeing the block header you submitted, but one similar mistake I was making when I wrote my miner in early stages was that I was changing the difficulty of the already mined block which in turn changed the hash that was found and made it incorrect.

You also keep saying "leading zeros", we don't deal with "leading zeros" at all in difficulty. Instead we deal with numbers. 05 is bigger than 03 but both have 1 leading zero, which could be another reason why your block was rejected.
You have to convert your hash to an integer in little-endian format and your target to its 256-form and then compare these two integers.

Also if you want to get rid of the complications and high difficulty of TestNet simply use RegTest instead. The difficulty is already low and you can mine the blocks in a blinking of an eye. Then submit it to your local blockchain and see if it is accepted or not.

[ranochigo]

the difficulty will drop to 1 for 2016 blocks and I suppose that happens quite too often because miners could be mining blocks in the future by adding 20 mins to their node timestamp just to cause a difficulty drop, but there are only 2016 blocks to be mined and some people abuse the testnet with S9s, so there is no guarantee that you will manage to hit a block before the difficulty gets adjusted again, but if your hashrate is good enough to solve a block with 1 diff in a few mins, then I can't see why not!

Nope. The difficulty reset doesn't last for 2016 blocks.

It only happens when the time elapsed since the previous block has exceeded 20 minutes and the client will accept any blocks with a difficulty of 1. Right after it gets mined, the difficulty will jump back to it's previous difficulty. The trick that loads of people use is to just mine a block with a difficulty of 1 and right at the moment when it hits 20 minutes, they will broadcast the block.

[McStone]

Thanks for all your replies!

...one similar mistake I was making when I wrote my miner in early stages was that I was changing the difficulty of the already mined block which in turn changed the hash that was found and made it incorrect.

yeah I'm aware of this fact, this is not the problem.

You also keep saying "leading zeros", we don't deal with "leading zeros" at all in difficulty. Instead we deal with numbers. 05 is bigger than 03 but both have 1 leading zero, which could be another reason why your block was rejected.

yeah, also about this I'm aware, I just call it "leading zeros" because for humans it's easier to check.
But my script is calculating the target out of the difficulty bits, and comparing the height of the hash (as number) against it.
So also this is not the problem.

I quickly was setting up a local Regtest Net with two nodes, and the mining is working fine and the blocks are accepted.
Seems on testnet, it is/was just a timing issue that some other guy submitted his block faster...


Cheers

[mikeywith]

Nope. The difficulty reset doesn't last for 2016 blocks.

Then I stand corrected and edited my previous post accordingly, and in that case, with OP's hash rate it's going to take forever to solve a block even on the testnet, perhaps he should just use Regtest as CE suggested.

Seems on testnet, it is/was just a timing issue that some other guy submitted his block faster...

Since it's only a single block to be accepted at diff of 1, chances are pretty slim, you are going to need a bit more hashrate.