Couple of questions of mining in testnet.

[firstbt]

Hello,

I am just a beginner ...so I just wanted to see some actual happening in test environment.
Installed Bitcoin-qt 0.21.0 latest version in my mac OS. I run with testnet to test some mining functionality.

1. Console window : I issued like "generatetoaddress 200 "myaddresss"...it returns empty array, which seems not working. I read somewhere on the internet saying this command only working for regtest, NOT for testnet. Well, actually I switched to regtest network, and now I see this command working fine. What I am curious here is that the reason why it is not working under testnet is because of high difficulty? Actually, I am testing my normal MAC GPU..so that means due to high difficulty, it can't find the right hash ..is that why I see it is not working in testnet?.OR simply just testnet network doesn't provide mining functionality at all basically, I don't think so though?

2. I wanted to play around with cgminer against my bitcoin-qt(just FYI, I obviously enabled server=1)... It asked me one pool address...
    is it asking me to enter JSONRPC server address or something else? ...I entered my JSONRPC server address, which is http://127.0.0.1:8332 (this port has setup in the bitcoin.conf)...and provide username and password...but the cgminer returns 500 internal server error? I tested against testnet and regtest both network..but cgminer I can't get it working....What I am missing here? BTW, my JSONRPC server is working fine..I can do curl and got response back. If I would need to provide some other address, what address I would need to provide to cgminer? Any help appreciated.

Thanks,

[nc50lc]

1. Console window : I issued like "generatetoaddress 200 "myaddresss"...it returns empty array, which seems not working. I read somewhere on the internet saying this command only working for regtest, NOT for testnet. Well, actually I switched to regtest network, and now I see this command working fine. What I am curious here is that the reason why it is not working under testnet is because of high difficulty? Actually, I am testing my normal MAC GPU..so that means due to high difficulty, it can't find the right hash ..is that why I see it is not working in testnet?.OR simply just testnet network doesn't provide mining functionality at all basically, I don't think so though?

Yes, it's made for Regtest because it doesn't retarget in regtest (source), but doesn't necessarily mean that it won't work in testnet or even mainnet.
Your test probably reached the maximum 1million tries since (yes) testnet's difficulty is still too much for a CPU to handle, you GPU won't be used by that command.
You can add "maxtries" to generatetoaddress to increase the number of tries before it give up but you need to be extremely lucky to mine a block in testnet using your MAC's CPU.

I'll leave #2 to anyone who mines in testnet and uses cgminer for CPU/GPU mining.

[firstbt]

Hello,

Thanks so much for your reply.

BTW, you said "it doesn't retarget in regtest"...actually, I looked at the source code..it looks like  "consensus.fPowNoRetargeting = true;" is only for regtest, the other network sets to false that I see in the source code....Can you possibly explain me easily what you meant by "it doesn't retarget in regtest"? retarget? ... Hope I can learn something new here as well.

Thanks.
 

[pooya87]

Can you possibly explain me easily what you meant by "it doesn't retarget in regtest"? retarget?

Blockchain starts with an initial target (nbits) in the genesis block, each subsequent block uses the same target until we reach 2016 blocks which is when the difficulty is adjusted. When the code calculates the next (readjusted) target it first checks if that boolean value is set and if it is (which is only true for RegTest) it simply skips the readjustment and difficulty remains the same as genesis block.
https://github.com/bitcoin/bitcoin/blob/c7ad94428ab6f54661d7a5441e1fdd0ebf034903/src/pow.cpp#L13-L72

[firstbt]

BTW, here is some additional information in regards to #2 question.

cgminer version is 4.9.2 latest.

curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "getdifficulty", "params": []}' -H 'content-type: text/plain;' http://127.0.0.1:8332/

Then I got the response back like this..

{"result":194619292.4408354,"error":null,"id":"curltest"}

cgminer runs on the same machine where the bitcoin-qt is running.

./cgminer -o http://127.0.0.1:8332 -O myusername:mypassword --btc-address myaddress -D -T --verbose

Here is the response.

[2021-02-23 23:20:43] Testing pool http://127.0.0.1:8332                    
[2021-02-23 23:20:43] Probing for GBT support                    
[2021-02-23 23:20:43] HTTP request failed: The requested URL returned error: 500 Internal Server Error                    
[2021-02-23 23:20:43] Failed to connect in json_rpc_call                    
[2021-02-23 23:20:43] No GBT coinbase + append support found, pool unusable if it has no stratum          

I *intentionally* provided wrong username and password...then the response I got..

Probing for GBT support                    
 [2021-02-23 23:26:41] HTTP request failed: The requested URL returned error: 401                    
 [2021-02-23 23:26:41] Failed to connect in json_rpc_call                    
 [2021-02-23 23:26:41] No GBT coinbase + append support found, pool unusable if it has no stratum                    

401 is unauthorized, which is correct because I provided wrong username and password...so I believe cgminer has no issue to contact the JSON RPC server, which is http://127.0.0.1:8332....

here is my bitcoin.conf

testnet=1
server=1
listen=1
rpcuser=myusername
rpcpassword=mypassword
rpctimeout=30

[test]
rpcport=8332        

Thanks,

[ranochigo]

CGminer was broken with Bitcoin Core's RPC when I upgraded from 0.19 to 0.20.0. The problem likely persisted till 0.21.0. Downgrading it back to 0.19.0 helped me to solve the issue.

I didn't try to fix the problem with 0.20.0 but that could probably be fixed by running a stratum proxy in between.

[firstbt]

Ok.. Thanks for the info..then is there any alternative tool I could use like cgminer that works with 0.21 Bitcoin core then?
I tried bfgminer(I guess this is sort of like brother with cgminer), but it doesn't still work...
Any other tools I could possibly try?

Thanks,

[firstbt]

Can you possibly explain me easily what you meant by "it doesn't retarget in regtest"? retarget?

Blockchain starts with an initial target (nbits) in the genesis block, each subsequent block uses the same target until we reach 2016 blocks which is when the difficulty is adjusted. When the code calculates the next (readjusted) target it first checks if that boolean value is set and if it is (which is only true for RegTest) it simply skips the readjustment and difficulty remains the same as genesis block.
https://github.com/bitcoin/bitcoin/blob/c7ad94428ab6f54661d7a5441e1fdd0ebf034903/src/pow.cpp#L13-L72

Oh...I got it...so that is why under regtest, the difficulty is extremely low..so we could create a block instantly. Is my understanding correct?

Thanks!

[pooya87]

Oh...I got it...so that is why under regtest, the difficulty is extremely low..so we could create a block instantly. Is my understanding correct?

Yes, you don't want to waste any time generating blocks on regtest so the difficulty is set in a way that most of the times when you set the nonce to zero (as you start the mining process) that is already a valid hash and there is no need to increment it. Worst case scenario your nonce goes to 2 so you basically compute 1 hash for almost all blocks to mine them.

[firstbt]

BTW, I was jut curious about some detailed error message of 500 ...Is there any place where I can possibly look at more detailed error message why the bitcoin-qt returns 500 ?
I noticed that under testnet3 folder, there is a debug.log... Is this something I can look at here? but it looks like this file wasn't updated when i made a request from cgminer then...
Any help would be appreciated someone could point out that if there are some logs file I could look at?

Thanks,

[ranochigo]

Ok.. Thanks for the info..then is there any alternative tool I could use like cgminer that works with 0.21 Bitcoin core then?
I tried bfgminer(I guess this is sort of like brother with cgminer), but it doesn't still work...
Any other tools I could possibly try?

I'm not exactly sure. I downgraded by Bitcoin Core and stopped trying to mine after that. CGminer hasn't been maintained for quite awhile as well.

BTW, I was jut curious about some detailed error message of 500 ...Is there any place where I can possibly look at more detailed error message why the bitcoin-qt returns 500 ?
I noticed that under testnet3 folder, there is a debug.log... Is this something I can look at here? but it looks like this file wasn't updated when i made a request from cgminer then...
Any help would be appreciated someone could point out that if there are some logs file I could look at?

Thanks,

Add debug=rpc into bitcoin.conf in that testnet folder and try to connect again. The logs from RPC interaction should be logged into the debug.log in that folder.

[firstbt]

Thanks alot

I see some "ThreadRPCServer method=getblocktemplate user=myusername" and "ThreadRPCServer method=getwork user=myusername"... I believe this behavior should be correct to probe to decide which mining protocol should be chosen....getblocktemplate should be selected because that is the latest protocol...according to a doc I read, cgminer is the one that support GBT as well....
Well...I am kinda lost...."No GBT coinbase + append support found, pool unusable if it has no stratum " ..this message I guess it doesn't kinda make sense why cgminer is showing...

Hope someone can clarify...BTW, I tried old version of cgminer....but it has different error message, which is 404, but basically it doesn't work either...

Thanks,

[firstbt]

I just came up one additional question.

Can miners select which transactions they want to include to mine in order to create a block?
I just looked at the payload of getblocktemplate...it contains the array of transactions. ..I believe these should be transactions that need to be included in next block...

1. When the request is made, the bitcoin-qt server sends all unconfirmed transactions from mempool to the miner client? or there is some behind logic that the server choose some transactions based on the blocksize(which is 1mb)..then send only up to that size?...if so..how the bitcoin server determines which transactions should be included to send out?..based on time?? based on fees?.....

2. So basically, what the miner tool does is that they take all these transactions that they received to try to create a block?


I looked at the couple of miner tools, but I don't see some way to select some transactions based on maybe some fees?

3. So...If they can select some transactions, does that mean there are some other GUI tool out there that shows those transactions received from the server...so they can check checkbox to include only some they want..maybe?..then start mining for those selection only?


Thanks a lot.

[ranochigo]

I see some "ThreadRPCServer method=getblocktemplate user=myusername" and "ThreadRPCServer method=getwork user=myusername"... I believe this behavior should be correct to probe to decide which mining protocol should be chosen....getblocktemplate should be selected because that is the latest protocol...according to a doc I read, cgminer is the one that support GBT as well....
Well...I am kinda lost...."No GBT coinbase + append support found, pool unusable if it has no stratum " ..this message I guess it doesn't kinda make sense why cgminer is showing...

I'll revisit the issue in a bit and see if I can get a fix for it.

Can miners select which transactions they want to include to mine in order to create a block?

They can. Most pools uses their own software which has specific parameters to choose the specific transactions to mine.

1. When the request is made, the bitcoin-qt server sends all unconfirmed transactions from mempool to the miner client? or there is some behind logic that the server choose some transactions based on the blocksize(which is 1mb)..then send only up to that size?...if so..how the bitcoin server determines which transactions should be included to send out?..based on time?? based on fees?.....

See getblocktemplate RPC. Bitcoin Core assembles the transaction to send to the miner. By default, Bitcoin Core selects the highest fee rates first, if the user does not specify transactions to be prioritized over the rest.

2. So basically, what the miner tool does is that they take all these transactions that they received to try to create a block?

To assemble the block header and start hashing it while changing the nonce or other parameters during each try.

I looked at the couple of miner tools, but I don't see some way to select some transactions based on maybe some fees?

3. So...If they can select some transactions, does that mean there are some other GUI tool out there that shows those transactions received from the server...so they can check checkbox to include only some they want..maybe?..then start mining for those selection only?

The assembly of block for Bitcoin Core is defined here: https://github.com/bitcoin/bitcoin/blob/master/src/miner.cpp.

I don't think any of the mining software will intentionally exclude any transactions. Any transaction exclusion should be done from the server, not on the mining client itself.

[firstbt]

In order to get balance of a particular address, it iterates every block in block chain..then compare the address that matched....then calculate the balance?
Then isn't that really taking so much time? Under my testnet, I issued getbalance ..then it pretty much quickly returns my current balance...however, the total blocks is abut 1937956...so each block..there should be lots of transactions as well... then how could that getbalance operation returns so quickly to show my balance?..to iterate all blocks and transactions...and compare my public address....etc..it seems like taking so much time..then...is that because it calculated my own wallet?..somehow, they keep track of my own balance separately in somehow wallet structure so they don't have to iterate all blocks somehow? is that why? then if I try to calculate a balance of some other public address then... is my understanding correct that could be taking huge amount time then?

Thanks,

[pooya87]

In order to get balance of a particular address, it iterates every block in block chain..then compare the address that matched....then calculate the balance?

Essentially yes.

When you download the blockchain as your node verifies that block it also checks each input/output against your wallet and each time any of your addresses were involved (received payment or spent coins) it updates your wallet too. So your balance keeps updating as your blockchain grows.
This is why you see your balance, unspent outputs, history,... quickly as you demand it. They are all updated and written in your wallet file.

If you import a new key/address you have to do it all over again minus the block verification so it is faster but it still takes a long time because it has to go through all the blocks from first block (called rescan in bitcoin core) and update your wallet file again.