My transactions get stuck in memory pool !!!

[chameleon123]

Several weeks ago I asked same question. I created my private chain with a very low difficulty (for implementing some tests) and when testing the network and creating transactions they get lost some how!!! Here's a scenario with images attached:

1. I created two node s on same machine, mined one of them till my custom coinbase maturity (I changed it to 10 blocks).
2. I made a transaction from first node to second node (2 btcs), then mined it with second node using command "generate  1". Till this stage it worked fine. See the image for results (first node is in the left and second node is in the right):



3.In this stage no matter from which side I create transactions. It gets stuck in memory  pool. No matter how hard i try to mine it, no peer would  add it to the chain. I created another transaction from first node to second node. See the picture.



I don't this the problem is with fee because with enough fee I still have the problem.
The link to my repository  is : https://github.com/chameleon1239/bitcoin.git

Any one has any idea? I didn't change the source code for confirming transactions  or memory pool.

[1714056598]

1714056598

[1714056598]

1714056598

[1714056598]

1714056598

[1714056598]

1714056598

[darosior]

Hi,

Are you sure your transaction is in the mempool ? Try

Code:

getrawmempool 1

to be sure and to know a little more about your tx.

[darosior]

Are you sure your transaction is in the mempool ? Try

Code:

getrawmempool 1

Hi, yes. I'm pretty sure. I used the command(It works without 1) and  I got that transaction ID. If you run it too, you can see it. Very strange!

The `1` parameter was in order to get more details http://chainquery.com/bitcoin-api/getrawmempool . It is indeed very strange, don't you modify any part of the code ? How did you set up the network ?

[chameleon123]

Are you sure your transaction is in the mempool ? Try

Code:

getrawmempool 1

Hi, yes. I'm pretty sure. I used the command(It works without 1) and  I got that transaction ID. If you run it too, you can see it. Very strange!

The `1` parameter was in order to get more details http://chainquery.com/bitcoin-api/getrawmempool . It is indeed very strange, don't you modify any part of the code ? How did you set up the network ?

I just changed some things according to https://bitcointalk.org/index.php?topic=3345808.msg35016844#msg35016844 guide.Created a new genesis block according to my new difficulty(nBits) which was  0x1f0fffff and powLimit in chainparams.cpp was changed to  0x000ffff...f .Also commented this line in mining.cpp :

Code:

if (IsInitialBlockDownload())
        throw JSONRPCError(RPC_CLIENT_IN_INITIAL_DOWNLOAD, "Bitcoin is downloading blocks...");

Finally changed coinbase maturity to 10.

[DaCryptoRaccoon]

Are you sure your transaction is in the mempool ? Try

Code:

getrawmempool 1

Hi, yes. I'm pretty sure. I used the command(It works without 1) and  I got that transaction ID. If you run it too, you can see it. Very strange!

The `1` parameter was in order to get more details http://chainquery.com/bitcoin-api/getrawmempool . It is indeed very strange, don't you modify any part of the code ? How did you set up the network ?

I just changed some things according to https://bitcointalk.org/index.php?topic=3345808.msg35016844#msg35016844 guide.Created a new genesis block according to my new difficulty(nBits) which was  0x1f0fffff and powLimit in chainparams.cpp was changed to  0x000ffff...f .Also commented this line in mining.cpp :

Code:

if (IsInitialBlockDownload())
        throw JSONRPCError(RPC_CLIENT_IN_INITIAL_DOWNLOAD, "Bitcoin is downloading blocks...");

Finally changed coinbase maturity to 10.

I guess you changed the peer magic values?
If your running in VM's on the same machine you may have firewall issues?

Post the outputs of the following in code tags from both consoles.

getmininginfo
getpeerinfo
getrawtransaction <TXIDHERE>

[chameleon123]

Are you sure your transaction is in the mempool ? Try

Code:

getrawmempool 1

Hi, yes. I'm pretty sure. I used the command(It works without 1) and  I got that transaction ID. If you run it too, you can see it. Very strange!

The `1` parameter was in order to get more details http://chainquery.com/bitcoin-api/getrawmempool . It is indeed very strange, don't you modify any part of the code ? How did you set up the network ?

I just changed some things according to https://bitcointalk.org/index.php?topic=3345808.msg35016844#msg35016844 guide.Created a new genesis block according to my new difficulty(nBits) which was  0x1f0fffff and powLimit in chainparams.cpp was changed to  0x000ffff...f .Also commented this line in mining.cpp :

Code:

if (IsInitialBlockDownload())
        throw JSONRPCError(RPC_CLIENT_IN_INITIAL_DOWNLOAD, "Bitcoin is downloading blocks...");

Finally changed coinbase maturity to 10.

I guess you changed the peer magic values?
If your running in VM's on the same machine you may have firewall issues?

Post the outputs of the following in code tags from both consoles.

getmininginfo
getpeerinfo
getrawtransaction <TXIDHERE>

I guess you changed the peer magic values? 

What are peer magic values ? I don't know anything about them.

If your running in VM's on the same machine you may have firewall issues?

As far as I know, No. I tested this scenario on another machine too but got same results.

[/quote]
Post the outputs of the following in code tags from both consoles.

getmininginfo
getpeerinfo
getrawtransaction <TXIDHERE>

[/quote]

For first node the out puts are :
getmininginfo

Code:

{
  "blocks": 13,
  "currentblockweight": 4000,
  "currentblocktx": 0,
  "difficulty": 1,
  "networkhashps": 0.009866452320361751,
  "pooledtx": 1,
  "chain": "main",
  "warnings": "This is a pre-release test build - use at your own risk - do not use for mining or merchant applications"
}

getpeerinfo

Code:


[
  {
    "id": 0,
    "addr": "127.0.0.1:58886",
    "addrbind": "127.0.0.1:6333",
    "services": "000000000000040d",
    "relaytxes": true,
    "lastsend": 1547002445,
    "lastrecv": 1547002445,
    "bytessent": 4939,
    "bytesrecv": 1080,
    "conntime": 1547001965,
    "timeoffset": 0,
    "pingtime": 0.000418,
    "minping": 0.000316,
    "version": 70015,
    "subver": "/Satoshi:0.17.99/",
    "inbound": true,
    "addnode": false,
    "startingheight": 0,
    "banscore": 0,
    "synced_headers": -1,
    "synced_blocks": -1,
    "inflight": [
    ],
    "whitelisted": false,
    "minfeefilter": 0.00001000,
    "bytessent_per_msg": {
      "block": 250,
      "cmpctblock": 3126,
      "feefilter": 32,
      "getheaders": 93,
      "headers": 212,
      "inv": 183,
      "ping": 160,
      "pong": 160,
      "sendcmpct": 66,
      "sendheaders": 24,
      "tx": 482,
      "verack": 24,
      "version": 127
    },
    "bytesrecv_per_msg": {
      "feefilter": 32,
      "getaddr": 24,
      "getdata": 244,
      "getheaders": 186,
      "ping": 160,
      "pong": 160,
      "sendcmpct": 99,
      "sendheaders": 24,
      "verack": 24,
      "version": 127
    }
  }
]

getrawtransaction 9df617feddf83c2eee4cb6c15b57471ad8f5070c46a8685a2984f463419adbf1

Code:

0200000000010175a55043385a47f725e5a4a6a11f987f51f43fb0b09832c9bc16d22b15db4e980000000017160014e62e2ee223556551100d9e999d2bc334dac83f42fdffffff0254d02c120100000017a9147e92ec13bb97a74588e11a7f9d33a9e68314044e8700c2eb0b0000000017a914184e89e76e84cc73561ff4920561ab038b058542870247304402203c28d4d768495fe8721fe17154dbc0ba8492a19790a4193aaaec28c75d4559b5022012e1c81d1c9d55fbc5d4b6b33e0e0a45d190d7d800d61c06e12feb75b46f51290121039dfb7a582ba7fd851d6ef4f9bf0092efafcad9f2fddd0b7d333cf11bd01e05460c000000

For second node :

getminiginfo

Code:

{
  "blocks": 13,
  "currentblockweight": 0,
  "currentblocktx": 0,
  "difficulty": 1,
  "networkhashps": 0.009866452320361751,
  "pooledtx": 1,
  "chain": "main",
  "warnings": "This is a pre-release test build - use at your own risk - do not use for mining or merchant applications"
}

getpeerinfo

Code:

21:48:42

{
  "blocks": 12,
  "currentblockweight": 0,
  "currentblocktx": 0,
  "difficulty": 1,
  "networkhashps": 0.009107671645756934,
  "pooledtx": 0,
  "chain": "main",
  "warnings": "This is a pre-release test build - use at your own risk - do not use for mining or merchant applications"
}


22:00:52

getmininginfo


22:00:52

{
  "blocks": 13,
  "currentblockweight": 0,
  "currentblocktx": 0,
  "difficulty": 1,
  "networkhashps": 0.009866452320361751,
  "pooledtx": 1,
  "chain": "main",
  "warnings": "This is a pre-release test build - use at your own risk - do not use for mining or merchant applications"
}


22:01:49

getpeerinfo


22:01:49

[
  {
    "id": 0,
    "addr": "127.0.0.1:6333",
    "addrbind": "127.0.0.1:58886",
    "services": "000000000000040d",
    "relaytxes": true,
    "lastsend": 1547002805,
    "lastrecv": 1547002805,
    "bytessent": 1272,
    "bytesrecv": 5131,
    "conntime": 1547001965,
    "timeoffset": 0,
    "pingtime": 0.000399,
    "minping": 0.00024,
    "version": 70015,
    "subver": "/Satoshi:0.17.99/",
    "inbound": false,
    "addnode": true,
    "startingheight": 0,
    "banscore": 0,
    "synced_headers": 13,
    "synced_blocks": 13,
    "inflight": [
    ],
    "whitelisted": false,
    "minfeefilter": 0.00001000,
    "bytessent_per_msg": {
      "feefilter": 32,
      "getaddr": 24,
      "getdata": 244,
      "getheaders": 186,
      "ping": 256,
      "pong": 256,
      "sendcmpct": 99,
      "sendheaders": 24,
      "verack": 24,
      "version": 127
    },
    "bytesrecv_per_msg": {
      "block": 250,
      "cmpctblock": 3126,
      "feefilter": 32,
      "getheaders": 93,
      "headers": 212,
      "inv": 183,
      "ping": 256,
      "pong": 256,
      "sendcmpct": 66,
      "sendheaders": 24,
      "tx": 482,
      "verack": 24,
      "version": 127
    }
  }
]

getrawtransaction 9df617feddf83c2eee4cb6c15b57471ad8f5070c46a8685a2984f463419adbf1

Code:

0200000000010175a55043385a47f725e5a4a6a11f987f51f43fb0b09832c9bc16d22b15db4e980000000017160014e62e2ee223556551100d9e999d2bc334dac83f42fdffffff0254d02c120100000017a9147e92ec13bb97a74588e11a7f9d33a9e68314044e8700c2eb0b0000000017a914184e89e76e84cc73561ff4920561ab038b058542870247304402203c28d4d768495fe8721fe17154dbc0ba8492a19790a4193aaaec28c75d4559b5022012e1c81d1c9d55fbc5d4b6b33e0e0a45d190d7d800d61c06e12feb75b46f51290121039dfb7a582ba7fd851d6ef4f9bf0092efafcad9f2fddd0b7d333cf11bd01e05460c000000

Ports of first node are 6333 and 6332 for rpc. Ports for second node are 13500 and 13501 for rpc.

[darosior]

What are peer magic values ? I don't know anything about them.

They are also called "magic bytes", these are bytes set that every message will start with so we can see that the data on the wire is a Bitcoin message. They are set to :

Code:

pchMessageStart[0] = 0xf9;
pchMessageStart[1] = 0xbe;
pchMessageStart[2] = 0xb4;
pchMessageStart[3] = 0xd9;

Link to the code

What is weird is that your nodes are connected, you can mine and blocks are transmitted. However when mining tx from the mempool are not added to the bloc the software is generating. I would recommend that you add some "LogPrintf()" in the mining code (which is located here) and then you do everything again, after having compiled. Then you would see informations about what is going on in "~/.bitcoin/debug.log". I don't have time to help you right now but I will if you haven't achieved it till then.

[DaCryptoRaccoon]

You will want to change the values to random non common ascii in chainparams.cpp

Code:

$ python
>>> import random
>>> hex(random.randint(127,255))
'0xd9'

Code:

pchMessageStart[0] = 0xf9;
pchMessageStart[1] = 0xbe;
pchMessageStart[2] = 0xb4;
pchMessageStart[3] = 0xd9;    <---- do all 4

Try this recompile and test again.

[chameleon123]

You will want to change the values to random non common ascii in chainparams.cpp

Code:

$ python
>>> import random
>>> hex(random.randint(127,255))
'0xd9'

Code:

pchMessageStart[0] = 0xf9;
pchMessageStart[1] = 0xbe;
pchMessageStart[2] = 0xb4;
pchMessageStart[3] = 0xd9;    <---- do all 4

Try this recompile and test again.

I already did that. I f you look at my code there are randome generated values for these parameters.

[chameleon123]

What are peer magic values ? I don't know anything about them.

They are also called "magic bytes", these are bytes set that every message will start with so we can see that the data on the wire is a Bitcoin message. They are set to :

Code:

pchMessageStart[0] = 0xf9;
pchMessageStart[1] = 0xbe;
pchMessageStart[2] = 0xb4;
pchMessageStart[3] = 0xd9;

Link to the code

What is weird is that your nodes are connected, you can mine and blocks are transmitted. However when mining tx from the mempool are not added to the bloc the software is generating. I would recommend that you add some "LogPrintf()" in the mining code (which is located here) and then you do everything again, after having compiled. Then you would see informations about what is going on in "~/.bitcoin/debug.log". I don't have time to help you right now but I will if you haven't achieved it till then.

Thanks a lot for all the help. I will certainly do that and let you know.Do you think I can solve it in two days? I have a deadline by that time.

[darosior]

What are peer magic values ? I don't know anything about them.

They are also called "magic bytes", these are bytes set that every message will start with so we can see that the data on the wire is a Bitcoin message. They are set to :

Code:

pchMessageStart[0] = 0xf9;
pchMessageStart[1] = 0xbe;
pchMessageStart[2] = 0xb4;
pchMessageStart[3] = 0xd9;

Link to the code

What is weird is that your nodes are connected, you can mine and blocks are transmitted. However when mining tx from the mempool are not added to the bloc the software is generating. I would recommend that you add some "LogPrintf()" in the mining code (which is located here) and then you do everything again, after having compiled. Then you would see informations about what is going on in "~/.bitcoin/debug.log". I don't have time to help you right now but I will if you haven't achieved it till then.

Thanks a lot for all the help. I will certainly do that and let you know.Do you think I can solve it in two days? I have a deadline by that time.

Depends how much time you put in and what outputs the debug.log ^^ But it is surely not impossible.

[chameleon123]

What are peer magic values ? I don't know anything about them.

They are also called "magic bytes", these are bytes set that every message will start with so we can see that the data on the wire is a Bitcoin message. They are set to :

Code:

pchMessageStart[0] = 0xf9;
pchMessageStart[1] = 0xbe;
pchMessageStart[2] = 0xb4;
pchMessageStart[3] = 0xd9;

Link to the code

What is weird is that your nodes are connected, you can mine and blocks are transmitted. However when mining tx from the mempool are not added to the bloc the software is generating. I would recommend that you add some "LogPrintf()" in the mining code (which is located here) and then you do everything again, after having compiled. Then you would see informations about what is going on in "~/.bitcoin/debug.log". I don't have time to help you right now but I will if you haven't achieved it till then.

Thanks for the hint. I did as you said and found something! The problem rises when  "TestPackageTransactions()" in miner.cpp returns false in this line : https://github.com/bitcoin/bitcoin/blob/c6806ee8694e9c864a29ef260c87b9fed77de84f/src/miner.cpp#L217
I tried to comment it so that it could accept my block but then "TestBlockValidity()" in validation.cpp threw an error in https://github.com/bitcoin/bitcoin/blob/c6806ee8694e9c864a29ef260c87b9fed77de84f/src/validation.cpp#L3594.
Then I tried to comment where this function was used. Then I got error in mining.cpp here : https://github.com/bitcoin/bitcoin/blob/master/src/rpc/mining.cpp#L144
If I comment this line as well I won't be able to mine anything and my balance doesn't not change.

It seems the  network doesn't accept my block as a valid block. I wonder why,It may have something to do with witness but it's all ambiguous to me !!!!

Link to my project with logs and without commenting those functions mentioned above : https://github.com/chameleon1239/bitcoin.git

The most weird thing is that it works for first transaction, then doesn't work for the second one. Then it works for third one and so on.
Please if you find some time take a look at it. I'll appreciate it so much.

[darosior]

What are peer magic values ? I don't know anything about them.

They are also called "magic bytes", these are bytes set that every message will start with so we can see that the data on the wire is a Bitcoin message. They are set to :

Code:

pchMessageStart[0] = 0xf9;
pchMessageStart[1] = 0xbe;
pchMessageStart[2] = 0xb4;
pchMessageStart[3] = 0xd9;

Link to the code

What is weird is that your nodes are connected, you can mine and blocks are transmitted. However when mining tx from the mempool are not added to the bloc the software is generating. I would recommend that you add some "LogPrintf()" in the mining code (which is located here) and then you do everything again, after having compiled. Then you would see informations about what is going on in "~/.bitcoin/debug.log". I don't have time to help you right now but I will if you haven't achieved it till then.

Thanks for the hint. I did as you said and found something! The problem rises when  "TestPackageTransactions()" in miner.cpp returns false in this line : https://github.com/bitcoin/bitcoin/blob/c6806ee8694e9c864a29ef260c87b9fed77de84f/src/miner.cpp#L217
I tried to comment it so that it could accept my block but then "TestBlockValidity()" in validation.cpp threw an error in https://github.com/bitcoin/bitcoin/blob/c6806ee8694e9c864a29ef260c87b9fed77de84f/src/validation.cpp#L3594.
Then I tried to comment where this function was used. Then I got error in mining.cpp here : https://github.com/bitcoin/bitcoin/blob/master/src/rpc/mining.cpp#L144
If I comment this line as well I won't be able to mine anything and my balance doesn't not change.

It seems the  network doesn't accept my block as a valid block. I wonder why,It may have something to do with witness but it's all ambiguous to me !!!!

Link to my project with logs and without commenting those functions mentioned above : https://github.com/chameleon1239/bitcoin.git

The most weird thing is that it works for first transaction, then doesn't work for the second one. Then it works for third one and so on.
Please if you find some time take a look at it. I'll appreciate it so much.

Hi,

don't change validation rules.

// Perform transaction-level checks before adding to block:
// - transaction finality (locktime)
// - premature witness (in case segwit transactions are added to mempool before
//   segwit activation)
bool BlockAssembler::TestPackageTransactions(const CTxMemPool::setEntries& package)
{
    for (CTxMemPool::txiter it : package) {
        if (!IsFinalTx(it->GetTx(), nHeight, nLockTimeCutoff))
            return false;
        if (!fIncludeWitness && it->GetTx().HasWitness())
            return false;
    }
    return true;
}

So the function that fails returns false in two cases
 : when the locktime is not finished or when you send a segwit tx when segwit is not activated.
The locktime of your transaction is :

Code:

0c000000

However according to the protocol spec if all your inputs have a sequence of sequence of

Code:

'0xffffffff'

This is ignored. However your sequence is :

Code:

'0xfffffffd'

So the tx wont be added to the bloc before 12 (0c000000) blocks. I don't know how comes that your tx have such a locktime and a sequence (which version of bitcoin-core did you fork ?), but you should try to mine 12 blocks then see if it is added to a block. If this doesn't work then it is an issue with Segwit activation (try using legacy addresses in this case).

To have your raw transaction decoded use

Code:

getrawtransaction <id> 1

[chameleon123]

which version of bitcoin-core did you fork ?

First, thank you so much for the help. I forked 16.2 from branch master. I mined coinbase transactions till 12 blocks further also but didn't work. I try to use legacy addresses and let you know ASAP. Thanks again!

[darosior]

which version of bitcoin-core did you fork ?

First, thank you so much for the help. I forked 16.2 from branch master. I mined coinbase transactions till 12 blocks further also but didn't work. I try to use legacy addresses and let you know ASAP. Thanks again!

Ah ! I think in 0.16, this is when Segwit had to be signaled for, and was treated as always active in 0.17. It should work with legacy addresses, and you should fork 0.17(.1) if you want to have it activated by default.  EDIT: this was completely wrong and confused.

[chameleon123]

which version of bitcoin-core did you fork ?

First, thank you so much for the help. I forked 16.2 from branch master. I mined coinbase transactions till 12 blocks further also but didn't work. I try to use legacy addresses and let you know ASAP. Thanks again!

Ah ! I think in 0.16, this is when Segwit had to be voted, and was treated as always active in 0.17. It should work with legacy addresses, and you should fork 0.17(.1) if you want to have it activated by default.

I did exactly as you said, forked from 0.17.1 . Same thing in the same place happened. By legacy addresses do you mean the ones that starts with 3.. or 1... ? . Bitcoin core wallet doesn't generate the ones that start with 1... . Is there any online generators ? I searched and found nothing.

[DaCryptoRaccoon]

Legacy start with 1  segwit starts with 3.

You should be able to launch the client with legacy addressing.

add this to your .conf file.

Code:

addresstype=legacy

It should then launch with legacy addressing.

[chameleon123]

Legacy start with 1  segwit starts with 3.

You should be able to launch the client with legacy addressing.

add this to your .conf file.

Code:

addresstype=legacy

It should then launch with legacy addressing.

Thank you so much!!!

[chameleon123]

which version of bitcoin-core did you fork ?

First, thank you so much for the help. I forked 16.2 from branch master. I mined coinbase transactions till 12 blocks further also but didn't work. I try to use legacy addresses and let you know ASAP. Thanks again!

Ah ! I think in 0.16, this is when Segwit had to be voted, and was treated as always active in 0.17. It should work with legacy addresses, and you should fork 0.17(.1) if you want to have it activated by default.

Finally I got the answer! Thank you for all the help. I truly appreciate it.
But It's still ambiguous to me why segwit address would fail creating block....

[darosior]

Finally I got the answer!

As I previously said you can use legacy addresses but it is not a long term viable solution. You should fork 0.17+ or mine enough blocks to activate Segwit.

Thank you for all the help. I truly appreciate it.

No problem  

But It's still ambiguous to me why segwit address would fail creating block....

Because Segwit was not activated yet so you could not (standardly) add Segwit transactions in a block.
Regarding the terminology that an "address would fail to create a block" : this is not the address which creates a block, it is the software. Block generation is (almost) unrelated to addresses.