|
Title: Do you need to manually reset the nodes every week? Post by: johnyj on February 18, 2016, 04:40:16 PM http://bitcointicker.co/networkstats/
You can see from this site, for more than 10 days, the mempool size steadily grew beyond 80MB, and the operator did a restart of the process, then it start to build from 0 again, but it always goes up no matter what is the queue size of transactions http://photo.mystisland.org/2016/mempoolgrow.png And this is not the same as I see on tradeblock, they seems to have it under 10MB most of the time Title: Re: Do you need to manually reset the nodes every week? Post by: tspacepilot on February 19, 2016, 06:12:26 AM There may be something going on here if you say that your stats are different from what you're seeing on other people's nodes. In any case, it's definitely not regular procedure to be restarting your nodes every week. Bitcoin is reasonably stable software and you should be able to get significant uptime if everything is configured correctly. On the other hand, 80MB doesn't really seem all that large (but you're right that that linear growth is troubling), I'm guessing something's configured incorrectly here. I think it's hard to diagnose further without more details.
Title: Re: Do you need to manually reset the nodes every week? Post by: johnyj on February 19, 2016, 08:01:02 AM There may be something going on here if you say that your stats are different from what you're seeing on other people's nodes. In any case, it's definitely not regular procedure to be restarting your nodes every week. Bitcoin is reasonably stable software and you should be able to get significant uptime if everything is configured correctly. On the other hand, 80MB doesn't really seem all that large (but you're right that that linear growth is troubling), I'm guessing something's configured incorrectly here. I think it's hard to diagnose further without more details. This is not my machine, but my machine is having similar problem and now it is over 225MB after 10 days of running. Maybe I'm just too lazy to build the bitcoind from source Title: Re: Do you need to manually reset the nodes every week? Post by: tspacepilot on February 19, 2016, 08:04:20 AM There may be something going on here if you say that your stats are different from what you're seeing on other people's nodes. In any case, it's definitely not regular procedure to be restarting your nodes every week. Bitcoin is reasonably stable software and you should be able to get significant uptime if everything is configured correctly. On the other hand, 80MB doesn't really seem all that large (but you're right that that linear growth is troubling), I'm guessing something's configured incorrectly here. I think it's hard to diagnose further without more details. This is not my machine, but my machine is having similar problem and now it is over 225MB after 10 days of running. Maybe I'm just too lazy to build the bitcoind from source I'm rebuilding bitcoin and bitcoin-qt because of another question in this board. I'll kick it off and run it for a while and tell you if I see something similar. I imagine that long before then someone else who's on here and has a node that's been up for a while can chime-in about whether 225MB is a lot. Is your memory usage still climbing on that linear pattern after 10 days? Title: Re: Do you need to manually reset the nodes every week? Post by: Foxpup on February 19, 2016, 10:48:08 AM What exactly is the problem? Linear mempool growth is normal and expected for a constant number of transactions per second, at least until old transactions stop being broadcast and an equilibrium is reached. Many have complained that said equilibrium ends up being too large, so starting from version 0.12 Bitcoin Core has a configurable limit on how large the mempool can grow, after which it will start dropping the transactions with the lowest fees.
For reference, my node's mempool usually hovers around 40k transactions in 1GB, which I think is typical. Title: Re: Do you need to manually reset the nodes every week? Post by: ScripterRon on February 19, 2016, 11:05:44 AM What exactly is the problem? Linear mempool growth is normal and expected for a constant number of transactions per second, at least until old transactions stop being broadcast and an equilibrium is reached. Many have complained that said equilibrium ends up being too large, so starting from version 0.12 Bitcoin Core has a configurable limit on how large the mempool can grow, after which it will start dropping the transactions with the lowest fees. My node has been hovering around 11,000 mempool transactions lately, although right now it is at 7100.For reference, my node's mempool usually hovers around 40k transactions in 1GB, which I think is typical. Title: Re: Do you need to manually reset the nodes every week? Post by: johnyj on February 20, 2016, 01:58:19 AM What exactly is the problem? Linear mempool growth is normal and expected for a constant number of transactions per second, at least until old transactions stop being broadcast and an equilibrium is reached. Many have complained that said equilibrium ends up being too large, so starting from version 0.12 Bitcoin Core has a configurable limit on how large the mempool can grow, after which it will start dropping the transactions with the lowest fees. For reference, my node's mempool usually hovers around 40k transactions in 1GB, which I think is typical. Thanks, good to know this is normal, because I have observed this phenomenon for years. I never cared too much since it is still small comparing to the total free memory. However I'm seeing tradeblock using a different way to calculate the mempool size, it is always related to the unconfirmed transactions in mempool, not growing linearly In your case, 40K transactions would result in 20M maximum mempool, not 1GB. Since you have 40K transactions, and each of them is 0.5KB maximum, what are all those other memories used for? Title: Re: Do you need to manually reset the nodes every week? Post by: Foxpup on February 20, 2016, 03:32:55 AM each of them is 0.5KB maximum What are you talking about? Even average transactions are larger than that, and mempool transactions generally aren't average anyway: most of these transactions are unconfirmed for a reason, and usually that reason is their size.Title: Re: Do you need to manually reset the nodes every week? Post by: johnyj on February 20, 2016, 04:25:17 AM each of them is 0.5KB maximum What are you talking about? Even average transactions are larger than that, and mempool transactions generally aren't average anyway: most of these transactions are unconfirmed for a reason, and usually that reason is their size.If a block contains 1MB data, and typically 2000 transactions, then each transaction is estimated at 0.5KB. In fact I have seen some blocks containing almost 3000 transactions, and cointape estimate average transaction to be 258 bytes right now I have checked the rawmempool and indeed found some very large transactions, but to match your number (40k transactions for 1GB), each of them should be 25K, which is almost 100 times larger than average transaction, all 100+ sigop transactions? Title: Re: Do you need to manually reset the nodes every week? Post by: Foxpup on February 20, 2016, 05:18:05 AM If a block contains 1MB data, and typically 2000 transactions, then each transaction is estimated at 0.5KB. In fact I have seen some blocks containing almost 3000 transactions, and cointape estimate average transaction to be 258 bytes right now I always heard it was around 600 bytes. Tradeblock says it's currently 540 bytes. Blocks containing 2,000 transactions are not typical.each of them should be 25K No, they should average 25kB. Most of the space is taken up by a very small number of exceptionally large transactions (possibly broadcast as DOS attacks), so most transactions will be far below average.Title: Re: Do you need to manually reset the nodes every week? Post by: shorena on February 21, 2016, 08:43:18 AM Not once a week, but I see similar things on my node. The TX that keep pilling up mostly pay a very low fee, so it could be avoided by setting limitfreerelay minrelaytxfee lower and high respectivly.
https://i.imgur.com/ElufFxM.png https://i.imgur.com/AAzF4gA.png Title: Re: Do you need to manually reset the nodes every week? Post by: tspacepilot on February 21, 2016, 10:08:38 PM Not once a week, but I see similar things on my node. The TX that keep pilling up mostly pay a very low fee, so it could be avoided by setting limitfreerelay minrelaytxfee lower and high respectivly. https://i.imgur.com/ElufFxM.png https://i.imgur.com/AAzF4gA.png Sho, is that drop on the 7th of feb where you're restarting your node to clear your mempool? If that's the case then it seems like at least one long-time node runner (you) *is* resetting your node regularly to avoid the pileup. Title: Re: Do you need to manually reset the nodes every week? Post by: shorena on February 22, 2016, 09:10:24 AM Not once a week, but I see similar things on my node. The TX that keep pilling up mostly pay a very low fee, so it could be avoided by setting limitfreerelay minrelaytxfee lower and high respectivly. https://i.imgur.com/ElufFxM.png https://i.imgur.com/AAzF4gA.png Sho, is that drop on the 7th of feb where you're restarting your node to clear your mempool? If that's the case then it seems like at least one long-time node runner (you) *is* resetting your node regularly to avoid the pileup. Its actually a crash with a quick reboot afterwards. 2 GB RAM is no longer enough to let it run just like that. Its about time again, maybe I should just reboot this time. The new server will have 6GB so its a matter for finding time to set up the new one. Title: Re: Do you need to manually reset the nodes every week? Post by: GamerSg on February 23, 2016, 11:29:41 AM Just set minrelaytxfee to 0.0001 and you will have a much smaller and stable mempool.
The default setting allows spam to be accepted into the mempool and sticks around forever because those transactions will never be mined. Title: Re: Do you need to manually reset the nodes every week? Post by: theymos on February 24, 2016, 02:00:07 AM I was occasionally having problems with this on one of my nodes, but 0.12.0's mempool limiter totally fixes it.
Title: Re: Do you need to manually reset the nodes every week? Post by: BlindMayorBitcorn on February 24, 2016, 02:43:52 AM I've personally never run anything more complicated than an SPV wallet. When MIT came aboard I had this I guess naive idea that soon enough every research University in the world would be running full nodes and that that would be more than enough. How wrong was I? Am I a part of the problem? Are SPV nodes a drag on the system, in that they make it that much more convenient to not run a full one?
Title: Re: Do you need to manually reset the nodes every week? Post by: theymos on February 24, 2016, 03:13:23 AM I've personally never run anything more complicated than an SPV wallet. When MIT came aboard I had this I guess naive idea that soon enough every research University in the world would be running full nodes and that that would be more than enough. How wrong was I? Am I a part of the problem? Are SPV nodes a drag on the system, in that they make it that much more convenient to not run a full one? If not enough of the economy is backed by full nodes, then Bitcoin is totally insecure. See here (https://bitcoin.org/en/bitcoin-core/features/validation) and here (https://en.bitcoin.it/wiki/Full_node#Economic_strength). Lightweight nodes are also a slight drain on the network's capacity, but that's a much less pressing and serious problem than the economic issue. |
