iCEBREAKER
Legendary
Offline
Activity: 2156
Merit: 1072
Crypto is the separation of Power and State.
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 12:04:40 AM |
|
Does CreateNewBlock currently take longer to execute if there are more TXs in a miner's mempool to pick from? If so, this would add credence to Cypherdoc's hunch that miner's are producing more empty blocks when mempool swells.
Yep, I already pointed that out to you specifically! It's superlinear in the mempool size (well, ignoring caching) But thats unrelated to f2pool/antpool and the other SPV miners, as they're not ever calling createnewblock in that case, as they're mining without even validating. One can mine on a validated chain with no transactions while waiting for createnewblock (which is what eligius does, for example). I also pointed out that this is trivially optimizable, but no one has bothered previously. Wait, how is an empty block created without calling createnewblock? Is there a createemptyblock thingy or what?
|
██████████ ██████████████████ ██████████████████████ ██████████████████████████ ████████████████████████████ ██████████████████████████████ ████████████████████████████████ ████████████████████████████████ ██████████████████████████████████ ██████████████████████████████████ ██████████████████████████████████ ██████████████████████████████████ ██████████████████████████████████ ████████████████████████████████ ██████████████ ██████████████ ████████████████████████████ ██████████████████████████ ██████████████████████ ██████████████████ ██████████ Monero
|
| "The difference between bad and well-developed digital cash will determine whether we have a dictatorship or a real democracy." David Chaum 1996 "Fungibility provides privacy as a side effect." Adam Back 2014
|
| | |
|
|
|
Peter R
Legendary
Offline
Activity: 1162
Merit: 1007
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 12:19:04 AM |
|
OK. Provide me with your estimates for the following (and explain how you arrived at your numbers) and I'll update my table using your numbers: 1. The cost per node to store 1 GB of additional blockchain data for 5 years, assume the outputs are spent. 2. The cost per node to store 1 GB of additional blockchain data for 5 years, assuming the outputs are unspent.
I may be missing the context as this thread is high volume and I've not read any of the backlog... But for a full verifying node, the on-going cost cost of 1GB of additional transactions with all outputs spent is 0; all the cost related to that 1GB of data is related to the bandwidth to get it to you and the verification cost, and for short term storage until its burried, after that it need not be stored. The cost for unspent is some non-zero number which depends on your estimation of storage costs. This thread can be hard to follow if you're not following it all the time! The question was in reference to a debate I was having with Odalv about these "order of magnitude" estimates shown in this table. I was suggesting that, under the conditions considered in the table, it is cheaper for miners to write the spam to the Blockchain and more costly for the spammer, than continually rejecting it: ![](https://ip.bitcointalk.org/?u=https%3A%2F%2Fi.imgur.com%2F9bJxcxp.png&t=665&c=ORMBhRo-qV10qA) Does CreateNewBlock currently take longer to execute if there are more TXs in a miner's mempool to pick from? If so, this would add credence to Cypherdoc's hunch that miner's are producing more empty blocks when mempool swells.
Yep, I already pointed that out to you specifically! It's superlinear in the mempool size (well, ignoring caching) But thats unrelated to f2pool/antpool and the other SPV miners, as they're not ever calling createnewblock in that case, as they're mining without even validating. One can mine on a validated chain with no transactions while waiting for createnewblock (which is what eligius does, for example). Sorry, yes I know you explained that. The point I'm trying to make is that if CreateNewBlock is super-linear in mempool size, then it would not be surprising to see more empty blocks (what Cypher was calling "defensive blocks") when mempool swells (the miners are mining on an empty block for longer while waiting for CreateNewBlock to finish). This was Cypher's point from the very beginning that many people, including myself, were suggesting was probably not the case! Furthermore, how can f2pool/antpool mine a non-empty block without calling createnewblock?
|
|
|
|
Adrian-x
Legendary
Offline
Activity: 1372
Merit: 1000
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 12:24:26 AM |
|
Spammer only spends money 1x during lifetime, but miner has to keep bigger disk-space forever (2 hard disc consume more electricity than single one)
OK. Provide me with your estimates for the following (and explain how you arrived at your numbers) and I'll update my table using your numbers: 1. The cost per node to store 1 GB of additional blockchain data for 5 years, assume the outputs are spent. 2. The cost per node to store 1 GB of additional blockchain data for 5 years, assuming the outputs are unspent. I'm sorry, I have no estimates. I know that bloat-chain is not solution There are better solutions how to organize blockchain. My full node just crashed ... all data are lost ... downloading 6 years 25 weeks. ... hmm I hope that it is only me and there will remain some copy of blockchain. :-) (not joking) your data management systems need updating if you lost the data. Bitcoin is a $3B network and if you dont have an invested interest in keeping the blockchain to prove you own X coin or care enough to manage the responsibility you shouldn't be dictating how we should use it. not everyone who can't backup data or doesn't want to should be running a node. I have friends who dont trust digital photos because they trust paper copies more. I'm surprised to find you fit the profile. I also think you're not weighting the cost and benefits correctly, the cost of 2 hard disc consuming more electricity than single one, is an inconsequential cost at the scale of executing this same attack with an 8MB block. (hell its inconsequential at the cost of the attack on the 1MB block.) limiting block space is not going to save anyone except you a little bandwidth because you dont back up.
|
Thank me in Bits 12MwnzxtprG2mHm3rKdgi7NmJKCypsMMQw
|
|
|
solex
Legendary
Offline
Activity: 1078
Merit: 1006
100 satoshis -> ISO code
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 01:25:48 AM |
|
I just want to highlight the comments here from bitcoin-dev today, which seem to indicate that these spamming attacks would have been 5.5x more expensive if a dust threshold software change had not been made. Perhaps there are deeper insights someone else has on this...? 16:53 wangchun why can those spam get confirmed. 0.00001 BTC vout below dust threshold right? 16:54 phantomcircuit wangchun, iirc the dust threshold is 546 satoshis 16:54 wangchun not 5460 satoshis? changed? 16:54 aschildbach wangchun: Yes it was cut by 10 a few months ago.
|
|
|
|
cypherdoc (OP)
Legendary
Offline
Activity: 1764
Merit: 1002
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 02:12:12 AM Last edit: July 09, 2015, 02:28:27 AM by cypherdoc |
|
The point I'm trying to make is that if CreateNewBlock is super-linear in mempool size, then it would not be surprising to see more empty blocks (what Cypher was calling "defensive blocks") when mempool swells (the miners are mining on an empty block for longer while waiting for CreateNewBlock to finish). This was Cypher's point from the very beginning that many people, including myself, were suggesting was probably not the case!
that is exactly what i was saying w/o knowing anything about CNB. it was just intuitive that constructing a new block would take longer for a large mempool so i deduced that a miner might decide "f*ck it" that he doesn't want to not only spend additional time validating a large incoming block but also not want to spend additional time creating a large new block so instead just automatically switch to 0 block mining upon arrival of a large block. the thing none of us anticipated was that he would not validate that incoming large block at all and switch to SPV mining while >50% of the network would also be SPV mining (Discus, Antpool, BTCGuild) and trigger an invalid SPV longest chain. that's some wild shit going on. given this newly recognized scenario, i still conclude however that this is all a result of the block size limit being hit continuously by attacking spammers and secondarily intentionally causing a bloated mempool which then causes these unanticipated defensive strategies and more intentionally, user disruption.
|
|
|
|
cypherdoc (OP)
Legendary
Offline
Activity: 1764
Merit: 1002
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 02:55:42 AM |
|
I just want to highlight the comments here from bitcoin-dev today, which seem to indicate that these spamming attacks would have been 5.5x more expensive if a dust threshold software change had not been made. Perhaps there are deeper insights someone else has on this...? 16:53 wangchun why can those spam get confirmed. 0.00001 BTC vout below dust threshold right? 16:54 phantomcircuit wangchun, iirc the dust threshold is 546 satoshis 16:54 wangchun not 5460 satoshis? changed? 16:54 aschildbach wangchun: Yes it was cut by 10 a few months ago.
i don't think it really matters, does it? since most of the mined blocks are filled with real demand trying to get into blocks with or without spamming, the only difference being that with spamming the real demand has to pay significantly higher fees to get in, the cost to fill up the rest of the block with spam is minimal as is the minimal cost to bloat the mempool to extraordinary levels (94K 0confs as i write). i say minimal cost for the mempool b/c most of it gets turned over after 24h deletion allowing that spam to be recycled w/o having to pay.
|
|
|
|
cypherdoc (OP)
Legendary
Offline
Activity: 1764
Merit: 1002
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 03:59:27 AM |
|
all the yearly silver and mining stocks have that sick top left to bottom right chart configuration. this is a horrible leading indicator for both gold and stocks: silver futures: ![](https://ip.bitcointalk.org/?u=https%3A%2F%2Fi.imgur.com%2F8gLb6NY.png&t=665&c=d9dtqrloIilrPA) SLW: ![](https://ip.bitcointalk.org/?u=https%3A%2F%2Fi.imgur.com%2FI8h6eYc.png&t=665&c=REsputBgP1hovA) PAAS: ![](https://ip.bitcointalk.org/?u=https%3A%2F%2Fi.imgur.com%2FTkQfzA5.png&t=665&c=F8RdmyisDY7czw) silverbox's GPL: ![](https://ip.bitcointalk.org/?u=https%3A%2F%2Fi.imgur.com%2FYbdkoVE.png&t=665&c=rQPGPqyjAtmnDg)
|
|
|
|
TPTB_need_war
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 04:08:36 AM |
|
When miner's mempool exceeds block size by some backlog factor, they need to send out a message to the peers they are connected to saying "don't propagate transactions to me with smaller than X fee per kilobyte until further notice". This will mitigate the impact that spam-that-can-never-be-added-to-a-block can have on the network, but it won't entirely eliminate it, because as the mempool backlog reduces, the miner will need to back off the notice to some unknown lower fee threshold. The miner could guess but it will never be 100% accurate. So spam will get verified and rejected. But perhaps those estimates can be close enough that the losses the spammers takes in fees for transactions it sends that do get into a block will be greater than the gains it values from effective spamming (what ever those gains are ![Huh](https://bitcointalk.org/Smileys/default/huh.gif) ). Appears to me the issue can be fixed. Afaik, the reason incoming transactions are verified before it is known whether they will be in a new block, is because it is considered spam to propagate (send out again to another peer) invalid transactions that would be rejected and never appear in a block.
|
|
|
|
cypherdoc (OP)
Legendary
Offline
Activity: 1764
Merit: 1002
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 04:27:03 AM |
|
When miner's mempool exceeds block size by some backlog factor, they need to send out a message to the peers they are connected to saying "don't propagate transactions to me with smaller than X fee per kilobyte until further notice". This will mitigate the impact that spam-that-can-never-be-added-to-a-block can have on the network, but it won't entirely eliminate it, because as the mempool backlog reduces, the miner will need to back off the notice to some unknown lower fee threshold. The miner could guess but it will never be 100% accurate. So spam will get verified and rejected. But perhaps those estimates can be close enough that the losses the spammers takes in fees for transactions it sends that do get into a block will be greater than the gains it values from effective spamming (what ever those gains are ![Huh](https://bitcointalk.org/Smileys/default/huh.gif) ). Appears to me the issue can be fixed. Afaik, the reason incoming transactions are verified before it is known whether they will be in a new block, is because it is considered spam to propagate (send out again to another peer) invalid transactions that would be rejected and never appear in a block. gmax already told us to use minrelaytxfee.
|
|
|
|
tvbcof
Legendary
Offline
Activity: 4676
Merit: 1276
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 04:47:58 AM |
|
Spammer only spends money 1x during lifetime, but miner has to keep bigger disk-space forever (2 hard disc consume more electricity than single one)
OK. Provide me with your estimates for the following (and explain how you arrived at your numbers) and I'll update my table using your numbers: 1. The cost per node to store 1 GB of additional blockchain data for 5 years, assume the outputs are spent. 2. The cost per node to store 1 GB of additional blockchain data for 5 years, assuming the outputs are unspent. I'm sorry, I have no estimates. I know that bloat-chain is not solution There are better solutions how to organize blockchain. My full node just crashed ... all data are lost ... downloading 6 years 25 weeks. ... hmm I hope that it is only me and there will remain some copy of blockchain. :-) (not joking) Don't worry. According to Mike Hearn Bitcoin can survive just fine with 4 or 6 (forgot which) copies of the blockchain worldwide.
|
sig spam anywhere and self-moderated threads on the pol&soc board are for losers.
|
|
|
domob
Legendary
Offline
Activity: 1135
Merit: 1170
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 05:28:52 AM |
|
Yes, but the ongoing spam shows clearly that all full nodes are handling the traffic just fine. We were told that they would crash and burn from overloaded memory. Not true. Wait, wasn't that exactly Hearn's main argument in his "the sky is falling if we don't increase the blocksize" blog post? So why do you see the fact that it did not prove true so far as supporting your position of increasing the block size?
|
Use your Namecoin identity as OpenID: https://nameid.org/Donations: 1 domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NC domobcmcmVdxC5yxMitojQ4tvAtv99pY BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS | GPG 0xA7330737
|
|
|
Peter R
Legendary
Offline
Activity: 1162
Merit: 1007
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 05:37:00 AM |
|
Yes, but the ongoing spam shows clearly that all full nodes are handling the traffic just fine. We were told that they would crash and burn from overloaded memory. Not true. Wait, wasn't that exactly Hearn's main argument in his "the sky is falling if we don't increase the blocksize" blog post? So why do you see the fact that it did not prove true so far as supporting your position of increasing the block size? It provided evidence that the network can handle the increased load, and it confirmed that people do get frustrated by the confirmation delays.
|
|
|
|
sidhujag
Legendary
Offline
Activity: 2044
Merit: 1005
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 07:43:52 AM |
|
Let's face it If we want adoption by a cb we need more tx/s however we can solve other probs first before raising limit to efficiently come to that conclusion.
|
|
|
|
gmaxwell
Staff
Legendary
Offline
Activity: 4228
Merit: 8644
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 09:36:30 AM |
|
I just want to highlight the comments here from bitcoin-dev today, which seem to indicate that these spamming attacks would have been 5.5x more expensive if a dust threshold software change had not been made. Perhaps there are deeper insights someone else has on this...? 16:53 wangchun why can those spam get confirmed. 0.00001 BTC vout below dust threshold right? 16:54 phantomcircuit wangchun, iirc the dust threshold is 546 satoshis 16:54 wangchun not 5460 satoshis? changed? 16:54 aschildbach wangchun: Yes it was cut by 10 a few months ago.
i don't think it really matters, does it? since most of the mined blocks are filled with real demand trying to get into blocks with or without spamming, Unfortunately, the majority in the blocks I checked earlier today have been the DOS attack-- e.g. transactions tracable from outputs of this transaction https://blockchain.info/tx/3bad15167c60de483cd32cb990d1e46f0a0d8ab380e3fc1cace01afc9c1bb5af and a few others. ... though the attack style has been shifting to evade filtering by miners. The change mentioned in the chat above is https://github.com/bitcoin/bitcoin/pull/3305 (you might find the comments there interesting), it's one of Mikes couple contributions to Bitcoin Core.
|
|
|
|
awemany
Newbie
Offline
Activity: 28
Merit: 0
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 10:43:47 AM |
|
So miners are filtering out the spam? What are they filtering for?
|
|
|
|
sickpig
Legendary
Offline
Activity: 1260
Merit: 1008
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 12:19:55 PM |
|
Jeff Garzik has just posted 2 pull requests for bitcoin core on github: https://github.com/bitcoin/bitcoin/pull/6405Remove TX priority and free transaction area from mempool, block creator.https://github.com/bitcoin/bitcoin/pull/6402 Floating network relay fee increase, if memory pool grows too large.the latter will introduce, if merged, "a relay fee that adjusts to floods which cause the memory pool to grow too large". It seems that core devs start becoming aware of the lack of of economic incentives for services provided by full nodes.
|
Bitcoin is a participatory system which ought to respect the right of self determinism of all of its users - Gregory Maxwell.
|
|
|
solex
Legendary
Offline
Activity: 1078
Merit: 1006
100 satoshis -> ISO code
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 12:21:03 PM |
|
Goddammit, Peter Todd hits the nail on the head! Kudos to him. Yes, that 10x reduction looks to be unwise in hindsight. I recall the painful and protracted debate on bct just getting the dust threshold at 5460 sats in the first place. So miners are filtering out the spam? What are they filtering for?
Hey, nice to see you in this thread. It appeared that BitFury was aggressively filtering by fee when producing small 138KB blocks, however they have just mined a 998KB block ( 364549) which I think is their first near this size.
|
|
|
|
TheRealSteve
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 01:30:20 PM |
|
It appeared that BitFury was aggressively filtering by fee when producing small 138KB blocks, however they have just mined a 998KB block ( 364549) which I think is their first near this size. Nah, there were also 364354, 364383, 364393 and 364398
|
|
|
|
awemany
Newbie
Offline
Activity: 28
Merit: 0
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 01:49:34 PM |
|
So miners are filtering out the spam? What are they filtering for?
Hey, nice to see you in this thread. It appeared that BitFury was aggressively filtering by fee when producing small 138KB blocks, however they have just mined a 998KB block ( 364549) which I think is their first near this size. Hi! Yeah, I was always somewhat put off by the presentation here, too confusing, threads with too many pages. Have a couple of old accounts that I lost all login data for. On the other hand, Reddit is sometimes a bit shallow and fast scrolling... So they did increase the fees then and not reduce their blocksize, it seems? That would make a lot more sense to me (from the miner's perspective).
|
|
|
|
newb4now
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 09, 2015, 02:14:38 PM |
|
In the short run, miners will keep putting up with spam transactions because most of their revenue (great majority) is coming from the BTC block reward. As rewards become less and less and transaction volume (spam or legitimate) continues to increase the balance will gradually start to change.
Eventually miners will start to ignore transactions without some minimum transaction fee (which will change over time based on block reward/transaction fee balance).
This is a self correcting problem which miners will correct on their own (by only verifying transactions with some minimum fee)
We all expected this from the beginning. The blockchain will eventually be supported by transactions fees as block reward keeps dropping. The only thing that has changed is that miners may respond sooner because of all these spam transactions. Spam will stop once the miners raise the minimum fee enough (or spam transactions will continue but never be confirmed)
|
|
|
|
|