Clipse
|
|
June 17, 2012, 12:29:25 PM |
|
Is there any specific reason that a large number of transactions would specifically cause orphans? Is it just because they are slower to be propagated to the network, or is it just too taxing to current pool softwares and algorithms to deal with? They propagate the network much slower. First, because the larger amount of data takes longer to upload. Then more, because Bitcoin nodes don't relay it until they've verified every transaction in the block. I am not sure I can follow. I understand how large blocks propagate slower through the network, and how this causes higher orphan rate. But how does not accepting *TXs* into your own to be mined blocks improve this scenario, as long as other pools still mine them? -coinft If I understand Luke, he said that when a pool mines a new block and announces it to the network, the block does't get relayed by any node until the node receives all the transactions that are listed inside the block. Is this really true? I would have thought that for my node receiving a transaction inside a block is the same as receiving it outside a block. Moreover, if there is a tx inside a block there shouldn't be anything pending for it - so why would it wait? I believe that is true and considering the size of deepbit its likely why he also decided to limit his block TX long ago and why they never suffered performance issues.
|
...In the land of the stale, the man with one share is king... >> ClipseWe pay miners at 130% PPS | Signup here : Bonus PPS Pool (Please read OP to understand the current process)
|
|
|
piotr_n
Legendary
Offline
Activity: 2053
Merit: 1356
aka tonikt
|
|
June 17, 2012, 12:32:43 PM |
|
Is there any specific reason that a large number of transactions would specifically cause orphans? Is it just because they are slower to be propagated to the network, or is it just too taxing to current pool softwares and algorithms to deal with? They propagate the network much slower. First, because the larger amount of data takes longer to upload. Then more, because Bitcoin nodes don't relay it until they've verified every transaction in the block. I am not sure I can follow. I understand how large blocks propagate slower through the network, and how this causes higher orphan rate. But how does not accepting *TXs* into your own to be mined blocks improve this scenario, as long as other pools still mine them? -coinft If I understand Luke, he said that when a pool mines a new block and announces it to the network, the block does't get relayed by any node until the node receives all the transactions that are listed inside the block. Is this really true? I would have thought that for my node receiving a transaction inside a block is the same as receiving it outside a block. Moreover, if there is a tx inside a block there shouldn't be anything pending for it - so why would it wait? I believe that is true and considering the size of deepbit its likely why he also decided to limit his block TX long ago and why they never suffered performance issues. But it wouldn't make any sense. It would mean that if I restart my client after one week being offline it executes getdata not only for the thousand of blocks, but also (in parallel) for every tx that is inside these blocks... I don't believe bitcoin devs would program it like this - bitcoin is not such a bad piece of code.
|
Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.PGP fingerprint: AB9E A551 E262 A87A 13BB 9059 1BE7 B545 CDF3 FD0E
|
|
|
Luke-Jr (OP)
Legendary
Offline
Activity: 2576
Merit: 1186
|
|
June 17, 2012, 12:42:29 PM |
|
Is there any specific reason that a large number of transactions would specifically cause orphans? Is it just because they are slower to be propagated to the network, or is it just too taxing to current pool softwares and algorithms to deal with? They propagate the network much slower. First, because the larger amount of data takes longer to upload. Then more, because Bitcoin nodes don't relay it until they've verified every transaction in the block. I am not sure I can follow. I understand how large blocks propagate slower through the network, and how this causes higher orphan rate. But how does not accepting *TXs* into your own to be mined blocks improve this scenario, as long as other pools still mine them? -coinft If I understand Luke, he said that when a pool mines a new block and announces it to the network, the block does't get relayed by any node until the node receives all the transactions that are listed inside the block. Is this really true? I would have thought that for my node receiving a transaction inside a block is the same as receiving it outside a block. Moreover, if there is a tx inside a block there shouldn't be anything pending for it - so why would it wait? The block itself contains the full data for each transaction. So if we fill the block to the max, that's a 1 MB block that each node needs to download (from another node, likely not a server with a big fat upload pipe) before it even starts validating it. Then once it's downloaded, it's up to 20,000 ECDSA cryptographic signature verifications before it decides the block is valid and begins relaying/uploading it to other nodes.
|
|
|
|
piotr_n
Legendary
Offline
Activity: 2053
Merit: 1356
aka tonikt
|
|
June 17, 2012, 12:43:49 PM |
|
Is there any specific reason that a large number of transactions would specifically cause orphans? Is it just because they are slower to be propagated to the network, or is it just too taxing to current pool softwares and algorithms to deal with? They propagate the network much slower. First, because the larger amount of data takes longer to upload. Then more, because Bitcoin nodes don't relay it until they've verified every transaction in the block. I am not sure I can follow. I understand how large blocks propagate slower through the network, and how this causes higher orphan rate. But how does not accepting *TXs* into your own to be mined blocks improve this scenario, as long as other pools still mine them? -coinft If I understand Luke, he said that when a pool mines a new block and announces it to the network, the block does't get relayed by any node until the node receives all the transactions that are listed inside the block. Is this really true? I would have thought that for my node receiving a transaction inside a block is the same as receiving it outside a block. Moreover, if there is a tx inside a block there shouldn't be anything pending for it - so why would it wait? The block itself contains the full data for each transaction. So if we fill the block to the max, that's a 1 MB block that each node needs to download (from another node, likely not a server with a big fat upload pipe) before it even starts validating it. Then once it's downloaded, it's up to 20,000 ECDSA cryptographic signature verifications before it decides the block is valid and begins relaying/uploading it to other nodes. Oh ok - now it does make sense.
|
Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.PGP fingerprint: AB9E A551 E262 A87A 13BB 9059 1BE7 B545 CDF3 FD0E
|
|
|
coinft
|
|
June 17, 2012, 01:25:36 PM |
|
Is there any specific reason that a large number of transactions would specifically cause orphans? Is it just because they are slower to be propagated to the network, or is it just too taxing to current pool softwares and algorithms to deal with? They propagate the network much slower. First, because the larger amount of data takes longer to upload. Then more, because Bitcoin nodes don't relay it until they've verified every transaction in the block. I am not sure I can follow. I understand how large blocks propagate slower through the network, and how this causes higher orphan rate. But how does not accepting *TXs* into your own to be mined blocks improve this scenario, as long as other pools still mine them? -coinft If I understand Luke, he said that when a pool mines a new block and announces it to the network, the block does't get relayed by any node until the node receives all the transactions that are listed inside the block. Is this really true? I would have thought that for my node receiving a transaction inside a block is the same as receiving it outside a block. Moreover, if there is a tx inside a block there shouldn't be anything pending for it - so why would it wait? The block itself contains the full data for each transaction. So if we fill the block to the max, that's a 1 MB block that each node needs to download (from another node, likely not a server with a big fat upload pipe) before it even starts validating it. Then once it's downloaded, it's up to 20,000 ECDSA cryptographic signature verifications before it decides the block is valid and begins relaying/uploading it to other nodes. Oh ok - now it does make sense. Yes makes sense. Our own mined blocks are slow too and someone may jump us. The bad news, this looks to be a global scalability problem. Maybe we need smaller and more blocks, but that's very unlikely to change, and has its own problems. Or a tiered network with small nodes not verifying everything -- again with problems.
|
|
|
|
piotr_n
Legendary
Offline
Activity: 2053
Merit: 1356
aka tonikt
|
|
June 17, 2012, 01:39:14 PM |
|
The bad news, this looks to be a global scalability problem. Maybe we need smaller and more blocks, but that's very unlikely to change, and has its own problems. Or a tiered network with small nodes not verifying everything -- again with problems. Before starting to cry, I would actually prefer to see some numbers - i.e. what was the percentage of orphaned blocks 3 months ago, comparing to what it has been for the last few weeks.
|
Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.PGP fingerprint: AB9E A551 E262 A87A 13BB 9059 1BE7 B545 CDF3 FD0E
|
|
|
organofcorti
Donator
Legendary
Offline
Activity: 2058
Merit: 1007
Poor impulse control.
|
|
June 17, 2012, 01:47:12 PM |
|
The bad news, this looks to be a global scalability problem. Maybe we need smaller and more blocks, but that's very unlikely to change, and has its own problems. Or a tiered network with small nodes not verifying everything -- again with problems. Before starting to cry, I would actually prefer to see some numbers - i.e. what was the percentage of orphaned blocks 3 months ago, comparing to what it has been for the last few weeks. Great idea. Eligius' block history is at: http://eligius.st/~artefact2/blocks/The 'Status' column shows whether or not a block was orphaned. See if there's been an increase here, then check other pools, and let us know
|
|
|
|
Luke-Jr (OP)
Legendary
Offline
Activity: 2576
Merit: 1186
|
|
June 17, 2012, 01:51:47 PM |
|
The bad news, this looks to be a global scalability problem. Maybe we need smaller and more blocks, but that's very unlikely to change, and has its own problems. Or a tiered network with small nodes not verifying everything -- again with problems. Before starting to cry, I would actually prefer to see some numbers - i.e. what was the percentage of orphaned blocks 3 months ago, comparing to what it has been for the last few weeks. Great idea. Eligius' block history is at: http://eligius.st/~artefact2/blocks/The 'Status' column shows whether or not a block was orphaned. See if there's been an increase here, then check other pools, and let us know No, it hasn't shown orphans at all for a while. I'm not sure a good way to see this info, considering most orphans are never seen by most of the network at all...
|
|
|
|
DeepBit
Donator
Hero Member
Offline
Activity: 532
Merit: 501
We have cookies
|
|
June 17, 2012, 02:28:07 PM |
|
The 'Status' column shows whether or not a block was orphaned. See if there's been an increase here, then check other pools, and let us know No, it hasn't shown orphans at all for a while. I'm not sure a good way to see this info, considering most orphans are never seen by most of the network at all... You can always check if the block was orphaned by using your block's hash information.
|
Welcome to my bitcoin mining pool: https://deepbit.net ~ 3600 GH/s, Both payment schemes, instant payout, no invalid blocks ! Coming soon: ICBIT Trading platform
|
|
|
muyuu
Donator
Legendary
Offline
Activity: 980
Merit: 1000
|
|
June 17, 2012, 04:14:22 PM |
|
So it was SatoshiDice. I was wondering what had changed to make mining profits so noticeably worse despite difficulty being in previously experienced ranges.
|
GPG ID: 7294199D - OTC ID: muyuu (470F97EB7294199D) forum tea fund BTC 1Epv7KHbNjYzqYVhTCgXWYhGSkv7BuKGEU DOGE DF1eTJ2vsxjHpmmbKu9jpqsrg5uyQLWksM CAP F1MzvmmHwP2UhFq82NQT7qDU9NQ8oQbtkQ
|
|
|
Gavin Andresen
Legendary
Offline
Activity: 1652
Merit: 2300
Chief Scientist
|
|
June 17, 2012, 04:19:58 PM |
|
So it was SatoshiDice. I was wondering what had changed to make mining profits so noticeably worse despite difficulty being in previously experienced ranges.
Luke is guessing it is SatoshiDice, it might just be a run of bad luck. Or it might be a side effect of Eligius accepting non-standard transactions. Does Eligius include transactions that have not been transmitted/relayed to the rest of the network? If so, it might be a side effect of that (and if that isn't a side effect now, it might be in the future if Eligius blocks take longer to verify as other nodes need to fetch transaction inputs from disk, instead of already having them in cache memory like transactions that ARE transmitted/relayed).
|
How often do you get the chance to work on a potentially world-changing project?
|
|
|
organofcorti
Donator
Legendary
Offline
Activity: 2058
Merit: 1007
Poor impulse control.
|
|
June 17, 2012, 04:25:55 PM |
|
At p2Pool, average orphaned blocks per day increased when average confirmation time started to increase. It's a very clear correlation. I haven't looked at other pools yet, though.
Interestingly there's not such a clear correlation between average Tx per block and the increase in orphans produced.
|
|
|
|
check_status
Full Member
Offline
Activity: 196
Merit: 100
Web Dev, Db Admin, Computer Technician
|
|
June 17, 2012, 04:36:00 PM |
|
So it was SatoshiDice. I was wondering what had changed to make mining profits so noticeably worse despite difficulty being in previously experienced ranges.
Luke is guessing it is SatoshiDice, it might just be a run of bad luck. Or it might be a side effect of Eligius accepting non-standard transactions. Does Eligius include transactions that have not been transmitted/relayed to the rest of the network? If so, it might be a side effect of that (and if that isn't a side effect now, it might be in the future if Eligius blocks take longer to verify as other nodes need to fetch transaction inputs from disk, instead of already having them in cache memory like transactions that ARE transmitted/relayed). Is this something that would affect all pools or is this something that could be targeted at specific pools by timing?
|
For Bitcoin to be a true global currency the value of BTC needs always to rise. If BTC became the global currency & money supply = 100 Trillion then ⊅1.00 BTC = $4,761,904.76. P2Pool Server List | How To's and Guides Mega List | 1 EndfedSryGUZK9sPrdvxHntYzv2EBexGA
|
|
|
Luke-Jr (OP)
Legendary
Offline
Activity: 2576
Merit: 1186
|
|
June 17, 2012, 04:50:30 PM |
|
So it was SatoshiDice. I was wondering what had changed to make mining profits so noticeably worse despite difficulty being in previously experienced ranges. Dice is abusing the network, but it isn't the root of the problem; the real problem is that Satoshi's design aimed at making the actual transaction processing have near zero cost has failed, and all the assumptions built on that premise collapse. Luke is guessing it is SatoshiDice, it might just be a run of bad luck.
Or it might be a side effect of Eligius accepting non-standard transactions.
Does Eligius include transactions that have not been transmitted/relayed to the rest of the network? If so, it might be a side effect of that (and if that isn't a side effect now, it might be in the future if Eligius blocks take longer to verify as other nodes need to fetch transaction inputs from disk, instead of already having them in cache memory like transactions that ARE transmitted/relayed). Bad luck is a normal part of mining, and almost certainly has its role at Eligius recently; but it's also certain that isn't the only thing going on. I did indeed check to be sure nobody was abusing non-standard transactions. Interestingly there's not such a clear correlation between average Tx per block and the increase in orphans produced. There's no way to get enough orphan data for good statistics. Just reading through the ones blockchain.info has seen, though, suggests a clear bias toward smaller blocks IMO.
|
|
|
|
piotr_n
Legendary
Offline
Activity: 2053
Merit: 1356
aka tonikt
|
|
June 17, 2012, 05:26:01 PM |
|
Interestingly there's not such a clear correlation between average Tx per block and the increase in orphans produced. There's no way to get enough orphan data for good statistics. Just reading through the ones blockchain.info has seen, though, suggests a clear bias toward smaller blocks IMO. So are you saying that Eligius doesn't have a list of all the blocks (or their hashes) we mined in the pool, regardless of them getting orphaned or not? It's kind of hard to believe... it sounds like Bitcoinica backups being stored in the same cloud as the server itself... Anyway, if you do have such a list, then please just publish it and I promise to draw a graph - checking which of them got orphaned and how the percentage has changed in time. Only then we can see if something really got screwed up in the network - or is it only a bad luck. Only numbers don't lie
|
Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.PGP fingerprint: AB9E A551 E262 A87A 13BB 9059 1BE7 B545 CDF3 FD0E
|
|
|
Luke-Jr (OP)
Legendary
Offline
Activity: 2576
Merit: 1186
|
|
June 17, 2012, 05:30:29 PM |
|
Interestingly there's not such a clear correlation between average Tx per block and the increase in orphans produced. There's no way to get enough orphan data for good statistics. Just reading through the ones blockchain.info has seen, though, suggests a clear bias toward smaller blocks IMO. So are you saying that Eligius doesn't have a list of all the blocks (or their hashes) we mined in the pool, regardless of them getting orphaned or not? Eligius-Ra's bitcoind in theory contains that data, but I'm not aware of any way to get it. If the coinbaser completed on the new block (which it almost always should) there should be a file in the JSON API directory. But even with the high orphanage rate we've been getting, Eligius doesn't have enough orphans to be statistically significant. Anyway, if you do have such a list, then please just publish it and I promise to draw a graph - checking which of them got orphaned and how the percentage has changed in time. Only then we can see if something really got screwed up in the network - or is it only a bad luck. Try looking through http://eligius.st/~luke-jr/raw/7/blocks/
|
|
|
|
piotr_n
Legendary
Offline
Activity: 2053
Merit: 1356
aka tonikt
|
|
June 17, 2012, 06:57:59 PM Last edit: June 17, 2012, 07:09:17 PM by piotr_n |
|
If all the mined blocks are there - then you must be right like hell. http://pastebin.com/mb14R9nR0% of orphaned blocks in January 1.0% in February 1.4% in March 0% in April 2.3% in May (though first one on the 13th) 12.7% in June
|
Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.PGP fingerprint: AB9E A551 E262 A87A 13BB 9059 1BE7 B545 CDF3 FD0E
|
|
|
Clipse
|
|
June 17, 2012, 07:11:49 PM |
|
If all the mined blocks are there - then you must be right like hell. http://pastebin.com/mb14R9nR0% of orphaned blocks in January 1.0% in February 1.4% in March 0% in April 2.3% in May (though first one on the 13th) 12.7% in June Jeez, effectively in June alone all miners would be -12.7% gauranteed on expected returns. That is definitely good enough to point to recent bloating of blockchain thats affecting other pools aswell.
|
...In the land of the stale, the man with one share is king... >> ClipseWe pay miners at 130% PPS | Signup here : Bonus PPS Pool (Please read OP to understand the current process)
|
|
|
piotr_n
Legendary
Offline
Activity: 2053
Merit: 1356
aka tonikt
|
|
June 17, 2012, 07:14:34 PM |
|
If all the mined blocks are there - then you must be right like hell. http://pastebin.com/mb14R9nR0% of orphaned blocks in January 1.0% in February 1.4% in March 0% in April 2.3% in May (though first one on the 13th) 12.7% in June Jeez, effectively in June alone all miners would be -12.7% gauranteed on expected returns. That is definitely good enough to point to recent bloating of blockchain thats affecting other pools aswell. Yeap. Now I understand why the difficulty is going up so slowly, despite of all the recent FPGA investments all around the world...
|
Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.PGP fingerprint: AB9E A551 E262 A87A 13BB 9059 1BE7 B545 CDF3 FD0E
|
|
|
bulanula
|
|
June 17, 2012, 08:03:57 PM |
|
If all the mined blocks are there - then you must be right like hell. http://pastebin.com/mb14R9nR0% of orphaned blocks in January 1.0% in February 1.4% in March 0% in April 2.3% in May (though first one on the 13th) 12.7% in June Jeez, effectively in June alone all miners would be -12.7% gauranteed on expected returns. That is definitely good enough to point to recent bloating of blockchain thats affecting other pools aswell. Yeap. Now I understand why the difficulty is going up so slowly, despite of all the recent FPGA investments all around the world... Are you guys basically saying that because of evorhees and his shitty gambling service all miners are getting 10% less income for some strange reason ? Screw SatoshiDice ! Someone better stop DDOSing the damn pools and start DDOSing the service ! ( I don't condone DDOS but I need my damn INCOME SatoshiVice )
|
|
|
|
|