[OLD] Eligius: ASIC, no registration, no fee CPPSRB BTC + 105% PPS NMC, 877 #

[Clipse]

Is there any specific reason that a large number of transactions would specifically cause orphans? Is it just because they are slower to be propagated to the network, or is it just too taxing to current pool softwares and algorithms to deal with?

They propagate the network much slower. First, because the larger amount of data takes longer to upload. Then more, because Bitcoin nodes don't relay it until they've verified every transaction in the block.

I am not sure I can follow. I understand how large blocks propagate slower through the network, and how this causes higher orphan rate. But how does not accepting *TXs* into your own to be mined blocks improve this scenario, as long as other pools still mine them?

-coinft

If I understand Luke, he said that when a pool mines a new block and announces it to the network, the block does't get relayed by any node until the node receives all the transactions that are listed inside the block.
Is this really true?
I would have thought that for my node receiving a transaction inside a block is the same as receiving it outside a block. Moreover, if there is a tx inside a block there shouldn't be anything pending for it - so why would it wait?

I believe that is true and considering the size of deepbit its likely why he also decided to limit his block TX long ago and why they never suffered performance issues.

[piotr_n]

Is there any specific reason that a large number of transactions would specifically cause orphans? Is it just because they are slower to be propagated to the network, or is it just too taxing to current pool softwares and algorithms to deal with?

They propagate the network much slower. First, because the larger amount of data takes longer to upload. Then more, because Bitcoin nodes don't relay it until they've verified every transaction in the block.

I am not sure I can follow. I understand how large blocks propagate slower through the network, and how this causes higher orphan rate. But how does not accepting *TXs* into your own to be mined blocks improve this scenario, as long as other pools still mine them?

-coinft

If I understand Luke, he said that when a pool mines a new block and announces it to the network, the block does't get relayed by any node until the node receives all the transactions that are listed inside the block.
Is this really true?
I would have thought that for my node receiving a transaction inside a block is the same as receiving it outside a block. Moreover, if there is a tx inside a block there shouldn't be anything pending for it - so why would it wait?

I believe that is true and considering the size of deepbit its likely why he also decided to limit his block TX long ago and why they never suffered performance issues.

But it wouldn't make any sense.
It would mean that if I restart my client after one week being offline it executes getdata not only for the thousand of blocks, but also (in parallel) for every tx that is inside these blocks...
I don't believe bitcoin devs would program it like this - bitcoin is not such a bad piece of code.

[Luke-Jr]

Is there any specific reason that a large number of transactions would specifically cause orphans? Is it just because they are slower to be propagated to the network, or is it just too taxing to current pool softwares and algorithms to deal with?

They propagate the network much slower. First, because the larger amount of data takes longer to upload. Then more, because Bitcoin nodes don't relay it until they've verified every transaction in the block.

I am not sure I can follow. I understand how large blocks propagate slower through the network, and how this causes higher orphan rate. But how does not accepting *TXs* into your own to be mined blocks improve this scenario, as long as other pools still mine them?

-coinft

If I understand Luke, he said that when a pool mines a new block and announces it to the network, the block does't get relayed by any node until the node receives all the transactions that are listed inside the block.
Is this really true?
I would have thought that for my node receiving a transaction inside a block is the same as receiving it outside a block. Moreover, if there is a tx inside a block there shouldn't be anything pending for it - so why would it wait?

The block itself contains the full data for each transaction. So if we fill the block to the max, that's a 1 MB block that each node needs to download (from another node, likely not a server with a big fat upload pipe) before it even starts validating it. Then once it's downloaded, it's up to 20,000 ECDSA cryptographic signature verifications before it decides the block is valid and begins relaying/uploading it to other nodes.

[piotr_n]

Is there any specific reason that a large number of transactions would specifically cause orphans? Is it just because they are slower to be propagated to the network, or is it just too taxing to current pool softwares and algorithms to deal with?

They propagate the network much slower. First, because the larger amount of data takes longer to upload. Then more, because Bitcoin nodes don't relay it until they've verified every transaction in the block.

I am not sure I can follow. I understand how large blocks propagate slower through the network, and how this causes higher orphan rate. But how does not accepting *TXs* into your own to be mined blocks improve this scenario, as long as other pools still mine them?

-coinft

If I understand Luke, he said that when a pool mines a new block and announces it to the network, the block does't get relayed by any node until the node receives all the transactions that are listed inside the block.
Is this really true?
I would have thought that for my node receiving a transaction inside a block is the same as receiving it outside a block. Moreover, if there is a tx inside a block there shouldn't be anything pending for it - so why would it wait?

The block itself contains the full data for each transaction. So if we fill the block to the max, that's a 1 MB block that each node needs to download (from another node, likely not a server with a big fat upload pipe) before it even starts validating it. Then once it's downloaded, it's up to 20,000 ECDSA cryptographic signature verifications before it decides the block is valid and begins relaying/uploading it to other nodes.

Oh ok - now it does make sense.

[coinft]

Is there any specific reason that a large number of transactions would specifically cause orphans? Is it just because they are slower to be propagated to the network, or is it just too taxing to current pool softwares and algorithms to deal with?

They propagate the network much slower. First, because the larger amount of data takes longer to upload. Then more, because Bitcoin nodes don't relay it until they've verified every transaction in the block.

I am not sure I can follow. I understand how large blocks propagate slower through the network, and how this causes higher orphan rate. But how does not accepting *TXs* into your own to be mined blocks improve this scenario, as long as other pools still mine them?

-coinft

If I understand Luke, he said that when a pool mines a new block and announces it to the network, the block does't get relayed by any node until the node receives all the transactions that are listed inside the block.
Is this really true?
I would have thought that for my node receiving a transaction inside a block is the same as receiving it outside a block. Moreover, if there is a tx inside a block there shouldn't be anything pending for it - so why would it wait?

The block itself contains the full data for each transaction. So if we fill the block to the max, that's a 1 MB block that each node needs to download (from another node, likely not a server with a big fat upload pipe) before it even starts validating it. Then once it's downloaded, it's up to 20,000 ECDSA cryptographic signature verifications before it decides the block is valid and begins relaying/uploading it to other nodes.

Oh ok - now it does make sense.

Yes makes sense. Our own mined blocks are slow too and someone may jump us.

The bad news, this looks to be a global scalability problem. Maybe we need smaller and more blocks, but that's very unlikely to change, and has its own problems. Or a tiered network with small nodes not verifying everything -- again with problems.

[piotr_n]

The bad news, this looks to be a global scalability problem. Maybe we need smaller and more blocks, but that's very unlikely to change, and has its own problems. Or a tiered network with small nodes not verifying everything -- again with problems.

Before starting to cry, I would actually prefer to see some numbers - i.e. what was the percentage of orphaned blocks 3 months ago, comparing to what it has been for the last few weeks.

[organofcorti]

The bad news, this looks to be a global scalability problem. Maybe we need smaller and more blocks, but that's very unlikely to change, and has its own problems. Or a tiered network with small nodes not verifying everything -- again with problems.

Before starting to cry, I would actually prefer to see some numbers - i.e. what was the percentage of orphaned blocks 3 months ago, comparing to what it has been for the last few weeks.

Great idea.

Eligius' block history is at: http://eligius.st/~artefact2/blocks/

The 'Status' column shows whether or not a block was orphaned. See if there's been an increase here, then check other pools, and let us know

[Luke-Jr]

The bad news, this looks to be a global scalability problem. Maybe we need smaller and more blocks, but that's very unlikely to change, and has its own problems. Or a tiered network with small nodes not verifying everything -- again with problems.

Before starting to cry, I would actually prefer to see some numbers - i.e. what was the percentage of orphaned blocks 3 months ago, comparing to what it has been for the last few weeks.

Great idea.

Eligius' block history is at: http://eligius.st/~artefact2/blocks/

The 'Status' column shows whether or not a block was orphaned. See if there's been an increase here, then check other pools, and let us know

No, it hasn't shown orphans at all for a while. I'm not sure a good way to see this info, considering most orphans are never seen by most of the network at all...

[DeepBit]

The 'Status' column shows whether or not a block was orphaned. See if there's been an increase here, then check other pools, and let us know

No, it hasn't shown orphans at all for a while. I'm not sure a good way to see this info, considering most orphans are never seen by most of the network at all...

You can always check if the block was orphaned by using your block's hash information.

[muyuu]

So it was SatoshiDice. I was wondering what had changed to make mining profits so noticeably worse despite difficulty being in previously experienced ranges.

[Gavin Andresen]

So it was SatoshiDice. I was wondering what had changed to make mining profits so noticeably worse despite difficulty being in previously experienced ranges.

Luke is guessing it is SatoshiDice, it might just be a run of bad luck.

Or it might be a side effect of Eligius accepting non-standard transactions.

Does Eligius include transactions that have not been transmitted/relayed to the rest of the network?  If so, it might be a side effect of that (and if that isn't a side effect now, it might be in the future if Eligius blocks take longer to verify as other nodes need to fetch transaction inputs from disk, instead of already having them in cache memory like transactions that ARE transmitted/relayed).

[organofcorti]

At p2Pool, average orphaned blocks per day increased when average confirmation time started to increase. It's a very clear correlation. I haven't looked at other pools yet, though.

Interestingly there's not such a clear correlation between average Tx per block and the increase in orphans produced.

[check_status]

So it was SatoshiDice. I was wondering what had changed to make mining profits so noticeably worse despite difficulty being in previously experienced ranges.

Luke is guessing it is SatoshiDice, it might just be a run of bad luck.

Or it might be a side effect of Eligius accepting non-standard transactions.

Does Eligius include transactions that have not been transmitted/relayed to the rest of the network?  If so, it might be a side effect of that (and if that isn't a side effect now, it might be in the future if Eligius blocks take longer to verify as other nodes need to fetch transaction inputs from disk, instead of already having them in cache memory like transactions that ARE transmitted/relayed).

Is this something that would affect all pools or is this something that could be targeted at specific pools by timing?

[Luke-Jr]

So it was SatoshiDice. I was wondering what had changed to make mining profits so noticeably worse despite difficulty being in previously experienced ranges.

Dice is abusing the network, but it isn't the root of the problem; the real problem is that Satoshi's design aimed at making the actual transaction processing have near zero cost has failed, and all the assumptions built on that premise collapse.

Luke is guessing it is SatoshiDice, it might just be a run of bad luck.

Or it might be a side effect of Eligius accepting non-standard transactions.

Does Eligius include transactions that have not been transmitted/relayed to the rest of the network?  If so, it might be a side effect of that (and if that isn't a side effect now, it might be in the future if Eligius blocks take longer to verify as other nodes need to fetch transaction inputs from disk, instead of already having them in cache memory like transactions that ARE transmitted/relayed).

Bad luck is a normal part of mining, and almost certainly has its role at Eligius recently; but it's also certain that isn't the only thing going on. I did indeed check to be sure nobody was abusing non-standard transactions.

Interestingly there's not such a clear correlation between average Tx per block and the increase in orphans produced.

There's no way to get enough orphan data for good statistics. Just reading through the ones blockchain.info has seen, though, suggests a clear bias toward smaller blocks IMO.

[piotr_n]

Interestingly there's not such a clear correlation between average Tx per block and the increase in orphans produced.

There's no way to get enough orphan data for good statistics. Just reading through the ones blockchain.info has seen, though, suggests a clear bias toward smaller blocks IMO.

So are you saying that Eligius doesn't have a list of all the blocks (or their hashes) we mined in the pool, regardless of them getting orphaned or not?
It's kind of hard to believe... it sounds like Bitcoinica backups being stored in the same cloud as the server itself...

Anyway, if you do have such a list, then please just publish it and I promise to draw a graph - checking which of them got orphaned and how the percentage has changed in time.
Only then we can see if something really got screwed up in the network - or is it only a bad luck. Only numbers don't lie

[Luke-Jr]

Interestingly there's not such a clear correlation between average Tx per block and the increase in orphans produced.

There's no way to get enough orphan data for good statistics. Just reading through the ones blockchain.info has seen, though, suggests a clear bias toward smaller blocks IMO.

So are you saying that Eligius doesn't have a list of all the blocks (or their hashes) we mined in the pool, regardless of them getting orphaned or not?

Eligius-Ra's bitcoind in theory contains that data, but I'm not aware of any way to get it. If the coinbaser completed on the new block (which it almost always should) there should be a file in the JSON API directory. But even with the high orphanage rate we've been getting, Eligius doesn't have enough orphans to be statistically significant.

Anyway, if you do have such a list, then please just publish it and I promise to draw a graph - checking which of them got orphaned and how the percentage has changed in time.
Only then we can see if something really got screwed up in the network - or is it only a bad luck.

Try looking through http://eligius.st/~luke-jr/raw/7/blocks/

[piotr_n]

Try looking through http://eligius.st/~luke-jr/raw/7/blocks/

If all the mined blocks are there - then you must be right like hell.
http://pastebin.com/mb14R9nR

Code:

0% of orphaned blocks in January
1.0% in February
1.4% in March
0% in April
2.3% in May (though first one on the 13th)
12.7% in June

[Clipse]

Try looking through http://eligius.st/~luke-jr/raw/7/blocks/

If all the mined blocks are there - then you must be right like hell.
http://pastebin.com/mb14R9nR

Code:

0% of orphaned blocks in January
1.0% in February
1.4% in March
0% in April
2.3% in May (though first one on the 13th)
12.7% in June

Jeez, effectively in June alone all miners would be -12.7% gauranteed on expected returns. That is definitely good enough to point to recent bloating of blockchain thats affecting other pools aswell.

[piotr_n]

Try looking through http://eligius.st/~luke-jr/raw/7/blocks/

If all the mined blocks are there - then you must be right like hell.
http://pastebin.com/mb14R9nR

Code:

0% of orphaned blocks in January
1.0% in February
1.4% in March
0% in April
2.3% in May (though first one on the 13th)
12.7% in June

Jeez, effectively in June alone all miners would be -12.7% gauranteed on expected returns. That is definitely good enough to point to recent bloating of blockchain thats affecting other pools aswell.

Yeap. Now I understand why the difficulty is going up so slowly, despite of all the recent FPGA investments all around the world...

[bulanula]

Try looking through http://eligius.st/~luke-jr/raw/7/blocks/

If all the mined blocks are there - then you must be right like hell.
http://pastebin.com/mb14R9nR

Code:

0% of orphaned blocks in January
1.0% in February
1.4% in March
0% in April
2.3% in May (though first one on the 13th)
12.7% in June

Jeez, effectively in June alone all miners would be -12.7% gauranteed on expected returns. That is definitely good enough to point to recent bloating of blockchain thats affecting other pools aswell.

Yeap. Now I understand why the difficulty is going up so slowly, despite of all the recent FPGA investments all around the world...

Are you guys basically saying that because of evorhees and his shitty gambling service all miners are getting 10% less income for some strange reason ?

Screw SatoshiDice ! Someone better stop DDOSing the damn pools and start DDOSing the service ! ( I don't condone DDOS but I need my damn INCOME SatoshiVice )