Bitcoin Forum
December 10, 2016, 01:04:27 PM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: [1] 2 3 4 5 »  All
  Print  
Author Topic: High orphan rate and long confirmation time discussion  (Read 8770 times)
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2002


Ruu \o/


View Profile WWW
June 18, 2012, 09:48:31 AM
 #1

So it appears pretty clear that most pools are suffering from a significant rise in orphan rates and many pool owners believe the issue appears to be related to slow block solve propagation due to the massive increase in transaction volume (mostly due to satoshidice).  Now it's clear that while only a small number of us are gamblers and care about satoshidice's success, the fact is most users of sd are obeying the transaction fees as determined by bitcoin (which are rather small now, but still present). So we're hitting some scalability issue with the network sooner than imagined. Some say that bitcoin was never meant to be a system for micro-transactions but that actually appears completely wrong to me if we're to assume widespread acceptance of btc.

I've seen massively delayed longpolls from various pools now and some just accept the shares long after the block has changed before they're sending out the longpoll, while other pools have a deathly pause, not returning a response about shares for up to 25 seconds, then sending out a longpoll and 25 seconds worth of rejected work. Some say that deepbit's transaction limits means they're avoiding this problem, but I'm not informed enough to know if this is true, but it would seem to me this must incur significant transaction delays. Some solution needs to be worked on that benefits all - pools, miners, and bitcoin at large. I don't want to see transactions delayed for up to 6 hours when even transaction fees are paid, but nor do I want my pools to become a massive orphanage.

Please discuss.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
Pooled mine at kano.is, solo mine at solo.ckpool.org
-ck
1481375067
Hero Member
*
Offline Offline

Posts: 1481375067

View Profile Personal Message (Offline)

Ignore
1481375067
Reply with quote  #2

1481375067
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481375067
Hero Member
*
Offline Offline

Posts: 1481375067

View Profile Personal Message (Offline)

Ignore
1481375067
Reply with quote  #2

1481375067
Report to moderator
1481375067
Hero Member
*
Offline Offline

Posts: 1481375067

View Profile Personal Message (Offline)

Ignore
1481375067
Reply with quote  #2

1481375067
Report to moderator
Graet
VIP
Legendary
*
Offline Offline

Activity: 980



View Profile WWW
June 18, 2012, 09:53:12 AM
 #2

Well said.

We are working on this issue for Ozcoin today, if it is suitable and works we will share our solution, I know others are trying different things too Smiley

| Ozcoin Pooled Mining Pty Ltd https://ozcoin.net Double Geometric Reward System https://lc.ozcoin.net for Litecoin mining DGM| https://crowncloud.net VPS and Dedicated Servers for the BTC community
Shadow383
Sr. Member
****
Offline Offline

Activity: 336


View Profile
June 18, 2012, 09:54:49 AM
 #3

It is a major issue - if you look at the sort of transaction volume most stock exchanges or major payment processors get, it seems doubtful that the network as it stands could cope.
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1960


Poor impulse control.


View Profile WWW
June 18, 2012, 10:40:00 AM
 #4

This is a very interesting question and I'm glad you made a thread for it. From the little investigation I've done it seems that there's been an increase in orphans since about March, which coincides with a sudden increase in confirmation times. The strange thing about this is the sudden increase in confirmation times does not coincide with the increase in transactions per block which started to increase from May, two months later.

This can be seen here:

http://blockchain.info/charts/avg-confirmation-time
http://blockchain.info/charts/n-transactions-per-block

So the increase in tx per block does not seem to have caused the increase in confirmation times. Satoshidice doesn't look like the culprit to me.

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2002


Ruu \o/


View Profile WWW
June 18, 2012, 10:42:07 AM
 #5

This is a very interesting question and I'm glad you made a thread for it. From the little investigation I've done it seems that there's been an increase in orphans since about March, which coincides with a sudden increase in confirmation times. The strange thing about this is the sudden increase in confirmation times does not coincide with the increase in transactions per block which started to increase from May, two months later.

This can be seen here:

http://blockchain.info/charts/avg-confirmation-time
http://blockchain.info/charts/n-transactions-per-block

So the increase in tx per block does not seem to have caused the increase in confirmation times. Satoshidice doesn't look like the culprit to me.
Interesting... is this BIP16 then?

Primary developer/maintainer for cgminer and ckpool/ckproxy.
Pooled mine at kano.is, solo mine at solo.ckpool.org
-ck
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1960


Poor impulse control.


View Profile WWW
June 18, 2012, 10:44:56 AM
 #6

Interesting... is this BIP16 then?

Don't think so - BIP16 was introduced 1st April, wasn't it? Increased confirmation times began in March. Can you think of anything else that changed then?

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2002


Ruu \o/


View Profile WWW
June 18, 2012, 10:50:56 AM
 #7

Interesting... is this BIP16 then?

Don't think so - BIP16 was introduced 1st April, wasn't it? Increased confirmation times began in March. Can you think of anything else that changed then?
It was accepted April 1, but there were pools that had already adopted it if I recall correctly.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
Pooled mine at kano.is, solo mine at solo.ckpool.org
-ck
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1960


Poor impulse control.


View Profile WWW
June 18, 2012, 11:02:45 AM
 #8

Interesting... is this BIP16 then?

Don't think so - BIP16 was introduced 1st April, wasn't it? Increased confirmation times began in March. Can you think of anything else that changed then?
It was accepted April 1, but there were pools that had already adopted it if I recall correctly.

Do you know of any way to find out when pools started to apply BIP16, without having to trawl the threads? I could look at every pool and try to correlate orphan rate with BIP16 take up.

Why would BIP16 cause increased confirmation times?

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2002


Ruu \o/


View Profile WWW
June 18, 2012, 11:06:11 AM
 #9

Do you know of any way to find out when pools started to apply BIP16, without having to trawl the threads? I could look at every pool and try to correlate orphan rate with BIP16 take up.

Why would BIP16 cause increased confirmation times?
I have no idea but it is the one most significant change to bitcoin that correlates very strongly with that time frame, unlike satoshidice's increase in transaction volume. Is BIP16 the same time we started accepting smaller transaction fees? They're allegedly 1/20th of what they were, but I don't know when that change hit. Maybe they really did need to be bigger?

Primary developer/maintainer for cgminer and ckpool/ckproxy.
Pooled mine at kano.is, solo mine at solo.ckpool.org
-ck
doublec
Legendary
*
Offline Offline

Activity: 1078


View Profile
June 18, 2012, 11:06:27 AM
 #10

Why would BIP16 cause increased confirmation times?
IIRC a bitcoin release candidate was used by some pools that had a BIP 16 switch on time 2 weeks earlier than what the official time eventually became. Pools and individual miners using this RC would have produced a lot of orphans until they noticed. This could cause increased confirmation times for a brief period when a chunk of the network was effectively offline. I don't think this was very long, nor was it a big chunk.
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2002


Ruu \o/


View Profile WWW
June 18, 2012, 11:09:49 AM
 #11

Why would BIP16 cause increased confirmation times?
IIRC a bitcoin release candidate was used by some pools that had a BIP 16 switch on time 2 weeks earlier than what the official time eventually became. Pools and individual miners using this RC would have produced a lot of orphans until they noticed. This could cause increased confirmation times for a brief period when a chunk of the network was effectively offline. I don't think this was very long, nor was it a big chunk.
Interesting, but the increased confirmation times and orphans persisted, so perhaps it's something about the protocol itself rather than it interacting with the non-BIP16 world? The timing of 2 weeks before correlates almost exactly with the graph showing increased transaction time.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
Pooled mine at kano.is, solo mine at solo.ckpool.org
-ck
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2002


Ruu \o/


View Profile WWW
June 18, 2012, 11:13:34 AM
 #12

The number of BIP16 transactions in the blockchain is very small.
I doubt BIP16 is the root cause.
I'm not implying it's actual BIP16 transactions that are responsible, but whatever code changes or protocol changes were required to support them since the correlation with rising transaction time is surprisingly strong.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
Pooled mine at kano.is, solo mine at solo.ckpool.org
-ck
doublec
Legendary
*
Offline Offline

Activity: 1078


View Profile
June 18, 2012, 11:14:16 AM
 #13

Interesting, but the increased confirmation times and orphans persisted, so perhaps it's something about the protocol itself rather than it interacting with the non-BIP16 world? The timing of 2 weeks before correlates almost exactly with the graph showing increased transaction time.
After the two weeks you get the miners who hadn't switched to the BIP 16 enabled code getting orphans. Maybe there's a bunch of standalone miners running non-upgraded code that are unattended (remains of botnets, unattended servers, etc?)
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1960


Poor impulse control.


View Profile WWW
June 18, 2012, 11:31:21 AM
 #14

Edit: as gmaxwell suggested I contacted Blockchain.info and indeed they only started recording Avg tx conformation time In February. That being the case, it doesn't seem that there's a sudden large increase after all.


Just to make it clear what we're talking about:





In case anyone can think of anything that occurred in conjunction with the increase in tx conf times, the start of the current tx conf times is 2012-02-26, with previous spikes on 2012-02-06 and 2012-02-15 to 2012-02-16.

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1960


Poor impulse control.


View Profile WWW
June 18, 2012, 11:53:19 AM
 #15

Is BIP16 the same time we started accepting smaller transaction fees? They're allegedly 1/20th of what they were, but I don't know when that change hit. Maybe they really did need to be bigger?

Smaller tx fees started with Bitcoin vers 0.3.23, which was available from June 2011.

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
Syke
Legendary
*
Offline Offline

Activity: 2100


View Profile
June 18, 2012, 12:01:19 PM
 #16

I'm looking over some of the orphan races at:

http://blockchain.info/orphaned-blocks

Races where 2 blocks are found with timestamps a few seconds apart is to be expected. But I'm seeing quite a few races where block are minutes apart. If it's taking blocks minutes to propate across the network, we've got a problem.

Like here's a particularly nasty one.

http://blockchain.info/block-index/232898/0000000000000873fda1001c4986936dfa3697ec84ff74e48c11fe9d355be07f

vs.

http://blockchain.info/block-index/232896/000000000000069c27242a928e33e45ddd27d6fd014c77946ba4d3d2891f5398

Some poor miner finds a block, and 30 minutes later DeepBit still hasn't received that block and thus orphans it when they find a block. 30 minutes!

Buy & Hold
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2002


Ruu \o/


View Profile WWW
June 18, 2012, 12:04:39 PM
 #17

I'm looking over some of the orphan races at:

http://blockchain.info/orphaned-blocks

Races where 2 blocks are found with timestamps a few seconds apart is to be expected. But I'm seeing quite a few races where block are minutes apart. If it's taking blocks minutes to propate across the network, we've got a problem.

Like here's a particularly nasty one.

http://blockchain.info/block-index/232898/0000000000000873fda1001c4986936dfa3697ec84ff74e48c11fe9d355be07f

vs.

http://blockchain.info/block-index/232896/000000000000069c27242a928e33e45ddd27d6fd014c77946ba4d3d2891f5398

Some poor miner finds a block, and 30 minutes later DeepBit still hasn't received that block and thus orphans it when they find a block. 30 minutes!

Bear in mind that block generation times can be not remotely accurate with rolltime enabled. cgminer limits rolling to about 10 minutes, so the time could be 10 minutes out from the real local time, but apparently luke-jr has removed all limits on his rolltime on eligius meaning work from there could appear to be from any time in the future.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
Pooled mine at kano.is, solo mine at solo.ckpool.org
-ck
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1960


Poor impulse control.


View Profile WWW
June 18, 2012, 01:02:58 PM
 #18

Full history of orphaned blocks at p2Pool, ordered by numbers of blocks solved:



Although it looks like this could just be because more blocks are being solved, there's no obvious correlation between orphans and hashrate:



I'd like to get data from a much more established pool though.

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218


Gerald Davis


View Profile
June 18, 2012, 01:35:01 PM
 #19

Just to make it clear what we're talking about:

(snip)

Thanks for the charts it illustrates the symptoms clearly.  That sharp rise in avg confirmation time can't be explained by increased tx volume.
Luke-Jr
Legendary
*
Offline Offline

Activity: 2100



View Profile
June 18, 2012, 01:39:13 PM
 #20

Just to make it clear what we're talking about:

(snip)

Thanks for the charts it illustrates the symptoms clearly.  That sharp rise in avg confirmation time can't be explained by increased tx volume.
The avg confirmation time isn't the (immediate) problem. The problem is block propagation time. If that isn't solved, you will likely see another sharp rise in avg confirmation time soon.

Pages: [1] 2 3 4 5 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!