Bitcoin Forum
July 09, 2020, 02:33:33 PM *
News: Latest Bitcoin Core release: 0.20.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: « 1 ... 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 [107] 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 ... 814 »
  Print  
Author Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool  (Read 2587116 times)
kano
Legendary
*
Offline Offline

Activity: 3192
Merit: 1284


Linux since 1997 RedHat 4


View Profile
April 28, 2012, 02:18:48 PM
 #2121

...
If a share is DOA or orphan it will still count as a real block. If you look at the source in main.got_response(header, request) the first thing it does is check if it is a real block. Only after that does it bother to check for DOA. CGminer will submit dead shares if you use --submit_stale, or if you are running a fairly recent version it will do this automatically. If you are using an old version of cgminer, and not using --submit_stale, then you could discard a valid block before it even reaches P2Pool.
So, you can confirm that a 10 second share that is effectively late (stale/rejected) will still make it all the way to bitcoind if it is a valid block?
So P2Pool already has this 2 way path?
A share that isn't valid for the share chain will still pay out to everyone and go to bitcoind if it is a block?
(or the share chain can also store invalid shares?)

If forrestv will confirm this - then that's good.

Pool: https://kano.is - lowest fee PPLNS 3 Days Here on Bitcointalk: Forum
Discord support invite at https://kano.is/ Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
1594305213
Hero Member
*
Offline Offline

Posts: 1594305213

View Profile Personal Message (Offline)

Ignore
1594305213
Reply with quote  #2

1594305213
Report to moderator
1594305213
Hero Member
*
Offline Offline

Posts: 1594305213

View Profile Personal Message (Offline)

Ignore
1594305213
Reply with quote  #2

1594305213
Report to moderator
1594305213
Hero Member
*
Offline Offline

Posts: 1594305213

View Profile Personal Message (Offline)

Ignore
1594305213
Reply with quote  #2

1594305213
Report to moderator
The forum was founded in 2009 by Satoshi and Sirius. It replaced a SourceForge forum.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1594305213
Hero Member
*
Offline Offline

Posts: 1594305213

View Profile Personal Message (Offline)

Ignore
1594305213
Reply with quote  #2

1594305213
Report to moderator
1594305213
Hero Member
*
Offline Offline

Posts: 1594305213

View Profile Personal Message (Offline)

Ignore
1594305213
Reply with quote  #2

1594305213
Report to moderator
1594305213
Hero Member
*
Offline Offline

Posts: 1594305213

View Profile Personal Message (Offline)

Ignore
1594305213
Reply with quote  #2

1594305213
Report to moderator
twmz
Hero Member
*****
Offline Offline

Activity: 737
Merit: 500



View Profile
April 28, 2012, 03:13:00 PM
 #2122

...
If a share is DOA or orphan it will still count as a real block. If you look at the source in main.got_response(header, request) the first thing it does is check if it is a real block. Only after that does it bother to check for DOA. CGminer will submit dead shares if you use --submit_stale, or if you are running a fairly recent version it will do this automatically. If you are using an old version of cgminer, and not using --submit_stale, then you could discard a valid block before it even reaches P2Pool.
So, you can confirm that a 10 second share that is effectively late (stale/rejected) will still make it all the way to bitcoind if it is a valid block?
So P2Pool already has this 2 way path?
A share that isn't valid for the share chain will still pay out to everyone and go to bitcoind if it is a block?
(or the share chain can also store invalid shares?)

If forrestv will confirm this - then that's good.


Here is the explaination forrest has given in IRC in the past...  If a share is stale/doa but still meets the current bitcoin difficulty requirements (aka it is a block), it is sent to the local p2pool node's bitcoind.  It is also sent to all of the node's peers.  Each of the peers also send the block to their own local bitcoind instances (in order to increase the speed of propagation through the bitcoin network).  However, each of the peers does not relay the stale share to any of their peers.

So it propagates partially through the p2pool p2p network and should be sent to at least 11 bitcoin nodes (1 bitcoin node from the finder of the block and the bitcoin nodes of the block finder's 10 outgoing p2pool connections).  If the block finder has any incoming p2pool connections, it will go to those p2pool peers as well.

A side effect of the fact that the stale share is not fully propagated through the p2pool network is that stale share/valid blocks are sometimes not announced on IRC because the IRC bots sometimes never hear about the stale share (because they are not a direct peer of the block finder).  We have seen this side effect several times (p2pool blocks have been found and not announced in IRC by any of the half dozen or so p2pool nodes that are connected to IRC and announcing blocks).


Was I helpful?  1TwmzX1wBxNF2qtAJRhdKmi2WyLZ5VHRs
WoT, GPG

Bitrated user: ewal.
Smoovious
Hero Member
*****
Offline Offline

Activity: 504
Merit: 500

Scattering my bits around the net since 1980


View Profile
April 28, 2012, 08:14:33 PM
 #2123

is it possible to show what version of p2pool the node is running, or is that not available?

also... woohoo! lowest ping time on your list! Cheesy

138ms! Cheesy

-- Smoov

I'm starting to think those are ping times from the client to the p2pool nodes, not the node to the other nodes. Loading the page from the miner would be most accurate.

Yes, it is always the ping from your current computer to the node. Everyone will get different numbers and if your node is on your machine or on the same network your ping will be very low. In effect it shows you the node that will be best for you and you alone to connect to.

The version number isn't in the data below so I don't display it.
http://p2pool.hopto.org:9332/local_stats
well, I thought the page was showing the ping times from his location since it was his page I was looking at.

Oh well.

If it is possible to lift the version # of the p2pools in the list, might be good to have it there, if you wanna pick a pool that is current.

-- Smoov
kano
Legendary
*
Offline Offline

Activity: 3192
Merit: 1284


Linux since 1997 RedHat 4


View Profile
April 29, 2012, 02:16:52 AM
 #2124

...
Here is the explaination forrest has given in IRC in the past...  If a share is stale/doa but still meets the current bitcoin difficulty requirements (aka it is a block), it is sent to the local p2pool node's bitcoind.  It is also sent to all of the node's peers.  Each of the peers also send the block to their own local bitcoind instances (in order to increase the speed of propagation through the bitcoin network).  However, each of the peers does not relay the stale share to any of their peers.

So it propagates partially through the p2pool p2p network and should be sent to at least 11 bitcoin nodes (1 bitcoin node from the finder of the block and the bitcoin nodes of the block finder's 10 outgoing p2pool connections).  If the block finder has any incoming p2pool connections, it will go to those p2pool peers as well.

A side effect of the fact that the stale share is not fully propagated through the p2pool network is that stale share/valid blocks are sometimes not announced on IRC because the IRC bots sometimes never hear about the stale share (because they are not a direct peer of the block finder).  We have seen this side effect several times (p2pool blocks have been found and not announced in IRC by any of the half dozen or so p2pool nodes that are connected to IRC and announcing blocks).

So the fact that such a block doesn't make it to the share chain for most P2Poolers doesn't skew the statistics in any way?
... again in relations to the low stats people have been discussing ...

There's no difficulty looking through the block chain to find all the recent P2Pool shares,
(not sure about identifying blocks early on when P2Pool first started of course)
but does P2Pool itself record blocks that didn't get into the share chain - that it isn't notified of via the share chain?
(I've no idea - that's why I ask)

Pool: https://kano.is - lowest fee PPLNS 3 Days Here on Bitcointalk: Forum
Discord support invite at https://kano.is/ Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1007


Gerald Davis


View Profile
April 29, 2012, 03:02:05 AM
 #2125

So the fact that such a block doesn't make it to the share chain for most P2Poolers doesn't skew the statistics in any way?
... again in relations to the low stats people have been discussing ...

It likely does just as if the pool is unaware of a orphaned share (which isn't a block) but the effect would be negligible.  

Even at avg diff of 800 only one in 700 shares is a block solution so the amount that not knowing about that share can affect stats is <1%.

If a node is unaware of any orphaned share it will affect things like luck calculations (but not payouts).

A block whose share is stale is no different than any other orphaned or DOA share with the exception that it is very rare.

Quote
There's no difficulty looking through the block chain to find all the recent P2Pool shares,
(not sure about identifying blocks early on when P2Pool first started of course)
but does P2Pool itself record blocks that didn't get into the share chain - that it isn't notified of via the share chain?
(I've no idea - that's why I ask)

p2pool doesn't need to know that it found a block because that doesn't affect
a) its work
b) its estimate on hashrate
c) reward split

sites like p2pool.info keep track of found blocks because miners like to know when to expect the next payment.  If your node was never notified on any p2pool block via the p2pool network it could still operate just fine (it will always be notified by bitcoind just like it is on any other found block).

Still I would think found blocks who's shares are oprhaned/DOA sould be replicated to every single p2pool node.  Getting the block broadcast from as many nodes as possible can only help block orphan rates.  Right?

JayCoin
Sr. Member
****
Offline Offline

Activity: 409
Merit: 251


Crypt'n Since 2011


View Profile WWW
April 30, 2012, 12:12:49 AM
 #2126

What is going on?  It seems very strange that we don't solve a block all day and then solve three blocks in a row!?

Hello There!
TheHarbinger
Sr. Member
****
Offline Offline

Activity: 378
Merit: 250


Why is it so damn hot in here?


View Profile
April 30, 2012, 12:14:13 AM
 #2127

To take a break from the serious conversations going on in this thread right now...


I just want you to know, I'm the one who found the block that broke the 23h 49m without a block drought.  No thanks necessary. But tips are, pay me bitches!  Grin

12Um6jfDE7q6crm1s6tSksMvda8s1hZ3Vj
TheHarbinger
Sr. Member
****
Offline Offline

Activity: 378
Merit: 250


Why is it so damn hot in here?


View Profile
April 30, 2012, 12:15:02 AM
 #2128

What is going on?  It seems very strange that we don't solve a block all day and then solve three blocks in a row!?

Variance.  It explains all.

12Um6jfDE7q6crm1s6tSksMvda8s1hZ3Vj
ChanceCoats123
Hero Member
*****
Offline Offline

Activity: 682
Merit: 500



View Profile
April 30, 2012, 01:12:34 AM
 #2129

To take a break from the serious conversations going on in this thread right now...


I just want you to know, I'm the one who found the block that broke the 23h 49m without a block drought.  No thanks necessary. But tips are, pay me bitches!  Grin

Lololol
kano
Legendary
*
Offline Offline

Activity: 3192
Merit: 1284


Linux since 1997 RedHat 4


View Profile
April 30, 2012, 02:20:04 AM
 #2130

To take a break from the serious conversations going on in this thread right now...


I just want you to know, I'm the one who found the block that broke the 23h 49m without a block drought.  No thanks necessary. But tips are, pay me bitches!  Grin
P2Pool gave you a tip for finding it Smiley
Not sure why more pools don't do that ... Sad

Pool: https://kano.is - lowest fee PPLNS 3 Days Here on Bitcointalk: Forum
Discord support invite at https://kano.is/ Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
jothan
Full Member
***
Offline Offline

Activity: 184
Merit: 100


Feel the coffee, be the coffee.


View Profile
April 30, 2012, 04:10:15 AM
 #2131

I posted a pseudo-spec on what would be required to mine on P2Pool with BFL hardware, thought this was relevant.

https://bitcointalk.org/index.php?topic=78540

Thanks !

Bitcoin: the only currency you can store directly into your brain.

What this planet needs is a good 0.0005 BTC US nickel.
Tittiez
Hero Member
*****
Offline Offline

Activity: 686
Merit: 500



View Profile
May 01, 2012, 12:00:21 AM
 #2132

Its nice to see luck increasing a little.
echris1
Full Member
***
Offline Offline

Activity: 125
Merit: 100


View Profile
May 01, 2012, 01:22:33 AM
 #2133

Its nice to see luck increasing a little.

No, ah!  Shouldn't have said anything. 

Heh, just kidding.  Luck does seem to be getting a bit better, plus I just found a block (178015) which always makes me feel lucky =)  I think that is four blocks ever found for me, unless I missed one somewhere. 
Tittiez
Hero Member
*****
Offline Offline

Activity: 686
Merit: 500



View Profile
May 02, 2012, 02:00:41 AM
 #2134

Its nice to see luck increasing a little.
No, ah!  Shouldn't have said anything. 

Ah shit oops.
echris1
Full Member
***
Offline Offline

Activity: 125
Merit: 100


View Profile
May 02, 2012, 02:27:15 AM
 #2135

So I checked my namecoin balance recently (something I tend to forget to do)  and noticed some coins.  So I dug through the log file and came up with this:


$grep submittal ~/p2pool/data/bitcoin/log
2012-04-27 16:34:07.646242 > Merged block submittal result: False Expected: True
2012-04-29 09:16:43.527938 Merged block submittal result: True
2012-04-30 20:00:57.030804 Merged block submittal result: True

Is that first one a namecoin block that was orphaned or invalid somehow? I'm guessing its not orphan, since as I understand it you wouldn't know about that right at submittal.  I have only ever found a few namecoin blocks so I'm not sure what the log usually looks like.  Just curious really, anyone know?
ChanceCoats123
Hero Member
*****
Offline Offline

Activity: 682
Merit: 500



View Profile
May 02, 2012, 02:35:27 AM
 #2136

Its nice to see luck increasing a little.

Oh god, all is lost! I ruined it last time and now it's happened again... New P2pool thread rule: NO TALKING ABOUT THE LUCK!
Aseras
Hero Member
*****
Offline Offline

Activity: 658
Merit: 500


View Profile
May 02, 2012, 12:42:37 PM
 #2137

Its nice to see luck increasing a little.

Oh god, all is lost! I ruined it last time and now it's happened again... New P2pool thread rule: NO TALKING ABOUT THE LUCK!

yeah this morning it was terrible. 5m blocks. just plain suck. lol
twmz
Hero Member
*****
Offline Offline

Activity: 737
Merit: 500



View Profile
May 02, 2012, 02:46:17 PM
 #2138

There have been on and off discussions of if the p2pool.info luck calculations are somehow wrong (i.e. too low) because of some error in the estimated hashrate, etc.

I did an experiment with independent data to see how it compared and here are the results.

Instead of looking at anything from p2pool, I looked at my payments over the past 30 days.  My personal p2pool hashrate has been unchanged during the past 30 days and I have a detailed record of every share I have found, so I am certain what my hashrate was during that duration.  Note, I did happen to be the block finder for one of our blocks in the past 30 days, and so I did even receive a bonus payment to counter balance all the other blocks where I didn't receive the bonus payment, so that effect is mostly washed out.

Taking the difficulty changes into account, I calculated what my expected payment should have been over the past 30 days given my hashrate.  I compared that with what I was actually paid.  Over the past 30 days, I received 78% of amount that I was expected to have received given my hashrate.  The luck calculation from p2pool.info (which is based on average pool hashrate and blocks found) calculates that the past 30 day luck is 77%.

77% global luck vs 78% personal luck.

That makes me pretty comfortable that the p2pool.info luck calculation approach is not fundamentally broken and off by 10% or something. 

If anyone else sees a different pattern in their own personal payouts, that would be interesting.

Was I helpful?  1TwmzX1wBxNF2qtAJRhdKmi2WyLZ5VHRs
WoT, GPG

Bitrated user: ewal.
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1007


Gerald Davis


View Profile
May 02, 2012, 04:18:07 PM
Last edit: May 02, 2012, 04:41:01 PM by DeathAndTaxes
 #2139

Thanks for doing that twmz.  Sadly that isn't what I wanted to hear.

While 80% over 90 days is plausible looking beyond 90 days the likelihood of such a low % becoming increasingly unlikely.

Since inception the pool has found 372 blocks.  Stats for the first block are unknown so we will exclude that.  

# Blocks: 371
Estimated shares in blocks:  619,152,160
Expected shares in blocks:  547,529,631
Lifetime Luck 87.9%
Shares per block relative to expected.  1.13x

I calculate
Actual # of blocks: 371
Expected # of blocks: 422

Mining is a possion distribution.
The odds of having 371 or less events when 422 are expected is 0.71%
http://www.sbrforum.com/betting-tools/poisson-calculator/

There is only a 0.71% chance we are just facing bad luck and a 99.29% chance something is resulting in less blocks found.

I am certain that "something" isn't malicious intent by developer however something is causing us to find less blocks than expected.

A number of theories:
a) some nodes are accidentally not submitting blocks.
b) something in the code is causing valid blocks to not be detected.
c) someone with ~40GH/s is intentionally withholding blocks to damage the network.
d) the stats are inaccurate and twmz observation is coincidental (I don't believe it just included it for completeness)
e) we are finding more blocks but they are being orphaned at a higher rate (>10%) and those orphans aren't being seen by p2pool.info node.
f) there is some bias in SHA-256 hash (unlikely but not impossible) such that low nonce values are less likely to find blocks that higher ones, due to LP p2pool searches more low nonce values
f) someone has found a method to "cheat" the p2pool reward algorithm

Note not all of these are equally likely I am just kinda brainstorming and putting down any possibility no matter how remote.

A couple suggestions:
a) found blocks should be forwarded to all nodes regardless of their staleness (this will aid in troubleshooting if nothing else)
b) some mechanism of recording the exact # of shares per round would be useful.

That being said I can no longer afford to take a 20% haircut on revenue so I will be temporarily leaving p2pool.

On edit:  I will leave one rig on p2pool using new address and attempt to gather detailed stats on shares vs payout.
twmz
Hero Member
*****
Offline Offline

Activity: 737
Merit: 500



View Profile
May 02, 2012, 04:30:15 PM
 #2140

A number of theories:
e) we are finding more blocks but they are being orphaned at a higher rate (>10%) and those orphans aren't being seen by p2pool.info node.

It's certainly possible, but I wanted to add that p2pool.info now also asks blockchain.info for a list of all blocks (including orphaned blocks) that include the p2pool donation address.  I trust that blockchain.info is much more likely to have seen all orphaned blocks (excluding the recent downtime) given that they connect to almost every publicly accessible node.  I think the likelyhood of unnoticed  orphaned blocks is very low at this point.

A couple suggestions:
a) found blocks should be forwarded to all nodes regardless of their staleness (this will aid in troubleshooting)

I believe forrest just commited this change to the git tree last night.

Was I helpful?  1TwmzX1wBxNF2qtAJRhdKmi2WyLZ5VHRs
WoT, GPG

Bitrated user: ewal.
Pages: « 1 ... 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 [107] 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 ... 814 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!