That's looking better, even though it is early doors. How did you manage to get connected to so many peers (30 &24 in)? I've never managed to get above 11 & 0 in.....
That's 6 outgoing (which is the recently revised default) and those always fill up quickly. The incoming ones always take a few hours to re-arrive after I restart p2pool. The only way another node will connect to me is if it is restarted, itself (and therefore needs to re-establish it's 6 outgoing connections) or if it loses one of its 6 outgoing connections. Also, nodes prefer to connect to "old" peers that have been around for a long time, and my node has been running consistently for about a year, so I don't usually have to wait more than a day to be full on incoming connections again. i changed the code to create 30 outgoing connections, its fairly easy! There is a reason it has a low default and is limited to 10: Outgoing connections are a limited resource, since few nodes are set up to accept incoming connections. See the discussion of the "Hub" patch to Bitcoin.  Besides that, you don't need that many connections. Bandwidth usage scales linearly with number of connections, but the benefit of increased connections on your orphan rate is small. |
|
|
|
|
Things should get better tomorrow when the last of the old shares fall off the end of the sharechain and I make a new release that fixes several critical issues.
|
|
|
|
So back to my original statement then.
The pool hash rate reported is below the actual hash rate of the miners (like all pools)
However, since p2pool has 60x the number of LP's compared to other pools, the actual hash rate is quite a bit higher than reported - whereas with a normal pool it shouldn't be more than about 1% higher (I typically get <0.4% for GPU/BFL/ICA and ~0.7% for MMQ)
Yet in those ignored shares, if there is a valid Block, it will of course get through, so rather than the number of blocks being representative of the number of valid shares, it is in fact representative of the number of all work generated by all p2pool miners which is of course higher than the reported hash rate.
Work done during that time is counted as DOA shares, as cabin said: One thing that might mitigate some of this is that p2pool does send a header that tells the miner to submit all work, even if it is considered dead right after a long poll. This work should show up in the number of DOA shares the pool reports. So we might be counting most of the hash rate that others pools miss right after a long poll.
So when people are reporting that p2pool is getting 110% luck consistently (which of course isn't possible - the probability of getting 110% over even 1 month is excessively unlikely) they are in fact not comparing the correct numbers.
Yes p2pool may well now be averaging 100% as expected, but those > 100% numbers are the result of not comparing the correct numbers
No, the probability of us getting >110% luck for a given month is 21% (assuming we should get 2.3 blocks/day and a month has 30 days). In[23]:= expected = 2.3*30;
In[24]:= 1 - CDF[PoissonDistribution[expected], expected*1.1] // N
Out[24]= 0.214626
Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion)
Is it just a flag or a counter? We have whole network stats for DOA and Orphans too, are there 2 flags/counters? With a counter the stale estimation should be pretty good. If it's a flag, multiple stales between shares would be miscounted as an unique stale. What might be underestimated too is the amount of work from nodes which stop mining after stale shares. It's a flag, but P2Pool buffers the number of stale shares, setting the flag as many times as it needs to. Nodes leaving before getting a share with the flag set is a problem, yes, but I think it's a small one. |
|
|
|
So - how is the pool hash rate calculated?
It includes orphans and dead on arrival shares as they are seen by every node. Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion) So it doesn't include all the 1-diff shares generated by the miner ... ? No... pool hash rate is only determined by actual shares, which is accurate enough. |
|
|
|
So - how is the pool hash rate calculated?
It includes orphans and dead on arrival shares as they are seen by every node. Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion) |
|
|
|
It should work like this but it doesn't. It's handing 3+ diff shares to a miner of mine that is 2 150 mhash cards and every share is dead for >24 hours.
I've gone back to 8.2.
There need to be a way to toggle or control it, at least for now for the slower workers.
A high difficulty shouldn't cause dead shares (it should only reduce how often pseudoshares are sent back to your P2Pool node), so this is either a bug in P2Pool or a bug in your miner. Can you tell me what mining software that miner was using? And send me p2pool's log file? |
|
|
|
BUGZZZZZ I didn't do anything abnormal, using bcoin0.71 Started p2pool like I do normally, but then I saw that an invalid hash message poped up, and then the bug showed up. I think this caused the bug: 2012-11-16 13:14:45.342000 invalid hash for 99.162.89.78 'remember_tx' 248345 04cac86c 86dc28a6********(rest of hash omitted because.....the text was too long for forum "The message exceeds the maximum allowed length (64000 characters). " And this forum has no way to attach text files. I am not sure if the invalid hash LENGTH is what cased the bug, because it was over 64,000 characters in length+ So perhaps this is just a bug that is caused by a hash that was too long, overloading a buffer or related. This might just be a bug caused by getting hashes from peers that are wayyyy too long? (no idea how that would happen unless someone is malicious, or their pc flipped out/program crashed....) I think it's due to a packet sent over the network getting corrupted somehow. Did P2Pool continue to work afterwards? It should have. If it didn't, this is a real bug ... but otherwise it's just a rare failure that was handled correctly. |
|
|
|
Yes, that's new. Preparation for ASICs! It tries to regulate the work's difficulty to keep it at one response per second.
Is there any way to configure the period server or client side? Not currently, no. Do you have a need to? |
|
|
|
Twice now I've seen pools saying their stratum implementation will not support merged mining. (BTCGuild and Oz). Will that be the same with p2pool?
No. P2Pool will support simultaneous Stratum and Merged mining. I don't believe that there's any technical reason for pools omitting MM support from their Stratum implementation (perhaps just a lack of incentive...) |
|
|
|
Does the new v9 retarget miner difficulty dynamically? On a fresh start it's this New work for worker! Difficulty: 0.999985 Share difficulty: 725.187715 Total block value: 50.148150 BTC including 165 transactions after a while it start showing this New work for worker! Difficulty: 3.033185 Share difficulty: 633.860115 Total block value: 50.542400 BTC including 716 transactions
New work for worker! Difficulty: 2.537561 Share difficulty: 637.237788 Total block value: 50.542400 BTC including 717 transactions
New work for worker! Difficulty: 1.820309 Share difficulty: 738.198270 Total block value: 50.063150 BTC including 63 transactions
Yes, that's new. Preparation for ASICs! It tries to regulate the work's difficulty to keep it at one response per second. |
|
|
|
That's expected - Just don't go to that page like that. The URL should always have a share hash on the end, like http://vps.forre.st:9332/static/share.html#00000000004506b7e27ad78c40fedc063bbe417936b99316856c0e7338ac5250 The error happening isn't harmful. How fast should a miner ask for updates from P2 pool? I can do 1ms to 10,000 ms.
Your miner should support long polling so that it doesn't have to do timed polling. Failing that, 100ms or lower, depending on how much overhead you can cope with. One small thing.. the links in the first post in this thread still point to version 8.2.. I guess it should be changed to 9 now?
Thanks for telling me, I forgot. Fixed. |
|
|
|
Heh - I was arguing that it should just work then got corrected (pointed out I was wrong) and worked out why back here: https://bitcointalk.org/index.php?topic=18313.msg1311688#msg1311688You now seem to be saying it will 'just' work Implementing Stratum in P2Pool without adding the extra p2pool coinbase rules to the miner will be no different to using GetWork as far as I see. Since without those rules, the only thing the miner can do is roll the nonce and the time. So in reality it will just be 'getwork' not 'stratum' going to the miner. You will still only be using roll-n-time in the miner until the miner knows the P2Pool rules to add to Stratum - since stratum updates the secondary nonce in the coinbase but of course doesn't update anything else in there, since for non-share chain, nothing else is required - but for p2pool you do have to change something for the share-chain (I've no idea exactly where it is in there) It didn't just work - It required a change to P2Pool's rules to allow a 4-byte nonce in the last txout script. I mentioned the possibility here: However, I discovered yesterday that Stratum is flexible enough to roll any data in the coinbase transaction at all, though, so rolling something in the last txout script is probably possible (though it would require a change to P2Pool's rules).
|
|
|
|
* Stratum support in the P2Pool protocol. Stratum mining will be enabled in a future patch to v9.
I think this is great news, allowing p2pool to scale to any size in the future, taking a lot of load off the local p2pool software itself, allowing it to concentrate on the business of getting those blocks and transactions out ASAP. Sending higher difficulty targets to the miner will take it even further. I agree. What I'm curious about is how it's there, but not enabled, and why? I simply haven't written the code for a Stratum server yet. Getting support into the P2Pool protocol and everyone switched over so it will work once implemented was the priority. |
|
|
|
(may i ask why we need stratum support when you are acutally using some kind of stratum when mining on P2Pool)
More efficient communication between your P2Pool node and your miner. One Stratum work unit could supply an ASIC miner with a virtually infinite amount of work, as opposed to a getwork work unit, which gives 4 GH of work. |
|
|
|
P2Pool release 9.0 tag: 9.0 hash: 2cb4d8381e179f71ea2075cdce948ea83cf0dc55 HARDFORK: Upgrade is required! Hardfork will happen after 95% of the pool's hash rate has upgraded. Everybody having not upgraded will be split off into a tiny P2Pool. Windows binary: http://u.forre.st/u/urxoztyg/p2pool_win32_9.0.zipSource zipball: https://github.com/forrestv/p2pool/zipball/9.0Source tarball: https://github.com/forrestv/p2pool/tarball/9.0Changes: * Transaction preforwarding Transactions that you're mining are sent to peers before you get a share, so any block solution you find can be broadcast virtually instantaneously. This could theoretically get our invalid block rate below any other pool's thanks to our large network of nodes. This was implemented in v8, but was prevented from taking effect due to a bug being discovered.* Stratum support in the P2Pool protocol. Stratum mining will be enabled in a future patch to v9.
|
|
|
|
What does "Punishing share for 'Block-stale detected!" mean? I gather it happens after a new block appears on the network.. but what is the punishment?
First, this has always been done, but I've just recently made the message more visible. It used to be simply "Stale detected! (long hex number) < (long hex number)". In order to punish the share, your node tries to orphan it by building on that share's parent instead of on top of it. If you find a share, you'll have made a fork, and other nodes will prefer your share to the original, since yours wasn't block-stale, resulting in the original share not being integrated into the chain. |
|
|
|
Um - it's the actual coinbase field itself - the same place where you put all that other random data.
If you can randomly throw merged mining data it there, you should be able to add a 4 byte fields in there that you can roll instead of the ntime field.
I'm not sure what I'm missing here?
You could easily roll something in the coinbase script, yes, but it wouldn't result in a valid p2pool share. The miner would need to be smart enough to update the data in the last txout, which is impractical. |
|
|
|
The earlier argument (^) about remote miners not working was a bit of a farce, as shown by the above numbers.
The thing to focus on in order to make sure things work is the timestamp rolling support in ASIC mining software. We need to ensure that it can roll ahead of the current time (and potentially backwards, though that would require an extension to getwork).
...
Sorry - I don't get the point of this comment. Why do you need rolling the time? Unless I've completely missed your point, you don't need to roll the time with Stratum or GBT, you roll a value in the coinbase instead, named (IMO badly) the secondary nonce instead of screwing with the block timestamp (which is a hack) unnecessary P2Pool can't handle simply altering the coinbase script. P2Pool internally has a structure that describes how to recreate the coinbase transaction, including the coinbase script, but that can't be changed because the hash of that structure is stuck in the last txout. P2Pool uses it to create short, self-contained proofs of work by storing the SHA256 midstate with shares and computing the generation transaction's hash using that midstate and the hash of that internal structure. TL;DR: If the coinbase script is altered, the hash in the last txout of the generation transaction has to be changed too. However, I discovered yesterday that Stratum is flexible enough to roll any data in the coinbase transaction at all, though, so rolling something in the last txout script is probably possible (though it would require a change to P2Pool's rules). |
|
|
|
The main problem I see coming is that even the slowest announced ASIC (BFL's Jalapeno) can go faster than one getwork per second. Most devices are a lot faster. Pulling the work away from the pools and closer to the actual device is a decent looking way to deal with that. I'm not aware of any huge issues with timestamp rolling as long as they can get enough getwork blocks, especially after a longpoll when they need more new pieces of work to start working on the new block (small surges of getworks every 10 seconds when a block comes out seems excessive).
On my desktop, P2Pool can supply ~130 getworks/second, or enough work for 520 GH/s (130/s * 4 GH), without any timestamp rolling. With timestamp rolling one minute backwards and forwards, it can supply 62 TH/s (520 GH/s * 120) of work, or 480 GH/s (1/s * 4 GH * 120) of work at a rate of one getwork per second. I don't think ASICs will need any special support. P2Pool can provide getwork results fast enough for hundreds of GH/s (from a normal computer) and could be optimized for more. In addition, any timestamp rolling multiplies that.
What about mining on a remote node? It seems like ASICs could kill P2P Mining. The earlier argument (^) about remote miners not working was a bit of a farce, as shown by the above numbers. The thing to focus on in order to make sure things work is the timestamp rolling support in ASIC mining software. We need to ensure that it can roll ahead of the current time (and potentially backwards, though that would require an extension to getwork). I looked at GBT, and I don't see it improving anything. It requires sending the entire block template (potentially up to 1 MB) to miners every 10 seconds, which is definitely impractical for remote miners. An extension to GBT that allowed only sending the merkle root would avoid this, though. |
|
|
|
|
Show Posts
