Bitcoin - we have a problem.

[DeathAndTaxes]

And as was pointed out, there is a large variation in the "time to solve", but perhaps if the 2 weeks to adjust were modified to something akin to a Kalman filter, see
http://en.wikipedia.org/wiki/Kalman_filter
one might be able to shorten the "time to change the difficulty" to a very short window?

One could adjust the difficulty every block without any complex mathematics or issues with synchronizing times. Now mining is random we only know the time of the average block which is 10 minutes however each individual blocks time to a solution will fall into a bell curved (visualize a bell curve with peak at 10 minutes and a standard deviation of 10 minutes).   If you simply looked at the last block the difficulty adjustment would just be tracking the normal random walk of bitcoin solution times.  Bitcoin "solves" that by looking at a 2016 block window and adjusting it every 2016 blocks.  However it isn't required that the adjustment interval and the averaging interval be the same.

For example a CC could look at time for last 2016 blocks to get the average time per block (and thus the necessary change in difficulty) but do that every block.

For example:
for block 2017 the difficulty is computed by looking at the time interval from block 1 to block 2016
for block 2018 the difficulty is computed by looking at the time interval from block 2 to block 2017
for block 2019 the difficulty is computed by looking at the time interval from block 2 to block 2018
...

[1715096527]

1715096527

[1715096527]

1715096527

[1715096527]

1715096527

[notme]

The issue with p2ppool is ASIC miners will not work on them

https://bitcointalk.org/index.php?topic=90658.0

Please stop spreading this lie.  The issue with p2pool is people who might try it are actively discouraged by uninformed people like you. ASICMINER hw works, BFL hw works, and soon Avalon hw will work.

https://bitcointalk.org/index.php?topic=153232.0:

Hardware latencies

ASICMINER Block Erupter blades: they seem to work correctly according to this post

Avalon: I've ordered one in batch 2 but don't have any experience with it yet... There have been various informations floating around but no definite conclusion. Here is what is reported to influence how Avalon and P2Pool behave, you may want to try different combinations of these suggestions:

• Test different versions of the Avalon firmware, beginning with the latest one.
• Avalons apparently have a limitation with stratum that a new branch of p2pool tries to circumvent: you may want to try this branch
• try to use cgminer's "--fix-protocol" to avoid Stratum
• contact the p2pool devs on the #p2pool IRC channel to report feedback (and maybe get some tips)
• force a high difficulty for the shares submitted by Avalon by adding +n at the end of the username used to connect to P2Pool (try n=16, 32, 64, ...)

Please report your experience (hashrate, DOA and orphan percentages and efficiency) if you can. Current reports are incomplete and seem to indicate that Avalons don't reach 100% of their mining efficiency on P2Pool.

BFL: if you have a BFL Single, an early FPGA MiniRig (cgminer has a parameter for later ones to fix them, check its documentation) don't waste their hashrate on P2Pool, they have huge latencies and can't perform well on P2Pool. Put them on a traditional pool.  If you have a BFL SC (ASIC), ckolivas and kano reported the same problem, see ckolivas post. At least two users reported around 100% efficiency: here and here. You might want to test it for yourself for at least 24h and report your results here (please include the details from cgminer's API if possible).

Additionally, with the upcoming hardfork (code released, waiting on 95% of hashpower to upgrade), p2pool should work reasonably well for all ASICs (including Avalon), and optimally for most.  Even without the hardfork, I'm mining away with 61 AM USBs with 111% efficiency.

[mateo]

Perhaps we should target, e.g. 10 or 20 pools, each with no more than 10% or 5%.

Surely that's achievable.

It is impossible to enforce - pool operators want as many users as possible so they can collect more fees. Even if most agree - some will not and lots of users will end up on them because the payouts are more consistent.

The trouble is we are greedy bastards by our very nature

Agreed. The Tragedy of Commons - one of the thorns of the free market.

[rovchris]

The issue with p2ppool is ASIC miners will not work on them

https://bitcointalk.org/index.php?topic=90658.0

Please stop spreading this lie.  The issue with p2pool is people who might try it are actively discouraged by uninformed people like you. ASICMINER hw works, BFL hw works, and soon Avalon hw will work.

https://bitcointalk.org/index.php?topic=153232.0:

Hardware latencies

ASICMINER Block Erupter blades: they seem to work correctly according to this post

Avalon: I've ordered one in batch 2 but don't have any experience with it yet... There have been various informations floating around but no definite conclusion. Here is what is reported to influence how Avalon and P2Pool behave, you may want to try different combinations of these suggestions:

• Test different versions of the Avalon firmware, beginning with the latest one.
• Avalons apparently have a limitation with stratum that a new branch of p2pool tries to circumvent: you may want to try this branch
• try to use cgminer's "--fix-protocol" to avoid Stratum
• contact the p2pool devs on the #p2pool IRC channel to report feedback (and maybe get some tips)
• force a high difficulty for the shares submitted by Avalon by adding +n at the end of the username used to connect to P2Pool (try n=16, 32, 64, ...)

Please report your experience (hashrate, DOA and orphan percentages and efficiency) if you can. Current reports are incomplete and seem to indicate that Avalons don't reach 100% of their mining efficiency on P2Pool.

BFL: if you have a BFL Single, an early FPGA MiniRig (cgminer has a parameter for later ones to fix them, check its documentation) don't waste their hashrate on P2Pool, they have huge latencies and can't perform well on P2Pool. Put them on a traditional pool.  If you have a BFL SC (ASIC), ckolivas and kano reported the same problem, see ckolivas post. At least two users reported around 100% efficiency: here and here. You might want to test it for yourself for at least 24h and report your results here (please include the details from cgminer's API if possible).

Additionally, with the upcoming hardfork (code released, waiting on 95% of hashpower to upgrade), p2pool should work reasonably well for all ASICs (including Avalon), and optimally for most.  Even without the hardfork, I'm mining away with 61 AM USBs with 111% efficiency.

The issue is when you search to see if they work the first results say they don't - What percentage of people are going to keep looking? If they already mining at 100% on an existing pool why are they going to take the risk?

Nobody is going to read through 300 pages of posts - only people that have been closely following that thread will.

After having a look through it is not clear what is going on with the BFL devices. When the guy who wrote cgminer is saying there is a problem what conclusion do you expect people to come to. It looks like the 5ghs devices work but not the bigger units.

To quote ckolivas the author of cgminer

Check the BFL forums. We asked them why they didn't implement the command because we tried it and it doesn't work on BFL SC devices.

They did not respond. I think BFL are too busy drowning in fail to respond to petty questions like these.

Avalon's as you say also have a problem

So at the moment it is only the USB miners and the BFL low ghs units that appear to work. So in essence no big miners are going to use it.

I also don't understand why so many people on this forum immediately attack each other - anyone that is not involved in Bitcoins is going to be reluctant to join or even post due to all the demeaning comments that are fired at you when you misunderstand something - which would will be the majority of new users.

You guys need to be more patient and just explain why something may be incorrect.

[rovchris]

Reading more about the p2pool I have to say it does appear to be a very good solution to decentralisation.

https://en.bitcoin.it/wiki/P2Pool

The issue is - it is far more complex than mining at a traditional pool.

Network latency plays a big part in how efficiently you mine which is not such an issue for normal mining pools.

You need to install the p2pool node software either on your miner or on a dedicated machine.

Bitcoin-qt / bitcoind has to be installed and the full block chain downloaded. This then needs to be monitored to make sure it is up and running as if your were solo mining.

The share difficulty is much higher therefore there is a greater variance in your earnings which is one of the reasons miners go to pools that use the pay per share payment model.

The P2Pool difficulty is hundreds of times higher than on other pools. It can take time to get a share. P2Pool displays an estimate of how long you have to wait in the console output.

The above quote is immediately going to discourage people.

After reading through that wiki page I can say that a huge number of people are going to have trouble understanding it and wonder why it is worth the effort.

Only dedicated people that understand the importance of decentralisation and have a good understanding of diagnosing network problems will take it on but this is very small percentage of miners.

It has to be looked at from the layman's perspective - it is far simpler to use a traditional mining pool and does not require any real technical expertise or understanding and you are far less likely to have an issue with stale shares, payment variance, mining efficiency, setting up long polls and the other issues that go with it.

When you see questions being posted asking what host do they need for their ASIC miner - you know at that point they are looking for the easiest possible way to generate bitcoins and are not interested in the ethos of the coin. They want to run them from a host that uses as little power as possible to maximise their profit. If you told them they have to run a PC to host the bitcoin-qt and the p2pool software the first thought will be why bother when I can use a mining pool and not have to do any of that. Have a look through the BFL forum and you will see where I am coming from.

For the p2pool to really succeed it needs to be as easy to use as the traditional mining pools otherwise people will not make the effort to switch.

After having a look at this chart http://p2pool.info/ there was an increase in the number of users from April to the middle of May but from there it has been in a steady decline even though the hash rate has increased. This is indicative of a hardcore group of users that have just been increasing their hash rate and not a growth in the user base or popularity.

I would like something explained - what is stopping you submitting the same share to the p2pnode and your bitcoin-qt client at the same time? (I am assuming that it is actually your bictoin-qt client that is providing the getwork response to the p2pnode sofware that then forwards it to your miner).You can easily write a proxy point your miner at that and this just duplicates and submits the same share to two different address simultaneously. If it was of valid difficulty the chances are the bitcoin-qt client would receive it first as a direct submission than through the p2pnode and then to the bitcoin-qt client. This does not work with mining pools because of the Merkle tree being different for 2 different bitcoin-qt servers and the share is immediately rejected.

[notme]

For the Layman:
http://p2pool.hostv.pl/

Pick one, point your miner at it, done.

Shares always have to payout to the current payout list to be valid on p2pool.  The software does send it to bitcoind if it is a valid block to make sure it is broadcast as soon as possible.  There is no way to change the way the payout is done.  If you change that, you have to start the proof of work all over again.

As for high variance, pooling hashpower to a single p2pool address is possible.  In fact, I ran such a pool for a couple of weeks.  Instead of needing to find a block on its own, my pool paid out whenever p2pool found a block.  I may revive it at some point, but there was almost no demand when I tried it.

The issue is when you search to see if they work the first results say they don't

Because people who don't know what they are talking about keep repeating it.  Soon the hardfork will take place, and p2pool will be compatible with all ASICs.  Now is the time to break this meme.

I also don't understand why so many people on this forum immediately attack each other - anyone that is not involved in Bitcoins is going to be reluctant to join or even post due to all the demeaning comments that are fired at you when you misunderstand something - which would will be the majority of new users.

You guys need to be more patient and just explain why something may be incorrect.

Maybe it is because people talk with confidence about things they don't full understand.  This leaves the uneducated with false belief, which they continue to repeat.  When you hear the same lie repeated 50 times because everybody takes hearsay as fact, it gets annoying.

[rovchris]

For the Layman:
http://p2pool.hostv.pl/

Pick one, point your miner at it, done.

I don't see how that is any different to using a mining pool?

I thought the point of the p2pool is that every miner runs a p2p node?

[notme]

For the Layman:
http://p2pool.hostv.pl/

Pick one, point your miner at it, done.

I don't see how that is any different to using a mining pool?

I thought the point of the p2pool is that every miner runs a p2p node?

Because every one of those nodes connects you to the same pool.  You can't have it both ways.  Either you use someone else's node, or you get your hands dirty and run your own.

[rovchris]

The issue is when you search to see if they work the first results say they don't

Because people who don't know what they are talking about keep repeating it.  Soon the hardfork will take place, and p2pool will be compatible with all ASICs.  Now is the time to break this meme.

I could understand your point if it was a comment by a random user but when it was a BFL Engineer what else can you be expected to think - They can not be classed as not knowing what they are talking about.

Will the new units have the same problem as the old Singles where it wouldn't stop working on a problem for 5 seconds making it useless for p2pool?

The answer is yes.


Regards,
BF Labs Inc.

So to clarify when using P2pool it is your local bitcoind that is issuing the response to the getwork request that is then relayed through the p2p node and then to your miner?

The point I am trying to make here is this - If for example you are mining against a traditional PPS pool and you have written a proxy that sits between your miner and the pool. This proxy checks the difficulty of the solved block before forwarding it to the pool. If the proxy receives a solved block that meets the difficulty it then forwards it to your local bitcoind. The local bitcoind will reject this solved block because the Merkle tree is different for the two bitcoind servers preventing you from "stealing" the block.

What I am trying to understand here is when using p2pool if your local bitcoind is issuing the getwork then using the proxy to intercept a solved share of required difficulty from your miner and sending that straight to the local bitcoind and not back to the p2pnode why would it not be accepted and the wallet be credited with bitcoins and the p2pnode would be none the wiser. So effectively you would be getting paid for shares and then take the full reward for yourself?

[notme]

Cool story bro.  I've debated this to death in other threads and the fact is people with BFL hardware have been able to make it work by tweaking a few things.

Edit: removed quotes since you fucked them up

[rovchris]

Cool story bro.  I've debated this to death in other threads and the fact is people with BFL hardware have been able to make it work by tweaking a few things.

Edit: removed quotes since you fucked them up

I don't understand why you are taking issue with me over this? -  I read a post by a BFL engineer that said they would have a problem with P2pool and somehow I am suppose to know that is not the case.

Generally when a manufacturer says something about their own products which would be detrimental to selling them why would you not believe them and consider it fact, a BFL engineer has far more authority on this matter than anyone else.

He may have been incorrect though so why do they not remove or update that post? Other people will see it and assume it to be the case as well and they are far more likely to listen to someone that works for the BFL than random people saying it does work.

Why you feel the need to be condescending about it I am unsure.

[notme]

Cool story bro.  I've debated this to death in other threads and the fact is people with BFL hardware have been able to make it work by tweaking a few things.

Edit: removed quotes since you fucked them up

I don't understand why you are taking issue with me over this? -  I read a post by a BFL engineer that said they would have a problem with P2pool and somehow I am suppose to know that is not the case.

Generally when a manufacturer says something about their own products which would be detrimental to selling them why would you not believe them and consider it fact, a BFL engineer has far more authority on this matter than anyone else.

He may have been incorrect though so why do they not remove or update that post? Other people will see it and assume it to be the case as well and they are far more likely to listen to someone that works for the BFL than random people saying it does work.

Why you feel the need to be condescending about it I am unsure.

BFL has no interest in getting to work.  If it isn't plug in play, they will say "no it won't work" and then get back to something that makes them money.

People who do have an interest in getting it to work have been able to do so.  Why you feel like beating a dead horse I am unsure.  I have seen every post you have tried to offer as "proof".  Sorry, but you are wrong.  No please quit telling people incorrect things, or at least HINT that you might not know instead of saying things like this:

The issue with p2ppool is ASIC miners will not work on them

[rovchris]

I see you conveniently edited out the link that was there pointing to a thread discussing the issue. Showing that it was not my assumption but based on information posted in this forum.

[notme]

I see you conveniently edited out the link that was there pointing to a thread discussing the issue. Showing that it was not my assumption but based on information posted in this forum.

I conveniently edited out everything except the once sentence I was referring to.

Your assumption was that you took information posted on the forum as fact without verifying it.  Learn 2 Internet.  Forums are not a valid reference.

[rovchris]

I see you conveniently edited out the link that was there pointing to a thread discussing the issue. Showing that it was not my assumption but based on information posted in this forum.

I conveniently edited out everything except the once sentence I was referring to.

Your assumption was that you took information posted on the forum as fact without verifying it.  Learn 2 Internet.  Forums are not a valid reference.

Yet you expect people to take your word that BFL products work properly on p2pool which is none other than posted on a forum. What gives you more authority than the manufacturer who stated that there is a problem.

[old c coder]

I see. Like when a nuclear power station goes offline rather than a couple of wind turbines.

How long does it take to adjust then?

2 weeks at 10 min per block, so it takes 30 min for each block difficulty wont be adjusted downwards for 6 weeks. It's an issue at the mo because we're in the transition from off the shelf hardware to dedicated hardware and ASICminer happens to be leading that transition. When more ASIC manufacturers have products ready for immediate sale and difficulty levels out its unlikely to be a major issue but it will be a rough ride upto that point as hardware prices will need to establish a predictable ROI time.

And as was pointed out, there is a large variation in the "time to solve", but perhaps if the 2 weeks to adjust were modified to something akin to a Kalman filter, see
http://en.wikipedia.org/wiki/Kalman_filter
one might be able to shorten the "time to change the difficulty" to a very short window?
...
Ron

I had to google Kalman filter as I had no idea what it was - it made for some interesting reading even though it is some pretty hardcore mathematics!

What applications were you writing that you are even aware of it, I would be quite interested to know.

The change at the moment is simply done on the block count every 2016 blocks I believe.

I also totally agree about bitcoin-qt more friendly for the masses - have you looked into the "signing" of messages yet? There is absolutely no way the man on the street is ever going to get their head around that - it is far to "geek" to be completely honest.

It makes a pleasant change to actually have some interesting ideas posted that can address some of the issues.

The Kalman filter was used in a fortran program, compiled to run on a 1Mhz 16 bit minicomputer, with 4K of rom, that was the first collision avoidance computer that lived on oil tankers, about 1977. It (the filter) was used in the satellite navigation autopilot s/w that plotted the course and steered the ship. Not well I might add. I just installed and serviced them.

Ron

[old c coder]

And as was pointed out, there is a large variation in the "time to solve", but perhaps if the 2 weeks to adjust were modified to something akin to a Kalman filter, see
http://en.wikipedia.org/wiki/Kalman_filter
one might be able to shorten the "time to change the difficulty" to a very short window?

One could adjust the difficulty every block without any complex mathematics or issues with synchronizing times. Now mining is random we only know the time of the average block which is 10 minutes however each individual blocks time to a solution will fall into a bell curved (visualize a bell curve with peak at 10 minutes and a standard deviation of 10 minutes).   If you simply looked at the last block the difficulty adjustment would just be tracking the normal random walk of bitcoin solution times.  Bitcoin "solves" that by looking at a 2016 block window and adjusting it every 2016 blocks.  However it isn't required that the adjustment interval and the averaging interval be the same.

For example a CC could look at time for last 2016 blocks to get the average time per block (and thus the necessary change in difficulty) but do that every block.

For example:
for block 2017 the difficulty is computed by looking at the time interval from block 1 to block 2016
for block 2018 the difficulty is computed by looking at the time interval from block 2 to block 2017
for block 2019 the difficulty is computed by looking at the time interval from block 2 to block 2018
...

But isn't that, then, just a moving average, all weighted the same. So an instant drop in hashing power, which is the question I think, would only affect the average slightly. If the latest, perhaps few, had more weight in the moving average, the "response" would be faster, I should think? This, in some analogous sense, is what a Kalman filter does.

I am looking at GetNextWorkRequired() in main.cpp and without comments, I will have to study it for some time to figure out how it is setting the difficulty, but it seems to be just the difference in time of the last 2016 blocks against the two week norm, limited to 1/4 if less and 4 times if more, with equal weighting of all blocks.

I like the idea of adjusting the difficulty more often than ~two weeks, though, as a start.

Ron

[rbdrbd]

Interesting thread. Not meaning to go OT, but in the context of solving these issues (or at least having a more refined alternative), what is everyone's opinion on Proof of Stake (as implemented in PPCoin, namely), as well as PPCoin's mint-rate adjustment algorithm as compared to Bitcoin (i.e. where PPC relies on difficulty, not time/block height)?

Seems like PPC would have a lot to gain on both fronts if this difficulty/centralization scenario plays out. It's looking like (at least to me) that Bitcoin POW energy efficiency will progressively become more and more of an issue. On one end you have the move to ASICs (and more efficient ASICs) which have reduced energy demands over GPUs and CPUs at a per-watt level, but on the other, if you are POW-only, like Bitcoin, the only real direction you can move in for increased network security is more hashrate. That hashrate will only stick around if there is profit to be made, either through minting operations (relying on this ASIC armsrace) or transaction fees (which may scare away users in the face of other viable cryptocurrencies).

I have noticed this POW centralization effect lately and it is a bit concerning to me. With PPC, given that it's still SHA256, I don't see how it would become less decentralized from a POW perspective than BTC, but at least your network security wouldn't depend 100% on POW hashpower.

[notme]

I see you conveniently edited out the link that was there pointing to a thread discussing the issue. Showing that it was not my assumption but based on information posted in this forum.

I conveniently edited out everything except the once sentence I was referring to.

Your assumption was that you took information posted on the forum as fact without verifying it.  Learn 2 Internet.  Forums are not a valid reference.

Yet you expect people to take your word that BFL products work properly on p2pool which is none other than posted on a forum. What gives you more authority than the manufacturer who stated that there is a problem.

No, I expect people to not make claims they can't back up with actual experience.  Your claim is patently false and I can prove it because I AM RUNNING ASICs ON P2POOL, AND SO ARE OTHERS.  I don't ask you or anyone else to believe me or the other posters.  I am merely asking you to stop making claims based on rumors because I happen to know are not true.  But, you'd rather be thick headed and point at quotes from a company who has been 2 weeks from shipping for over a year and who's engineers dramatically underestimated critical requirements, such as power usage.  Sorry, but I'll take my personal experience over BFL's "engineers" any day.  Again, I don't ask you to do the same.  I'm just asking that you stop spreading rumors you have not verified.

[DPoS]

(which may scare away users in the face of other viable cryptocurrencies).

that is such a reach..  btc with all its popularity is still a long way from having daily users and you assume that these few adopters would rather go to other coins because of some tiny transaction fees that are still way lower than any fiat, credit, etc systems?

You are just looking at it from a miner side with no real care about 'users'...  classic pump and dumper

lol at 'other viable cryptocurrencies'..  that is a looong way away and only if btc paves the road