I am far from suggesting there is any malicious intent in this. This does not make sense at all and contradicts everything I see in this thread.
Of course there won't be a "hey, I found a haxx0r backdoor in line 713 which steals our monies!!". And no, I don't plan to read the code, as I am not knowledgeable to do so.

But, obviously for me, there is a general and (now) constant problem nevertheless. I don't write this to hurt p2pool, the opposite is the case. There are enough outcries and "have to go back to xypool" here to justify a closer look, even without the problem being visible in the luck chart for weeks now.

Ente

All right.

This is what the "luck" graph looks like:

(as can be seen here: http://u.forre.st/p2pool/luck.png)


This is what "variability and luck" would make me expect it look instead:



But, in fact, it looks much more like this, which is not "variability and luck", but a inherent problem besides variability:


The red line is like the average block-finding rate. Do you see the angle between the black "hashes calculated" graph and the red "blocks found" graph? THAT is what I am talking about, not the stochastic ripple in the original blue line, which I crudely smoothed out with the red line.

It does not seem to be caused by block orphans, as people stated before that these are in the low % range. Neither has it anything to do with local orphans or local deads, as these don't add to the (black) "hashes calculated" neither.


I am not confident enough to calculate the "we are losing x % hashingpower" from this graphs, maybe someone else is. To me it seems clear to be a constant "loss", therefore the two graphs resemble two lines with an angle.
My wild guess would be we are effectively losing 15% hashingpower, in the sense that we have 15% less blocks, since 2 months.

Ente

edit: fixed images

No, of course not "compensated" as a reaction to the bad luck streak. But "evened out" in the sense that this single bad luck streak has less and less influence as the time or number of draws rise.

I am pretty sure we all are more or less talking about the same thing, and actually agreeing. How about we say "there are bad times, there are good times, everything is fine", and leave that page-long non-p2pool-related statistics/probability stuff?

The still unanswered question remains:
Is p2pool in fact statistically "working", in the sense that 100 Th will yield the [statistically] same amount of blocks as 100 Th on a single bitcoind or centralized pool?

Ente

http://www.geekwire.com/2012/bitcoin-startup-coinlab-lands-funding-tim-draper-monetize-games/


Wow, things are starting to move! :-)

Ente

:-)

Ente

There is no memory in bitcoin hashing.

Exactly!
That is exactly why it evens out after some time.
If you roll dice, and have a particulary unlucky streak with four "1" in a row, that is bad luck. And happens quite rarely too. But as you go on, after some time, you will eventually hit four "6" in a row. Then it will have evened out. Sure, you will have dozens of "three 1" and "three 6" in between, maybe even another "four 1" too. But if you roll long enough, and the dice *really* dont remember anything (i.e. not broken or manipulated), you will eventually have an average of exactly 3. It may take a few hundreds or many thousands of rolls, depending on how many decimals you want to converge to "3".

Anyway, I think we are talking about the same here ;-)

(And to me it is not entirely sure yet that the *whole* p2pool setup is working entirely as expected and calculated. There seem to be an additional, constant small-% loss in the way.. observing the "luck.png" graphs..)

Ente

..let me know a few hours before your rig goes down, next time, ok? I would like a "warning, blockfinding-spree ahead!" warning system! ;-)

It all evens out after some time.

Ente

That got me thinking.
Yes, this seems to be a good idea!
How to adjust the individual share difficulty?

- To have it similar to now (one share each 10 sec), each node would poll the number of nodes and the p2pool hashrate. If the individual hashrate is "high" compared to the average node's hashrate, make the local share-difficulty higher.
Quite complicated, maybe unstable, and all in all not too elegant, in my view..

- Drop the "one share every 10 sec" altogether. Make every node find one share, say, every 10 mins. Big miners will have a high local difficulty, small miners a low one. Depending on the amount of nodes, the sharechain would grow faster or slower than now. With those 10 mins per local share and approx 200 users at the moment, there would be a new share every 3 secs. This probably is too short, so we may make it "node, find a share every 30 mins!" for an approx sharechain-growth of one share every 10 secs.

This would not scale too far neither. But it would be a lot easier and with less drastic changes than the other, older proposals (sub-p2pool, multi-p2pool etc). So it might be a bridge-technology, as we say here.

Ente

"Worth"? That assumes some gain by doing that. To drive away p2pool-users? It would be easier by a magnitude to use those 30Gh/s to "normally" mine for profit, and attack a regular centralized pool with DDOS. That would drive off their users to some degree, maybe attracting some of them for his own pool.

All data is there already anyway. We see which nodes exist, which nodes contribute how many shares (to add to the p2pool net hashpower) and which nodes do or do not find bitcoin blocks (to reduce the luck). If someone wants, he should be able to dig through the stats to verify/falsify this. Even if those 30Gh/s were on dozens of individual nodes, they probably come from the same IP. Even if not, with 30Gh/s you should be able to find anything, any clue.

You see, even with this "bad luck", people are sticking to p2pool, as can be seen in the stats. Not because p2pool is necessarily paying out better than all other pools at any time, but because p2pool is superior and healthier for the whole bitcoin universe.

But yes. I believe p2pool has a problem which may have stayed unidentified yet. The "hashing power reflected by hashes" and "hashing power reflected by blocks" are two fairly straight lines, but clearly diverging. Its not two overlapping lines, its not one line being crossed by the other all the time.
Connectivity seems to be one problem for the individual miner. Not for the global p2pool efficiency though.

Ente

That is a good idea, actually!
..And this tag would obviously stick a lot longer to someone.

Ente

Interesting! I wouldn't have expected this.. Nice find! Report back in a week, curious how this turns out!

Ente

..I was more asking about the bigger picture.. The one at which's end a multi-million player game is, you know.. ;-)
No i2p here, yet. Maybe later though..

Ente

I don't understand all genre words you use, nor do I understand all concepts you lay down. But what I do understand does really sound great! :-)
I never thought about the "initial population problem" before. I was a WoW addict many, many years ago, and consider myself cured by now. But this, for the good cause, would get me into MMORPG again! :-)

I much appreciate your work and effort.

So, in easy words: what would be the next (first?) steps? Contacting small gamedeveloper-companies? Raise money? Get more (bitcointalk-) people together? Work on basic concepts?

Ente

As long as noone knows this "password generation scheme" (because it is the standard or someone watched me typing one key three dozen times and counting to 30) it doesnt make a worse password than another 37 random small-letter password.
You can either attack with a dictionary, permutations of words from a dictionary, or every combination through bruteforce. 30*k+bitcoin will only be found with brute-force, I would say.
Oh, but there may be funny details in your *implementation*. Like the old windows passwords as well as icq passwords would truncate after 8 letters.. ;-)

And, to add another heresy: I prefer to choose a long, strong password, and barely change it at all. Some of them I use for years now. If its only used for one login/website, I see no reason to change it at all, ever.

edit: I like that kid. Please buy it icecream and give it a hug.

Ente

However, it might be a good idea to write down how exactly the wallet was created. So the wallet could be recreated later, or even the program could be recreated to create the same wallet with the same passphrase.

Of course, the whole point is to not have a wallet, but a passphrase in memory. No idea where to write the instructions "how to recreate the wallet" in that case ;-)

Oh, I know! If its a standard, and all clients/services use the same way to create the wallet? duh!

Ente

Nice to hear you are on this as well!
Create a human-rememberable output from 256- or even 512 bit randomness? I dont believe this is doable in that way. The human-languages-keyspace would be too small or the resulting token ridiculously long. You could brute-force many random keys until you find one which has a corresponding rememberable token. But this would be no different to, for example, hash a human-rememberable token and use the hash as a pseudo-random key..

May I suggest KISS? :-)

Ente

All wrong? So you say it is not possible to calculate the "strength" of a passsphrase in bits? Easy way: calculate the bits for a brute-force case, from length and keyspace. More clever way: Let the user enter a few additional infos, like "there are three real-world words in that phrase". I am not saying this is a must or necessarily worth the effort, but I don't agree "they are all wrong" at all.


Of course "I can choose my passphrase" is not enough by itself for me to trust it. But in such a case I could (and might) skim through the sourcecode. If there is nothing hinting it transfers data through the internet as well as it seems reasonable it does what it states (i.e. use the installed openssl libraries to make hashes), I am reasonably sure. If then I dont find any warnings online, find some recommendations, and use it on a live-cd, I feel safe enough to trust it with some value.

If said program creates the passphrase (or a list of words), it doesn't have to transfer any data online. If there is a backdoor which makes the combination of the words non-random, it is close to impossible to find in the sourcecode as well as close to impossible to detect in the created passphrase. I would (and did) read through sourcecode. But I will definitely not make a statistical analysis of so-called random passphrases a program generates for me.

I will not trust any program/service/page/algo if I can not reproduce the output with a different method. Easy with pywallet and the like, I verify its working as intended with online sha256- and Hex/Base56 generators. If both ways lead to the same privatekey, I use the program in a secure (offline) way to create the privatekey I finally use for funds.


Does that mean the user chooses a password and the seed is generated from this as well, or
the user chooses a password and gets an additional passphrase from the program?

Ente

exactly. And then you are at the beginning, again, where the user is responsible to create a secure passphrase, by which means he chooses. If everyone uses the same (given) scheme to generate the passphrase, it gets substantially more insecure..

Ente

This adds almost no security.


This adds security.


I dont believe it is that bad. Display how any bits are used in the user's passphrase, maybe with an estimate of how quick it could be cracked.

I, however, would be very sceptical of a program/service where the passphrase is generated for me, and that passphrase gives immediate access to substancial value. I would either skim through the sourcecode, or drop that program right away.

Ente

Yes, I agree to "do 10000 hashes of the phrase", but only if the number is given by the standard. Make it changeagle in "expert mode", if you like. But its trivial for a computer to generate 100000 times the hash of a phrase, and stop when one of the hashes matches a known address. It would not add security, but be another thing the user has to remember.

Ente