Lyddite
Member
Offline
Activity: 98
Merit: 10


August 30, 2013, 09:56:49 AM Last edit: August 30, 2013, 09:46:44 PM by Lyddite 

FYI primecoin.org home page now has a link to the hp client download. Hope the miner icon I add is likeable I think this helps new users to find it easier. Once again let's give Mikael a big round of applause for his hard work, dedication and selfless contribution! I like it! Any reason to not only use the hp client? Even if not mining?

 Lyddite 



cabin


August 30, 2013, 01:20:25 PM 

Looking at the code, unless there is a bug in the algorithm that is say rejecting 10.5 length chains at the moment, it *should* behave linearly. Diff 9.999 should be same as 10, but but from the charts it is obviously not. I think the distribution of the Fermat's remainder is skewed. Sunny mentioned that briefly in his paper too. The skewed distribution is probably caused by the repeated exponentiation done during the Fermat's test. I wonder if this is because all the current miners are optimized to sieve for 9 chains at the moment? I wonder if the the difficulty would look more linear if miners switch to sieving for 10 chains now that most 9 chains won't solve a block. I would love for a sievelength param to be added to the build that would sieve for chains of a set length. I would do it myself and experiment but I am having some difficulty building the project on windows, and I suspect it is at least as hard as bitcoin to build.
I can build the ypool miner easily and from those experiments the sieve length is quite effective at optimizing for chains of a desired length. I'm a bit doubtful that switching the target difficulty to 10 would be beneficial at this point. It might become beneficial when difficulty is around 9.9 but I haven't really investigated that. Also it's good to remember that optimal mining parameters are probably not the same for ypool and solo mining. Ypool miners are concerned with maximizing the value of their shares. Previously that system was exploited using shorter chains. Now that the system was changed radically it could be possible to exploit it with longer chains. The building procedure for the stock Primecoin client should be the same as for Bitcoin. For my version you also need libgmp on top of that. Ah ok, if the distribution is heavily skewed, and the number of miners was increasing rapidly, that could explain some of it. Well, if the parameter is added I would try it out now even if it is too early and see what happens. I did eventually get it to build on mingw64 with the gmp 5.1.0 source. For fun I collected stats on the decimal part of the distribution of all 5+ chains, separated into 1% bins. It looks fairly flat: 66 54 53 55 52 56 44 70 68 53 53 57 54 33 53 46 61 68 66 67 55 69 59 58 39 60 73 58 51 61 56 56 53 56 66 50 72 76 58 57 55 55 61 55 62 67 53 58 46 44 59 60 53 62 39 46 47 55 55 48 69 77 47 56 75 52 65 74 41 49 67 53 74 47 49 63 55 57 47 61 59 63 45 59 54 55 61 62 48 61 55 62 63 55 70 62 50 81 55 64
Would I be correct in saying the chance to find a prime in a set that is sieved is about 0.095 and the chance of finding a prime just randomly is 1/ ln(2^255) = 0.00568 ?




mikaelh (OP)


August 30, 2013, 05:54:14 PM 

Looking at the code, unless there is a bug in the algorithm that is say rejecting 10.5 length chains at the moment, it *should* behave linearly. Diff 9.999 should be same as 10, but but from the charts it is obviously not. I think the distribution of the Fermat's remainder is skewed. Sunny mentioned that briefly in his paper too. The skewed distribution is probably caused by the repeated exponentiation done during the Fermat's test. I wonder if this is because all the current miners are optimized to sieve for 9 chains at the moment? I wonder if the the difficulty would look more linear if miners switch to sieving for 10 chains now that most 9 chains won't solve a block. I would love for a sievelength param to be added to the build that would sieve for chains of a set length. I would do it myself and experiment but I am having some difficulty building the project on windows, and I suspect it is at least as hard as bitcoin to build.
I can build the ypool miner easily and from those experiments the sieve length is quite effective at optimizing for chains of a desired length. I'm a bit doubtful that switching the target difficulty to 10 would be beneficial at this point. It might become beneficial when difficulty is around 9.9 but I haven't really investigated that. Also it's good to remember that optimal mining parameters are probably not the same for ypool and solo mining. Ypool miners are concerned with maximizing the value of their shares. Previously that system was exploited using shorter chains. Now that the system was changed radically it could be possible to exploit it with longer chains. The building procedure for the stock Primecoin client should be the same as for Bitcoin. For my version you also need libgmp on top of that. Ah ok, if the distribution is heavily skewed, and the number of miners was increasing rapidly, that could explain some of it. Well, if the parameter is added I would try it out now even if it is too early and see what happens. I did eventually get it to build on mingw64 with the gmp 5.1.0 source. For fun I collected stats on the decimal part of the distribution of all 5+ chains, separated into 1% bins. It looks fairly flat: 66 54 53 55 52 56 44 70 68 53 53 57 54 33 53 46 61 68 66 67 55 69 59 58 39 60 73 58 51 61 56 56 53 56 66 50 72 76 58 57 55 55 61 55 62 67 53 58 46 44 59 60 53 62 39 46 47 55 55 48 69 77 47 56 75 52 65 74 41 49 67 53 74 47 49 63 55 57 47 61 59 63 45 59 54 55 61 62 48 61 55 62 63 55 70 62 50 81 55 64
Would I be correct in saying the chance to find a prime in a set that is sieved is about 0.095 and the chance of finding a prime just randomly is 1/ ln(2^255) = 0.00568 ? Yeah, I actually did some testing myself too and observed similar results. The distribution of the remainder is indeed not (significantly) skewed. So I was wrong there. I originally thought that the difficulty curve looked logarithmic which led to me guessing that the distribution would be skewed. I looked at the difficulty curve again and it's not logarithmic. The amount of chains that pass the Fermat's remainder test is simply inversely proportional to the decimal part of the difficulty. That means 20% of 9chains pass at difficulty 9.8 and only 10% pass at difficulty 9.9. The probability of finding a prime around 2^255 is indeed about 1 / ln(2^255). Sieving improves the odds significantly. According to my miner's statistics, about 4.3% of candidates are primes on mainnet (7% on testnet). And SlyWax's earlier formula block rate estimate based on difficulty was indeed correct. I was wrong there too. But please remember that chains/day is not an accurate estimate of actual performance.




cabin


August 30, 2013, 07:48:13 PM Last edit: August 30, 2013, 10:04:19 PM by cabin 

Looking at the code, unless there is a bug in the algorithm that is say rejecting 10.5 length chains at the moment, it *should* behave linearly. Diff 9.999 should be same as 10, but but from the charts it is obviously not. I think the distribution of the Fermat's remainder is skewed. Sunny mentioned that briefly in his paper too. The skewed distribution is probably caused by the repeated exponentiation done during the Fermat's test. I wonder if this is because all the current miners are optimized to sieve for 9 chains at the moment? I wonder if the the difficulty would look more linear if miners switch to sieving for 10 chains now that most 9 chains won't solve a block. I would love for a sievelength param to be added to the build that would sieve for chains of a set length. I would do it myself and experiment but I am having some difficulty building the project on windows, and I suspect it is at least as hard as bitcoin to build.
I can build the ypool miner easily and from those experiments the sieve length is quite effective at optimizing for chains of a desired length. I'm a bit doubtful that switching the target difficulty to 10 would be beneficial at this point. It might become beneficial when difficulty is around 9.9 but I haven't really investigated that. Also it's good to remember that optimal mining parameters are probably not the same for ypool and solo mining. Ypool miners are concerned with maximizing the value of their shares. Previously that system was exploited using shorter chains. Now that the system was changed radically it could be possible to exploit it with longer chains. The building procedure for the stock Primecoin client should be the same as for Bitcoin. For my version you also need libgmp on top of that. Ah ok, if the distribution is heavily skewed, and the number of miners was increasing rapidly, that could explain some of it. Well, if the parameter is added I would try it out now even if it is too early and see what happens. I did eventually get it to build on mingw64 with the gmp 5.1.0 source. For fun I collected stats on the decimal part of the distribution of all 5+ chains, separated into 1% bins. It looks fairly flat: 66 54 53 55 52 56 44 70 68 53 53 57 54 33 53 46 61 68 66 67 55 69 59 58 39 60 73 58 51 61 56 56 53 56 66 50 72 76 58 57 55 55 61 55 62 67 53 58 46 44 59 60 53 62 39 46 47 55 55 48 69 77 47 56 75 52 65 74 41 49 67 53 74 47 49 63 55 57 47 61 59 63 45 59 54 55 61 62 48 61 55 62 63 55 70 62 50 81 55 64
Would I be correct in saying the chance to find a prime in a set that is sieved is about 0.095 and the chance of finding a prime just randomly is 1/ ln(2^255) = 0.00568 ? Yeah, I actually did some testing myself too and observed similar results. The distribution of the remainder is indeed not (significantly) skewed. So I was wrong there. I originally thought that the difficulty curve looked logarithmic which led to me guessing that the distribution would be skewed. I looked at the difficulty curve again and it's not logarithmic. The amount of chains that pass the Fermat's remainder test is simply inversely proportional to the decimal part of the difficulty. That means 20% of 9chains pass at difficulty 9.8 and only 10% pass at difficulty 9.9. The probability of finding a prime around 2^255 is indeed about 1 / ln(2^255). Sieving improves the odds significantly. According to my miner's statistics, about 4.3% of candidates are primes on mainnet (7% on testnet). And SlyWax's earlier formula block rate estimate based on difficulty was indeed correct. I was wrong there too. But please remember that chains/day is not an accurate estimate of actual performance. Ah ok, so EstimateCandidatePrimeProbability() returns around 9%, but in reality it is more like 4% when you actually divide primes found by the number of tests. Interesting. My calculations also showed it would be a terrible idea to start search for 10 chains already. Edit: actually the number of tests isn't a good stat for this.. it is the number of chains tested and is incremented even when the sieve is empty and no prime tests are done. I ran some numbers on real tests at each chain length, and it looks like it will always the same, around 0.094 of candidates are prime. I'll run it for longer though to get better results for the longer chains. 0.09442 0.094065 0.094624 0.094806 0.048345 0.099099 0.18182 0 1.#IND 1.#IND 1.#IND 1.#IND 1.#IND 1.#IND 1.#IND 1.#IND 1.#IND 1.#IND 1.#IND 1.#IND This would also make a good block estimator: double nineFraction = (currently around 0.17) double blocksPerDay = 24*(d7ChainsPerHour*0.094^2*nineFraction + d7ChainsPerHour*0.094^3);




wibtc
Newbie
Offline
Activity: 32
Merit: 0


August 31, 2013, 12:03:53 AM 

hp10 ubuntu x64 build has the following error everyday. Anybody has the same problem?
primecoind: checkqueue.h:171: CCheckQueueControl<T>::CCheckQueueControl(CCheckQueue<T>*) [with T = CScriptCheck]: Assertion `pqueue>nTotal == pqueue>nIdle' failed.




mhps


August 31, 2013, 03:06:04 AM 

This would also make a good block estimator: double nineFraction = (currently around 0.17) double blocksPerDay = 24*(d7ChainsPerHour*0.094^2*nineFraction + d7ChainsPerHour*0.094^3);
Perhaps either ignore the high order terms (the method I use) or sum up to infinity which gives in a cleaner form of 24*d7ChainsPerHour*0.094^2/(1nineFraction) However my observation is that slower machines nest an unproportional less number of chains that are longer than 7.




SlyWax


August 31, 2013, 05:07:03 AM 

hp10 ubuntu x64 build has the following error everyday. Anybody has the same problem?
primecoind: checkqueue.h:171: CCheckQueueControl<T>::CCheckQueueControl(CCheckQueue<T>*) [with T = CScriptCheck]: Assertion `pqueue>nTotal == pqueue>nIdle' failed.
Yes same error, same OS.




patapato
Member
Offline
Activity: 93
Merit: 10


August 31, 2013, 10:20:02 AM 

For fun I collected stats on the decimal part of the distribution of all 5+ chains, separated into 1% bins. It looks fairly flat: 66 54 53 55 52 56 44 70 68 53 53 57 54 33 53 46 61 68 66 67 55 69 59 58 39 60 73 58 51 61 56 56 53 56 66 50 72 76 58 57 55 55 61 55 62 67 53 58 46 44 59 60 53 62 39 46 47 55 55 48 69 77 47 56 75 52 65 74 41 49 67 53 74 47 49 63 55 57 47 61 59 63 45 59 54 55 61 62 48 61 55 62 63 55 70 62 50 81 55 64
mean is 57.54 median is 56 standard deviation (sigma) is 8.94 skewness is 25.13 = third moment / sigma^3 = sum( ((sample  mean)/sigma)^3 ) / total samples nonparametric skewness is 0.17 = (mean  median) / sigma I don't know if it is much or few skewness for our problem, just I hope that it could help to think.




mikaelh (OP)


August 31, 2013, 10:33:05 AM 

Ah ok, so EstimateCandidatePrimeProbability() returns around 9%, but in reality it is more like 4% when you actually divide primes found by the number of tests. Interesting. My calculations also showed it would be a terrible idea to start search for 10 chains already.
Edit: actually the number of tests isn't a good stat for this.. it is the number of chains tested and is incremented even when the sieve is empty and no prime tests are done. I ran some numbers on real tests at each chain length, and it looks like it will always the same, around 0.094 of candidates are prime. I'll run it for longer though to get better results for the longer chains.
0.09442 0.094065 0.094624 0.094806 0.048345 0.099099 0.18182 0 1.#IND 1.#IND 1.#IND 1.#IND 1.#IND 1.#IND 1.#IND 1.#IND 1.#IND 1.#IND 1.#IND 1.#IND
This would also make a good block estimator: double nineFraction = (currently around 0.17) double blocksPerDay = 24*(d7ChainsPerHour*0.094^2*nineFraction + d7ChainsPerHour*0.094^3);
Good work. There is indeed a small bug with the number of tests. It's not very significant since every sieve should contain at least dozens of candidates. I tried skipping the BiTwin candidates and then I'm getting the following stats on testnet: 20130831 10:25:06 primemeter 51090434 prime/h 514555696 test/h 4380 5chains/h 11031.065686 chain/d So the probability is about 0.0992. The estimated probability is also around 0.1 which matches with the stats. The theoretical probability goes down slightly for the subsequent numbers in a chain.




cabin


August 31, 2013, 12:24:36 PM 

So the probability is about 0.0992. The estimated probability is also around 0.1 which matches with the stats. The theoretical probability goes down slightly for the subsequent numbers in a chain.
That all makes sense. But I do see one weird thing.. even after 10 hours, the probability of finding the 5th prime in a chain remains abnormally low. For now I'll assume it is a bug in my code.. but it is weird. chains 1 to 10: 0.094342 0.094123 0.093681 0.093306 0.055778 0.079268 0.15385 0 1.#IND 1.#IND
the code that generates this is around line 650 here: https://github.com/rsweny/primecoin/blob/master/src/prime.cppI suspect there are thread issues with this but I'm not sure why it would only skew the 5's.




mikaelh (OP)


August 31, 2013, 01:54:37 PM 

So the probability is about 0.0992. The estimated probability is also around 0.1 which matches with the stats. The theoretical probability goes down slightly for the subsequent numbers in a chain.
That all makes sense. But I do see one weird thing.. even after 10 hours, the probability of finding the 5th prime in a chain remains abnormally low. For now I'll assume it is a bug in my code.. but it is weird. chains 1 to 10: 0.094342 0.094123 0.093681 0.093306 0.055778 0.079268 0.15385 0 1.#IND 1.#IND
the code that generates this is around line 650 here: https://github.com/rsweny/primecoin/blob/master/src/prime.cppI suspect there are thread issues with this but I'm not sure why it would only skew the 5's. I'm guessing you're seeing the effect of BiTwin chains in your fifth bin. The test for the tenth prime in a BiTwin will succeed with a much lower probability. The tenth prime of a BiTwin chain corresponds to the fifth prime of a 2CC chain. And that's why you are seeing a lower rate of primes in the fifth bin.




Krusher33
Newbie
Offline
Activity: 37
Merit: 0


August 31, 2013, 02:18:39 PM 

hp10 ubuntu x64 build has the following error everyday. Anybody has the same problem?
primecoind: checkqueue.h:171: CCheckQueueControl<T>::CCheckQueueControl(CCheckQueue<T>*) [with T = CScriptCheck]: Assertion `pqueue>nTotal == pqueue>nIdle' failed.
I'm using Ubuntu 13.04 64bit and haven't seen this yet.




ReCat


August 31, 2013, 02:23:26 PM 

Well, I've had a primecoin client running on my personallyowned rackserver that I have in a datacenter(In washington), and it's been a proper 24 hours now. The client hasn't synchronized. It's Win Server 08R2, as well as a precompiled build.
Do I need to manually add some peers, or something? Although I also noticed that it was giving that error that everyone else here reports. So I restarted it and it seems to be doing better now. (few hours to sync)
Also, I port forwarded the port 9911 for PrimeCoin. Is there anything else I can do to make my client be of more use to the network? I heard something about super nodes but I'm not sure how that works.
Cheers.

BTC: 1recatirpHBjR9sxgabB3RDtM6TgntYUW Hold onto what you love with all your might, Because you can never know when  Oh. What you love is now gone.



mechs


August 31, 2013, 08:04:25 PM 

Well, I've had a primecoin client running on my personallyowned rackserver that I have in a datacenter(In washington), and it's been a proper 24 hours now. The client hasn't synchronized. It's Win Server 08R2, as well as a precompiled build.
Do I need to manually add some peers, or something? Although I also noticed that it was giving that error that everyone else here reports. So I restarted it and it seems to be doing better now. (few hours to sync)
Also, I port forwarded the port 9911 for PrimeCoin. Is there anything else I can do to make my client be of more use to the network? I heard something about super nodes but I'm not sure how that works.
Cheers.
The Linux versions seem to be more stable.




crendore


August 31, 2013, 08:07:00 PM 

The Linux versions seem to be more stable.
+1 don't know why people even bother with windows when it comes to rack mounted servers. Shows a true level of ignorance of what a server actually does. "oh well i use windows on my desktop, so i better use windows on my server". lol.




mechs


August 31, 2013, 08:17:43 PM 

The Linux versions seem to be more stable.
+1 don't know why people even bother with windows when it comes to rack mounted servers. Shows a true level of ignorance of what a server actually does. "oh well i use windows on my desktop, so i better use windows on my server". lol. I think their may be a place for a Windows server for certain applications such as web hosting. But I agree, Linux is usually superior




ReCat


September 01, 2013, 12:04:50 AM 

The Linux versions seem to be more stable.
+1 don't know why people even bother with windows when it comes to rack mounted servers. Shows a true level of ignorance of what a server actually does. "oh well i use windows on my desktop, so i better use windows on my server". lol. We are running Windows Server 2008 R2 because we need to run some legacy Windows Server applications. There is a large amount of software that is windowsonly. If it were up to me, I would have chosen linux. Anyways. That was a oneoff bug. Right after writing the post, client synced fully, and I am now mining properly.

BTC: 1recatirpHBjR9sxgabB3RDtM6TgntYUW Hold onto what you love with all your might, Because you can never know when  Oh. What you love is now gone.



Sy
Legendary
Offline
Activity: 1484
Merit: 1003
Bounty Detective


September 03, 2013, 06:54:28 AM 

So  how are all the amazon and vps user doing? Still profitable?




Trillium


September 03, 2013, 06:57:17 AM 

So  how are all the amazon and vps user doing? Still profitable?
Not likely at current exchange rates and difficulty..

BTC:1AaaAAAAaAAE2L1PXM1x9VDNqvcrfa9He6



cryptrol


September 03, 2013, 08:54:14 AM 

After the gpu miner apparent failure, it seems that difficulty is kinda stagnated. "blocks" : 147833, "difficulty" : 9.85310471, "blocks" : 147834, "difficulty" : 9.85307074, "blocks" : 147837, "difficulty" : 9.85310775, "blocks" : 147838, "difficulty" : 9.85310000, "blocks" : 147839, "difficulty" : 9.85312480, "blocks" : 147841, "difficulty" : 9.85307729, "blocks" : 147843, "difficulty" : 9.85294372,
Even getting lower.




