Just got back in, so I agree primarily with everything you say, and I recall going back to the original miner, I used the x11 hash value as an input for the original biblehash, because it was a reliable strong distinct blockhash; my fear was, with business logic going into the biblehash itself (as we raise the bar), there are conditions where the biblehash could possibly return the same value for two blocks (or worse, a faulty hash if someone was not synced), and I wanted to make sure the blockindex would stay safe. Otherwise, I would have completely switched out the x11 algo with the biblehash. Im glad we did, so now we can put some business logic in the PoBH area (like the tithe rule and tx lookups).
Anyway, yes, Im all for removing the x11 from the POW.
Btw, whoever compiled the latest and is trying to test with me in testnet, please do a get on the 1021c; the version had two required changes and I just checked it in.
So, now our newest version eliminates x11 from the miner in testnet after block 1350, and keeps x11 for blockindex references, and uses 100% of the PoBh for POW in the miner.
(This way, we wont risk breaking the blockchain storage, txlookups, and all the places that rely on a block->GetHash. Yet the security is still there because the POW of each block is protected by the biblehash output).
Lets see how this behaves in testnet, the new version is out (for testnet only). Re-building windows now.
Hmm, I think something's wrong in the latest commit. When you enable mining, biblepay_generate is set to true but hashps is 0 and there's no CPU load. Issue is present for both pool and solo mining. Hmm, I retested it and it seems to be mining for me in test & prod. Can u please ensure you have pulled the head (835*) 1021c? On a side note windows is now compiled. I will test it and post it only here if it works and then we can test it in testnet with the new anti-x11 feature. |
|
|
|
I like your idea, about adding the 12th element, the biblehash directly to the x11 algo. But for now, I have already created a testnet version where we lower the X11 component by 300%.
On a side note, I am potentially setting the mandatory for block 7000, to give time for ccex. (And also, in case we can squeeze in the txindexlookup system in PoBh to raise the bar even further before block 7000 hits).
The testnet version should be ready by block 1350 - but Windows will take the rest of the day to compile.
In the next testnet version, we lower the x11 difficulty by 300% as of block 1350, thereby putting the onus on PoBh.
Im testing this now; will let you know when linux is ready as Ill need help testing this in testnet.
That's some fast work  Looking forward to testing this, I'm hoping it'll solve a lot of the issues we've been experiencing. I was thinking a little more on the possibility of a hybrid GPU miner. Even if all the difficulty was loaded onto the Biblehash algorithm, if the GPU mined a batch of X11 solutions, sent those to the CPU to be processed, then mined the next batch while the CPU was working, it might still provide an advantage? I'm very interested in seeing some metrics on X11 and Biblehash. If we assume that after the update there's exactly one X11 hash for every Biblehash iteration, and Biblehash requires 1000x as many cycles as X11, you'd get at most a 0.1% performance gain (realistically less, reading back from a GPU is slow). Obviously, in that case there'd be absolutely no point in doing this and you'd be better off CPU mining Biblepay and using your GPU for something else. If, however, the metrics aren't quite that lopsided, it might be a different story. Of course, it might start begging the question at some point about why include X11 if it contributes so little to overall mining process. I think it still makes sense, though, since it adds a lot of complexity to any attempt at creating a Biblepay GPU or ASIC miner beyond the obstacles posed by Biblehash itself. ASICs especially, since complexity is directly tied to manufacturing cost. Just got back in, so I agree primarily with everything you say, and I recall going back to the original miner, I used the x11 hash value as an input for the original biblehash, because it was a reliable strong distinct blockhash; my fear was, with business logic going into the biblehash itself (as we raise the bar), there are conditions where the biblehash could possibly return the same value for two blocks (or worse, a faulty hash if someone was not synced), and I wanted to make sure the blockindex would stay safe. Otherwise, I would have completely switched out the x11 algo with the biblehash. Im glad we did, so now we can put some business logic in the PoBH area (like the tithe rule and tx lookups). Anyway, yes, Im all for removing the x11 from the POW. Btw, whoever compiled the latest and is trying to test with me in testnet, please do a get on the 1021c; the version had two required changes and I just checked it in. So, now our newest version eliminates x11 from the miner in testnet after block 1350, and keeps x11 for blockindex references, and uses 100% of the PoBh for POW in the miner. (This way, we wont risk breaking the blockchain storage, txlookups, and all the places that rely on a block->GetHash. Yet the security is still there because the POW of each block is protected by the biblehash output). Lets see how this behaves in testnet, the new version is out (for testnet only). Re-building windows now. |
|
|
|
What is the difference between "Hashes Per Sec" vs "Hashes Per Sec2" in the leaderboard? Hashespersec is what the box is reporting for its speed, hashpersec2 is what the pool is measuring; payments are based on hashpersec2. |
|
|
|
log1k, Ive heard there are 550+ miners, network hash is super high now, with 700k hps I think youll get 1 block a day (my understanding is that current network hash is 100x higher than it currently shows) theres been a lot of push for dev to make a hash pool because solo mining has gotten so hard, the pool is mostly working now  Sorry, should've stated that i'm on the pool - still havent received a payout. Has been at least 15 blocks movement. Log1, did you find yourself in the block distribution history? I dont see a user named log1, if not what is your username or minername? |
|
|
|
|
Ok guys this is just a notice for a testnet release of the next upcoming version 1021:
1.0.2.1 - Non-Mandatory Release
- For upcoming mandatories rule @1350(testnet)||@7000(Prod), Testnet features: test anti-block-clump retarget rule, slow_block_threshhold = 30 mins rule,
and lowering x11 difficulty by 300%, and ratio of x11:biblehashes in getmininginfo
- In wallet Bible Reader (Help | Read Bible). NOTE: You currently must choose the Book after the page loads.
- Fixes for in-wallet pool-miner (fixes diminishing KHS)
- adjustment to getnetworkhasps
Note that Windows will take another 6 hours to compile and I will be gone, so that wont be out til tomorrow morning.
In the mean time heres what I need help on: I cant mine the next block in testnet on my linux rig as we left the diff too high. So if anyone wants to get the latest from github on linux and jump on testnet it would be appreciated.
What I am trying to test with you guys next is bullet item #1. At block 1350, we start decreasing the diff of the X11 hash by 300%. I have a couple new features in there that will show how much % of the onus is on the biblehash as of the new version, Ill explain where that is once we mine a few new blocks.
Tomorrow if this version mines, we can release the windows version and that version allows the pool to work properly (without the diminishing HPS).
|
|
|
|
at any rate i seem to be ruining it for everyone else, will drop the vps' off now.. additionally i seem to have discovered a slow memleak in the daemon
I raised the IP ban threshold by 400% if you want to try it again; threadlimit is still 24 as we still need that in. |
|
|
|
I now want to add: tackle the diff readjustment per algo, and remove the difficulty of the x11, another words, we should put All of the onus on solving the block basing it off of the PoBh difficulty level only. I believe we can accomplish this by lowering the difficulty of X11 down to miniscule amount, and testing this in testnet.
Sounds like a sensible solution. It'd essentially be like adding a 12th algorithm to X11 and would simplify difficulty retargeting and network hash calculations. That'd also eliminate any risk of a hybrid GPU miner, since the overhead of reading back from the GPU every single iteration would far outweigh just performing a low difficulty X11 on the CPU. I was looking through the DakGravityWave difficulty readjustment algo, adjusting that to try to balance two difficulties for two hashes would be messy. It'd be highly dependent on metrics on both hashing algos and end up being really fragile if anything changed in either of them. Always best to go simple when possible. I like your idea, about adding the 12th element, the biblehash directly to the x11 algo. But for now, I have already created a testnet version where we lower the X11 component by 300%. On a side note, I am potentially setting the mandatory for block 7000, to give time for ccex. (And also, in case we can squeeze in the txindexlookup system in PoBh to raise the bar even further before block 7000 hits). The testnet version should be ready by block 1350 - but Windows will take the rest of the day to compile. In the next testnet version, we lower the x11 difficulty by 300% as of block 1350, thereby putting the onus on PoBh. Im testing this now; will let you know when linux is ready as Ill need help testing this in testnet. |
|
|
|
i've 6 separate vps, each identifying as a separate worker (ye1, ye2, ye3, ye4, ye5 & ye6)
initially i thought the workerid was main username - so i limited each vps to 4 threads only (6x4=24).
still getting banned?
or is it 24 threads per workerid?
seems to flick on/off pool mining..
"poolinfo1": "B9LeH2D3pxTHcssNhgBqseJekonk1q36w8; B9LeH2D3pxTHcssNhgBqseJekonk1q36w8; B9LeH2D3pxTHcssNhgBqseJekonk1q36w8; ",
"poolinfo2": "POOL DOWN-REVERTING TO SOLO MINING; POOL DOWN-REVERTING TO SOLO MINING; POOL DOWN-REVERTING TO SOLO MINING; POOL DOWN-REVERTING TO SOLO MINING; POOL DOWN-REVERTING TO SOLO MINING; ",
"poolinfo3": "MINER IP BANNED; MINER IP BANNED; MINER IP BANNED; MINER IP BANNED; MINER IP BANNED; ",
Yeah, we might still have a limit of 100 hits per IP per minute also; try temporarily lowering the threads for a couple days and once the new version comes out we can raise the limit per IP per minute. |
|
|
|
We need more exchange platform not just c-cex
It would be nice eventually-- but --its also nice to be a hidden diamond and ensure all of our kinks are worked out and let everyone buy us cheap for a while. |
|
|
|
I agree with all this, except that I think the pool is receiving the correct amount of blocks for the userbases network hash level, as I really feel that only 5% of our base came over. Since we have not actually advertised it yet as ready for prod, I think 95% is still solo mining, but then I could be wrong about that.
Yeah, that's what I meant. The pool was finding too few blocks relative to the network hash rate, meaning that the network hash rate must be wrong. Network hash rate approximations can be wrong, but everyone is mining against the same algorithm so the rate at which you're finding blocks over a statistically relevant period of time is the most objective reference. A little off-topic, but about the time spent on X11 vs time spent on Biblehash - since X11 comes first and is independent of Biblehash, do you think it's possible someone could build a hybrid GPU miner? Using a GPU to find X11 solutions, and simultaneously feeding those into the CPU to run Biblehash on? Whether there'd any benefit to that would likely depend on where the CPU is spending the majority of its time, I suppose. Since it sounds like Biblehash is the bottleneck it's probably not a concern, but if it turns out that the opposite is true then it could be an issue. Also, it looks like the X11 hash and the Biblehash hash are being compared against the same hashtarget value in the miner code. Do you know if the difficulty readjustment takes this into consideration? This is way outside of my field of knowledge, so maybe I'm thinking about this wrong, but imagine a scenario like this: >1 out of 100 hashes are solutions at difficulty X >0.01 probability of finding an X11 solution >0.0001 probability of finding an X11 solution that also produces a Biblehash solution >1 out of 200 hashes are solutions at difficulty 2X >0.005 probability of finding an X11 solution >0.000025 probability of finding an X11 solution that also produces a Biblehash solution So, the difficulty adjustment system might have expected this to double the block time, but doubling the difficulty actually increased it by 4 times. Just for the record, I haven't looked at the code related to calculating difficulty at all, I'm just wondering. -edit- Block time probably isn't a good term to use there, I mean doubling the difficulty so the block time would remain consistent after readjusting for a doubled network hash rate. Yeah, you must be reading my mind. Good finds/observations. Well, this is sort of an eye opening experience actually. The difficulty readjustment does not take into account independent adjustments for each algorithm. I agree, with the test harness metrics, someone with a lot of time on their hands could offload the X11 hash (as of when we released the wallet with the faster hashing speed and the outer X11 loop). I mentioned two things in the next mandatory that would make the wallet smoother: preventing the block clumping, (thats a constant retarget), and lowering the stuck block threshhold. I now want to add: tackle the diff readjustment per algo, and remove the difficulty of the x11, another words, we should put All of the onus on solving the block basing it off of the PoBh difficulty level only. I believe we can accomplish this by lowering the difficulty of X11 down to miniscule amount, and testing this in testnet. I would consider this an emergency, but we need to test thoroughly, and we dont want to tick off ccex. We need to give them at least a 2 week notice. Ill get to work on a new testnet version for us with these mandatories features to be ready by Friday for testing, then on the weekened we can announce an upgrade to them for about 14 days later. In the mean time, I believe the pool problem will be fixed, it passed UAT and now I need to compile the windows wallet. |
|
|
|
There seems to be an issue left where the pool is not receiving every solution.
This is why ocasionally, you see a miner lagging behind further and further and finally diminishing to 0 and falling off leaderboard.
This can currently be solved by restarting the node (not doing a setgenerate X). This is because the pool creates new share guids when you do that and then your hashps slowly rises back to its value. This appears to only be happening to about 10% of the users.
So, today I will work on fixing that and implementing the metric to allow us to see the ratio of x11:biblehashes.
On a side note, I have been working on adding the ability to read the bible from the wallet. We already have the entire bible compiled in the wallet, so I know we need this feature inside the wallet for the GUI.
(You can already read a verse from the RPC with 'run readverse booknumber chapternumber versenumber'.)
I really love the 'proof of bible', because of both the theoretical concept and the specific content. I can also envision why this would create the need for calculating the network_khashps in a different way. I hope you can find a way I also had a miner lagging behind and eventually dropping off (sjappie_miner). It was only mining at around 20khashps, maybe that's why it was more likely to drop out? I think this was the second time it dropped of the leaderboard. The first time the miner reappeared on it's own as far as I know, but now it's been off the leaderboard for quite some time. As far as I understand, this has to do with the 'Hashes Per Sec2', right? Can anyone please explain to me what this number means? Does it have to do with the ability of the miner to report it's 'proof of work' to the pool? A lot of the time, the 'Hashes Per Sec2' from my miners is around half of the 'Hashes Per Sec' (one miners is at around 20% right now in terms of 'Hashes Per Sec' vs 'Hashes Per Sec2'. Sadly, I don't have access to my miners right now, so I can't restart the nodes. So, you think this problems has it's source in the pool or in the individual miners? Thanks, I hope in general people like the project and then interest will grow and we become a household name. So I think to answer this question we can do a live test. I have one miner bible_pay 'pc' showing right now at 10% in hashespersec2. Rebooting the node (its been mining all night). If the hashespersec2 increases to 90% of the HPS, then we know the answer will be not the pool and in the minerthread report to pool for stale share logic. If you want to restart your slow node right now we can do a side by side test. |
|
|
|
Can only use 24 thread in pool
Yes, for now be sure to use 24 threads or less in the pool. During all the debugging I had to limit it as I thought we were ddossing ourselves. We can work on that later once we solve this diminishing KHPS issue. Is this limit per worker or per user? Per workerid. |
|
|
|
Whats interesting about that is I had that problem this morning, so I bounced it and was able to get in. However when You had that problem I was able to log in. Ill put some logging in there and see if we can fix that. So if anyone cant login to the pool, click logout a few times and then login and then logout-login-login etc. I have logging in that area now. |
|
|
|
Can only use 24 thread in pool Yes, for now be sure to use 24 threads or less in the pool. During all the debugging I had to limit it as I thought we were ddossing ourselves. We can work on that later once we solve this diminishing KHPS issue. |
|
|
|
Whats interesting about that is I had that problem this morning, so I bounced it and was able to get in. However when You had that problem I was able to log in. Ill put some logging in there and see if we can fix that. |
|
|
|
|
So about the pool in general:
Im fairly happy overall about most of it and slightly dissapointed about a small part of it.
On the bright side, its not the server that is getting overloaded, looking at all the logs I think one server can handle the pool (although in the long term, it will be nice to have more than one pool), I can see that every time a miner is ready for more work (IE they send the readytomine packet), we receive and send 100% of the time, and have no issues.
There seems to be an issue left where the pool is not receiving every solution.
This is why ocasionally, you see a miner lagging behind further and further and finally diminishing to 0 and falling off leaderboard.
This can currently be solved by restarting the node (not doing a setgenerate X). This is because the pool creates new share guids when you do that and then your hashps slowly rises back to its value. This appears to only be happening to about 10% of the users.
So, today I will work on fixing that and implementing the metric to allow us to see the ratio of x11:biblehashes.
On a side note, I have been working on adding the ability to read the bible from the wallet. We already have the entire bible compiled in the wallet, so I know we need this feature inside the wallet for the GUI.
(You can already read a verse from the RPC with 'run readverse booknumber chapternumber versenumber'.)
|
|
|
|
Haven't had any issues with the pool overnight. First block reward should be maturing soon and we'll see how withdrawals go.
The rate at which we're getting blocks seems a little low. Summing the 'hashes per sec' column on the leaderboard is currently giving 8,257,214 H/s. That's nearly 63% of the network hash (going by 1.0.1.9's network_khashps value of 13,142,217 H/s). If you go based on HPS from the block distribution reward table, it's about 6,216,672 H/s, or ~47% of the network hash as of block 3225.
I kind of suspect that the reported network hash rate is significantly under-reporting. I really doubt 47-63% the network's hash power has jumped into a test phase pool. That also seems a little more centralized than I'd expect from a CPU-mined coin, with 24 users running about 80 nodes contributing 1/2 to 2/3 of the network hash when even a low end machine should be putting out ~50 KH/s (0.4% of the reported network hashrate).
I agree with all this, except that I think the pool is receiving the correct amount of blocks for the userbases network hash level, as I really feel that only 5% of our base came over. Since we have not actually advertised it yet as ready for prod, I think 95% is still solo mining, but then I could be wrong about that. If 5% really did come over, the wallet hash rate would be about 20* higher than the sum of the pool. (You sum to 8.2 Mh/s). That would put us at 165,144,280 or 165Mh/s total network (8257214*20). The old getnetworkhashps (btw that command still works) bears 12,882 currently in prod - if we multiply * 10000, we arrive at 128,820,000 which would be closer to what I expect. Looking at the code for networkhashps, it just subtracts the old chainwork (from the beginning of the sample period) from the latest chainwork (from the end of the sample period) and does some averaging to arrive at how many hashes it should take to do that much chainwork. What is probably happening is the PoBh algorithm is much harder to solve than the standard sha256 or x11 hash. Apparently 1000* harder. I believe to solve this, we will need to allow more of the network to jump in the pool and get a feel for how this thing goes, and then release a fix for networkhashps. If all goes well we will just multiply it * 10000, but in the mean time, let me look at the actual work done in the loop by the biblehash vs the x11 loop. Its leading me to believe that a CPU must perform 1000 iterations of the loop to hash one bible hash as compared to one x11 hash. That actually sounds relatively feasible. EDIT: I have an idea that might help shed light. I can put a counter metric inside our PoBh mining loop that will shed light on how many x11 hashes it takes to hash one bible hash, and that should clarify this a great degree. |
|
|
|
I have 3 laptops running the miner. Normally I find at least one block every day. Last Thursday I found 5 - woohoo. Not a lot of horse power, but getting the job done. I didn't find any blocks Sunday, so I guess it was a day of rest for my wallets.
Love BiblePay!!! Great concept. Dev is doing a great job.
Wow  you are very lucky to find 5 blocks with current difficulty. So, you just mentioned no block on Sunday, do you mean that you found blocks on other days besides Thursday and Sunday? I normally find a block or 2 a day on average. My 3 laptops get about 300K hash total. I am receiving a lot of questions about the reported total hash rate and questions about who is mining all the blocks. I dont want to jump to any conclusions, my primary logical explanation is still that without the pool, we have 550 connections to the node.biblepay.org node, and with the pool, we have about 25 of those miners who have solved 6 blocks in 12 hours. That leads me to believe that we currently have 5% of our userbase in the test pool (running against prod) (550nodes/25testminers=.05%). 5% of our daily block emission (205*.05) = 10.2 blocks per day. They solved 6 in the pool in 1/2 a day, which sounds right. But I agree, that would mean our networkhashps reading in the wallet is too low. Coinforge, btw, what genproclimit setting do you use on your laptops? I just want to ensure multithreading is not hurting our ability to solve blocks. |
|
|
|
|
We can probably ask RandomMiner how many blocks per day he normally averages since he is bringing 1.18 MHS to the pool, it will be an interesting comparison.
|
|
|
|
|
Show Posts
