[ANN][RIC] Riecoin: constellations POW *CPU* HARD FORK successful, world record

[PttnMe]

Great clo1, thank you very much for helping the project and providing your work.

On Debian 9 (Sid), I have many Boost/Bignum errors while compiling. Do you have any idea why this is happening? I installed libboost-all-dev. Errors look like

Code:

./bignum.h:57:24: error: invalid use of incomplete type ‘BIGNUM’ {aka ‘struct bignum_st’}
/usr/include/openssl/ossl_typ.h:80:16: note: forward declaration of ‘BIGNUM’ {aka ‘struct bignum_st’} typedef struct bignum_st BIGNUM;
./bignum.h:62:9: error: ‘BN_init’ was not declared in this scope
./bignum.h:68:30: error: cannot convert ‘CBigNum*’ to ‘BIGNUM*’ {aka ‘bignum_st*’}
/usr/include/openssl/bn.h:219:25: note:   initializing argument 1 of ‘BIGNUM* BN_copy(BIGNUM*, const BIGNUM*)’
The last 2 lines multiple times

On Debian 9 stable, there is no Bignum error, but I get:

Code:

test/test_riecoin.cpp: In member function ‘CBlock TestChain100Setup::CreateAndProcessBlock(const std::vector<CMutableTransaction>&, const CScript&)’:
test/test_riecoin.cpp:151:84: error: could not convert ‘(& chainparams)->CChainParams::GetConsensus()’ from ‘const Consensus::Params’ to ‘offsetType {aka arith_uint256}’
     while (!CheckProofOfWork(block.GetHash(), block.nBits, chainparams.GetConsensus())) ++block.nNonce;
~~~~~~~~~~~~~~~~~~~~~~~~^~
test/test_riecoin.cpp:151:97: error: ‘class CBlock’ has no member named ‘nNonce’
     while (!CheckProofOfWork(block.GetHash(), block.nBits, chainparams.GetConsensus())) ++block.nNonce;

I am however able to compile the latest Bitcoin Core in both computers, without any issue.

I guess that for now, I will need to install Ubuntu 16.04 to test your code.

Getwork is no longer supported in bitcoin. Pttn, it would be awesome if you could add GetBlockTemplate support to rieMiner. I’ve temporarily added getwork support to the code for testing although the format is slightly different. It takes an address as an input.

Wallet mining was removed from bitcoin several versions ago. I recently noticed there is some mining code for testing purposes (generateblocks and generatetoaddress). I haven’t implemented this. Is this needed and if so what code should be used here? I know the code currently in the core is outdated.

GetBlockTemplate support was already added a few days ago and blocks were successfully mined with it (also works if it contains transactions). Example: Block 947897 with transaction fees. As soon as I am able to compile your code, I will test rieMiner with your update to make sure that rieMiner still works, and adapt the code if needed. I will see what it needed to change when the BIPs are activated. And if I have more time, I will test further your code.

Stratum support for pooled mining, at least experimental, will come today as well. Miners are welcome to test the code and submit bugs/open GitHub issues.
Once everything is finished, fixed and tested, we will be able to upgrade xptMiner as well. rieMiner provides a much more developer friendly code for people wanting to improve the algorithm.

[PttnMe]

rieMiner 0.9α3 (2018-09-09) - Stratum support added ! You can now solo + pool mine (although Stratum support is still experimental)

Testing, feedbacks, or even contributions are welcome to fix remaining bugs and improve the miner.

The miner is working for RiePool and XPoolX. However, there is a bug that produces many duplicate shares for powerful CPUs. This problem is present if many threads are used, but seems to disappear if mining with just 4 threads or less (and it seems possible to launch 4 instances using 4 threads for a 16 threads CPU without issue). I will investigate this.

I do not have an uBlock.it account, so I was not able to test for this pool.

Pools owners are also invited to test the miner and make suggestions or contributions.

rieMiner's GitHub repository, issues
rieMiner's thread on Riecoin-Community.com forum

[clo1]

Thanks for testing this.

If you got to the test units there is a good chance you already built riecoind and possibly also riecoin-qt.

The test units have not been updated and won't compile. To eliminate the errors you need to re-configure.

./configure --disable-tests
make

I'm not sure why it didn't work on debian sid. My plan for this week is to install ubuntu 18.04 and try to build for windows. I'll see if I get a similar error.

The binaries are over 100 MB so they can't be placed on github. Maybe we can put them on the community site.

I'll take a look at GetBlockTemplate a little later.

[hopguide]

Awesome Clo1 and PttnMe! I am looking forward to your stable releases! Keep the good work up!

Who is the idiot who dumped at 50 while 2 devs are releasing great advancements?

[clo1]

The code builds on both my systems - 16.04 and 17.10.

PttnMe, can you tell me what version of OpenSSL you are using on Debian 9 (Sid)? I'm using 1.02g. Were all your errors related to bignum on Debian 9 Sid?

Bitcoin has completely removed bignum and replaced it with a custom class (arith_uint256). I used this whenever possible but couldn't use it for the prime validation code. My first thought was to use GMP, but I ended up staying with bignum so I could use the exact same validation code as in the original. I'm a little concerned now that we might have compatibility problems later on if we stick with bignum. Does anyone have any thoughts on this?

[czakris]

The miner is working for RiePool and XPoolX. However, there is a bug that produces many duplicate shares for powerful CPUs. This problem is present if many threads are used, but seems to disappear if mining with just 4 threads or less (and it seems possible to launch 4 instances using 4 threads for a 16 threads CPU without issue). I will investigate this.

AFAIR it can be something similar to that issue when you try to generate random numbers using loop having only one variable to store them you will get same numbers changing every second. Every CPU thread needs unique variable. I think to fix this you need to declare as many variables as possible at the very beginning of your code and then limit them to what you need.

[PttnMe]

The test units have not been updated and won't compile. To eliminate the errors you need to re-configure.

./configure --disable-tests
make

I'm not sure why it didn't work on debian sid. My plan for this week is to install ubuntu 18.04 and try to build for windows. I'll see if I get a similar error.

Oh, sorry, I missed this part of your text. So I was able to compile your wallet after disabling the testing part on Debian 9 stable, and execute it on Debian 9 Sid. It is so great to run this version of Riecoin, you did a fantastic job, and I wish you good luck for the bounties! I am also using libdb 5.3.

I am currently syncing. This is really slow for a network with very few transactions, but this seems more a CPU bottleneck than a network one, as one core is used at 100% during sync.
Do you think that the POW validation could be optimized like the miner was, in order to sync much faster?

PttnMe, can you tell me what version of OpenSSL you are using on Debian 9 (Sid)? I'm using 1.02g. Were all your errors related to bignum on Debian 9 Sid?

Bitcoin has completely removed bignum and replaced it with a custom class (arith_uint256). I used this whenever possible but couldn't use it for the prime validation code. My first thought was to use GMP, but I ended up staying with bignum so I could use the exact same validation code as in the original. I'm a little concerned now that we might have compatibility problems later on if we stick with bignum. Does anyone have any thoughts on this?

apt-cache policy gives for respectively Sid and Stretch :

Code:

openssl: Installed: 1.1.1~~pre9-1
libssl-dev: Installed: 1.1.1~~pre9-1

openssl: Installed: 1.1.0f-3+deb9u2
libssl1.0-dev: Installed: 1.0.2l-2+deb9u3

Note: I installed libssl-dev instead for Stretch (version 1.1.0f-3+deb9u3) and got Bignum errors as well.
I think that installing libssl1.0-dev in Sid will fix the problem.

I only got Bignum errors. Here is a pastebin of the errors that I got.

The miner currently uses GMP, so you might consider using it as well in the wallet and remove the Bignum usage like the Bitcoin devs did.
It should be Ok as long as the proof of work is mathematically the same: checking if n, n + 4, n + 6, n + 10, n + 12, and n + 16 are prime using Rabin-Miller primality tests (with 40 iterations if I remember correctly).
You could even take this opportunity to optimize the POW validation to speed up the syncing as I said, if possible.

Until segwit is enabled you must run with addresstype=legacy in your riecoin.conf file. Without this segwit addresses will be used. I believe the transactions will be accepted but you won't be able to spend them until segwit is enabled. When generating an address, make sure it starts with 'R' (or 'r' on testnet) and not '3' (or '2' on testnet).

Do you plan to fully support the "3"/"bc1" addresses for Riecoin, and change the prefixes (I would suggest "3" -> "T" (2nd letter after R) and "bc1" -> "bcR" (1 became R for legacy addresses))?
The closer we are to the Bitcoin features support, the easier we will be able catch up its future updates, and there will be better compatibility for exchanges listing and other services.

The binaries are over 100 MB so they can't be placed on github. Maybe we can put them on the community site.

IGJ will certainly gladly upload the binaries on his website.

The miner is working for RiePool and XPoolX. However, there is a bug that produces many duplicate shares for powerful CPUs. This problem is present if many threads are used, but seems to disappear if mining with just 4 threads or less (and it seems possible to launch 4 instances using 4 threads for a 16 threads CPU without issue). I will investigate this.

AFAIR it can be something similar to that issue when you try to generate random numbers using loop having only one variable to store them you will get same numbers changing every second. Every CPU thread needs unique variable. I think to fix this you need to declare as many variables as possible at the very beginning of your code and then limit them to what you need.

I was thinking in this direction as well and might have found a solution.

Edit: this problem was fixed and will be commited soon.

[military_force]

Your project sounds good, it looks very interesting and promising and your team work was so impressive. I hope it will be a great project in the future so keep it up guys and all the best for your bright future.

[clo1]

I am currently syncing. This is really slow for a network with very few transactions, but this seems more a CPU bottleneck than a network one, as one core is used at 100% during sync.
Do you think that the POW validation could be optimized like the miner was, in order to sync much faster?

It took me a little over a day to sync which was a big improvement over the 5 days it took me on the original code. I think the speed improvement comes because the proof of work is only checked on 20% of blocks if the block is older than the latest checkpoint. The last checkpoint is now almost 4 years old in the original code. One possibility would be to only check each prime a single time instead of multiple times if the block is older than the latest checkpoint. In either case we should regularly add checkpoints.

I think the miner improvement comes from needing to check fewer primes, not from speeding up the checks themselves.

Do you plan to fully support the "3"/"bc1" addresses for Riecoin, and change the prefixes (I would suggest "3" -> "T" (2nd letter after R) and "bc1" -> "bcR" (1 became R for legacy addresses))?
The closer we are to the Bitcoin features support, the easier we will be able catch up its future updates, and there will be better compatibility for exchanges listing and other services.

I haven't taken anything out so if it is in Bitcoin it should be in the new code. The '3' and '2' come from the SCRIPT_ADDRESS that gatra set.

[PttnMe]

The duplicate shares problem is now fixed and rieMiner seems now be usable for pooled mining. Miners, please try rieMiner and give some feedbacks.

And lucky you XPoolX, I even found a block during testing...

I haven't taken anything out so if it is in Bitcoin it should be in the new code. The '3' and '2' come from the SCRIPT_ADDRESS that gatra set.

I did not know that these "3" addresses already existed in 2014. But in this case, we should change the prefixes anyway to avoid confusion with Bitcoin "3" and "bc1" addresses.

Edit: I finished syncing and tested quickly rieMiner with your wallet via GetBlockTemplate. Unfortunately, I am getting "CheckProofOfWork() : not valid pow" messages in Testnet when finding blocks, but the miner seems to be able to extract correctly the mining information (else, it would not mine anything, yet it is actually working and finding/submitting things).

The GetBlockTemplate command in 0.16 seems to give the same results for the needed fields (previousblockhash, coinbasevalue, bits,...), so I think that the miner should still have been compatible without any changes. The version changed, but submissions are still rejected if I hard change it to 2. Can you see in your side what might have changed when the wallet receives proof of work?

[IGJ]

Hello guys

Very good work clo1 !!! And to you pttn !

I wrote few words in community board about openssl 1.1 and created a patch for 16.0 you can check there in developers corner.

For faster sync I don't think is good idea to easier the miller-rabin test, we should keep it as close to 10.2 as possible, because this may lead to forks. miller-rabin is probabilistic test, so some nodes may accept the prime the others may reject, as many test is done more sure is, it will be accept by all.

GMPlib have high optimized miller-rabin test as far as I know, and I believe it will be faster than openssl, but strong tests will be needed to see if openssl and gmplib will give same result for all blocks we have by now.

[cryptapus]

I've posted my code on github.
https://github.com/clo-prime/riecoin-0.16

...

Greetings clo1,

Thanks for posting your code. I've skimmed it and would like to assist if possible (I can perhaps help with gitian, builds, and tests). In the interest of more easily auditing/maintenance would it be possible to add your commits on top of the latest 0.16 code from upstream, preserving the git history? Also, it might be easier to not unnecessarily rename files to ease future merges.

For instance, I've created a new github organization:

https://github.com/riecointeam/riecoin

If you put your changes on the master branch it might make things more clear. (Or gatra can update riecoin/riecoin.git master to 0.16, whatever.)

FWIW, I don't have an interest in taking a share of your rightly deserved bounty.

Nice work!

[R3D5H1RT]

I love CPU only coins!

[Pon13]

rieMiner 0.9α3 (2018-09-09) - Stratum support added ! You can now solo + pool mine (although Stratum support is still experimental)

Testing, feedbacks, or even contributions are welcome to fix remaining bugs and improve the miner.

The miner is working for RiePool and XPoolX. However, there is a bug that produces many duplicate shares for powerful CPUs. This problem is present if many threads are used, but seems to disappear if mining with just 4 threads or less (and it seems possible to launch 4 instances using 4 threads for a 16 threads CPU without issue). I will investigate this.

I do not have an uBlock.it account, so I was not able to test for this pool.

Pools owners are also invited to test the miner and make suggestions or contributions.

rieMiner's GitHub repository, issues
rieMiner's thread on Riecoin-Community.com forum

hi man !
just to inform you that i compiled a x64 windows version of your miner and it works perfectly in ublock.it as well !
https://imgur.com/a/tk0pfeO
https://imgur.com/a/RXjqDBj
* mine.ublock.it = mine.blockocean.com

Many thanks for your work !!

EDIT: Since compiling stuff is not my area of expertise (its been a loooong time from uni), i'd like to ask if there is a way to export the needed .dll in/from MinGW in order to have a stand alone version running straight from windows?

[xpoolx]

Whoah! nice changes this week on riecoin.

let's see if gatra aproves the new wallet to start working on the pool implementation for it.

And thanks a lot Prtnme for the new miner. Im going to test it right now

[clo1]

Edit: I finished syncing and tested quickly rieMiner with your wallet via GetBlockTemplate. Unfortunately, I am getting "CheckProofOfWork() : not valid pow" messages in Testnet when finding blocks, but the miner seems to be able to extract correctly the mining information (else, it would not mine anything, yet it is actually working and finding/submitting things).

The GetBlockTemplate command in 0.16 seems to give the same results for the needed fields (previousblockhash, coinbasevalue, bits,...), so I think that the miner should still have been compatible without any changes. The version changed, but submissions are still rejected if I hard change it to 2. Can you see in your side what might have changed when the wallet receives proof of work?

I think the GetBlockTemplate problem (actually it was submitblock) is fixed.

[clo1]

I've posted my code on github.
https://github.com/clo-prime/riecoin-0.16

...

Greetings clo1,

Thanks for posting your code. I've skimmed it and would like to assist if possible (I can perhaps help with gitian, builds, and tests). In the interest of more easily auditing/maintenance would it be possible to add your commits on top of the latest 0.16 code from upstream, preserving the git history? Also, it might be easier to not unnecessarily rename files to ease future merges.

For instance, I've created a new github organization:

https://github.com/riecointeam/riecoin

If you put your changes on the master branch it might make things more clear. (Or gatra can update riecoin/riecoin.git master to 0.16, whatever.)

FWIW, I don't have an interest in taking a share of your rightly deserved bounty.

Nice work!

Help in these areas would be great.

Just to be clear, is the code you listed the latest bitcoin 0.16 code? I can add my code to this, but I want to check a couple more things before I start.

Most of the filenames I changed were also changed in the original code ... images, riecoind, riecoin-cli, a few others. I might have changed a couple extra in the qt code (bitcoingui.cpp -> riecoingui.cpp).

Similarly, a lot of the changes are text substitutions of bitcoin for riecoin. This was true in the original code and given how much the code has grown since then is worse now. It would be easy for me to do a separate commit of all the text substitutions so that the main riecoin changes are more obvious.

My approach was to change all filenames and all text (except copyrights and a couple others). Let me know if this is a problem. Changing some, but not others will make things more difficult.

I'm not sure if I will get the bounty but if I do I plan to share a portion with others contributing to Riecoin.

[cryptapus]

...

Just to be clear, is the code you listed the latest bitcoin 0.16 code? I can add my code to this, but I want to check a couple more things before I start.

Most of the filenames I changed were also changed in the original code ... images, riecoind, riecoin-cli, a few others. I might have changed a couple extra in the qt code (bitcoingui.cpp -> riecoingui.cpp).

Similarly, a lot of the changes are text substitutions of bitcoin for riecoin. This was true in the original code and given how much the code has grown since then is worse now. It would be easy for me to do a separate commit of all the text substitutions so that the main riecoin changes are more obvious.

My approach was to change all filenames and all text (except copyrights and a couple others). Let me know if this is a problem. Changing some, but not others will make things more difficult.

I'm not sure if I will get the bounty but if I do I plan to share a portion with others contributing to Riecoin.

Yes, https://github.com/riecointeam/riecoin master branch should be even with upstream bitcoin 0.16 branch (now v0.16.2).

Here is a great repo that nzsquirrell put together on minimal changes necessary that I find quite good. It's for 0.14 but is very close to 0.16:

https://github.com/nzsquirrell/testcoin14/commits/master

especially this commit:

https://github.com/nzsquirrell/testcoin14/commit/3a2665eb070ed901ed28722e345ad61d75027d4c

There are surprisingly few name changes that need to be made, most of the missing ones from the above commit are in src/rpc for the rpc calls. We can always knock those out once the core logic is in place.

[PttnMe]

rieMiner 0.9β1 (2018-09-14) - Benchmark mode added ! Also, some bug fixes, and a huge code refactoring.

The miner should now be stable enough to replace the older fastrie/xptMiner without problem. There are also instructions on how to build a standalone executable on Windows.

I am waiting the official Riecoin Wallet update and letting some time to find and fix remaining bugs and do some minor refactoring before releasing the stable rieMiner 0.9. At this moment, official standalone binaries for Windows and probably Linux will be provided.

As always, feedbacks and suggestions are welcome, do not hesitate.
And to pool owners, if you could tell about rieMiner in your homepages, it would really help me and I would be grateful; we would have more people testing rieMiner.

hi man !
just to inform you that i compiled a x64 windows version of your miner and it works perfectly in ublock.it as well !
https://imgur.com/a/tk0pfeO
https://imgur.com/a/RXjqDBj
* mine.ublock.it = mine.blockocean.com

Many thanks for your work !!

EDIT: Since compiling stuff is not my area of expertise (its been a loooong time from uni), i'd like to ask if there is a way to export the needed .dll in/from MinGW in order to have a stand alone version running straight from windows?

Great! Thank you for letting me know that it also works on uBlock.it. It should also be the case for rieMiner 0.9β1. And now, you have an answer about including the Dlls when building.

Whoah! nice changes this week on riecoin.

let's see if gatra aproves the new wallet to start working on the pool implementation for it.

And thanks a lot Prtnme for the new miner. Im going to test it right now

I hope that you are satisfied with rieMiner. Else, please report bugs to improve the software until the stable release. Some were fixed in 0.9β1.

I think the GetBlockTemplate problem (actually it was submitblock) is fixed.

I confirm this, solo mining with GetBlockTemplate via your wallet seems to work fine now.

[czakris]

Hi PttnMe,
Right now I am testing your miner using fx8350, 32GB of RAM and W7 64bits trying solo mining with these rieMiner.conf settings:

Host = 127.0.0.1;
Port = 28332;
User = (my wallet address);
Pass = (my password);
Protocol = GetBlockTemplate;
Address =  (my wallet address);
Tuples = 6;
Threads = 8;
Sieve = 4073741824; (because why not?  )
Refresh = 30;

Miner works fine, trashing my CPU  at full speed, but still, I am unable to switch into GetBlockTemplate Protocol. (Benchmark only). Any idea how to turn it on?