Large Bitcoin Collider Thread 2.0

[daserpent]

Can I participate from Windows?? Does it require GPU power or CPU power?

[1715198555]

1715198555

[1715198555]

1715198555

[1715198555]

1715198555

[1715198555]

1715198555

[1715198555]

1715198555

[dcw312]

GPU version of LBC usually works only on Linux, not on Windows. With your Dockerfile is it possibile to exploit GPU on Win?

A quick Google search indicates that Nvidia/Docker does not support Windows. For Linux, see this project here for Nvidia+Docker https://github.com/NVIDIA/nvidia-docker/tree/2.0

[KeepingScore]

No downloads available on site?

[rico666]

Just to make it clear for everybody:

Yes, the project is still on, I haven't abandoned it.

I just happen to spend my energy currently on the BURST cryptocurrency, so there is not much time left for the LBC project at the moment.
I feel the need to make that clarification here, as there are 1st "stalkers" who use Burst discussions on Reddit to ask me about LBC
https://www.reddit.com/r/burstcoin/comments/75urgi/i_want_to_set_another_node_but_this_time_on_a_vps/dob21xk/

Please don't do that. There is a nice Discord Channel at https://discord.gg/AyEfZrY

Where you can get more interactive support from the great LBC community.

Also, there are at the moment some showstoppers that would need too much of my time to look into (like a broken blockparser, see https://github.com/znort987/blockparser/issues/65) if someone wants to give me a real nudge to update some things LBC (and I do have some pretty nice updates pending), fix that blockparser issue or find an alternative.

Arulbero has sent me a new version of his ECC lib which may be (I haven't tested it yet, but I believe his tests/claims) 20% faster.
I would want to include that too. So someone please look into the blockexplorer.

[arulbero]

Arulbero has sent me a new version of his ECC lib which may be (I haven't tested it yet, but I believe his tests/claims) 20% faster.

A little explanation (and update) for the LBC users:

LBC generator has 3 components:

1) ECC generator --> 2) sha256 + ripemd160 --> 3) bloom search

1) this component computes the generation from the private key of the corresponding public key. We have only a CPU version of this library.

2) each single public key produces 2 160-bit strings (addresses = hash of the public key), uncompressed and compressed addresses; Rico developed 2 optimized versions of this component, one for CPU and one for GPU. In the GPU case the output of the CPU is sent to the GPU.

3) check if the addresses generated in the step 2) are or not in the bloom filter (CPU version and GPU version).


For the systems GPU limited (GPU used at 90% or more) it is crucial to improve the component 2).

For the systems CPU limited (example: CPU used at 100% and GPU at 50%) it is important instead to improve the speed of the ECC generator, i.e. the rate at which the CPU feeds the GPU.

ECC generator is the component  I have been working on in the past months.
I'm improving the ECC library as much as I can. At this moment the library is almost 30% faster than the current one. On my CPU Kabylake, it takes only 1,16 s to generate 16,7 M public keys against the 1,64 s of the latest release.

For a system cpu limited, the new library could take a +30% speedup (exploiting completely the GPU's speed).
In that case I think that the LBC generator will be faster than oclvanitygen on all GPUs (for now the LBC generator is faster only on middle/slow gpu, but not on fast gpu).

For a system gpu limited, the only advantage is that you could use 3 cores to get (almost) the same performance as 4 cores now.

For a system without GPU there is a small speedup, because in the CPU generator only a 10% of the cpu work is about the ECC arithmetic, about 65% is for the sha256/ripemd160 task and about 25% for the bloom search.

[arulbero]

Also, there are at the moment some showstoppers that would need too much of my time to look into (like a broken blockparser, see https://github.com/znort987/blockparser/issues/65) if someone wants to give me a real nudge to update some things LBC (and I do have some pretty nice updates pending), fix that blockparser issue or find an alternative.

I would want to include that too. So someone please look into the blockexplorer.

This is a Python version to extract data from utxo: https://bitcoin.stackexchange.com/questions/56655/getting-a-crypto-proof-utxo-set
https://github.com/sr-gi/bitcoin_tools

Like Wuille noted, Bitcoin Core 0.15 has changed db format of the Utxo set. Maybe you could try with a old version of Core.
  

[kknd]

tks arulbero for work 
I was thinking that the project was abandoned

[rico666]

For a system without GPU there is a small speedup, because in the CPU generator only a 10% of the cpu work is about the ECC arithmetic, about 65% is for the sha256/ripemed160 task and about 25% for the bloom search.

No, the bloom search (more precisely: bloom check) accounts for about 1% of the total work.

[arulbero]

For a system without GPU there is a small speedup, because in the CPU generator only a 10% of the cpu work is about the ECC arithmetic, about 65% is for the sha256/ripemd160 task and about 25% for the bloom search.

No, the bloom search (more precisely: bloom check) accounts for about 1% of the total work.

On the CPU generator the bloom check is only 1%?
I made a little program with:

1) my aecc
2) sha256 and ripemd160 from supervanitygen
3) bloom check from brainflayer

and I got: 1,16s for aecc, 12,4s for sha256+ripemd160, 4s for bloom check => total: 17,55s for 16,7 Mkeys

Then: 6,6% for aecc, 70,6% for sha256 and 22,8% for bloom check. Maybe I'm not using the bloom check in the best way, but 23% against 1% is too much.

[rico666]

On the CPU generator the bloom check is only 1%?
I made a little program with:

1) my aecc
2) sha256 and ripemd160 from supervanitygen
3) bloom check from brainflayer

and I got: 1,16s for aecc, 12,4s for sha256+ripemd160, 4s for bloom check => total: 17,55s for 16,7 Mkeys

Then: 6,6% for aecc, 70,6% for sha256 and 22,8% for bloom check. Maybe I'm not using the bloom check in the best way, but 23% against 1% is too much.

Hard to say when I do not see the little program. On my machine the LBC generator takes like 0.2 s for both compressed and uncompressed checks of 16.7M keys.
No - I do not use the code from brainflayer anymore, but I believe even the original code didn't take so long on my CPU (more like 0.4s instead of 4s)


On a different topic:

I updated the views. The download link to the LBC Appliance is updated. Careful, it's a 1.8GB download. It has an updated Arch Linux, newest available BLF, client and a haswell generator.
You will now see also a Crowdfunding page, which is merely there to stipulate a discussion about improving GPU performance (see what arulbero wrote above about GPU limited issues).
By using an improved ECC lib and symmetries, we may run sooner into a GPU limited situation in the future.

[rico666]

Success!

(25.99 Mkeys/s) on a machine (Skylake + M2000M) that gave me 22.7 Mkeys/s max with the previous generator.
Thanks to the new Arulbero ECC library.

A nice side effect of arulbero completely ditching the libgmp requirement (by providing his own tailored bignum math) is, that the generator binaries are now only half the size of the previous ones.

243184 Oct 16 14:20 kardashev-skylake

237KiB - we may cast that soon into a FPGA   - just kidding... Or am I?

Still, please do have patience before I push the new binaries on the server, I would like to get rid of the FTP server first and move all to a HTTPS communication.


edit: with some newer versions I was able to squeeze 27.26 Mkeys/s out of my machine with around 95/96% GPU usage. Certainly more will be possible in the future.

kardashev-skylake is now available in the new versions, those of you who have skylake machines with GPU, please test. (client should auto-update the generator upon restart).

Generators for the other instruction sets will follow soon.

[gizmoh]

Hurrah! over 30% jump from  ~31.5  Mkeys to ~41.6  Mkeys on a 7700k with 1080Ti

Gpu usage from ~83% now to ~98%

Thanks Arulbero for the optimizations and Rico for quick implementation  

[rico666]

Hurrah! over 30% jump from  ~31.5  Mkeys to ~41.6  Mkeys on a 7700k with 1080Ti

Gpu usage from ~83% now to ~98%

Thanks Arulbero for the optimizations and Rico for quick implementation  

28% for GPU clients that were not GPU limited - to be precise. 
Your observation is consistent with what is seen on these machines.

i7-6700 CPU @ 3.40GHz + 1080 : 32.47 Mkeys/s -> 42.36 Mkeys/s

That makes the overall collider speed an equivalent of 6 such machines.
If we had 600 of these colliders, the next puzzle transaction privkey would be here in less than 24 hours - worst case.

[arulbero]

Hurrah! over 30% jump from  ~31.5  Mkeys to ~41.6  Mkeys on a 7700k with 1080Ti

If your system is more or less like this:

https://bitcointalk.org/index.php?topic=1573035.msg18373053#msg18373053

or like this https://bitcointalk.org/index.php?topic=1573035.msg18446472#msg18446472

then we are now faster than oclvanitygen on fast GPUs too. Remember that 41.6 Mkeys/s means 41.6 M compressed keys + 41.6 M uncompressed keys per second, over 83 Maddresses/s!.  Could you test oclvanitygen on your machine?

Gpu usage from ~83% now to ~98%

I'm afraid that almost all systems are now GPU limited.

Only multi GPU systems can take advantage from other ecc improvements (if there will be) and from n-k symmetry.  I think that a GPU version of ECC library would be not very useful at the moment. We need above all to speedup sha256/ripemd160 on GPU.

[rico666]

I'm afraid that almost all systems are now GPU limited.

Only multi GPU systems can take advantage from other ecc improvements (if there will be) and from n-k symmetry.  I think that a GPU version of ECC library is not very useful at the moment. We need overall to speedup sha256/ripemd160 on GPU.

You are right: Multi-GPU systems are not GPU limited and the client has been tested already on Systems with as many as 16 GPUs. (currently the Amazon p16.xlarge delivers 175 Mkeys/s)

I have already a n-k symmetry prototype running - in fact had this already May, 25th:
https://twitter.com/LBC_collider/status/867657663987015680

The single showstopper to enable n-k symmetry is accounting work that has to be done on the server.

In theory, with n-k symmetry and better hash160 OpenCL code (which I believe is possible - see also https://lbc.cryptoguru.org/crowdfunding), we may see 2x speedups on the same hardware.

[KeepingScore]

Solution to the VM's "loadable library and perl binaries are mismatched" bug when you run "./collider/LBC -x" is to run "cpan -r" and let it update everything.

[CrytpSig]

I'm afraid that almost all systems are now GPU limited.

Only multi GPU systems can take advantage from other ecc improvements (if there will be) and from n-k symmetry.  I think that a GPU version of ECC library is not very useful at the moment. We need overall to speedup sha256/ripemd160 on GPU.

You are right: Multi-GPU systems are not GPU limited and the client has been tested already on Systems with as many as 16 GPUs. (currently the Amazon p16.xlarge delivers 175 Mkeys/s)

I have already a n-k symmetry prototype running - in fact had this already May, 25th:
https://twitter.com/LBC_collider/status/867657663987015680

The single showstopper to enable n-k symmetry is accounting work that has to be done on the server.

In theory, with n-k symmetry and better hash160 OpenCL code (which I believe is possible - see also https://lbc.cryptoguru.org/crowdfunding), we may see 2x speedups on the same hardware.

Hi I am having issues with the Arch Linux virtual machine on windows Problem connecting to server https://lbc.cryptoguru.org/static/client(status: 500 Can't connect to lbc.cryptoguru.org:443 (certificate verify failed). retries left 29

[rico666]

I'm having issues with the Arch Linux virtual machine on windows Problem connecting to server https://lbc.cryptoguru.org/static/client(status: 500 Can't connect to lbc.cryptoguru.org:443 (certificate verify failed). retries left 29

maybe a

> pacman -S ca-certificates

will help. I remember (faintly) that the ca-certificates package said something like "valid until 20.10" or something like that "next check 20.10" - not sure.

Or you do a

> pacman -Syu

Which is the Arch Linux way of "apt-get update & apt-get upgrade"

edit:

Or you have a simple networking problem. Try a

> ping 8.8.8.8

if that works. If yes, update as said above. If no, you have to get your VM networking right  first. But that is a matter of the VM configuration, not of the OS running inside the VM.

[CrytpSig]

I'm having issues with the Arch Linux virtual machine on windows Problem connecting to server https://lbc.cryptoguru.org/static/client(status: 500 Can't connect to lbc.cryptoguru.org:443 (certificate verify failed). retries left 29

maybe a

> pacman -S ca-certificates

will help. I remember (faintly) that the ca-certificates package said something like "valid until 20.10" or something like that "next check 20.10" - not sure.

Or you do a

> pacman -Syu

Which is the Arch Linux way of "apt-get update & apt-get upgrade"

edit:

Or you have a simple networking problem. Try a

> ping 8.8.8.8

if that works. If yes, update as said above. If no, you have to get your VM networking right  first. But that is a matter of the VM configuration, not of the OS running inside the VM.

Ping came back fine updated on those commands previously still not working. I think this is an issue with the Perl certificate implementation. I had a similar issue running a perl script on windows with he same error that i could not fix.

[Suzobo]

Might be time to start up the server again and see what a nVidia Quadro card can do.
I know I did around a million keys pr sec with the cpu alone.