lightweight database, for brute force using publickeys-32Mk =3.81MB(secp256k1)

[mcdouglasx]

Something strange is definitely happening.
I have to calmly review the error.
-definitely not the collision margin.
-python expresses results wrong sometimes (I don't think so)
-the script gives good results but in certain cases it doesn't.
-Does the "bitstring" module sometimes fail?
- At a certain point in the database, bits are skipped and the calculations are not correct?
-does the script misinterpret some bytes?

Pk: 1033162084
Pk: 1033162084
Pk: 1033379684
Pk: 1033162084
Pk: 1033162084
Pk: 1033597284
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033379684
Pk: 1033488484
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033597284
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033488484
Pk: 1033597284
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033597284
Pk: 1033162084

edit:
These lines are definitely interfering.

Code:

Low_m= 16

lm= num // Low_m

[WanderingPhilospher]

Something strange is definitely happening.
I have to calmly review the error.
-definitely not the collision margin.
-python expresses results wrong sometimes (I don't think so)
-the script gives good results but in certain cases it doesn't.
-Does the "bitstring" module sometimes fail?
- At a certain point in the database, bits are skipped and the calculations are not correct?
-does the script misinterpret some bytes?

Pk: 1033162084
Pk: 1033162084
Pk: 1033379684
Pk: 1033162084
Pk: 1033162084
Pk: 1033597284
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033379684
Pk: 1033488484
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033597284
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033488484
Pk: 1033597284
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033597284
Pk: 1033162084

edit:
These lines are definitely interfering.

Code:

Low_m= 16

lm= num // Low_m

Can you not rewrite lowmem script to something easier (less code)
And without bringing in the Bitcoin library?

Something like NotATether was mentioning?

[mcdouglasx]

Something strange is definitely happening.
I have to calmly review the error.
-definitely not the collision margin.
-python expresses results wrong sometimes (I don't think so)
-the script gives good results but in certain cases it doesn't.
-Does the "bitstring" module sometimes fail?
- At a certain point in the database, bits are skipped and the calculations are not correct?
-does the script misinterpret some bytes?

Pk: 1033162084
Pk: 1033162084
Pk: 1033379684
Pk: 1033162084
Pk: 1033162084
Pk: 1033597284
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033379684
Pk: 1033488484
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033597284
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033488484
Pk: 1033597284
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033162084
Pk: 1033597284
Pk: 1033162084

edit:
These lines are definitely interfering.

Code:

Low_m= 16

lm= num // Low_m

Can you not rewrite lowmem script to something easier (less code)
And without bringing in the Bitcoin library?

Something like NotATether was mentioning?

fixed, please delete previous db.

[WanderingPhilospher]

fixed, please delete previous db.

So when using num and Low_m, we should use numbers that leave no remainder, correct?
Such as 2^32 / 2^26 = 64; this would be ok, right?

[mcdouglasx]

fixed, please delete previous db.

So when using num and Low_m, we should use numbers that leave no remainder, correct?
Such as 2^32 / 2^26 = 64; this would be ok, right?

yes.


A quick way to try without complete DB is:

You load your configuration.
example:

You choose a PK within the range:

PK: 4294960000

NUM = 4294967296
Low_m = 67108864

You start the script
After a minute cancel.

Then in the search script you choose a range close to your test PK.
So:
#range
START = 4294950000
END = 4294990000

You start the search
If you get
PK: 4294960000

It means that your database will be executed without errors and without false positives.

[WanderingPhilospher]

After numerous, 100s of tests, I really do not understand the scan script.

If I generate 2^20 keys using the database script (with a subtraction of 1); it should take my target pubkey and generate/subtract 2^20 consecutive keys from my target pubkey. OK, I got that.

But when I run the scan script, if the random hits anywhere in the range of target pubkey - 2^20, it should result in a hit/match/key found. However, it does not.

I even tested with incrementing the pk (private key/integer position), versus random. Again, if I have 2^20 generated keys in database, I should be able to start pk position at 0 or 1, and increment by 2^20 private keys, and always get a hit/match/key found. But that was not the case.

Help me understand why this is so, thanks.

Also, why does the script need a margin collision on the scan script? My understanding is takes the pk that the rand generator has landed on, and does a subtraction loop 64 times (default). Again, If I land on any key within target pub - 2^20, it should result in a hit. Why would I need to do 64 subtraction loops.

Thanks.

[mcdouglasx]

After numerous, 100s of tests, I really do not understand the scan script.

If I generate 2^20 keys using the database script (with a subtraction of 1); it should take my target pubkey and generate/subtract 2^20 consecutive keys from my target pubkey. OK, I got that.

But when I run the scan script, if the random hits anywhere in the range of target pubkey - 2^20, it should result in a hit/match/key found. However, it does not.

I even tested with incrementing the pk (private key/integer position), versus random. Again, if I have 2^20 generated keys in database, I should be able to start pk position at 0 or 1, and increment by 2^20 private keys, and always get a hit/match/key found. But that was not the case.

Help me understand why this is so, thanks.

Also, why does the script need a margin collision on the scan script? My understanding is takes the pk that the rand generator has landed on, and does a subtraction loop 64 times (default). Again, If I land on any key within target pub - 2^20, it should result in a hit. Why would I need to do 64 subtraction loops.

Thanks.

We import the modules.

Code:

#@mcdouglasx
import secp256k1 as ice
import random
from bitstring import BitArray

We select the range where our privatekey is supposed to be located.

Code:

#range
start= 1033100000
end=   1033200000
       

we open the loop

Code:

while True:

We generate a random pk within the chosen range and generate the upub.

    

Code:

pk= random.randint(start, end)
   
    target = ice.scalar_multiplication(pk)

Here we subtract from the pk 64 times "G" or the number that we chose to subtract when we created the database (it must be the same one with which you created the database)

 

Code:

   num = 64 # collision margin.

    sustract= 1 # #amount to subtract each time.

    sustract_pub= ice.scalar_multiplication(sustract)



    res= ice.point_loop_subtraction(num, target, sustract_pub)
   
    binary = ''
   
    for t in range (num):
       
        h= (res[t*65:t*65+65]).hex()
        hc= int(h[2:], 16)
       
       
        if str(hc).endswith(('0','2','4','6','8')):
            A="0"
            binary+= ''.join(str(A))
           
        if str(hc).endswith(('1','3','5','7','9')):
            A="1"
            binary+= ''.join(str(A))
   
       
    my_str = binary

    

Why 64 in collision margin?

because we need to store 64 pubkeys identified with 1s and 0s.
so as we will see:

Code:

my_str ="1010100010110101000101101010001010101000101101000101101010101010"

Why not less than 64?, 16?

Code:

my_str ="1010100010110101"

because if it is less, you can find equal sequences belonging to other pks (false positives).

-we transform this sequence into bytes:

Code:

 b = bytes(BitArray(bin=my_str))  

we look for a match in the database

    

Code:

file = open("data-base.bin", "rb")

    dat = bytes(file.read())
   
    if b  in dat:
        s = b
        f = dat

We find the position of "b" in the database and multiply by the chosen subtraction.

Code:

inx = f.find(s)*sustract
inx_0=inx

we get the correct pk

adding (pk obtained randomly + (position of the target in the database* the subtraction)) + (position of the target in the database* the subtraction)* 7)
where seven is the maximum number of bits in a byte that are counted like this (01234567).

Code:

Pk = (int(pk) + int(inx_0))+int(inx_0)*7

we save the result.

Code:

 data = open("win.txt","a")
data.write("Pk:"+" "+str(Pk)+"\n")
data.close()
break

[WanderingPhilospher]

Ok, but you didn't answer my most important question lol.

If my target pubkey's priv key is (TPK) is 123456789 and I generate 2^20 keys, with subtraction value of 1, then the database now contains TPK - 2^20 consecutive keys. In other words, TPK - 1, TPK - 2, TPK - 3,...TPK - 2^20.

So in the database now are the corresponding pubkeys to, 123456788, 123456787, 123456786, ...., 122408213.

Now, when I run the scan and I land on any priv keys from 123456788 to 122408213, I should get a match/solved key.

But that has not been the case.

When I run an increment test (only way I know the above doesn't result in a match); I should be able to start my priv key counter at 1 and increment by 2^20 and eventually get a match.

priv key = 1; no match
priv key = 1+1048576 (last position of priv key plus increment of 2^20); no match
...
priv key = 122683392 (increment * 117); should be a match because 122683392 is within the range of 123456788 to 122408213. But program skips the match and keeps on going.

Why would this be?

[mcdouglasx]

Ok, but you didn't answer my most important question lol.

If my target pubkey's priv key is (TPK) is 123456789 and I generate 2^20 keys, with subtraction value of 1, then the database now contains TPK - 2^20 consecutive keys. In other words, TPK - 1, TPK - 2, TPK - 3,...TPK - 2^20.

So in the database now are the corresponding pubkeys to, 123456788, 123456787, 123456786, ...., 122408213.

Now, when I run the scan and I land on any priv keys from 123456788 to 122408213, I should get a match/solved key.

But that has not been the case.

When I run an increment test (only way I know the above doesn't result in a match); I should be able to start my priv key counter at 1 and increment by 2^20 and eventually get a match.

priv key = 1; no match
priv key = 1+1048576 (last position of priv key plus increment of 2^20); no match
...
priv key = 122683392 (increment * 117); should be a match because 122683392 is within the range of 123456788 to 122408213. But program skips the match and keeps on going.

Why would this be?

I'm going to test it.
--
update:

On my side everything is going well, you should receive matches unless there is an error or you search in the wrong range.

You can share this data to emulate your search:

for the database.

target=
num =
subtract=
Low_m=

edit= 2

for your example

start= 122408213-64

end= 123456789-64

they all match.

[WanderingPhilospher]

I’m not doing a random like in your initial scan script. I am doing an increment.

Did you do an increment test?

There is no range; start at priv key 1 and increment by number of keys generated during database build.

[mcdouglasx]

I’m not doing a random like in your initial scan script. I am doing an increment.

Did you do an increment test?

There is no range; start at priv key 1 and increment by number of keys generated during database build.

for your example

pk= 123456789

start= 122408213+64

end= 123456789-64

they all match.

If you start from pk 1 you will have a first match at

start= 122408213+64


If you want to test a single pk it would be like this:

Code:

#@mcdouglasx
import secp256k1 as ice
import random
from bitstring import BitArray



print("Scanning Binary Sequence")



pk= 123456789-64
    
target = ice.scalar_multiplication(pk)

num = 64 # collision margin.

sustract= 1 # #amount to subtract each time.

sustract_pub= ice.scalar_multiplication(sustract)

res= ice.point_loop_subtraction(num, target, sustract_pub)
    
binary = ''
    
for t in range (num):
        
    h= (res[t*65:t*65+65]).hex()
    hc= int(h[2:], 16)
        
        
    if str(hc).endswith(('0','2','4','6','8')):
        A="0"
        binary+= ''.join(str(A))
            
    if str(hc).endswith(('1','3','5','7','9')):
        A="1"
        binary+= ''.join(str(A))
    
        
my_str = binary

b = bytes(BitArray(bin=my_str))
    
    

file = open("data-base.bin", "rb")

dat = bytes(file.read())
    
if b  in dat:
    s = b
    f = dat
    inx = f.find(s)*sustract
    inx_0=inx
    Pk = (int(pk) + int(inx_0))+int(inx_0)*7
        
    data = open("twin.txt","a")
    data.write("Pk:"+" "+str(Pk)+"\n")
    data.close()

If you want to obtain incremental matching you must change the order of the database, because its order is reversed.

111000
incremental you get
000111

do not match.

[WanderingPhilospher]

If you want to obtain incremental matching you must change the order of the database, because its order is reversed.

111000
incremental you get
000111

do not match.

I'm still lost lol.

You are still giving an example of a range. I am not using a range. I am using the start as pk = 0; then increment by the number of keys generated. If increment was 7 (if I only generated 7 keys in the database), the pk's would look like:
0
7
14
21
28
35
...
763
770

If the target pubkey's pk was 777; when my increment reaches pk 770, that should be a match (777 - 770 = 7)

[mcdouglasx]

If you want to obtain incremental matching you must change the order of the database, because its order is reversed.

111000
incremental you get
000111

do not match.

I'm still lost lol.

You are still giving an example of a range. I am not using a range. I am using the start as pk = 0; then increment by the number of keys generated. If increment was 7 (if I only generated 7 keys in the database), the pk's would look like:
0
7
14
21
28
35
...
763
770

If the target pubkey's pk was 777; when my increment reaches pk 770, that should be a match (777 - 770 = 7)

What are you increasing?

if you create an incremental database
1:1000000

for search
you must do sums in loop of +64

and you will get matches

from
1 to (1000000-64)

[WanderingPhilospher]

What are you increasing?

The private key!

Your script has a random range for the private key:

Code:

start= 1033100000
end=   1033200000
       

while True:

    pk= random.randint(start, end)

I am incrementing the pk!

pk = 0
------
pk = pk + 7 (or however many keys created during database creation.

Again, I am talking about your scan / search script; the script that searches for a match lol.

[mcdouglasx]

What are you increasing?

The private key!

Your script has a random range for the private key:

Code:

start= 1033100000
end=   1033200000
       

while True:

    pk= random.randint(start, end)

I am incrementing the pk!

pk = 0
------
pk = pk + 7 (or however many keys created during database creation.

Again, I am talking about your scan / search script; the script that searches for a match lol.

I know you are talking about the search script but I mention the database because they work in pairs, you cannot subtract 1 million from the target and then search incrementally.
because in database the bits obtained would be inverted

database sequence
01011110
incremental
01111010

so you won't get matches.

[WanderingPhilospher]

What are you increasing?

The private key!

Your script has a random range for the private key:

Code:

start= 1033100000
end=   1033200000
       

while True:

    pk= random.randint(start, end)

I am incrementing the pk!

pk = 0
------
pk = pk + 7 (or however many keys created during database creation.

Again, I am talking about your scan / search script; the script that searches for a match lol.

I know you are talking about the search script but I mention the database because they work in pairs, you cannot subtract 1 million from the target and then search incrementally.
because in database the bits obtained would be inverted

database sequence
01011110
incremental
01111010

so you won't get matches.

I don’t understand.

Why does it matter how I reached (my example above with 777) pk 770, whether I reached it by landing on it incrementally or by random?

That makes no sense. If I land on 770 regardless of how I landed on it, I should get a match.

If I am incrementing my search rather than random, when I land on 770, the script should calculate 770-64 and that should be a match.

[mcdouglasx]

What are you increasing?

The private key!

Your script has a random range for the private key:

Code:

start= 1033100000
end=   1033200000
       

while True:

    pk= random.randint(start, end)

I am incrementing the pk!

pk = 0
------
pk = pk + 7 (or however many keys created during database creation.

Again, I am talking about your scan / search script; the script that searches for a match lol.

I know you are talking about the search script but I mention the database because they work in pairs, you cannot subtract 1 million from the target and then search incrementally.
because in database the bits obtained would be inverted

database sequence
01011110
incremental
01111010

so you won't get matches.

I don’t understand.

Why does it matter how I reached (my example above with 777) pk 770, whether I reached it by landing on it incrementally or by random?

That makes no sense. If I land on 770 regardless of how I landed on it, I should get a match.

If I am incrementing my search rather than random, when I land on 770, the script should calculate 770-64 and that should be a match.

if you only modify:

Code:

pk= random.randint(start, end)

you create your database from 777:1

in search
incremental match
65, 66,67,68......until 777

[WanderingPhilospher]

I give up lol.

It doesn’t work with incrementing pk versus random pk.

[mcdouglasx]

I give up lol.

It doesn’t work with incrementing pk versus random pk.

You are doing something wrong in your incremental script.
If you post it maybe I'll see your mistake.

[WanderingPhilospher]

I give up lol.

It doesn’t work with incrementing pk versus random pk.

You are doing something wrong in your incremental script.
If you post it maybe I'll see your mistake.

I just tried again lol.

Use this as target pub = 0209c58240e50e3ba3f833c82655e8725c037a2294e14cf5d73a5df8d56159de69

Create 4,000,000 keys in database with subtraction of 1, and Low_m = 1

Start your pk at 0 and then increment by 4,000,000

I use:
pk = 0 (start)
and then in the while true or the main:
pk = pk + 4000000 (increment)

I ran a test in the 28 bit range with 1 million keys and incremented and it worked. But the above test did not. ?