Bitcoin Forum
April 03, 2026, 03:30:34 PM *
News: Latest Bitcoin Core release: 30.2 [Torrent]
 
   Home   Help Search Login Register More  
Pages: « 1 2 3 [4]  All
  Print  
Author Topic: Ultra-Lightweight Database with Public Keys (for puzzle btc)  (Read 2082 times)
nomachine
Full Member
***
Offline Offline

Activity: 812
Merit: 134



View Profile
February 22, 2025, 07:49:32 AM
Last edit: February 23, 2025, 10:19:46 AM by nomachine
Merited by mcdouglasx (2), iceland2k14 (1)
 #61

How to create a baby table file using your method here?

Code:
import math
import time
import sys
import os
import pickle
import subprocess
import gc
from gmpy2 import mpz, powmod, invert, jacobi
import xxhash
from sortedcontainers import SortedDict

# Clear screen and initialize
os.system("cls||clear")
t = time.ctime()
sys.stdout.write(f"\033[?25l\033[01;33m[+] BSGS: {t}\n")
sys.stdout.flush()

# Elliptic Curve Parameters (secp256k1)
modulo = mpz(0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEFFFFFC2F)
order = mpz(0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141)
Gx = mpz(0x79BE667EF9DCBBAC55A06295CE870B07029BFCDB2DCE28D959F2815B16F81798)
Gy = mpz(0x483ADA7726A3C4655DA4FBFC0E1108A8FD17B448A68554199C47D08FFB10D4B8)
PG = (Gx, Gy)

# Point Addition on Elliptic Curve
def add(P, Q):
    if P == (0, 0):
        return Q
    if Q == (0, 0):
        return P
    Px, Py = P
    Qx, Qy = Q
    if Px == Qx:
        if Py == Qy:
            inv_2Py = invert((Py << 1) % modulo, modulo)
            m = (3 * Px * Px * inv_2Py) % modulo
        else:
            return (0, 0)
    else:
        inv_diff_x = invert(Qx - Px, modulo)
        m = ((Qy - Py) * inv_diff_x) % modulo
    x = (m * m - Px - Qx) % modulo
    y = (m * (Px - x) - Py) % modulo
    return (x, y)

# Scalar Multiplication using Montgomery Ladder
def mul(k, P=PG):
    R0, R1 = (0, 0), P
    for i in reversed(range(k.bit_length())):
        if (k >> i) & 1:
            R0, R1 = add(R0, R1), add(R1, R1)
        else:
            R1, R0 = add(R0, R1), add(R0, R0)
    return R0

# Point Subtraction
def point_subtraction(P, Q):
    Q_neg = (Q[0], (-Q[1]) % modulo)
    return add(P, Q_neg)

# Compute Y from X using curve equation
def X2Y(X, y_parity, p=modulo):
    X3_7 = (pow(X, 3, p) + 7) % p
    if jacobi(X3_7, p) != 1:
        return None
    Y = powmod(X3_7, (p + 1) >> 2, p)
    return Y if (Y & 1) == y_parity else (p - Y)

# Convert point to compressed public key
def point_to_cpub(point):
    x, y = point
    y_parity = y & 1
    prefix = '02' if y_parity == 0 else '03'
    compressed_pubkey = prefix + format(x, '064x')
    return compressed_pubkey

# Hash a compressed public key using xxhash and store only the first 8 characters
def hash_cpub(cpub):
    return xxhash.xxh64(cpub.encode()).hexdigest()[:8]

# Optimized function to pack the hash into bits using segments
def pack_hash_into_bits(cpub, segment_size=8):
    # Hash the compressed public key
    hash_result = hash_cpub(cpub)
   
    # Convert the hash result to bytes
    hash_bytes = bytes.fromhex(hash_result)
   
    # Initialize an empty list to store the bits
    binary = []
   
    # Iterate over the hash bytes in chunks of segment_size
    for i in range(0, len(hash_bytes), segment_size):
        segment = hash_bytes[i:i + segment_size]
       
        # Convert the segment to a single integer
        segment_int = int.from_bytes(segment, byteorder='big')
       
        # Extract the least significant bit (LSB) and append to the list
        binary.append((segment_int & 1) == 1)
   
    return binary

# Save baby table to a file compressed with pigz at compression level 7 and custom block size
def save_baby_table(baby_table, filename='baby_table.pgz', block_size=128):
    serialized_data = pickle.dumps(baby_table, protocol=pickle.HIGHEST_PROTOCOL)
    with subprocess.Popen(['pigz', '-7', '-b', str(block_size), '-c'], stdin=subprocess.PIPE, stdout=open(filename, 'wb')) as proc:
        proc.communicate(input=serialized_data)
    print(f"[+] Baby table saved")

# Load baby table from a compressed file
def load_baby_table(filename='baby_table.pgz'):
    print(f"[+] Loading baby table...")
    try:
        with subprocess.Popen(['pigz', '-d', '-c', filename], stdout=subprocess.PIPE) as proc:
            serialized_data = proc.stdout.read()
        baby_table = pickle.loads(serialized_data)
        print(f"[+] Baby table loaded successfully with {len(baby_table)} entries")
        return baby_table
    except Exception as e:
        print(f"[error] Failed to load baby table: {e}")
        return None

# Delete existing baby table if it exists
def delete_existing_table(filename='baby_table.pgz'):
    if os.path.exists(filename):
        os.remove(filename)

# Main Script
if __name__ == "__main__":
    # Puzzle Parameters
    puzzle = 40
    start_range = 2**(puzzle-1)
    end_range = (2**puzzle) - 1
    puzzle_pubkey = '03a2efa402fd5268400c77c20e574ba86409ededee7c4020e4b9f0edbee53de0d4'

    # Parse Public Key
    if len(puzzle_pubkey) != 66:
        print("[error] Public key length invalid!")
        sys.exit(1)
    prefix = puzzle_pubkey[:2]
    X = mpz(int(puzzle_pubkey[2:], 16))
    y_parity = int(prefix) - 2
    Y = X2Y(X, y_parity)
    if Y is None:
        print("[error] Invalid compressed public key!")
        sys.exit(1)
    P = (X, Y)  # Uncompressed public key

    # Precompute m and mP for BSGS
    m = int(math.floor(math.sqrt(end_range - start_range)))
    m_P = mul(m)

    # Delete existing baby table if it exists
    delete_existing_table('baby_table.pgz')

    # Create Baby Table with SortedDict
    print('[+] Creating babyTable...')
    baby_table = SortedDict()
    Ps = (0, 0)  # Start with the point at infinity
    for i in range(m + 1):
        cpub = point_to_cpub(Ps)
        cpub_hash = hash_cpub(cpub)
        binary_bits = pack_hash_into_bits(cpub)
        baby_table[cpub_hash] = (i, binary_bits)
        Ps = add(Ps, PG)

    # Save the baby table to a file
    save_baby_table(baby_table, 'baby_table.pgz', block_size=128)

    # Force reload from file by deleting the in-memory table and freeing memory
    del baby_table
    gc.collect()

    # Load the baby table from the file
    baby_table = load_baby_table('baby_table.pgz')
    if baby_table is None:
        print("[error] Failed to load baby table. Exiting...")
        sys.exit(1)

    # BSGS Search
    print('[+] BSGS Search in progress')
    S = point_subtraction(P, mul(start_range))
    step = 0
    st = time.time()
    while step < (end_range - start_range):
        cpub = point_to_cpub(S)
        cpub_hash = hash_cpub(cpub)
        if cpub_hash in baby_table:
            b, binary_bits = baby_table[cpub_hash]
            k = start_range + step + b
            if point_to_cpub(mul(k)) == puzzle_pubkey:
                print(f'[+] m={m} step={step} b={b}')
                print(f'[+] Key found: {k}')
                print("[+] Time Spent : {0:.2f} seconds".format(time.time() - st))
                sys.exit()
        S = point_subtraction(S, m_P)
        step += m

    print('[+] Key not found')
    print("[+] Time Spent : {0:.2f} seconds".format(time.time() - st))

  • BSGS: Sat Feb 22 08:27:19 2025
  • Creating babyTable...
  • Baby table saved
  • Loading baby table...
  • Baby table loaded successfully with 741383 entries
  • BSGS Search in progress
  • m=741455 step=453895024440 b=574622
  • Key found: 1003651412950
  • Time Spent : 2.78 seconds

The script hashes a compressed public key using xxhash, truncating the result to the first 8 characters to create a short, unique identifier for efficient indexing.

It then converts the hashed public key into a list of binary bits by processing segments of the hash, creating a compact binary representation based on McDoUgLaSx's code.

Finally, it serializes and compresses the baby table (a SortedDict of precomputed values) into a file using pigz (note: you need to install pigz for this functionality).

For Puzzle 40, the size of the baby table is approximately 6MB....

This implementation is designed for small puzzles (up to puzzle 52). Solving larger puzzles requires significant computational resources, and the code must be modified to include the Bloom Filter and other optimizations, especially in C++. Do not attempt puzzle 66 or higher unless you have at least 256GB of RAM and 500GB of free storage space. Otherwise, you may crash your PC. It is intended for educational and testing purposes.

BTC: bc1qdwnxr7s08xwelpjy3cc52rrxg63xsmagv50fa8
Akito S. M. Hosana
Jr. Member
*
Offline Offline

Activity: 420
Merit: 8


View Profile
February 22, 2025, 08:04:52 AM
 #62

  • Time Spent : 2.78 seconds
Holy moly, this is faster than that Kangaroo script in Python.  Grin
mcdouglasx (OP)
Hero Member
*****
Offline Offline

Activity: 952
Merit: 532



View Profile WWW
February 22, 2025, 03:43:06 PM
 #63

  • Time Spent : 2.78 seconds
Holy moly, this is faster than that Kangaroo script in Python.  Grin

We are awaiting an improvement in the efficient handling of bits, which is the greatest problem faced by this binary method. If such a barrier did not exist, BSGS scope could be exponentially increased due to the low resource consumption required by binary methods to identify an enormous set of public keys.
iceland2k14
Member
**
Offline Offline

Activity: 76
Merit: 89


View Profile
February 23, 2025, 09:13:19 AM
Merited by Halab (2), mcdouglasx (1)
 #64

I would like to think about this Light Weight Database approach to see if that helps... (Later)
Currently without this and just using the BSGS script (15 MB Table) solves Puzzles #40 in 0.49 sec.

Code:
import math
import time
import os
import secp256k1 as ice


# Main Script
if __name__ == "__main__":
    # Puzzle Parameters
    puzzle = 45
    start_range = 2**(puzzle-1)
    end_range = (2**puzzle) - 1
    puzzle_pubkey = '026ecabd2d22fdb737be21975ce9a694e108eb94f3649c586cc7461c8abf5da71a'

    P = ice.pub2upub(puzzle_pubkey)

    # Precompute m and mP for BSGS
    m = int(math.floor(math.sqrt(end_range - start_range)))
    m = m * 20  # for use in bsgs_2nd
    m_P = ice.scalar_multiplication(m)
    m_Pn = ice.point_negation(m_P)
    m2 = math.ceil((end_range - start_range)/m)
    print(f'[+] m={m} m2={m2} Puzzle={puzzle}')
   
    if os.path.isfile('baby_table.2nd') == False:
        print(f'[+] Preparing for BSGS Table with {m} elements in RAM')
        ice.bsgs_2nd_check_prepare(m)
        ice.dump_bsgs_2nd('baby_table.2nd', True)
    else:
        ice.load_bsgs_2nd('baby_table.2nd', True)
       


    # BSGS Search
    print('[+] BSGS Search in progress')
    st = time.time()
    S = ice.point_subtraction(P, ice.scalar_multiplication(start_range))
    found, pvk = ice.bsgs_2nd_check(S, 1)
    if found == True:
        k = start_range + int.from_bytes(pvk, 'big')
        print(f'FOUND PrivateKey: {k:064x}')
        print(f"[+] Time Spent : {time.time() - st:.2f} seconds")
        exit()
       
    ice.init_P2_Group(m_Pn) # Negative for P2_mcpu
    SL = ice.point_sequential_increment_P2_mcpu(m2, S) # succesive decrement
    found, pvk = ice.bsgs_2nd_check_mcpu(SL, 1)
    if int.from_bytes(found, 'big') > 0:
        elapsed = time.time() - st
        for i in range(m2):
            if found[i] == 1:
                print(f'start=[0x{start_range:0x}] [i={i}] idx=0x{int.from_bytes(pvk[i*32:(i+1)*32], "big"):0x} ')
                k = start_range + (i+1) * m + int.from_bytes(pvk[i*32:(i+1)*32], 'big')
                print(f'FOUND PrivateKey: {k:064x}')
                print(f"[+] Time Spent : {elapsed:.2f} seconds")
                exit()

    print('[+] Key not found')
    print(f"[+] Time Spent : {time.time() - st:.2f} seconds")

Result for Puzzle #45 is

Code:
python ULWD_bsgs.py
[+] m=83886060 m2=209716 Puzzle=45
[+] Preparing for BSGS Table with 83886060 elements in RAM
Calculating      : 100 %
[+] [N2:4194500, N3:209725, N4:10486]  Vec4th Size    : 10486
[+] [bloom2nd (14 MB)] [bloom3rd (0 MB)] [bloom4th (0 MB)]
[+] BSGS Search in progress
start=[0x100000000000] [i=28660] idx=0x11cfb29
FOUND PrivateKey: 0000000000000000000000000000000000000000000000000000122fca143c05
[+] Time Spent : 2.80 seconds

For bigger puzzles lightweight strategy could help with iteration in primary collision and then secondary check for confirmation using the table.
nomachine
Full Member
***
Offline Offline

Activity: 812
Merit: 134



View Profile
February 23, 2025, 05:04:42 PM
Last edit: February 23, 2025, 05:32:25 PM by nomachine
 #65

Your Python implementation using .so libraries is likely more optimized and includes advanced features written in C, which is great for performance.  Wink

Here is my C++ implementation of the same script in Python. It is designed to be minimalistic, focusing only on the core BSGS algorithm without adding extra features like multicore support or Bloom filters. This makes it easier to understand and experiment with—no mumbo-jumbo.....  Grin

bsgs.cpp
Code:
#include <iostream>
#include <fstream>
#include <vector>
#include <unordered_map>
#include <cmath>
#include <gmpxx.h>
#include <chrono>
#include <cassert>
#include <cstdio>
#include <sys/stat.h>
#include <xxhash.h>

using namespace std;

typedef array<mpz_class, 2> Point;

const mpz_class modulo("FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEFFFFFC2F", 16);
const mpz_class order("FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141", 16);
const mpz_class Gx("79BE667EF9DCBBAC55A06295CE870B07029BFCDB2DCE28D959F2815B16F81798", 16);
const mpz_class Gy("483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8", 16);
const Point PG = {Gx, Gy};
const Point Z = {0, 0};

auto starttime = chrono::high_resolution_clock::now();

inline Point add(const Point& P, const Point& Q, const mpz_class& p = modulo) {
    if (P == Z) return Q;
    if (Q == Z) return P;
    const mpz_class& P0 = P[0];
    const mpz_class& P1 = P[1];
    const mpz_class& Q0 = Q[0];
    const mpz_class& Q1 = Q[1];
    mpz_class lmbda, num, denom, inv;
    if (P != Q) {
        num = Q1 - P1;
        denom = Q0 - P0;
    } else {
        if (P1 == 0) return Z;
        num = 3 * P0 * P0;
        denom = 2 * P1;
    }
    mpz_invert(inv.get_mpz_t(), denom.get_mpz_t(), p.get_mpz_t());
    lmbda = (num * inv) % p;
    mpz_class x = (lmbda * lmbda - P0 - Q0) % p;
    if (x < 0) x += p;
    mpz_class y = (lmbda * (P0 - x) - P1) % p;
    if (y < 0) y += p;
    return {x, y};
}

inline Point mul(const mpz_class& k, const Point& P = PG, const mpz_class& p = modulo) {
    Point R0 = Z, R1 = P;
    unsigned long bit_length = mpz_sizeinbase(k.get_mpz_t(), 2);
    for (unsigned long i = bit_length - 1; i < bit_length; --i) {
        if (mpz_tstbit(k.get_mpz_t(), i)) {
            R0 = add(R0, R1, p);
            R1 = add(R1, R1, p);
        } else {
            R1 = add(R0, R1, p);
            R0 = add(R0, R0, p);
        }
    }
    return R0;
}

inline Point point_subtraction(const Point& P, const Point& Q) {
    Point Q_neg = {Q[0], (-Q[1]) % modulo};
    return add(P, Q_neg);
}

inline mpz_class X2Y(const mpz_class& X, int y_parity, const mpz_class& p = modulo) {
    mpz_class X_cubed = (X * X * X) % p;
    mpz_class tmp = (X_cubed + mpz_class(7)) % p;
    mpz_class Y;
    mpz_class exp = (p + mpz_class(1)) / mpz_class(4);
    mpz_powm(Y.get_mpz_t(), tmp.get_mpz_t(), exp.get_mpz_t(), p.get_mpz_t());
    if ((Y % 2) != y_parity) {
        Y = p - Y;
    }
    return Y;
}

inline std::string point_to_cpub(const Point& point) {
    mpz_class x = point[0], y = point[1];
    int y_parity = y.get_ui() & 1;
    std::string prefix = y_parity == 0 ? "02" : "03";
    char buffer[65];
    mpz_get_str(buffer, 16, x.get_mpz_t());
    return prefix + std::string(buffer);
}

inline std::string hash_cpub(const std::string& cpub) {
    XXH64_hash_t hash = XXH64(cpub.c_str(), cpub.size(), 0);
    char buffer[17];
    snprintf(buffer, sizeof(buffer), "%016lx", hash);
    return std::string(buffer, 8);
}

void save_baby_table(const std::unordered_map<std::string, int>& baby_table, const std::string& filename = "baby_table") {
    std::ofstream outfile(filename, std::ios::binary);
    if (!outfile) {
        std::cerr << "[error] Failed to open file for writing." << std::endl;
        return;
    }
    for (const auto& entry : baby_table) {
        outfile.write(entry.first.c_str(), 8);
        int index = entry.second;
        outfile.write(reinterpret_cast<const char*>(&index), sizeof(index));
    }
    std::cout << "[+] Baby table saved" << std::endl;
    std::string command = "pigz -7 -b 128 -f " + filename;
    FILE* pipe = popen(command.c_str(), "r");
    if (!pipe) {
        std::cerr << "[error] Failed to compress file." << std::endl;
        return;
    }
    pclose(pipe);
    std::cout << "[+] File compressed: " << filename << ".gz" << std::endl;
}

std::unordered_map<std::string, int> load_baby_table(const std::string& filename = "baby_table") {
    std::string command = "pigz -d -c " + filename + ".gz";
    FILE* pipe = popen(command.c_str(), "r");
    if (!pipe) {
        std::cerr << "[error] Failed to decompress file." << std::endl;
        return {};
    }
    std::unordered_map<std::string, int> baby_table;
    char key_buffer[9];
    int index;
    while (fread(key_buffer, 8, 1, pipe) == 1) {
        key_buffer[8] = '\0';
        fread(&index, sizeof(index), 1, pipe);
        baby_table[key_buffer] = index;
    }
    pclose(pipe);
    std::cout << "[+] Baby table loaded successfully with " << baby_table.size() << " entries" << std::endl;
    return baby_table;
}

void delete_existing_table(const std::string& filename = "baby_table") {
    struct stat buffer;
    if (stat(filename.c_str(), &buffer) == 0) {
        if (std::remove(filename.c_str()) != 0) {
            std::cerr << "[error] Failed to delete existing table." << std::endl;
        }
    }
    std::string compressed_filename = filename + ".gz";
    if (stat(compressed_filename.c_str(), &buffer) == 0) {
        if (std::remove(compressed_filename.c_str()) != 0) {
            std::cerr << "[error] Failed to delete compressed table." << std::endl;
        }
    }
}

int main() {
    int puzzle = 40;
    mpz_class start_range = mpz_class(1) << (puzzle - 1);
    mpz_class end_range = (mpz_class(1) << puzzle) - 1;
    std::string puzzle_pubkey = "03a2efa402fd5268400c77c20e574ba86409ededee7c4020e4b9f0edbee53de0d4";
  
    time_t currentTime = time(nullptr);
    cout << "\r\033[01;33m[+]\033[32m BSGS: \033[01;33m" << ctime(&currentTime) << "\033[0m" << "\r";
    cout << "[+] [Puzzle]: " << puzzle << endl;
  
    if (puzzle_pubkey.length() != 66) {
        std::cerr << "[error] Public key length invalid!" << std::endl;
        return 1;
    }
    std::string prefix = puzzle_pubkey.substr(0, 2);
    mpz_class X(puzzle_pubkey.substr(2), 16);
    int y_parity = std::stoi(prefix) - 2;
    mpz_class Y = X2Y(X, y_parity);
    if (Y == 0) {
        std::cerr << "[error] Invalid compressed public key!" << std::endl;
        return 1;
    }
    Point P = {X, Y};

    mpz_class m = sqrt(end_range - start_range);
    Point m_P = mul(m);

    delete_existing_table("baby_table");

    std::cout << "[+] Creating babyTable..." << std::endl;
    std::unordered_map<std::string, int> baby_table;
    Point Ps = Z;
    for (unsigned long i = 0; i <= m.get_ui(); ++i) {
        std::string cpub = point_to_cpub(Ps);
        std::string cpub_hash = hash_cpub(cpub);
        baby_table[cpub_hash] = i;
        Ps = add(Ps, PG);
    }

    save_baby_table(baby_table, "baby_table");

    baby_table.clear();

    baby_table = load_baby_table("baby_table");
    if (baby_table.empty()) {
        std::cerr << "[error] Failed to load baby table. Exiting..." << std::endl;
        return 1;
    }

    std::cout << "[+] BSGS Search in progress" << std::endl;
    Point S = point_subtraction(P, mul(start_range));
    mpz_class step = 0;
    auto st = std::chrono::high_resolution_clock::now();
    while (step < (end_range - start_range)) {
        std::string cpub = point_to_cpub(S);
        std::string cpub_hash = hash_cpub(cpub);
        auto it = baby_table.find(cpub_hash);
        if (it != baby_table.end()) {
            int b = it->second;
            mpz_class k = start_range + step + b;
            if (point_to_cpub(mul(k)) == puzzle_pubkey) {
                std::cout << "[+] m=" << m << " step=" << step << " b=" << b << std::endl;
                std::cout << "[+] Key found: " << k << std::endl;
                auto et = std::chrono::high_resolution_clock::now();
                std::chrono::duration<double> elapsed = et - st;
                std::cout << "[+] Time Spent : " << elapsed.count() << " seconds" << std::endl;
                return 0;
            }
        }
        S = point_subtraction(S, m_P);
        step += m;
    }

    std::cout << "[+] Key not found" << std::endl;
    auto et = std::chrono::high_resolution_clock::now();
    std::chrono::duration<double> elapsed = et - st;
    std::cout << "[+] Time Spent : " << elapsed.count() << " seconds" << std::endl;
    return 0;
}


You need the g++ compiler and make to build application:
Code:
sudo apt install build-essential
GMP Library:
Code:
sudo apt install libgmp-dev
XXHash Library:
Code:
sudo apt install libxxhash-dev
Pigz:
Code:
sudo apt install pigz


Compile:
Code:
g++ -o bsgs bsgs.cpp -m64 -march=native -mtune=native -Wall -ftree-vectorize -flto -O3 -funroll-loops -fomit-frame-pointer -fno-rtti -lgmp -lgmpxx -lxxhash

  • BSGS: Sun Feb 23 18:02:02 2025
  • [Puzzle]: 40
  • Creating babyTable...
  • Baby table saved
  • File compressed: baby_table.gz
  • Baby table loaded successfully with 741055 entries
  • BSGS Search in progress
  • m=741455 step=453895024440 b=574622
  • Key found: 1003651412950
  • Time Spent : 1.6087 seconds

Single-core result. On a 4 year old PC.

BTC: bc1qdwnxr7s08xwelpjy3cc52rrxg63xsmagv50fa8
Akito S. M. Hosana
Jr. Member
*
Offline Offline

Activity: 420
Merit: 8


View Profile
February 23, 2025, 06:48:45 PM
Last edit: February 23, 2025, 07:37:36 PM by Akito S. M. Hosana
 #66

Single-core result. On a 4 year old PC.

  • [Puzzle]: 57
  • Creating babyTable...
  • Baby table saved
  • File compressed: baby_table.gz
  • Baby table loaded successfully with 260216853 entries
  • BSGS Search in progress
  • m=268435455 step=66188164767523695 b=105394861
  • Key found: 138245758910846492


Puzzle 57 has a 2.1 GB table size with this. How large will it be for Puzzle 66? 100GB? That would be impossible to solve without 128GB of RAM, right?
nomachine
Full Member
***
Offline Offline

Activity: 812
Merit: 134



View Profile
February 23, 2025, 06:52:19 PM
 #67

Puzzle 57 has a 2.1 GB table size with this. How large will it be for Puzzle 66? 100GB? That would be impossible to solve without 128GB of RAM, right?

Yes, you need to allocate 90% of the available RAM as the maximum size for the BSGS table to avoid overloading the system. You can create a new script to calculate the maximum number of entries (m) that can fit in the allocated memory, similar to how Keyhunt does it. You can use combinations with random ranges, Bloom filters, and other techniques. However, it's a losing battle for anything above puzzle 80. Someone will need to figure out a way to pack an even smaller table.

BTC: bc1qdwnxr7s08xwelpjy3cc52rrxg63xsmagv50fa8
Geshma
Newbie
*
Offline Offline

Activity: 19
Merit: 0


View Profile
February 24, 2025, 08:16:49 AM
 #68

  • BSGS with Two Grumpy Giants and a Baby: Mon Feb 24 08:04:32 2025
  • Creating babyTable...
  • Baby table saved
  • Loading baby table...
  • Baby table loaded successfully with 741383 entries
  • BSGS Search in progress with Two Grumpy Giants and a Baby
  • m=741455 step=612168 b=574622
  • Key found: 1003651412950
  • Time Spent : 4.15 seconds

number of steps have reduced drastically
Akito S. M. Hosana
Jr. Member
*
Offline Offline

Activity: 420
Merit: 8


View Profile
February 24, 2025, 09:10:57 AM
Last edit: February 24, 2025, 09:46:16 AM by Akito S. M. Hosana
 #69

Code:
    mpz_class m = sqrt(end_range - start_range);
    m = m * 4;
    Point m_P = mul(m);

If you make m 4 times bigger then the result is less than a second. But the table is 25MB  Undecided

C++
  • BSGS: Mon Feb 24 09:06:40 2025
  • [Puzzle]: 40
  • Creating babyTable...
  • Baby table saved
  • File compressed: baby_table.gz
  • Baby table loaded successfully with 2964220 entries
  • BSGS Search in progress
  • m=2965820 step=453895024440 b=574622
  • Key found: 1003651412950
  • Time Spent : 0.420612 seconds

Python
  • BSGS: Mon Feb 24 09:12:13 2025
  • Creating babyTable...
  • Baby table saved
  • Loading baby table...
  • Baby table loaded successfully with 2964806 entries
  • BSGS Search in progress
  • m=2965820 step=453895024440 b=574622
  • Key found: 1003651412950
  • Time Spent : 0.73 seconds
nomachine
Full Member
***
Offline Offline

Activity: 812
Merit: 134



View Profile
February 24, 2025, 09:22:37 AM
 #70

A balance must be found between the size of the baby table and the time required to solve it. If the table is too big, then you don't have enough RAM, if it is too small, then you don't have enough time. It's the curse of this hobby. Grin

BTC: bc1qdwnxr7s08xwelpjy3cc52rrxg63xsmagv50fa8
Akito S. M. Hosana
Jr. Member
*
Offline Offline

Activity: 420
Merit: 8


View Profile
February 24, 2025, 10:44:54 AM
 #71

We can manipulate RAM in various ways using different approaches. The goal of this topic is to make the table or database as small and compressed as possible. This can be achieved through binary methods, truncating the public key, or limiting the index. We could compress everything further how NoMachine does it. So far, I have not found a better improvement that doesn't increase the number of false positives or risk a complete crash of the application or even the entire PC. Cry
nomachine
Full Member
***
Offline Offline

Activity: 812
Merit: 134



View Profile
February 24, 2025, 10:54:12 AM
 #72

Maybe the solution is to ask AI less for advice and use your own brain more. It's useful for fixing some bugs in code, but it's not meant for making any concrete progress. Trust me. Grin

BTC: bc1qdwnxr7s08xwelpjy3cc52rrxg63xsmagv50fa8
Pages: « 1 2 3 [4]  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!