Bitcoin Forum
May 27, 2024, 04:42:15 PM
Welcome,
Guest
. Please
login
or
register
.
News
: Latest Bitcoin Core release:
27.0
[
Torrent
]
Home
Help
Search
Login
Register
More
Bitcoin Forum
>
Bitcoin
>
Development & Technical Discussion
>
What data have Bitcoin Core blockchain files?
Pages: [
1
]
« previous topic
next topic »
Print
Author
Topic: What data have Bitcoin Core blockchain files? (Read 138 times)
pbies
(OP)
Full Member
Offline
Activity: 254
Merit: 126
What data have Bitcoin Core blockchain files?
June 12, 2022, 03:11:24 AM
#1
I am interested in writing a Python script to parse BerkeleyDB in which is blockchain on disk working.
I know that this is old BerkeleyDB and I need to use bsddb3 with Python 3.9 or earlier.
What data can I find in these folders?
1. blocks+index
2. chainstate
3. indexes
How should I instruct Python to connect to whole blockchain in these files and then query it?
Any help appreciated.
BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
I have 9900K and 1080 Ti, gathering funds for new desktop PC for Bitcoin operations - 12900K and RTX 4090
pbies
(OP)
Full Member
Offline
Activity: 254
Merit: 126
Re: What data have Bitcoin Core blockchain files?
June 12, 2022, 03:28:24 PM
#2
Quote from: ETFbitcoin on June 12, 2022, 11:24:54 AM
1. Use Bitcoin Core RPC JSON to obtain block data.
That is too slow for me. I need access to whole data in hours, not months.
One thing I would like to do is to dump all public addresses with any additional data.
BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
I have 9900K and 1080 Ti, gathering funds for new desktop PC for Bitcoin operations - 12900K and RTX 4090
pooya87
Legendary
Offline
Activity: 3458
Merit: 10594
Re: What data have Bitcoin Core blockchain files?
June 13, 2022, 03:17:05 AM
Merited
by
ABCbits
(1)
#3
Quote from: pbies on June 12, 2022, 03:28:24 PM
One thing I would like to do is to dump all public addresses with any additional data.
Unless you are trying to do something special that is not to extract pubkeys to brute force private keys lol you can use existing lists that contain all bitcoin addresses that have ever been used[1] or list of addresses with a balance[2].
[1]
https://bitcointalk.org/index.php?topic=5265993.0
[2]
https://bitcointalk.org/index.php?topic=5254914.0
.
.
B
L
A
C
K
J
A
C
K
♠
F
U
N
.
███
▄
██
██
██
█████
██
██
██
███
▀
██
████████
██
███
██
██████
██
██
██
██
██
████████
██
██
▄▄
░██
████
██
█████
▀░▀▀
██
████████████
████
░██
██████████
██
██████
█
██
███
██
███
▄
░████
██████
██
██
▀
███████
███
██
██
██
███
██████
██░██
██████
██
██
CRYPTO
CASINO
&
SPORTS
BETTING
│
.
REWARD SYSTEM
.
E
L
I
T
E
S
E
A
T
..
│
▄▄███████▄▄
▄████
███████
████▄
███
█████████████
███
██
█████
█████████
███
██
██
█████
██
██
██
████████
██
██
█████
███████████
█████
██
██
█████
██
███
██
█████████
██
██
██████
██████
█████████
██
██
█████████
██
████████
██
██
████████
██
███████
██
███
████
███
██████
███
▀████
███████
████▀
█
█
█
█
█
█████████
█
█
█
█
█
.
Whitepaper
J
A
C
K
NotATether
Legendary
Offline
Activity: 1610
Merit: 6813
bitcoincleanup.com / bitmixlist.org
Re: What data have Bitcoin Core blockchain files?
June 13, 2022, 03:59:22 AM
#4
The most reliable way to parse the BDB files is simply to copy the DB read/write code of Bitcoin Core and make a CLI frontend that will let you query any data. That's because drivers like db_dump(1) and others are not guarranteed to work, as within the BDB file format, the program can write it in however manner it likes (i.e. it's technically a BDB format but it's unreadable to other applications).
.
.
B
L
A
C
K
J
A
C
K
♠
F
U
N
.
███
▄
██
██
██
█████
██
██
██
███
▀
██
████████
██
███
██
██████
██
██
██
██
██
████████
██
██
▄▄
░██
████
██
█████
▀░▀▀
██
████████████
████
░██
██████████
██
██████
█
██
███
██
███
▄
░████
██████
██
██
▀
███████
███
██
██
██
███
██████
██░██
██████
██
██
CRYPTO
CASINO
&
SPORTS
BETTING
│
.
REWARD SYSTEM
.
E
L
I
T
E
S
E
A
T
..
│
▄▄███████▄▄
▄████
███████
████▄
███
█████████████
███
██
█████
█████████
███
██
██
█████
██
██
██
████████
██
██
█████
███████████
█████
██
██
█████
██
███
██
█████████
██
██
██████
██████
█████████
██
██
█████████
██
████████
██
██
████████
██
███████
██
███
████
███
██████
███
▀████
███████
████▀
█
█
█
█
█
█████████
█
█
█
█
█
.
Whitepaper
J
A
C
K
pbies
(OP)
Full Member
Offline
Activity: 254
Merit: 126
Re: What data have Bitcoin Core blockchain files?
June 13, 2022, 10:45:05 AM
#5
I am trying to write my own Python scripts to mess around with BerkeleyDB and Bitcoin Core blockchain.
I am not interested in ready solutions that provide list of addresses - they are always outdated, I need to take data from current blockchain.
To translate C/C++ code of Bitcoin Core to Python is last resort solution.
RPC is too slow to call it for whole blockchain, and it will eat too much resources. I have synced Bitcoin Core, the blockchain. I need to operate on BerkeleyDB directly.
Dolthub is a remote solution, I need local solution.
BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
I have 9900K and 1080 Ti, gathering funds for new desktop PC for Bitcoin operations - 12900K and RTX 4090
ABCbits
Legendary
Offline
Activity: 2884
Merit: 7524
Crypto Swap Exchange
Re: What data have Bitcoin Core blockchain files?
June 13, 2022, 12:36:11 PM
Merited
by
pooya87
(4)
#6
Quote from: pbies on June 13, 2022, 10:45:05 AM
RPC is too slow to call it for whole blockchain, and it will eat too much resources. I have synced Bitcoin Core, the blockchain. I need to operate on BerkeleyDB directly.
Since Bitcoin Core RPC-JSON doesn't meet your need, here are few pointers for reading the files directly.
1.
https://bitcoindev.network/understanding-the-data/
2.
https://github.com/bitcoin/bitcoin/issues/6613
3.
https://learnmeabitcoin.com/technical/blkdat
4.
https://bitcoin.stackexchange.com/a/48738
P.S. I would still recommend you to check self-hosted block explorer (e.g. mempool.space) and Electrum server (e.g.
Fulcrum
).
Quote from: pbies on June 13, 2022, 10:45:05 AM
Dolthub is a remote solution, I need local solution.
Only if you run the SQL query on their website rather than cloning the data.
█▀▀▀
█
█
█
█
█
█
█
█
█
█
█
█▄▄▄
▀▀▀▀▀▀▀▀▀▀▀
e
▄▄▄▄▄▄▄▄▄▄▄
████
█
█████████
▄
███
██
███████
▄███
██
▐███
████
▄█████▀
███
████
██
▄████▀
███
▐████▄███▀
████
▐██████▀
█████
▀█████
█████
██████▄
████
████████▄
██
▄█████▀█████▄
▄██████
███
▀█████▀
█
█████
██████
▀██▀
██
██▀
█████████
▀
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
c
.
h.
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▀▀▀█
█
█
█
█
█
█
█
█
█
█
█
▄▄▄█
▄█
█████▄▄▄
██
███████████▄▄
██
████
█████████
██
████
██
███████
██
████
██
███████
██
███████
██████
██
█░░████
█████
██
█▌▐████
█████
██
███████
████
██
█████████▀
██
████████▀
██
██████▀
░
▀
▀██▀▀
✔
Instant
✔
Non KYC
Crypto Swap Exchange
.
.
.
.
.
........
Bitcoin | Litecoin | Ethereum | Monero | Dash | ERC20 Tokens | more soon
........
.
.
.
.
.
▄███████████████████▄
███████
██
█
██
█████████
██▀▀▀▀▀
██
▀
██
▀▀███████
███
█████████████
▀████
█████
████
███▄
████
████
█████
████
███▀
████
████
█████
███████████
▀████
█████
████
████▄
████
███
█████
████
████▀
████
███
███
█████████████
▄████
ANN
THREAD
▄██████████████████▄
███████████▀▄▀██████
██████████▀
█
▄███████
████████▀▌
█
▄████████
████████▌▄
█
█████████
███████▀▄█
██
▀███████
████▀▄▄█
█
██
███
▀█████
███▀▄█▀▄██
█
█
████
████
███
█
█
█
█▀▄█▌▐
████
▐███
███
█
▌▐
█
█▀▌▐
█████
▐███
ONION
LINK
pbies
(OP)
Full Member
Offline
Activity: 254
Merit: 126
Re: What data have Bitcoin Core blockchain files?
June 13, 2022, 12:59:09 PM
#7
Quote from: ETFbitcoin on June 13, 2022, 12:36:11 PM
Quote from: pbies on June 13, 2022, 10:45:05 AM
RPC is too slow to call it for whole blockchain, and it will eat too much resources. I have synced Bitcoin Core, the blockchain. I need to operate on BerkeleyDB directly.
Since Bitcoin Core RPC-JSON doesn't meet your need, here are few pointers for reading the files directly.
1.
https://bitcoindev.network/understanding-the-data/
2.
https://github.com/bitcoin/bitcoin/issues/6613
3.
https://learnmeabitcoin.com/technical/blkdat
4.
https://bitcoin.stackexchange.com/a/48738
P.S. I would still recommend you to check self-hosted block explorer (e.g. mempool.space) and Electrum server (e.g.
Fulcrum
).
Quote from: pbies on June 13, 2022, 10:45:05 AM
Dolthub is a remote solution, I need local solution.
Only if you run the SQL query on their website rather than cloning the data.
Thank you very much! This is what I needed! Especially link no 1.
BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
I have 9900K and 1080 Ti, gathering funds for new desktop PC for Bitcoin operations - 12900K and RTX 4090
Pages: [
1
]
Print
Bitcoin Forum
>
Bitcoin
>
Development & Technical Discussion
>
What data have Bitcoin Core blockchain files?
« previous topic
next topic »
Jump to:
Please select a destination:
-----------------------------
Bitcoin
-----------------------------
=> Bitcoin Discussion
===> Legal
===> Press
===> Meetups
===> Important Announcements
=> Development & Technical Discussion
===> Wallet software
=====> Electrum
=====> Bitcoin Wallet for Android
=====> BitcoinJ
=====> Armory
=====> Mycelium
=====> Hardware wallets
=> Mining
===> Mining support
===> Pools
===> Mining software (miners)
===> Hardware
=====> Group buys
===> Mining speculation
=> Bitcoin Technical Support
=> Project Development
-----------------------------
Economy
-----------------------------
=> Economics
===> Speculation
=> Marketplace
===> Goods
=====> Computer hardware
=====> Digital goods
=======> Invites & Accounts
=====> Collectibles
===> Services
===> Currency exchange
===> Gambling
=====> Games and rounds
=====> Investor-based games
=====> Gambling discussion
===> Lending
=====> Long-term offers
===> Securities
===> Auctions
===> Service Announcements
=====> Micro Earnings
===> Service Discussion
=====> Web Wallets
=====> Exchanges
=> Trading Discussion
===> Scam Accusations
===> Reputation
-----------------------------
Other
-----------------------------
=> Meta
===> New forum software
===> Bitcoin Wiki
=> Politics & Society
=> Beginners & Help
=> Off-topic
=> Serious discussion
===> Ivory Tower
=> Archival
===> Корзина
===> CPU/GPU Bitcoin mining hardware
===> Chinese students
===> Obsolete (buying)
===> Obsolete (selling)
===> MultiBit
-----------------------------
Alternate cryptocurrencies
-----------------------------
=> Altcoin Discussion
=> Announcements (Altcoins)
===> Tokens (Altcoins)
=> Mining (Altcoins)
===> Pools (Altcoins)
=> Marketplace (Altcoins)
===> Service Announcements (Altcoins)
===> Service Discussion (Altcoins)
===> Bounties (Altcoins)
=> Speculation (Altcoins)
-----------------------------
Local
-----------------------------
=> العربية (Arabic)
===> العملات البديلة (Altcoins)
=====> النقاشات
===> إستفسارات و أسئلة المبتدئين
===> التعدين
===> النقاشات الأخرى
===> منصات التبادل
=> Bahasa Indonesia (Indonesian)
===> Marketplace (Bahasa Indonesia)
===> Mining (Bahasa Indonesia)
===> Altcoins (Bahasa Indonesia)
===> Trading dan Spekulasi
===> Ekonomi, Politik, dan Budaya
===> Topik Lainnya
=> Español (Spanish)
===> Mercado y Economía
=====> Servicios
=====> Trading y especulación
===> Hardware y Minería
===> Esquina Libre
===> Mercadillo
=====> Mexico
=====> Argentina
=====> España
=====> Centroamerica y Caribe
===> Primeros pasos y ayuda
===> Altcoins (criptomonedas alternativas)
=====> Minería de altcoins
=====> Servicios
=====> Tokens (Español)
=> 中文 (Chinese)
===> 跳蚤市场
===> 山寨币
===> 媒体
===> 挖矿
===> 离题万里
=> Hrvatski (Croatian)
===> Trgovina
===> Altcoins (Hrvatski)
=====> Announcements (Hrvatski)
===> Off-topic (Hrvatski)
=> Deutsch (German)
===> Anfänger und Hilfe
===> Mining (Deutsch)
===> Trading und Spekulation
===> Projektentwicklung
===> Off-Topic (Deutsch)
===> Treffen
===> Presse
===> Altcoins (Deutsch)
=====> Announcements (Deutsch)
===> Marktplatz
=====> Auktionen
=====> Suche
=====> Biete
=> Ελληνικά (Greek)
===> Αγορά
===> Mining Discussion (Ελληνικά)
===> Altcoins (Ελληνικά)
=====> Altcoin Announcements (Ελληνικά)
=====> Altcoin Mining (Ελληνικά)
=> עברית (Hebrew)
=> Français
===> Actualité et News
===> Débutants
===> Discussions générales et utilisation du Bitcoin
===> Mining et Hardware
===> Économie et spéculation
===> Place de marché
=====> Échanges
=====> Produits et services
=====> Petites annonces
===> Le Bitcoin et la loi
===> Wiki, documentation et traduction
===> Développement et technique
===> Vos sites et projets
===> Hors-sujet
===> Altcoins (Français)
=====> Annonces
=> India
===> Mining (India)
===> Marketplace (India)
===> Regional Languages (India)
===> Press & News from India
===> Alt Coins (India)
===> Buyer/ Seller Reputations (India)
===> Off-Topic (India)
=> Italiano (Italian)
===> Guide (Italiano)
===> Progetti
===> Discussioni avanzate e sviluppo
===> Trading, analisi e speculazione
===> Mercato
=====> Mercato valute
=====> Beni
=====> Servizi
=====> Esercizi commerciali
=====> Hardware/Mining (Italiano)
=====> Gambling (Italiano)
===> Accuse scam/truffe
===> Mining (Italiano)
===> Alt-Currencies (Italiano)
=====> Annunci
===> Raduni/Meeting (Italiano)
===> Crittografia e decentralizzazione
===> Off-Topic (Italiano)
=> 日本語 (Japanese)
===> アルトコイン
=> Nederlands (Dutch)
===> Markt
===> Gokken/lotterijen
===> Mining (Nederlands)
===> Beurzen
===> Alt Coins (Nederlands)
===> Off-topic (Nederlands)
===> Meetings (Nederlands)
=> Nigeria (Naija)
===> Politics and society (Naija)
===> Off-topic (Naija)
=> 한국어 (Korean)
===> 대체코인 Alt Coins (한국어)
=> Pilipinas
===> Altcoins (Pilipinas)
=====> Altcoin Announcements (Pilipinas)
===> Pamilihan
===> Others (Pilipinas)
=> Polski
===> Tablica ogłoszeń
===> Alternatywne kryptowaluty
=====> Nowe kryptowaluty i tokeny
=====> Tablica ogłoszeń (altcoiny)
=> Português (Portuguese)
===> Primeiros Passos (Iniciantes)
===> Economia & Mercado
===> Mineração em Geral
===> Desenvolvimento & Discussões Técnicas
===> Criptomoedas Alternativas
===> Brasil
===> Portugal
=> Русский (Russian)
===> Новички
===> Бизнес
=====> Барахолка
=====> Обменники
===> Идеи
===> Кодеры
===> Майнеры
===> Политика
===> Трейдеры
===> Альтернативные криптовалюты
=====> Токены
=====> Бayнти и aиpдpoпы
===> Хайпы
===> Работа
===> Разное
===> Oбcyждeниe Bitcoin
=====> Новости
=====> Юристы
=> Română (Romanian)
===> Anunturi importante
===> Offtopic
===> Market
=====> Discutii Servicii
===> Minerit
===> Tutoriale
===> Bine ai venit!
===> Presa
===> Altcoins (Monede Alternative)
=====> Anunturi Monede Alternative
=> Skandinavisk
=> Türkçe (Turkish)
===> Bitcoin Haberleri
===> Pazar Alanı
===> Madencilik
===> Ekonomi
===> Servisler
=====> Fonlar
===> Proje Geliştirme
===> Alternatif Kripto-Paralar
=====> Madencilik (Alternatif Kripto-Paralar)
=====> Duyurular (Alternatif Kripto-Paralar)
===> Konu Dışı
===> Yeni Başlayanlar & Yardım
===> Buluşmalar
=> Other languages/locations
Loading...