Bitcoin Forum
December 18, 2017, 07:55:07 AM *
News: Latest stable version of Bitcoin Core: 0.15.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: How does Bitcoin block index work?  (Read 998 times)
g2com
Sr. Member
****
Offline Offline

Activity: 252


View Profile
February 14, 2017, 07:58:04 PM
 #1

Hi, I can find few information on Bitcoin's storage. So I understand that blkindex.dat is for the block index and blk00x.dat for transactions. However I don't understand how is this index generated. I understand an index file speeds up data lookup. However LevelDB states it doesn't support index so how is this index used along with LevelDB data? Also, a transaction is a nested structure that contains multiple inputs and outputs. LevelDB is a key-value storage engine so how does it recognize the hierarchy of each transaction?
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
achow101
Moderator
Legendary
*
Offline Offline

Activity: 1246


17kKQppUsngUiByDsce4JXoZEjjpvX9bpR


View Profile WWW
February 14, 2017, 08:11:28 PM
 #2

The blk.dat file stores raw blocks, not just transactions. It only stores blocks and the transactions included within those blocks. AFAIK the index is an index of offsets for each block, basically where each block is located in which file and where in that file so that it can be read from the disk and processed if need be.

g2com
Sr. Member
****
Offline Offline

Activity: 252


View Profile
February 14, 2017, 10:50:37 PM
 #3

The blk.dat file stores raw blocks, not just transactions. It only stores blocks and the transactions included within those blocks. AFAIK the index is an index of offsets for each block, basically where each block is located in which file and where in that file so that it can be read from the disk and processed if need be.

So, the whole block is read into the memory when Bitcoin accesses it? How does Bitcoin lookup a transaction from a block file?
achow101
Moderator
Legendary
*
Offline Offline

Activity: 1246


17kKQppUsngUiByDsce4JXoZEjjpvX9bpR


View Profile WWW
February 14, 2017, 10:57:59 PM
 #4

So, the whole block is read into the memory when Bitcoin accesses it?
I believe so.

How does Bitcoin lookup a transaction from a block file?
Bitcoin Core normally does not need to lookup transactions. It can only do that if you enable the transaction index (-txindex CLI option) in which case it does basically the same thing as for blocks, it makes the database and has each value point to the location in a file of where that transaction is located.

For transactions that are in your own wallet, it stores the transaction data in your wallet.dat file.

The databases also include other things like the current chainstate and the UTXO set.

g2com
Sr. Member
****
Offline Offline

Activity: 252


View Profile
February 14, 2017, 11:05:54 PM
 #5

I saw that the serialized transaction is just a lump of data with a key, so how does Bitcoin find transactions associated with a single private key? Does it need to travserse all unspent tx to get the total balance?
g2com
Sr. Member
****
Offline Offline

Activity: 252


View Profile
February 14, 2017, 11:07:37 PM
 #6

So, the whole block is read into the memory when Bitcoin accesses it?
I believe so.

How does Bitcoin lookup a transaction from a block file?
Bitcoin Core normally does not need to lookup transactions. It can only do that if you enable the transaction index (-txindex CLI option) in which case it does basically the same thing as for blocks, it makes the database and has each value point to the location in a file of where that transaction is located.

For transactions that are in your own wallet, it stores the transaction data in your wallet.dat file.

The databases also include other things like the current chainstate and the UTXO set.

I see. So both wallet.dat and chainstate store unspent transaction data, but only those in wallet.dat are spendable right?
achow101
Moderator
Legendary
*
Offline Offline

Activity: 1246


17kKQppUsngUiByDsce4JXoZEjjpvX9bpR


View Profile WWW
February 14, 2017, 11:19:38 PM
 #7

I saw that the serialized transaction is just a lump of data with a key,
There is no key associated with a transaction.

so how does Bitcoin find transactions associated with a single private key? Does it need to travserse all unspent tx to get the total balance?
Yes. It goes through the UTXO set and finds which UTXOs can be spent by you.

I see. So both wallet.dat and chainstate store unspent transaction data, but only those in wallet.dat are spendable right?
Only those in the wallet.dat can be spent by you specifically. Nearly all the UTXOs can be spent by someone.

g2com
Sr. Member
****
Offline Offline

Activity: 252


View Profile
February 14, 2017, 11:36:33 PM
 #8

I saw that the serialized transaction is just a lump of data with a key,
There is no key associated with a transaction.

so how does Bitcoin find transactions associated with a single private key? Does it need to travserse all unspent tx to get the total balance?
Yes. It goes through the UTXO set and finds which UTXOs can be spent by you.

I see. So both wallet.dat and chainstate store unspent transaction data, but only those in wallet.dat are spendable right?
Only those in the wallet.dat can be spent by you specifically. Nearly all the UTXOs can be spent by someone.

Thanks for the answer. The key I meant the transaction hash, since you need to supply a key to retrieve a value.
g2com
Sr. Member
****
Offline Offline

Activity: 252


View Profile
February 15, 2017, 07:17:55 AM
 #9

I've looked into the Tx index. It contains three columns, nFile, nBlockPos and nTxPos. The last two variables are integers indicating where the transaction can be found in a file. I don't understand how does Bitcoin retrieve one transaction. Does the Bitcoin client read the entire index into memory, then create a map that establishes the transaction and its position?
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!