Bitcoin Forum
October 18, 2019, 05:16:36 AM *
News: Latest Bitcoin Core release: 0.18.1 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: Are Blocks in blockchain are indexed ?  (Read 260 times)
r1s2g3
Sr. Member
****
Offline Offline

Activity: 630
Merit: 388


I am alive but in hibernation.


View Profile
September 18, 2018, 02:54:45 AM
Merited by dbshck (2), NeuroticFish (1), ETFbitcoin (1)
 #1

When we do a transaction then it is made sure that whatever we are going to send to receiver are unspent outputs (UTXO). It means it need to be scanned in entire blockchain that it is not spent.
But I see that transaction are validated within a seconds (or few seconds). Since blockchain size is now is 170 GB plus so it look to me quite a fast speed to traverse all blocks in blockchain.

I understand each block has address of the previous block  so do it have kind of index to reach that last block and we traverse all block one by one?

.
Game that
pays for
Playing











A
blockchain
based game
Ask me anything
about the game
in Bitcointalk.











A game
that recognize
your ownership
Join the
exciting game
of splinterlands











              ▄▄▄▄▄▄██████▄▄▄▄▄▄
          ▄▄██████████████████████▄▄
        ███████▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀███████
     ████████▌    ▄▄▄▄▄▄▄▄    ▐████████
   ▄██████████▌  █████ ▀ ▀███  ▐██████████▄
  ▐███████████▌ ████▄▄ ██ ▐███ ▐███████████▌
  ████████████▌▐█████     ████▌▐████████████
 █████████████▌ ███▄▄ ██ ▐████ ▐█████████████
 █████████████▌  ███    ▄████  ▐█████████████
██████████████▌    ▀▀▀▀▀▀▀▀    ▐██████████████
██████████████████████████████████████████████
▀████████████████████████████████████████████▀
   ▐██████████████████████████████████████▌
   ▐█████████████▀▀▀▀▀▀▀▀▀▀▀▀█████████████▌
    ▀▀██████▀▀                  ▀▀██████▀▀
1571375796
Hero Member
*
Offline Offline

Posts: 1571375796

View Profile Personal Message (Offline)

Ignore
1571375796
Reply with quote  #2

1571375796
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1571375796
Hero Member
*
Offline Offline

Posts: 1571375796

View Profile Personal Message (Offline)

Ignore
1571375796
Reply with quote  #2

1571375796
Report to moderator
1571375796
Hero Member
*
Offline Offline

Posts: 1571375796

View Profile Personal Message (Offline)

Ignore
1571375796
Reply with quote  #2

1571375796
Report to moderator
ETFbitcoin
Legendary
*
Offline Offline

Activity: 1792
Merit: 2053

Use SegWit and enjoy lower fees.


View Profile WWW
September 18, 2018, 04:04:30 AM
Last edit: September 18, 2018, 04:24:24 AM by ETFbitcoin
Merited by dbshck (5), NeuroticFish (1)
 #2

Bitcoin core (other client might do things differently) build and use UTXO database which stored on storage (not RAM) which makes transaction verification is faster. The UTXO database updated every time node receive transaction/block or create and broadcast transaction.
Besides, AFAIK most transaction can be verified in hundred millisecond.

There's no way any developer would think scan entire database just to get a data. Indexing, database balancing, sorting or/and separate database for linking obviously used in here.

If you're curious with UTXO size/count, you can check https://statoshi.info/dashboard/db/unspent-transaction-output-set

nc50lc
Hero Member
*****
Online Online

Activity: 742
Merit: 636


Self-proclaimed Genius ㊙️


View Profile WWW
September 18, 2018, 06:30:03 AM
Merited by dbshck (5), Welsh (2), LeGaulois (1)
 #3

-snip-
By saying "indexed" was it meant like index in books and file storage systems?
Then, by comparing the speed between (ex. not indexed Window Explorer) and Bitcoin's Blockchain, this question pops in to your mind?

Clients have a blockchain_headers which contains something similar (for indexing) that enables it to easily find the transaction hashes which have the accurate "links" to where the UTXOs are stored:
Code:
"inputs": [
        {
            "addresses": [
                "39EK***********************"
            ],
            "age": 513486,
            "output_index": 82,
            "output_value": 10000,
            "prev_hash": "24dc16f626085e7c75e6fbdbd2533f27d98d884fe73f58fed46a3e319157****",
            "script": "160014d0ef607e4dc338d4dd67df92e2090b770a781***",
            "script_type": "pay-to-script-hash",
            "sequence": 4294967293,
            "witness":
  • prev_hash: the previous transaction where it belongs.
  • output_index: where it sits between all of the other outputs of the particular transaction.

File explorers and Blockchain works differently, this is just "in-relation".

Thirdspace
Hero Member
*****
Offline Offline

Activity: 1134
Merit: 717


Mixing reinvented for your privacy | chipmixer.com


View Profile
September 18, 2018, 10:36:25 PM
Merited by dbshck (5), r1s2g3 (1)
 #4

It means it need to be scanned in entire blockchain that it is not spent.
chainstate/*: this is a LevelDB database with a compact representation of all currently unspent transaction outputs and some metadata about the transactions they are from. The data here is necessary for validating new incoming blocks and transactions.
no it doesn't have to (re)scan the entire blockchain everytime,
wallet/node keeps track of current unspent transaction outputs in chainstate

I understand each block has address of the previous block  so do it have kind of index to reach that last block and we traverse all block one by one?
all databases always have indexes to make searching specific data faster
for bitcoin, metadata for blocks are stored under blocks/index/* directory
you can read more in details on bitcoin wiki Bitcoin Core 0.11 (ch 2): Data Storage

r1s2g3
Sr. Member
****
Offline Offline

Activity: 630
Merit: 388


I am alive but in hibernation.


View Profile
September 19, 2018, 01:33:29 AM
 #5

There's no way any developer would think scan entire database just to get a data.


Then, by comparing the speed between (ex. not indexed Window Explorer) and Bitcoin's Blockchain, this question pops in to your mind?

I am a Performance DBA, I know even if the table size in a couple of GB and you have to do Full table scan to fetch the data ,required data cannot be fetched within the seconds . (Debarring parallel query operations).

you can read more in details on bitcoin wiki Bitcoin Core 0.11 (ch 2): Data Storage

That link perfectly satisfy my query.


Thanks  to you all. Slightly off topic question, I already gone through all the chapters of "Mastering Bitcoin" book and I do not remember if I have read anything about it.  Did I missed in reading or there are some other source/book from which I should start.

.
Game that
pays for
Playing











A
blockchain
based game
Ask me anything
about the game
in Bitcointalk.











A game
that recognize
your ownership
Join the
exciting game
of splinterlands











              ▄▄▄▄▄▄██████▄▄▄▄▄▄
          ▄▄██████████████████████▄▄
        ███████▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀███████
     ████████▌    ▄▄▄▄▄▄▄▄    ▐████████
   ▄██████████▌  █████ ▀ ▀███  ▐██████████▄
  ▐███████████▌ ████▄▄ ██ ▐███ ▐███████████▌
  ████████████▌▐█████     ████▌▐████████████
 █████████████▌ ███▄▄ ██ ▐████ ▐█████████████
 █████████████▌  ███    ▄████  ▐█████████████
██████████████▌    ▀▀▀▀▀▀▀▀    ▐██████████████
██████████████████████████████████████████████
▀████████████████████████████████████████████▀
   ▐██████████████████████████████████████▌
   ▐█████████████▀▀▀▀▀▀▀▀▀▀▀▀█████████████▌
    ▀▀██████▀▀                  ▀▀██████▀▀
ETFbitcoin
Legendary
*
Offline Offline

Activity: 1792
Merit: 2053

Use SegWit and enjoy lower fees.


View Profile WWW
September 19, 2018, 04:15:34 AM
 #6

There's no way any developer would think scan entire database just to get a data.


Then, by comparing the speed between (ex. not indexed Window Explorer) and Bitcoin's Blockchain, this question pops in to your mind?

I am a Performance DBA, I know even if the table size in a couple of GB and you have to do Full table scan to fetch the data ,required data cannot be fetched within the seconds . (Debarring parallel query operations).

Sometimes telling obvious things is safer/better choice Roll Eyes

you can read more in details on bitcoin wiki Bitcoin Core 0.11 (ch 2): Data Storage

That link perfectly satisfy my query.


Thanks  to you all. Slightly off topic question, I already gone through all the chapters of "Mastering Bitcoin" book and I do not remember if I have read anything about it.  Did I missed in reading or there are some other source/book from which I should start.

This should be obvious, but the link is outdated and UTXO database/chainstate and LevelDB used by Bitcoin Core has been changed few times.
AFAIK 0.15.1 change chainstate format and there are multiple levelDB optimization.

"Mastering Bitcoin" mostly discuss about things necessary in Bitcoin protocol and database isn't part of it (except block header/TX format) since each client have different way to store/index blockchain/UTXO. A bit off-topic, you might find source/book you want in list that i mentioned in past

Since you have intermediate blockchain knowledge, surely these lists shouldn't too difficult for you :
https://github.com/decrypto-org/blockchain-papers
https://github.com/coderplex/awesome-blockchain

You also could check these cryptocurrency specific lists, but might be difficult for you (and me) :
https://github.com/igorbarinov/awesome-bitcoin
https://github.com/Scanate/EthList
https://github.com/jashmenn/bitcoin-reading-list

The lists i mentioned aren't courses, but i'm sure you would get more knowledge/info with shorter time.

Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!