Bitcoin Forum
October 16, 2018, 06:58:50 AM *
News: Make sure you are not using versions of Bitcoin Core other than 0.17.0 [Torrent], 0.16.3, 0.15.2, or 0.14.3. More info.
 
   Home   Help Search Donate Login Register  
Pages: « 1 [2]  All
  Print  
Author Topic: Fast parallel block validation without UTXO-index  (Read 4511 times)
gmaxwell
Moderator
Legendary
*
qt
Offline Offline

Activity: 2534
Merit: 1545



View Profile
April 11, 2017, 04:28:52 PM
 #21

This in contrast to Core's model which needs sequential reads and writes to the UTXO index, even if all scripts are already verified.
This is not how Bitcoin Core works. No access to the disk normally at all happens when a block is verified except for transactions that have not yet been seen, reading or writing.
I am not talking about disk reads and writes, I am talking about UTXO index reads and writes.
There are no UTXO index reads or writes in the normal case (transaction recently seen.) in Bitcoin Core.

Bitcoin will not be compromised
1539673130
Hero Member
*
Offline Offline

Posts: 1539673130

View Profile Personal Message (Offline)

Ignore
1539673130
Reply with quote  #2

1539673130
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1539673130
Hero Member
*
Offline Offline

Posts: 1539673130

View Profile Personal Message (Offline)

Ignore
1539673130
Reply with quote  #2

1539673130
Report to moderator
piotr_n
Legendary
*
Offline Offline

Activity: 1946
Merit: 1019


aka tonikt


View Profile WWW
April 11, 2017, 10:24:52 PM
 #22

I think what tomtomtom7 is trying to say is that the common approach (used not only by core) is that even though the node verifies the transactions before a block is announced, the actual moment when the UTXO database is updated (and the changes committed to it)  is during the block's validation.

Whilst his solution updates (appends)  TXO database while verifying a transaction in the first place. Later,  during the block's validation, it's just logging  the information of which of the outputs from the TXO database has been spent by the block.

Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.
PGP fingerprint: AB9E A551 E262 A87A 13BB  9059 1BE7 B545 CDF3 FD0E
gmaxwell
Moderator
Legendary
*
qt
Offline Offline

Activity: 2534
Merit: 1545



View Profile
April 11, 2017, 10:49:54 PM
 #23

I think what tomtomtom7 is trying to say is that the common approach (used not only by core) is that even though the node verifies the transactions before a block is announced, the actual moment when the UTXO database is updated (and the changes committed to it)  is during the block's validation.
That is what I understand him to be saying, but it isn't true for Bitcoin Core. For Core, the moment when the UTXO database is updated/and changes committed is _not_ during block validation.  It's after in separate batch steps which apply big batches of changes at a time; specifically to mostly get interacting with the database out of the critical path.

This is also why if you shut your node off uncleanly you'll see it going and reprocessing dozens of blocks-- their effects hadn't been applied to the database yet.

Bitcoin will not be compromised
cryptoanarchist
Legendary
*
Offline Offline

Activity: 1120
Merit: 1003



View Profile
April 11, 2017, 11:07:09 PM
 #24

I think what tomtomtom7 is trying to say is that the common approach (used not only by core) is that even though the node verifies the transactions before a block is announced, the actual moment when the UTXO database is updated (and the changes committed to it)  is during the block's validation.
That is what I understand him to be saying, but it isn't true for Bitcoin Core. For Core, the moment when the UTXO database is updated/and changes committed is _not_ during block validation.  It's after in separate batch steps which apply big batches of changes at a time; specifically to mostly get interacting with the database out of the critical path.

This is also why if you shut your node off uncleanly you'll see it going and reprocessing dozens of blocks-- their effects hadn't been applied to the database yet.

I don't understand the specifics of the code, so can you enlighten me? What you're saying sounds like semantics: "It's after in separate batch steps" sounds like it is still done when a block validates, just maybe not part of a certain function. I mean, it has to be done before the next block anyway. Is the UTXO updated before the node officially validates the block??

It sounds like this new way validates the TX as they happen, and this saves time later. I wish people could be a little more specific in a tech sub-forum -like with code snippets.

I'm grumpy!!
gmaxwell
Moderator
Legendary
*
qt
Offline Offline

Activity: 2534
Merit: 1545



View Profile
April 12, 2017, 09:56:58 AM
 #25

I don't understand the specifics of the code, so can you enlighten me? What you're saying sounds like semantics: "It's after in separate batch steps" sounds like it is still done when a block validates,

It's not semantics. When validating blocks the vast majority of the time the UTXO set is not used by bitcoin core; because the UTXOs are in the cache already (due to mempool) and nor are the result of the block's changes written out-- they're buffered until the node flushes-- and very often never get written at all because the newly created outputs are spent by a later block before the system flushes them.

OP suggests that the structure he proposes reduces the delay at the time a block is processed compared to Bitcoin Core's UTXO database, but it shouldn't-- at least not for that reason-- because the database is only used at block processing times in the exceptional case where a transaction shows up by surprise.

Bitcoin will not be compromised
tomtomtom7
Newbie
*
Offline Offline

Activity: 38
Merit: 0


View Profile
April 12, 2017, 11:22:37 AM
 #26

I don't understand the specifics of the code, so can you enlighten me? What you're saying sounds like semantics: "It's after in separate batch steps" sounds like it is still done when a block validates,

It's not semantics. When validating blocks the vast majority of the time the UTXO set is not used by bitcoin core; because the UTXOs are in the cache already (due to mempool) and nor are the result of the block's changes written out-- they're buffered until the node flushes-- and very often never get written at all because the newly created outputs are spent by a later block before the system flushes them.

OP suggests that the structure he proposes reduces the delay at the time a block is processed compared to Bitcoin Core's UTXO database, but it shouldn't-- at least not for that reason-- because the database is only used at block processing times in the exceptional case where a transaction shows up by surprise.

This is semantics. I am aware that Core uses a cache to make most UTXO read and writes faster, but I would still call them UTXO read writes (or CoinView read/writes?), and as I understand, they still need to be handled sequentially.

Any reasonable database will attempt to use RAM for frequently used items, regardless of whether this is managed by the application, by a dedicated database component or the OS.

gmaxwell
Moderator
Legendary
*
qt
Offline Offline

Activity: 2534
Merit: 1545



View Profile
April 12, 2017, 06:18:19 PM
 #27

This is semantics.
It isn't-- because the speedups you conjecture creating already exist.

You propose that your design moves accessing the UTXO data out of the critical path for block validation, on the basis that you will have already validated the transactions and so don't need to access that data.

Bitcoin Core does not have UTXO accesses in the critical path in the same case and doesn't write in the critical path either. So it would not see the benefit you expect for this reason. (not that it wouldn't potentially have benefit, just not the benefit you are suggesting at least not for the reason you suggest)

Cheers

Bitcoin will not be compromised
Pages: « 1 [2]  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!