Bitcoin Forum
June 24, 2024, 08:34:22 PM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
  Home Help Search Login Register More  
  Show Posts
Pages: « 1 [2]
21  Bitcoin / Development & Technical Discussion / Re: Fast parallel block validation without UTXO-index on: April 09, 2017, 06:53:40 AM
Oh, ok. Sorry, I didn't get the spend-index idea at first.

Can you please say something more about how it actually works inside?


I essentially transform transactions and outputs to a "hash" with:

Code:
hash(tx) = tx.filepos / 16
hash(output) = tx.filepos / 16 + 1 + output-index
(guarded for exceptional overflows by a (now empty) map of exceptions for if output-count +1> tx-size/16)

And use the "hash" as a direct index in the bit-index.

The spend-index represents the spent state a few blocks from the tip; a set bit at a location means the transaction exists there or an output is spent there. When the spend tree is searched and reaches the position spend-index, a positive lookup for transaction and a negative lookup for  output is enough.

The spent index grows with 1 bit per every every 16 bytes, or 1/128th of the blockchain size. Just like with the spend-tree, mostly the end is accessed, so the start can be kept on disk.

Code of the spend-index itself is pretty simple here: https://github.com/tomasvdw/bitcrust/blob/master/src/store/spend_index.rs
22  Bitcoin / Development & Technical Discussion / Re: Fast parallel block validation without UTXO-index on: April 09, 2017, 06:24:01 AM
I see the UTXO approach actually as a left over from the architecture before Compact-Blocks/XThin.

If all transaction are pre-synchronized,

Point of fact: transactions being validated in advance has always been true and is completely orthogonal with compact blocks.  Bitcoin has also directly exploited this fact via signature caching (which your benchmarks disables, btw, because you use the undocumented blocks only mode) since applying Satoshi's patch for it in May 2012, and further with the input cache since July 2012.

I've seen Peter R make the same argument a couple times and I've asked him why he presents it that way but AFAIR he has never responded to one of those messages.



I understand Core  pre validates signatures with our without Compact Blocks/XThin. I also understand the benches aren't comparing this, but only compare full block verification which I explain on that very page.

But without Compact Blocks/XThin, optimizing for order comparison is a a lot less relevant because of the high bandwidth cost.
23  Bitcoin / Development & Technical Discussion / Re: Fast parallel block validation without UTXO-index on: April 08, 2017, 06:37:25 PM
Thanks for your feedback.

The UTXO index is there for a reason.
Personally I'd rather see some work on improving the index and the UTXO-db itself, rather than getting rid of it, which I think is not the right way to go.

I see the UTXO approach actually as a left over from the architecture before Compact-Blocks/XThin.

If all transaction are pre-synchronized, we can distinguish base load script validation (when a tx comes in) and peak load order validation (when a block comes), the UTXO-index doesn't make all that much sense.

This is because for script validation, when output scripts are needed the distinction between spent and unspent doesn't really exists as it can be spent on one branch and not spent on another. Using a transactionally maintained UTXO index there   only complication things (with reorgs).

And for order validation, the output scripts are not needed at all.

Looking at the "performance results" one must remember that core (and the leveldb solution) use a limited amount of system memory, while this solution seems to be using all of it.

I don't really think this is the case. Bitcrust uses memory mapped pages which show up as lots of virtual memory, but are swapped by the OS as needed. If instead you just traditional disk IO, the data is cached by the OS as needed.

Either way, MRU pages are kept  in memory.

Also, when the core gets to verify a transaction that is trying to spend a non-existing input, it won't consume so much resources. This is a potential DoS problem.

It is never needed to scan the entire chain as after a few blocks the spend-index catches a seek and performs a direct lookup. There may still be potential DoS problems, but not with spending a non-existing output.
24  Bitcoin / Development & Technical Discussion / Re: Fast parallel block validation without UTXO-index on: April 07, 2017, 05:46:01 AM
Ok. But what about outputs that never existed in the first place. How do you find that they don't exist?

The spend tree is scanned; if the output is found it is a double spent, if the corresponding transaction is spent it is ok, and if nothing is found it is spending a non-existing output.

Similarly, if the spend-index "catches" the search, two lookups are done: presence of the transaction and absence of the output.

The 'cheat' of spending an output of an existing transaction with an index higher then the output count is caught by script validation.
25  Bitcoin / Development & Technical Discussion / Re: Fast parallel block validation without UTXO-index on: April 07, 2017, 01:12:20 AM
Current usage:

1.3G   ../data-cmp/block-index
5.6G   ../data-cmp/spend-tree
41M   ../data-cmp/headers
25G   ../data-cmp/transactions2
196M   ../data-cmp/spend-index
3.6G   ../data-cmp/tx-index
85G   ../data-cmp/transactions1
119G   ../data-cmp/

But this can be pruned, just like with Core.

As for memory, this uses memory mapped files which means lots of virtual memory, but leave it to the OS to swap. By using proper locality of reference, the data that isn't used is kept out of memory.

The problem of spending early outputs is solved by the spend-index, which lags behind the tips a few blocks to catch these. This is also explained under the "spend-index" section on the site.
26  Bitcoin / Development & Technical Discussion / Fast parallel block validation without UTXO-index on: April 06, 2017, 04:55:38 PM

I have build a storage engine as the core of a new Bitcoin Node in development. The idea is to abandon the UTXO index and use a fast concurrent spend tree instead.

The results are very promising. Further information is here: https://bitcrust.org

Performance results: https://bitcrust.org/results

Source code in Rust is available at https://github.com/tomasvdw/bitcrust
27  Bitcoin / Development & Technical Discussion / Re: Malleated Transactions on: March 13, 2017, 09:51:53 AM
To prevent a false dichotomy, I would like to note that Christian Decker's BIP 140 (https://github.com/bitcoin/bips/blob/master/bip-0140.mediawiki) also fixes maleability.

In my opinion in better way then both FlexTrans and SegWit.
28  Bitcoin / Development & Technical Discussion / Re: REWARD offered for hash collisions for SHA1, SHA256, RIPEMD160 and other on: February 25, 2017, 05:06:27 PM
You may not recognize or agree with a or any legal distinction between stealing someone's BTC and claiming a BTC reward as was done here in this thread or is being done in the puzzle transaction.

But there is a huge moral distinction between the two:  stealing someone's BTC is wrong, claiming a BTC reward is not wrong.

Just because you can do something does not make it morally right.



Not all cases are morally clear cut. What if someone disagrees with how someone else acquired the coins in the first place?

A more interesting example of how "stealing" becomes fuzzy is LN payment channels. What if I close a channel with an earlier stage to my advantage, and I succeed because my peer for some reason fails to monitor the blockchain?

Am I stealing? Not really. My gains  would be the result of an explicit and well known  clause of our contract. Would you consider this morally wrong?

I think we should value the programmatic rules of contracts and as such not judge these types of theft in the same way as we would do with theft in the "real" world.
29  Bitcoin / Development & Technical Discussion / Re: REWARD offered for hash collisions for SHA1, SHA256, RIPEMD160 and other on: February 25, 2017, 02:56:58 PM
Stealing someone's coins by breaking ECDSA is not the same as a reward specifically for breaking something.


That is an interesting distinction as this implies that the blockchain should recognize some form of legal or moral ownership defined outside of the blockchain.

An alternative view more in line with the decentralized nature of bitcoin, is that "ownership" is simply defined as being able to produce a valid input script for an output script.

As such, someone being able to find the private key of an early public key by whatever means, must be considered the "owner". It doesn't matter if the keys are found on an (owned) usb-stick or by trial and error,
30  Bitcoin / Development & Technical Discussion / Re: What we can do to confirm transactions faster? on: February 25, 2017, 02:44:35 PM

My favourite conspiracy theory is that it's some miner(s) who flood the network with transactions, forcing legitimate users to increase their fees in their normal transactions.

This works because the miner can just collect his own fees back. (of course only on the blocks they mine)  It also explains the hold up on scaling solutions.  Scaling solutions break their scam.

No proof, just fun speculation.


It doesn't work because although he collects his own fees, he doesn't collect additional fees for the space that these transaction use. As fees only need to be marginally higher then others, the end result is always less income then when including real transactions.
31  Bitcoin / Electrum / Re: electrum won't start on: June 14, 2014, 01:54:26 PM
Thanks a lot!

Reinstall and copy wallet.dat file didn't work.

Create new wallet from existing seed worked, copy seed from previous file worked!

 
32  Bitcoin / Electrum / electrum won't start on: June 13, 2014, 02:32:55 PM
Hello there,

All of a sudden, electrum.exe won't start anymore. When started it briefly shows up in Task manager and is gone after a second. I don't see logs or console output.

- I don't have the twelve word seed.
- I do have a wallet.dat and blockchain_headers in APP_DATA. wallet.dat looks valid.

What procedure should I follow? Reinstall electrum and then overwrite wallet.dat with my current one? Would that work?

Does wallet.dat contain my private keys without the need for the twelve word seed?

Thanks!
Tomas
33  Economy / Gambling / Re: [ANN] btcsoccer.com new loginless soccer betting service on: June 12, 2014, 10:11:48 AM
Hi there,

https://btcsoccer.com is ready for Brazil!

Give it a try, it's easy and fun.

Tomas
34  Economy / Gambling / Re: [ANN] btcsoccer.com new loginless soccer betting service on: April 07, 2014, 06:41:07 PM
The liquidity is still pretty small.

You can probably still make some easy money by betting on reasonable choices because current bets often seem to defy all reason Smiley

There is little risk in placing a bet on a game with no existing bets because:
- If there is no winner you get your money back
- If you win you only pay a fee over your profits; not over your own wager.




35  Economy / Gambling / Re: [ANN] btcsoccer.com new loginless soccer betting service on: April 02, 2014, 01:27:59 PM
Thanks for the heads up!

We are a soccer betting exchange.  We take Bitcoin bets and return the total wager - in Bitcoin - to the winner(s) of the game.

We made this service mainly because we really like the magic of betting without having to register or login. I think this makes it all really straightforward and transparent; you simply pick a game, select your result(s), checkout your BetSlip, pay and win! (or lose)

The trust issue is no bigger then any other bitcoin service: all transactions are uniquely tied to a bet & wallet address and transparently stored in the blockchain? Besides our 2mBTC minimum wager is small enough for anyone to start give it a spin I think?

Regarding the promotion: we're working with a 2% house edge on profits only (non-winner matches are 100% refunded to all bettors). You think it would entice people if we temporarily remove our commission?
36  Economy / Gambling / Re: [ANN] btcsoccer.com new loginless soccer betting service on: April 02, 2014, 08:29:21 AM
So....

How does this forum work?

Did I post in private mode? Or are you all too flabbergasted by my site to comment?

I am not sure why my soccer website seems to invoke less reactions then all these ponzi scheme sites
37  Economy / Gambling / [ANN] btcsoccer.com new loginless soccer betting service on: March 28, 2014, 02:32:35 PM
Hello,

We have created a new soccer betting website at https://btcsoccer.com.

Mainly because I really like the idea of gaming and gambling without having to login. It really feels magical to be able to play without all the hassle and trust, just by scanning a QR code.

Features

- Betting on all major European leagues.
- Live match updates
- Direct payout after the game
- 2% house edge, only on profits

Please check it out and let me know what you think...

Tomas
https://btcsoccer.com
38  Other / Beginners & Help / python-bitcoinrpc on: October 13, 2013, 07:55:00 PM
Hello,

I am having some trouble with creating transactions using python-bitcoinrpc (on python 2.7.x).

It seems that the library doesn't support the Decimal data type; when I do:

Code:
new_tx = btcoind.createrawtransaction([{ 'txid': mytx, 'vout': myvout}], { to_addr: Decimal('0.001') })

It raises:
Code:
...
  File "/usr/lib/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: Decimal('0.001') is not JSON serializable

I have seen some solutions (http://stackoverflow.com/questions/1960516/python-json-serialize-a-decimal-object?)
but they all seem to entail converting the decimal either to float or to string. The first is not an option due to the loss of precision and the second creates a json-rpc error.

How can I ensure the decimal is properly encoded to json?

Thanks,
Tomas
Pages: « 1 [2]
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!