Bitcoin Forum
December 09, 2016, 02:16:26 AM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: « 1 [2]  All
  Print  
Author Topic: Bitcoin client operating with a finite amount of disk space  (Read 4166 times)
etotheipi
Legendary
*
expert
Offline Offline

Activity: 1428


Core Armory Developer


View Profile WWW
July 28, 2011, 09:19:55 PM
 #21

Quote
I was thinking about how to remove transactions when all the outputs have been used, but it seems to me that at least the transaction hash must be kept, because without it, it's impossible to tell the difference between an orphan transaction and a double-spend.  Although since neither are fully valid, it might make sense to discard them both, and if the orphan later becomes a non-orphan, hope that it will be retransmitted.

I don't see why you need to keep that transaction.  The entire state of the network (and everyone's balances) can be determined solely by the set of unused TxOut objects and the hash/index of their parent Tx object.  And that's all the information you need to sign new transactions.  All the TxIns and previous TxOuts are only necessary for blockchain verification, but that is done by the miners before they include them in a block.  Your client can get away with storing just the TxOut information above, and trust that they are valid because they were part of the longest blockchain (which is difficult to fake), and would not be be there if they weren't valid.

You can store all the Tx's in a tree data structure, whose values are arrays of TxOut objects.  As TxOut's are spent, you can remove them from the array, saving about 40 bytes for each of them.  When the last TxOut in the array is spend, you can also remove the Tx node, which saves another 40 bytes (approx).  You would only need to keep that data (and/or its hash) if you were concerned about verifying the transaction history.

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
1481249786
Hero Member
*
Offline Offline

Posts: 1481249786

View Profile Personal Message (Offline)

Ignore
1481249786
Reply with quote  #2

1481249786
Report to moderator
1481249786
Hero Member
*
Offline Offline

Posts: 1481249786

View Profile Personal Message (Offline)

Ignore
1481249786
Reply with quote  #2

1481249786
Report to moderator
1481249786
Hero Member
*
Offline Offline

Posts: 1481249786

View Profile Personal Message (Offline)

Ignore
1481249786
Reply with quote  #2

1481249786
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
vector76
Member
**
Offline Offline

Activity: 70


View Profile
July 28, 2011, 09:39:06 PM
 #22

Quote
I was thinking about how to remove transactions when all the outputs have been used, but it seems to me that at least the transaction hash must be kept, because without it, it's impossible to tell the difference between an orphan transaction and a double-spend.  Although since neither are fully valid, it might make sense to discard them both, and if the orphan later becomes a non-orphan, hope that it will be retransmitted.

I don't see why you need to keep that transaction.  The entire state of the network (and everyone's balances) can be determined solely by the set of unused TxOut objects and the hash/index of their parent Tx object.  And that's all the information you need to sign new transactions.

I agree everything that needs to be known about current balances exists in unspent outputs, and I agree you can authenticate valid transactions without keeping the spent outputs or even the hashes of the spent Tx.

Are you saying it's possible to distinguish between a double-spend and an orphan?  Or that you don't need to distinguish between them?  If it's the latter, I would agree for space-constrained nodes you can just discard an orphan/double-spend without worrying which case it happens to be.  Maybe you could even give it the benefit of the doubt and hang on to it until you see the next block or two before you toss it out.

I'd be curious to know quantitatively what fraction of the space is taken by transactions whose outputs have all been used.
etotheipi
Legendary
*
expert
Offline Offline

Activity: 1428


Core Armory Developer


View Profile WWW
July 28, 2011, 09:53:24 PM
 #23

I'm saying the latter -- I don't see why the client ever needs to distinguish between those two, it only matters whether the transaction is valid. 

The only difference might be that a 0-confirmation transaction is a bit less trustworthy on a light node, because it doesn't have the ability to verify the transaction itself.  It only knows for sure when it sees that transaction in a block, or can guess with high confidence that it wouldn't have received that Tx data unless it was valid, since invalid Tx's don't get very far in the network.

I'm working on some block-chain analysis tools right now, and playing/testing with the 374 MB of data up to block 136496.  I'm not quite there yet, but I share your intrigue and might take a shot at that calculation in the next few days.

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
krepta3000
Member
**
Offline Offline

Activity: 92


View Profile
July 28, 2011, 11:10:56 PM
 #24

I'm not worried about block chain size nearly as much as log file size.  Is there some way to make bitcoin restrain it's log file to a certain size, deleting older log data as it goes?  I've had to switch storage spaces several times to accommodate the log file, or delete the log file after shutting down the bitcoin client.
etotheipi
Legendary
*
expert
Offline Offline

Activity: 1428


Core Armory Developer


View Profile WWW
July 28, 2011, 11:21:03 PM
 #25

What is even in the log file?  I never knew that it existed (and was a concern) until this thread...

I am much more concerned about the blockchain file, because that's a critical part of the protocol.  Presumably, the log file can be trimmed... but the blk0001.dat cannot.  And the more successful bitcoin becomes, the more that filesize is going to spiral out of control.

I guess there's no options for the miners, they're going to have to hold the whole file no matter what.  But for the users, a reduced set will become necessary.  Maybe not just yet, but eventually.  Once the BTC network starts processing 1000 transactions per block, the blockchain is going to grow about 10-30 MB per day... or possibly 10 GB per year.  It can still be handled by the miners, but the average user isn't going to want to hold that much data just to use the program.

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
vector76
Member
**
Offline Offline

Activity: 70


View Profile
July 28, 2011, 11:40:14 PM
 #26

I'm not worried about block chain size nearly as much as log file size.  Is there some way to make bitcoin restrain it's log file to a certain size, deleting older log data as it goes?  I've had to switch storage spaces several times to accommodate the log file, or delete the log file after shutting down the bitcoin client.
Just found that there is an undocumented -printtoconsole option that will attempt to write to stdout instead of to the log file.  It may or may not succeed in writing to stdout but it seems it does suppress appending to the log file.
Pages: « 1 [2]  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!