Bitcoin Forum
August 18, 2017, 02:28:15 PM *
News: Latest stable version of Bitcoin Core: 0.14.2  [Torrent].
 
   Home   Help Search Donate Login Register  
Poll
Question: Will you support Gavin's new block size limit hard fork of 8MB by January 1, 2016 then doubling every 2 years?
1.  yes
2.  no

Pages: « 1 ... 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 [519] 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 ... 1558 »
  Print  
Author Topic: Gold collapsing. Bitcoin UP.  (Read 1953438 times)
cypherdoc
Legendary
*
Offline Offline

Activity: 1764



View Profile
August 08, 2014, 04:47:11 PM
 #10361

iirc, the utxo set takes up about 75% of RAM required to run a node.  as i run 4 nodes myself, it takes a minimum of 1GB RAM to run smoothly meaning 750MB could be estimated for the utxo set itself.  that's pretty big to be including in a block...
Not in the block, alongside the block.

Just like when people merge mine Namecoin the nmc blocks aren't included in the btc blocks.

why would you do that?
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1503066495
Hero Member
*
Offline Offline

Posts: 1503066495

View Profile Personal Message (Offline)

Ignore
1503066495
Reply with quote  #2

1503066495
Report to moderator
1503066495
Hero Member
*
Offline Offline

Posts: 1503066495

View Profile Personal Message (Offline)

Ignore
1503066495
Reply with quote  #2

1503066495
Report to moderator
1503066495
Hero Member
*
Offline Offline

Posts: 1503066495

View Profile Personal Message (Offline)

Ignore
1503066495
Reply with quote  #2

1503066495
Report to moderator
domob
Legendary
*
Offline Offline

Activity: 980


View Profile WWW
August 08, 2014, 05:29:13 PM
 #10362

iirc, the utxo set takes up about 75% of RAM required to run a node.  as i run 4 nodes myself, it takes a minimum of 1GB RAM to run smoothly meaning 750MB could be estimated for the utxo set itself.  that's pretty big to be including in a block...
Not in the block, alongside the block.

Just like when people merge mine Namecoin the nmc blocks aren't included in the btc blocks.

why would you do that?

I may be mistaken, but isn't this basically what the "ultimate UTXO pruning" thing is about?  As far as I have heard, it allows new clients to be synced almost immediately (at least a lot faster than when downloading the full chain) with less required trust than SPV clients and the like (or even almost the same trustless-ness as a full node?).  I've never really dived into the details, though.

Use your Namecoin identity as OpenID: https://nameid.org/
Donations: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS | GPG 0xA7330737
flipperfish
Sr. Member
****
Offline Offline

Activity: 334


Dolphie Selfie


View Profile
August 08, 2014, 05:41:33 PM
 #10363

i think the UTXO set will have to have a pre-agreed order that will need to be synced amongst all miners in real-time.  then, the successful miner can publish the header hash along with some signal as to how far down in that list he went before he stopped adding tx's.  then, all other miners will know how to verify the published header hash and remove those tx's from the utxo set before moving on to the next block.

Yes, as I understand it, the trick is the deterministic order. The "signal as to how far down in that list" is the thing gavin tweeted about.

how reliable is a synced utxo set across all miners?

I don't think it is the utxo set, that has to be in an deterministic order, but the txs in the mempool. The utxo set is a result of the valid blockchain, so it should be synchronized already (except a node regards a block as valid, which later will become orphaned).

Gavin proposed to use "Invertible Bloom Lookup Tables". The lookup table tells the miner which transactions from its mempool should be considered for the verification of the published blockheader. Because now the miner knows which txs are used to form the block and in which order, he can reconstruct the merkle root and compare that against the published blockheader. However the proposed "Invertible Bloom Lookup Tables" are not 100% reliable. From a quick glance at the paper I found, that about <1% of keys are lost during an experiment with 10000 keys. Keys can be seen as txs in this case.
Now I ask myself what happens in case of the remaining 1%? Maybe the failure rate can be reduced by publishing multiple lookup tables of the same set?
sidhujag
Legendary
*
Offline Offline

Activity: 1540


View Profile
August 08, 2014, 06:01:16 PM
 #10364

iirc, the utxo set takes up about 75% of RAM required to run a node.  as i run 4 nodes myself, it takes a minimum of 1GB RAM to run smoothly meaning 750MB could be estimated for the utxo set itself.  that's pretty big to be including in a block...
Not in the block, alongside the block.

Just like when people merge mine Namecoin the nmc blocks aren't included in the btc blocks.

The merged-mine implementations that don't include the memory fix: (mutable vs imutable storage) and no auxpow data in cblockindex (blocks stored in memory) will eat up alot of RAM... so far only Devcoin and I0coin are the ones that have this fix and only Devcoin is running on 0.9.2 (currently in testing) with all the other fixes that bitcoin rolled up along the way included. So for most of these incompatible alt-coins who now wish to merge-mine (because I called that they would, and the ones that dont will die) are now suffering from either memory bloat issues or incorrect implementions/bugs and its next to impossible for a general programmer to port it over because its increasingly convoluted code to keep track of.

★☆★Syscoin - Decentralized Marketplace and Multisig Platform
Pay with Bitcoin, ZCash and many more
For more visit Syscoin.org  ★☆★
cypherdoc
Legendary
*
Offline Offline

Activity: 1764



View Profile
August 08, 2014, 06:04:19 PM
 #10365

iirc, the utxo set takes up about 75% of RAM required to run a node.  as i run 4 nodes myself, it takes a minimum of 1GB RAM to run smoothly meaning 750MB could be estimated for the utxo set itself.  that's pretty big to be including in a block...
Not in the block, alongside the block.

Just like when people merge mine Namecoin the nmc blocks aren't included in the btc blocks.

The merged-mine implementations that don't include the memory fix: (mutable vs imutable storage) and no auxpow data in cblockindex (blocks stored in memory) will eat up alot of RAM... so far only Devcoin and I0coin are the ones that have this fix and only Devcoin is running on 0.9.2 (currently in testing) with all the other fixes that bitcoin rolled up along the way included. So for most of these incompatible alt-coins who now wish to merge-mine (because I called that they would, and the ones that dont will die) are now suffering from either memory bloat issues or incorrect implementions/bugs and its next to impossible for a general programmer to port it over because its increasingly convoluted code to keep track of.

which is exactly why i've been warning everyone about these leech coins or sidechains that wish to piggyback themselves onto the Bitcoin mining network to secure themselves.
sidhujag
Legendary
*
Offline Offline

Activity: 1540


View Profile
August 08, 2014, 06:36:11 PM
 #10366

iirc, the utxo set takes up about 75% of RAM required to run a node.  as i run 4 nodes myself, it takes a minimum of 1GB RAM to run smoothly meaning 750MB could be estimated for the utxo set itself.  that's pretty big to be including in a block...
Not in the block, alongside the block.

Just like when people merge mine Namecoin the nmc blocks aren't included in the btc blocks.

The merged-mine implementations that don't include the memory fix: (mutable vs imutable storage) and no auxpow data in cblockindex (blocks stored in memory) will eat up alot of RAM... so far only Devcoin and I0coin are the ones that have this fix and only Devcoin is running on 0.9.2 (currently in testing) with all the other fixes that bitcoin rolled up along the way included. So for most of these incompatible alt-coins who now wish to merge-mine (because I called that they would, and the ones that dont will die) are now suffering from either memory bloat issues or incorrect implementions/bugs and its next to impossible for a general programmer to port it over because its increasingly convoluted code to keep track of.

which is exactly why i've been warning everyone about these leech coins or sidechains that wish to piggyback themselves onto the Bitcoin mining network to secure themselves.

Well merged-mining is still better than POS, and now with LTC asics you have scrypt merged-miners aswell... I think its a great thing. But those that have strong dev communites will survive. With Devcoin since its address compatible with bitcoin its more of a trivial task to keep the port and then the thin clients and external tools all just work... it was a smart decision to do so... so it really is a viable alternative but others may not be so lucky a few years down the road with some major structural changes to the codebase. I would much rather support merged-mine coins over some random pos coin because of the inability of those pos coins to keep up with the development power of bitcoin.

If you think about it conceptually from a bigger picture.. none of these alts can exist without bitcoin anyway, so it makes sense that without bitcoin hash power they would simply be not secure... but they are flexible to find another high hash parent if that ever happens.

★☆★Syscoin - Decentralized Marketplace and Multisig Platform
Pay with Bitcoin, ZCash and many more
For more visit Syscoin.org  ★☆★
domob
Legendary
*
Offline Offline

Activity: 980


View Profile WWW
August 08, 2014, 06:59:28 PM
 #10367

iirc, the utxo set takes up about 75% of RAM required to run a node.  as i run 4 nodes myself, it takes a minimum of 1GB RAM to run smoothly meaning 750MB could be estimated for the utxo set itself.  that's pretty big to be including in a block...
Not in the block, alongside the block.

Just like when people merge mine Namecoin the nmc blocks aren't included in the btc blocks.

The merged-mine implementations that don't include the memory fix: (mutable vs imutable storage) and no auxpow data in cblockindex (blocks stored in memory) will eat up alot of RAM... so far only Devcoin and I0coin are the ones that have this fix and only Devcoin is running on 0.9.2 (currently in testing) with all the other fixes that bitcoin rolled up along the way included. So for most of these incompatible alt-coins who now wish to merge-mine (because I called that they would, and the ones that dont will die) are now suffering from either memory bloat issues or incorrect implementions/bugs and its next to impossible for a general programmer to port it over because its increasingly convoluted code to keep track of.

Note that Namecoin also doesn't keep the auxpow in memory (CBlockIndex) or in the blkindex.dat file any longer.  I have removed this artefact (from times before I hacked on the code) recently.

Use your Namecoin identity as OpenID: https://nameid.org/
Donations: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS | GPG 0xA7330737
sidhujag
Legendary
*
Offline Offline

Activity: 1540


View Profile
August 08, 2014, 07:07:13 PM
 #10368

iirc, the utxo set takes up about 75% of RAM required to run a node.  as i run 4 nodes myself, it takes a minimum of 1GB RAM to run smoothly meaning 750MB could be estimated for the utxo set itself.  that's pretty big to be including in a block...
Not in the block, alongside the block.

Just like when people merge mine Namecoin the nmc blocks aren't included in the btc blocks.

The merged-mine implementations that don't include the memory fix: (mutable vs imutable storage) and no auxpow data in cblockindex (blocks stored in memory) will eat up alot of RAM... so far only Devcoin and I0coin are the ones that have this fix and only Devcoin is running on 0.9.2 (currently in testing) with all the other fixes that bitcoin rolled up along the way included. So for most of these incompatible alt-coins who now wish to merge-mine (because I called that they would, and the ones that dont will die) are now suffering from either memory bloat issues or incorrect implementions/bugs and its next to impossible for a general programmer to port it over because its increasingly convoluted code to keep track of.

Note that Namecoin also doesn't keep the auxpow in memory (CBlockIndex) or in the blkindex.dat file any longer.  I have removed this artefact (from times before I hacked on the code) recently.

You still need it in blnkindex.dat unless you dont care about not having any thin clients or external clients using "getheaders"

★☆★Syscoin - Decentralized Marketplace and Multisig Platform
Pay with Bitcoin, ZCash and many more
For more visit Syscoin.org  ★☆★
justusranvier
Legendary
*
Offline Offline

Activity: 1400



View Profile WWW
August 08, 2014, 07:13:37 PM
 #10369

why would you do that?
https://bitcointalk.org/index.php?topic=88208.0

If it ever becomes possible to do so without risking the integrity of the ledger, I would like Bitcoin to be able to discard transaction history.

Right now storing 100% of history is the easiest way to safeguard the ledger, but it also means we're subsidizing surveillance.

In a perfect world we'd only need to keep a P2P-validated UTXO set and enough history to detect tampering, and let the people who want to store everything forever so they can perform graph analysis pay for that themselves.
molecular
Donator
Legendary
*
Offline Offline

Activity: 2338



View Profile
August 08, 2014, 07:16:23 PM
 #10370

i think the UTXO set will have to have a pre-agreed order that will need to be synced amongst all miners in real-time.  then, the successful miner can publish the header hash along with some signal as to how far down in that list he went before he stopped adding tx's.  then, all other miners will know how to verify the published header hash and remove those tx's from the utxo set before moving on to the next block.

Yes, as I understand it, the trick is the deterministic order. The "signal as to how far down in that list" is the thing gavin tweeted about.

how reliable is a synced utxo set across all miners?

I don't think it is the utxo set, that has to be in an deterministic order, but the txs in the mempool. The utxo set is a result of the valid blockchain, so it should be synchronized already (except a node regards a block as valid, which later will become orphaned).

Gavin proposed to use "Invertible Bloom Lookup Tables". The lookup table tells the miner which transactions from its mempool should be considered for the verification of the published blockheader. Because now the miner knows which txs are used to form the block and in which order, he can reconstruct the merkle root and compare that against the published blockheader. However the proposed "Invertible Bloom Lookup Tables" are not 100% reliable. From a quick glance at the paper I found, that about <1% of keys are lost during an experiment with 10000 keys. Keys can be seen as txs in this case.

I just read the relevant parts of a paper explaining ITLB (Invertible Bloom Lookup Tables)... quite an interesting data structure. It has a constant size (pre-chosen) and allows storing (key, value) pairs. "insert" and "delete" operations are efficient. "get" may fail and "list" may return partial results.

I can see how an ITLB of some constant size might be used for block announcements by miners.

Some things I don't quite get, though:

  • why does this require deterministic ordering of txs in mempool?
  • does this require that all txs a miner includes in a block are properly broadcast beforehand?
  • would this require a hardfork or can it be used off-chain (so to say) by a subgroup of willing miners?

Now I ask myself what happens in case of the remaining 1%? Maybe the failure rate can be reduced by publishing multiple lookup tables of the same set?

maybe the publishing miner can include the 1% explicitly (he can determine which ones are affected himself and include the remainder). This would destroy the "constant size" property, but it would lessen the propagation cost of block size by ~99%, right?

PGP key molecular F9B70769 fingerprint 9CDD C0D3 20F8 279F 6BE0  3F39 FC49 2362 F9B7 0769
flipperfish
Sr. Member
****
Offline Offline

Activity: 334


Dolphie Selfie


View Profile
August 08, 2014, 09:56:30 PM
 #10371

Some things I don't quite get, though:

  • why does this require deterministic ordering of txs in mempool?
  • does this require that all txs a miner includes in a block are properly broadcast beforehand?
  • would this require a hardfork or can it be used off-chain (so to say) by a subgroup of willing miners?

1) This is needed to construct the merkle tree. Txs are grouped pairwise to build the nodes of the next level in the tree. The pairs are concatenated for the hashing, so the order is relevant. Currently the order is defined by the list of txs in the published block.

2) Yes, although I think the receiving miner could request missing tx(s).

3) To my understanding this does not need a hardfork, because the blocks are still built by the same rules.


Now I ask myself what happens in case of the remaining 1%? Maybe the failure rate can be reduced by publishing multiple lookup tables of the same set?

maybe the publishing miner can include the 1% explicitly (he can determine which ones are affected himself and include the remainder). This would destroy the "constant size" property, but it would lessen the propagation cost of block size by ~99%, right?


Ah, of course, this makes sense!
cypherdoc
Legendary
*
Offline Offline

Activity: 1764



View Profile
August 08, 2014, 10:55:21 PM
 #10372

Hey look!

Dow goes up, Dow goes down,  gold goes up, gold goes down. Meanwhile;

Bitcoin goes UP!
hdbuck
Legendary
*
Offline Offline

Activity: 1274



View Profile
August 08, 2014, 11:03:34 PM
 #10373

The US government has nearly as much debt as the world has wealth:
http://dollarvigilante.com/blog/2014/6/10/the-us-government-has-nearly-as-much-debt-as-the-world-has-w.html
tvbcof
Legendary
*
Offline Offline

Activity: 2226


View Profile
August 09, 2014, 06:19:49 AM
 #10374

Hey look!

Dow goes up, Dow goes down,  gold goes up, gold goes down. Meanwhile;

Bitcoin goes UP!

Chart I'm looking at has BTC trending down for a month.  The bad news, though, is that the imbalance is heavy on the supply side.  At this point I'll be surprised to see a selling opportunity which works for me in 2014, and not at all surprised to seen some significant downside in the intrim.  Oh well.


sidhujag
Legendary
*
Offline Offline

Activity: 1540


View Profile
August 09, 2014, 07:23:12 AM
 #10375

Hey look!

Dow goes up, Dow goes down,  gold goes up, gold goes down. Meanwhile;

Bitcoin goes UP!

Chart I'm looking at has BTC trending down for a month.  The bad news, though, is that the imbalance is heavy on the supply side.  At this point I'll be surprised to see a selling opportunity which works for me in 2014, and not at all surprised to seen some significant downside in the intrim.  Oh well.



LTC didnt reach target for me means its dragging it all down. Physcologically we are not ready for the next uptick, for me, until that level is tested, unless we go up and then come crash down later on which is not bullish long term... anyways just what worked for me in trading forex over the years... you build up an intuition of levels that must test before the real trend resumes. Something you cant really teach.

★☆★Syscoin - Decentralized Marketplace and Multisig Platform
Pay with Bitcoin, ZCash and many more
For more visit Syscoin.org  ★☆★
domob
Legendary
*
Offline Offline

Activity: 980


View Profile WWW
August 09, 2014, 08:32:32 AM
 #10376

iirc, the utxo set takes up about 75% of RAM required to run a node.  as i run 4 nodes myself, it takes a minimum of 1GB RAM to run smoothly meaning 750MB could be estimated for the utxo set itself.  that's pretty big to be including in a block...
Not in the block, alongside the block.

Just like when people merge mine Namecoin the nmc blocks aren't included in the btc blocks.

The merged-mine implementations that don't include the memory fix: (mutable vs imutable storage) and no auxpow data in cblockindex (blocks stored in memory) will eat up alot of RAM... so far only Devcoin and I0coin are the ones that have this fix and only Devcoin is running on 0.9.2 (currently in testing) with all the other fixes that bitcoin rolled up along the way included. So for most of these incompatible alt-coins who now wish to merge-mine (because I called that they would, and the ones that dont will die) are now suffering from either memory bloat issues or incorrect implementions/bugs and its next to impossible for a general programmer to port it over because its increasingly convoluted code to keep track of.

Note that Namecoin also doesn't keep the auxpow in memory (CBlockIndex) or in the blkindex.dat file any longer.  I have removed this artefact (from times before I hacked on the code) recently.

You still need it in blnkindex.dat unless you dont care about not having any thin clients or external clients using "getheaders"

Nope.  Currently, Namecoin doesn't allow "getheaders" since no client uses it and I didn't want to leave it there completely untested.  But in a previous development version, I implemented it just fine - you can just load the full block from disk if you really need it (someone sends a "getheaders") - and optionally cache it in memory, subject to your favourite heuristics for removing the data from memory again.

Use your Namecoin identity as OpenID: https://nameid.org/
Donations: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS | GPG 0xA7330737
cypherdoc
Legendary
*
Offline Offline

Activity: 1764



View Profile
August 09, 2014, 09:25:41 AM
 #10377

hopefully, this lull will kill off several (a few dozen) altcoins.
cypherdoc
Legendary
*
Offline Offline

Activity: 1764



View Profile
August 09, 2014, 09:33:28 AM
 #10378

the charts of all the alts look horrific.  most investors in cryptocurrencies will lose money.
Wekkel
Legendary
*
Offline Offline

Activity: 1582



View Profile
August 09, 2014, 10:37:07 AM
 #10379

Not kill, just low prices.

Like this post? you can tip me (BTC) 18j7UBNfhWWfvwGwrtzWfUrp1v6RDerFkY or (XEM) NBXGH5-MXQPNL-T5TA3R-QCQYUX-3FIEXC-LGIKGT-H7XJ
cypherdoc
Legendary
*
Offline Offline

Activity: 1764



View Profile
August 09, 2014, 05:08:10 PM
 #10380

stick with it, boys.
Pages: « 1 ... 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 [519] 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 ... 1558 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!