Bitcoin Forum
December 07, 2016, 08:50:26 AM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Poll
Question: Will you support Gavin's new block size limit hard fork of 8MB by January 1, 2016 then doubling every 2 years?
1.  yes
2.  no

Pages: « 1 ... 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 [520] 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 ... 1560 »
  Print  
Author Topic: Gold collapsing. Bitcoin UP.  (Read 1805432 times)
flipperfish
Sr. Member
****
Offline Offline

Activity: 312


Dolphie Selfie


View Profile
August 08, 2014, 05:41:33 PM
 #10381

i think the UTXO set will have to have a pre-agreed order that will need to be synced amongst all miners in real-time.  then, the successful miner can publish the header hash along with some signal as to how far down in that list he went before he stopped adding tx's.  then, all other miners will know how to verify the published header hash and remove those tx's from the utxo set before moving on to the next block.

Yes, as I understand it, the trick is the deterministic order. The "signal as to how far down in that list" is the thing gavin tweeted about.

how reliable is a synced utxo set across all miners?

I don't think it is the utxo set, that has to be in an deterministic order, but the txs in the mempool. The utxo set is a result of the valid blockchain, so it should be synchronized already (except a node regards a block as valid, which later will become orphaned).

Gavin proposed to use "Invertible Bloom Lookup Tables". The lookup table tells the miner which transactions from its mempool should be considered for the verification of the published blockheader. Because now the miner knows which txs are used to form the block and in which order, he can reconstruct the merkle root and compare that against the published blockheader. However the proposed "Invertible Bloom Lookup Tables" are not 100% reliable. From a quick glance at the paper I found, that about <1% of keys are lost during an experiment with 10000 keys. Keys can be seen as txs in this case.
Now I ask myself what happens in case of the remaining 1%? Maybe the failure rate can be reduced by publishing multiple lookup tables of the same set?
1481100626
Hero Member
*
Offline Offline

Posts: 1481100626

View Profile Personal Message (Offline)

Ignore
1481100626
Reply with quote  #2

1481100626
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481100626
Hero Member
*
Offline Offline

Posts: 1481100626

View Profile Personal Message (Offline)

Ignore
1481100626
Reply with quote  #2

1481100626
Report to moderator
1481100626
Hero Member
*
Offline Offline

Posts: 1481100626

View Profile Personal Message (Offline)

Ignore
1481100626
Reply with quote  #2

1481100626
Report to moderator
sidhujag
Legendary
*
Offline Offline

Activity: 1288


View Profile
August 08, 2014, 06:01:16 PM
 #10382

iirc, the utxo set takes up about 75% of RAM required to run a node.  as i run 4 nodes myself, it takes a minimum of 1GB RAM to run smoothly meaning 750MB could be estimated for the utxo set itself.  that's pretty big to be including in a block...
Not in the block, alongside the block.

Just like when people merge mine Namecoin the nmc blocks aren't included in the btc blocks.

The merged-mine implementations that don't include the memory fix: (mutable vs imutable storage) and no auxpow data in cblockindex (blocks stored in memory) will eat up alot of RAM... so far only Devcoin and I0coin are the ones that have this fix and only Devcoin is running on 0.9.2 (currently in testing) with all the other fixes that bitcoin rolled up along the way included. So for most of these incompatible alt-coins who now wish to merge-mine (because I called that they would, and the ones that dont will die) are now suffering from either memory bloat issues or incorrect implementions/bugs and its next to impossible for a general programmer to port it over because its increasingly convoluted code to keep track of.
cypherdoc
Legendary
*
Offline Offline

Activity: 1764



View Profile
August 08, 2014, 06:04:19 PM
 #10383

iirc, the utxo set takes up about 75% of RAM required to run a node.  as i run 4 nodes myself, it takes a minimum of 1GB RAM to run smoothly meaning 750MB could be estimated for the utxo set itself.  that's pretty big to be including in a block...
Not in the block, alongside the block.

Just like when people merge mine Namecoin the nmc blocks aren't included in the btc blocks.

The merged-mine implementations that don't include the memory fix: (mutable vs imutable storage) and no auxpow data in cblockindex (blocks stored in memory) will eat up alot of RAM... so far only Devcoin and I0coin are the ones that have this fix and only Devcoin is running on 0.9.2 (currently in testing) with all the other fixes that bitcoin rolled up along the way included. So for most of these incompatible alt-coins who now wish to merge-mine (because I called that they would, and the ones that dont will die) are now suffering from either memory bloat issues or incorrect implementions/bugs and its next to impossible for a general programmer to port it over because its increasingly convoluted code to keep track of.

which is exactly why i've been warning everyone about these leech coins or sidechains that wish to piggyback themselves onto the Bitcoin mining network to secure themselves.
sidhujag
Legendary
*
Offline Offline

Activity: 1288


View Profile
August 08, 2014, 06:36:11 PM
 #10384

iirc, the utxo set takes up about 75% of RAM required to run a node.  as i run 4 nodes myself, it takes a minimum of 1GB RAM to run smoothly meaning 750MB could be estimated for the utxo set itself.  that's pretty big to be including in a block...
Not in the block, alongside the block.

Just like when people merge mine Namecoin the nmc blocks aren't included in the btc blocks.

The merged-mine implementations that don't include the memory fix: (mutable vs imutable storage) and no auxpow data in cblockindex (blocks stored in memory) will eat up alot of RAM... so far only Devcoin and I0coin are the ones that have this fix and only Devcoin is running on 0.9.2 (currently in testing) with all the other fixes that bitcoin rolled up along the way included. So for most of these incompatible alt-coins who now wish to merge-mine (because I called that they would, and the ones that dont will die) are now suffering from either memory bloat issues or incorrect implementions/bugs and its next to impossible for a general programmer to port it over because its increasingly convoluted code to keep track of.

which is exactly why i've been warning everyone about these leech coins or sidechains that wish to piggyback themselves onto the Bitcoin mining network to secure themselves.

Well merged-mining is still better than POS, and now with LTC asics you have scrypt merged-miners aswell... I think its a great thing. But those that have strong dev communites will survive. With Devcoin since its address compatible with bitcoin its more of a trivial task to keep the port and then the thin clients and external tools all just work... it was a smart decision to do so... so it really is a viable alternative but others may not be so lucky a few years down the road with some major structural changes to the codebase. I would much rather support merged-mine coins over some random pos coin because of the inability of those pos coins to keep up with the development power of bitcoin.

If you think about it conceptually from a bigger picture.. none of these alts can exist without bitcoin anyway, so it makes sense that without bitcoin hash power they would simply be not secure... but they are flexible to find another high hash parent if that ever happens.
domob
Legendary
*
Offline Offline

Activity: 936


View Profile WWW
August 08, 2014, 06:59:28 PM
 #10385

iirc, the utxo set takes up about 75% of RAM required to run a node.  as i run 4 nodes myself, it takes a minimum of 1GB RAM to run smoothly meaning 750MB could be estimated for the utxo set itself.  that's pretty big to be including in a block...
Not in the block, alongside the block.

Just like when people merge mine Namecoin the nmc blocks aren't included in the btc blocks.

The merged-mine implementations that don't include the memory fix: (mutable vs imutable storage) and no auxpow data in cblockindex (blocks stored in memory) will eat up alot of RAM... so far only Devcoin and I0coin are the ones that have this fix and only Devcoin is running on 0.9.2 (currently in testing) with all the other fixes that bitcoin rolled up along the way included. So for most of these incompatible alt-coins who now wish to merge-mine (because I called that they would, and the ones that dont will die) are now suffering from either memory bloat issues or incorrect implementions/bugs and its next to impossible for a general programmer to port it over because its increasingly convoluted code to keep track of.

Note that Namecoin also doesn't keep the auxpow in memory (CBlockIndex) or in the blkindex.dat file any longer.  I have removed this artefact (from times before I hacked on the code) recently.

Use your Namecoin identity as OpenID: https://nameid.org/
Donations: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS | GPG 0xA7330737
sidhujag
Legendary
*
Offline Offline

Activity: 1288


View Profile
August 08, 2014, 07:07:13 PM
 #10386

iirc, the utxo set takes up about 75% of RAM required to run a node.  as i run 4 nodes myself, it takes a minimum of 1GB RAM to run smoothly meaning 750MB could be estimated for the utxo set itself.  that's pretty big to be including in a block...
Not in the block, alongside the block.

Just like when people merge mine Namecoin the nmc blocks aren't included in the btc blocks.

The merged-mine implementations that don't include the memory fix: (mutable vs imutable storage) and no auxpow data in cblockindex (blocks stored in memory) will eat up alot of RAM... so far only Devcoin and I0coin are the ones that have this fix and only Devcoin is running on 0.9.2 (currently in testing) with all the other fixes that bitcoin rolled up along the way included. So for most of these incompatible alt-coins who now wish to merge-mine (because I called that they would, and the ones that dont will die) are now suffering from either memory bloat issues or incorrect implementions/bugs and its next to impossible for a general programmer to port it over because its increasingly convoluted code to keep track of.

Note that Namecoin also doesn't keep the auxpow in memory (CBlockIndex) or in the blkindex.dat file any longer.  I have removed this artefact (from times before I hacked on the code) recently.

You still need it in blnkindex.dat unless you dont care about not having any thin clients or external clients using "getheaders"
justusranvier
Legendary
*
Offline Offline

Activity: 1400



View Profile WWW
August 08, 2014, 07:13:37 PM
 #10387

why would you do that?
https://bitcointalk.org/index.php?topic=88208.0

If it ever becomes possible to do so without risking the integrity of the ledger, I would like Bitcoin to be able to discard transaction history.

Right now storing 100% of history is the easiest way to safeguard the ledger, but it also means we're subsidizing surveillance.

In a perfect world we'd only need to keep a P2P-validated UTXO set and enough history to detect tampering, and let the people who want to store everything forever so they can perform graph analysis pay for that themselves.
molecular
Donator
Legendary
*
Offline Offline

Activity: 2128



View Profile
August 08, 2014, 07:16:23 PM
 #10388

i think the UTXO set will have to have a pre-agreed order that will need to be synced amongst all miners in real-time.  then, the successful miner can publish the header hash along with some signal as to how far down in that list he went before he stopped adding tx's.  then, all other miners will know how to verify the published header hash and remove those tx's from the utxo set before moving on to the next block.

Yes, as I understand it, the trick is the deterministic order. The "signal as to how far down in that list" is the thing gavin tweeted about.

how reliable is a synced utxo set across all miners?

I don't think it is the utxo set, that has to be in an deterministic order, but the txs in the mempool. The utxo set is a result of the valid blockchain, so it should be synchronized already (except a node regards a block as valid, which later will become orphaned).

Gavin proposed to use "Invertible Bloom Lookup Tables". The lookup table tells the miner which transactions from its mempool should be considered for the verification of the published blockheader. Because now the miner knows which txs are used to form the block and in which order, he can reconstruct the merkle root and compare that against the published blockheader. However the proposed "Invertible Bloom Lookup Tables" are not 100% reliable. From a quick glance at the paper I found, that about <1% of keys are lost during an experiment with 10000 keys. Keys can be seen as txs in this case.

I just read the relevant parts of a paper explaining ITLB (Invertible Bloom Lookup Tables)... quite an interesting data structure. It has a constant size (pre-chosen) and allows storing (key, value) pairs. "insert" and "delete" operations are efficient. "get" may fail and "list" may return partial results.

I can see how an ITLB of some constant size might be used for block announcements by miners.

Some things I don't quite get, though:

  • why does this require deterministic ordering of txs in mempool?
  • does this require that all txs a miner includes in a block are properly broadcast beforehand?
  • would this require a hardfork or can it be used off-chain (so to say) by a subgroup of willing miners?

Now I ask myself what happens in case of the remaining 1%? Maybe the failure rate can be reduced by publishing multiple lookup tables of the same set?

maybe the publishing miner can include the 1% explicitly (he can determine which ones are affected himself and include the remainder). This would destroy the "constant size" property, but it would lessen the propagation cost of block size by ~99%, right?

PGP key molecular F9B70769 fingerprint 9CDD C0D3 20F8 279F 6BE0  3F39 FC49 2362 F9B7 0769
flipperfish
Sr. Member
****
Offline Offline

Activity: 312


Dolphie Selfie


View Profile
August 08, 2014, 09:56:30 PM
 #10389

Some things I don't quite get, though:

  • why does this require deterministic ordering of txs in mempool?
  • does this require that all txs a miner includes in a block are properly broadcast beforehand?
  • would this require a hardfork or can it be used off-chain (so to say) by a subgroup of willing miners?

1) This is needed to construct the merkle tree. Txs are grouped pairwise to build the nodes of the next level in the tree. The pairs are concatenated for the hashing, so the order is relevant. Currently the order is defined by the list of txs in the published block.

2) Yes, although I think the receiving miner could request missing tx(s).

3) To my understanding this does not need a hardfork, because the blocks are still built by the same rules.


Now I ask myself what happens in case of the remaining 1%? Maybe the failure rate can be reduced by publishing multiple lookup tables of the same set?

maybe the publishing miner can include the 1% explicitly (he can determine which ones are affected himself and include the remainder). This would destroy the "constant size" property, but it would lessen the propagation cost of block size by ~99%, right?


Ah, of course, this makes sense!
cypherdoc
Legendary
*
Offline Offline

Activity: 1764



View Profile
August 08, 2014, 10:55:21 PM
 #10390

Hey look!

Dow goes up, Dow goes down,  gold goes up, gold goes down. Meanwhile;

Bitcoin goes UP!
hdbuck
Legendary
*
Offline Offline

Activity: 1134



View Profile
August 08, 2014, 11:03:34 PM
 #10391

The US government has nearly as much debt as the world has wealth:
http://dollarvigilante.com/blog/2014/6/10/the-us-government-has-nearly-as-much-debt-as-the-world-has-w.html
tvbcof
Legendary
*
Offline Offline

Activity: 1988


View Profile
August 09, 2014, 06:19:49 AM
 #10392

Hey look!

Dow goes up, Dow goes down,  gold goes up, gold goes down. Meanwhile;

Bitcoin goes UP!

Chart I'm looking at has BTC trending down for a month.  The bad news, though, is that the imbalance is heavy on the supply side.  At this point I'll be surprised to see a selling opportunity which works for me in 2014, and not at all surprised to seen some significant downside in the intrim.  Oh well.


sidhujag
Legendary
*
Offline Offline

Activity: 1288


View Profile
August 09, 2014, 07:23:12 AM
 #10393

Hey look!

Dow goes up, Dow goes down,  gold goes up, gold goes down. Meanwhile;

Bitcoin goes UP!

Chart I'm looking at has BTC trending down for a month.  The bad news, though, is that the imbalance is heavy on the supply side.  At this point I'll be surprised to see a selling opportunity which works for me in 2014, and not at all surprised to seen some significant downside in the intrim.  Oh well.



LTC didnt reach target for me means its dragging it all down. Physcologically we are not ready for the next uptick, for me, until that level is tested, unless we go up and then come crash down later on which is not bullish long term... anyways just what worked for me in trading forex over the years... you build up an intuition of levels that must test before the real trend resumes. Something you cant really teach.
domob
Legendary
*
Offline Offline

Activity: 936


View Profile WWW
August 09, 2014, 08:32:32 AM
 #10394

iirc, the utxo set takes up about 75% of RAM required to run a node.  as i run 4 nodes myself, it takes a minimum of 1GB RAM to run smoothly meaning 750MB could be estimated for the utxo set itself.  that's pretty big to be including in a block...
Not in the block, alongside the block.

Just like when people merge mine Namecoin the nmc blocks aren't included in the btc blocks.

The merged-mine implementations that don't include the memory fix: (mutable vs imutable storage) and no auxpow data in cblockindex (blocks stored in memory) will eat up alot of RAM... so far only Devcoin and I0coin are the ones that have this fix and only Devcoin is running on 0.9.2 (currently in testing) with all the other fixes that bitcoin rolled up along the way included. So for most of these incompatible alt-coins who now wish to merge-mine (because I called that they would, and the ones that dont will die) are now suffering from either memory bloat issues or incorrect implementions/bugs and its next to impossible for a general programmer to port it over because its increasingly convoluted code to keep track of.

Note that Namecoin also doesn't keep the auxpow in memory (CBlockIndex) or in the blkindex.dat file any longer.  I have removed this artefact (from times before I hacked on the code) recently.

You still need it in blnkindex.dat unless you dont care about not having any thin clients or external clients using "getheaders"

Nope.  Currently, Namecoin doesn't allow "getheaders" since no client uses it and I didn't want to leave it there completely untested.  But in a previous development version, I implemented it just fine - you can just load the full block from disk if you really need it (someone sends a "getheaders") - and optionally cache it in memory, subject to your favourite heuristics for removing the data from memory again.

Use your Namecoin identity as OpenID: https://nameid.org/
Donations: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS | GPG 0xA7330737
cypherdoc
Legendary
*
Offline Offline

Activity: 1764



View Profile
August 09, 2014, 09:25:41 AM
 #10395

hopefully, this lull will kill off several (a few dozen) altcoins.
cypherdoc
Legendary
*
Offline Offline

Activity: 1764



View Profile
August 09, 2014, 09:33:28 AM
 #10396

the charts of all the alts look horrific.  most investors in cryptocurrencies will lose money.
Wekkel
Legendary
*
Offline Offline

Activity: 1386



View Profile
August 09, 2014, 10:37:07 AM
 #10397

Not kill, just low prices.

Like this post? you can tip me (BTC) 1LGi2DMhectdFSkBid5srrnHk8WHgD3V1d or very WoW (Doge) D9p6FZQb1sKkq9hApy4tnjSduYfdnc74bb
cypherdoc
Legendary
*
Offline Offline

Activity: 1764



View Profile
August 09, 2014, 05:08:10 PM
 #10398

stick with it, boys.
Melbustus
Legendary
*
Offline Offline

Activity: 1554



View Profile
August 09, 2014, 05:28:33 PM
 #10399

hopefully, this lull will kill off several (a few dozen) altcoins.


I think it's gonna get worse before it gets rational. The mentality in the alt-scene reminds me of of the dotcom bubble. Everyone is screaming "New Paradigm!", "Old Rules Don't Apply!". It's the same phenomenon; they're correctly seeing that this new idea of blockchain-based ledgers opens up a whole new world of possibilities, but they're then incorrectly - and sometimes even explicitly - saying that core human economic principles no longer apply and its a completely new field, even in that regard. The consequence is that valuations will be hugely distorted relative to fundamentals, and many many participants will be burned when valuations inevitably (and quickly) correct.

Anyways, I think, yeah, the lull will probably kill off a few dozen alts. But there are hundreds of scam-coins; didn't someone count over 1000 recently? And besides, the "platform" coins are only just getting going. I like the tech Ethereum is trying to do, but I think it's likely a terrible investment given their distro model, what they've raised already, and the opportunity cost of bitcoin itself. Maybe an ultimate market-crash of such a "flagship" alt will be what triggers reality in the alt market.

Bitcoin is the first monetary system to credibly offer perfect information to all economic participants.
But Bitcointalk & /r/bitcoin are heavily censored. bitco.in/forum, forum.bitcoin.com, and /r/btc are open.
Best info on Casascius coins: http://spotcoins.com/casascius
cypherdoc
Legendary
*
Offline Offline

Activity: 1764



View Profile
August 09, 2014, 05:53:08 PM
 #10400

Like I said,  the majority of investors even in a promising investment space will lose money. That's because scammers are inevitably attracted. 
Pages: « 1 ... 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 [520] 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 ... 1560 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!