Gold collapsing. Bitcoin UP.

[cypherdoc]

ok, i'm not getting the bolded part.  this graph shows 37 MB worth of unconf tx's, no?:

No clue, no node I have access to is seeing that much-- they may have turned off the minfee rules (not unreasonable for a metrics thing)...

Even given that, again, 37MB doesn't explain your swap.

this is what they say on their website.  i should try to find out the exact #:

TradeBlock maintains an extensive bitcoin network data architecture with multiple nodes across geographies. With the ability to view and record every message broadcast to the network, including those that are not extensively relayed, unique insights regarding the network may be derived.

[TPTB_need_war]

Conclusions are valid in substance but in terms of not getting blocks for days, that's not really a big problem for a pool (or solo mining farm) being run as a business. Most expenses (hosting, etc) are going to run monthly, possibly somewhat more often (paying workers), but not daily. Once you look at a one-month window it is about the same as Litecoin: almost certain to find a block in a month. With some ability to draw on reserves or credit in the event of a few bad months it should still be viable.

I think you missed the economic reasoning, which is that pools have to compete at near 0 margins which means any losses due to overpaying some ephemeral miners during periods (e.g. any variant of pool hopping or just miners who come and go for any reason), is bankruptcy relative to those with larger hashrate share which don't incur that variance in profitability.

Variance such as in unit-of-exchange kills currency and it does the same concentrating effect on pools, as they are currently defined in the current network design for PoW coins (which btw I have a fix for[1]).

And that centralizing effect is exacerbated by the in-band (e.g. lower relative to hashrate latency and orphan rate) and out-of-band incentives (e.g. being paid to double-spend a 0-conf, etc) that may apply to any who may be running hidden monopolies via the Sybil attack, perhaps able to run at negative profit margins thus driving smaller pools extinct. Again I challenge any one to prove that pools are not Sybil attacked (meaning multiple pools being effectively owned by the same controlling entity). Since it can't be disproved, then Cypherdoc's assumption that miners are independent can not be proven to be true, yet it can be falsied by the math and economic reasoning I presented above (although some will refuse to admit the math because they can't falsify it with physical evidence).

All of this gets worse as bandwidth requirements are scaled up.

[1] my improvement over CN need not be anonymity (which could just be copied from CN), but rather network characteristics.

[Adrian-x]

Interesting!  
And this is why I like the empirical "block box" approach.  I don't care initially what the mechanism is.  I try to find a simple model that explains the effect, and then, later, ask what that mechanism might be.
But now why would the "latency of the mining process" depend on the size of the previous block?  That doesn't make sense to me, but we just showed empirically that F2Pool is indeed more likely to produce an empty block when the previous block is large.

It wouldn't expect the miner latency part to be size dependant: the miner can't even tell how big the prior block was.  I expect your function relating them to have a big constant term in it! (thats why I asked if you tried other regression approaches. )

I suppose there may be some dependance that is introduced by virtue of what percentage of the miners got the dummy work.  Would be pretty interesting to try to seperate that.

Another trap of empirical analysis in this kind of discussion is that we can only measure how the system is-- but then we use that to project the future;  e.g.  say we didn't have ECDSA caching today, you might then measure that it was taking >2 minutes to verify a maximum size block... and yet 100 lines of code and that cost vanishes; which is bad news if you were counting on it to maintain incentives.

Miners can't tell as soon as a block header is published, but they can tell once they get the block, according to Peters analysis bigger blocks take longer to propagate so SPV mining fills the gap until the block is known. So yes you're correct they can't know but not knowing isn't a good explanation as to why we see a higher percent of empty block after a big block.

[TPTB_need_war]

Another trap of empirical analysis in this kind of discussion is that we can only measure how the system is-- but then we use that to project the future;  e.g.  say we didn't have ECDSA caching today, you might then measure that it was taking >2 minutes to verify a maximum size block... and yet 100 lines of code and that cost vanishes; which is bad news if you were counting on it to maintain incentives.

Good point and more accurate is "we can only measure how we think it is". We don't even have absolute proof that someone isn't running a superior algorithm or hardware solution although we can often make very strong educated inductions.

[cypherdoc]

Interesting!  
And this is why I like the empirical "block box" approach.  I don't care initially what the mechanism is.  I try to find a simple model that explains the effect, and then, later, ask what that mechanism might be.
But now why would the "latency of the mining process" depend on the size of the previous block?  That doesn't make sense to me, but we just showed empirically that F2Pool is indeed more likely to produce an empty block when the previous block is large.

It wouldn't expect the miner latency part to be size dependant: the miner can't even tell how big the prior block was.  I expect your function relating them to have a big constant term in it! (thats why I asked if you tried other regression approaches. )

I suppose there may be some dependance that is introduced by virtue of what percentage of the miners got the dummy work.  Would be pretty interesting to try to seperate that.

Another trap of empirical analysis in this kind of discussion is that we can only measure how the system is-- but then we use that to project the future;  e.g.  say we didn't have ECDSA caching today, you might then measure that it was taking >2 minutes to verify a maximum size block... and yet 100 lines of code and that cost vanishes; which is bad news if you were counting on it to maintain incentives.

Miners can't tell as soon as a block header is published, but they can tell once they get the block, according to Peters analysis bigger blocks received take longer to propagate process so SPV mining fills the gap until the block is known has been processed. So yes you're correct they can't know but not knowing isn't a good explanation as to why we see a higher percent of empty block after a big block.

ftfy

[TPTB_need_war]

If you don't want to be called LeBron until the day Lord Satoshi moves His Holy Coins, I suggest apologizing for accusing the core devs (who write the free code you run) of impropriety/malfeasance/obstructionism/etc.

And you have the audacity to assert I am delusional for having faith in Armstrong's investment in his models and the performance I have observed thereof. At least I am not blind to the existence of my faith — as opposed to first hand knowledge of the open source, which you don't have either in the case of your Lord and I also doubt you've internalized all the source code that has been written by ongoing devs.

Haters gonna hate...

[cypherdoc]

ok, i'm not getting the bolded part.  this graph shows 37 MB worth of unconf tx's, no?:

No clue, no node I have access to is seeing that much-- they may have turned off the minfee rules (not unreasonable for a metrics thing)...

Even given that, again, 37MB doesn't explain your swap.

but turning off minfee rules would only be used to include 0 fee tx's.

look at how much BTC value is included in that 37MB.

[cypherdoc]

looks like they may have turned off SPV mining.  that would be good.

[TPTB_need_war]

I favor Adam Backamoto's extension block proposal.

The 1MB blocksize limit reminds me of the old 640k limit in DOS.

Rather than destroy Window's interoperability with the rich/valuable legacy of 8088 based software, that limit was extended via various hacks sublime software engineering.

Where can I find a specification for this extension blocks proposal, so I can determine how this proposal is differentiated from a pegged side chain?

Mike Hearn is so obviously for centralization it is a requirement to separate his objective points, and appears he may have a point about Lightning networks being not viable for much but very specialized interactions (which was also my upthread point about them).

I continue to think existing PoW designs are stuck between a rock and a hard place, thus I am very interested to read more detail about this proposal.

Edit: I found it, http://sourceforge.net/p/bitcoin/mailman/message/34157058/

The only way I see this is differentiated from pegged side chains, is it could be optionally a one-way transfer of BTC to the extension block chain, thus removes the need for the reliance on federated servers.

I understand that if address formats are differentiated between the two chains, then it is claimed one could pay from one chain to other but I can't see how that can work because the miners on the lower bandwidth chain would be SPV miners on the extension block chain. Thus all the dominos (due to orphaned chains) insecurity ramifications of pegged side chains which I argued upthread is untenable, unless you allow the BTC on each chain to have a different value. Afaics, the only reliable way to move pegged value back and forth between chains is as I wrote upthread with very long challenge periods on transferred funds and to use the Blockstream SPV proofs.

Thus one can view this proposal as a trojan horse to require Blockstream's pedded chains (and either the federated servers or the soft fork for the added OP code to eliminate the federated servers). Clever.

However, I still like the proposal for the same reasons I liked pegged side chains. But pleeeaaaase do it correctly. None of the mumbo jumbo about instant transfers between pegged chains. Sheesh.

And note this does nothing to solve the decentralization problem and rather just transfers the centralization to the extension block chain, because the 1MB chain can ultimately be 51% attacked because its relative block rewards will wither as debasement diminishes and transactions increase on the extension block chain (that is unless extension block transactions will have so extremely small fees and the 1MB chain sufficiently high enough fees which is very very unlikely because centralization by definition drives towards monopolistic pricing). Also because per math and economic reasoning above, solo mining is doomed any way.

[domob]

look at what we're facing with this latest spam attack.  note the little blip back on May 29 which was Stress Test 1.  Stress Test 2 is the blip in the middle with the huge spikes of the last couple of days on the far right.  this looks to me to be the work of a non-economic spammer looking to disrupt new and existing users via stuck tx's which coincides with the Grexit and trouble in general in the traditional fiat markets.  they want to discourage adoption of Bitcoin.  the fastest way to eliminate this attack on users is to lift the block size limit to alleviate the congestion and increase the expense of the spam:

So you think that it would be best to simply put all that spam on the blockchain, and have everyone that ever wants to validate it go through that spam for eternity?  Wouldn't it be better to simply let the network and miners adjust their fee policies instead as they see fit and make sure that the spam is not even mined unless they pay "enough" fees?

[Peter R]

So you think that it would be best to simply put all that spam on the blockchain, and have everyone that ever wants to validate it go through that spam for eternity?  Wouldn't it be better to simply let the network and miners adjust their fee policies instead as they see fit and make sure that the spam is not even mined unless they pay "enough" fees?

Who decides what enough fees is?

I think this whole blocksize debate comes down to ideology.  It's not a technical decision. Some people think we need to keep 1 degree of freedom of control to shape Bitcoin "correctly," while others think we should let destiny take the wheel.

[TPTB_need_war]

So you think that it would be best to simply put all that spam on the blockchain, and have everyone that ever wants to validate it go through that spam for eternity?  Wouldn't it be better to simply let the network and miners adjust their fee policies instead as they see fit and make sure that the spam is not even mined unless they pay "enough" fees?

Who decides what enough fees is?

Actually you ask precisely the correct question, if you want to find the correct answer for decentralization. Hint, hint. But that answer is more clever and complex than I expect you to find.

I think this whole blocksize debate comes down to ideology.  It's not a technical decision. Some people think we need to keep 1 degree of freedom of control to shape Bitcoin "correctly," while others think we should let destiny take the wheel.

You think that because you are blinded.

My well informed, prescient suggestion is wholeheartedly support the pegged side chain direction; otherwise prepare to exchange your BTC for another monetary unit or accept centralization.

[TPTB_need_war]

A clinic full of subordinate eye frying minions would be another...

Many of my co-workers kept a stock market ticker within easy visual reach and referred to them about 60 times per hour as they performed the tasks of the day.  That's fine for code monkeys, but...

+1 for dry humor style points  

If I were hiring cypherdoc to shill for me he would have been fired about a month ago when he reached an inflection point...

That was when I arrived in the thread.  

As long as he sticks to techno babble and steers clear of excessive ad hominem (against me for example that got his HF case revealed by vokain), then afaics he bolsters his case by remaining the owner of the most read thread on bitcointalk, i.e. in the bitcoin universe.

Note I am no longer fighting with him and I assume he has declared a truce with me also. Best for both of us. I have no skin in the HF debacle (although I feel for vokain et al, they need to learn and bounce back[1]). If he has cajones, Slypherdoc will invest some of his 3000 BTC in NewCoin and perhaps make off like a $billionaire bandit (untraceable). As for justice, don't people who fall for scams have some culpability too? The universe has a way of handing out justice. I lost my eye because of some decisions I had made. I had to own up to responsibility for it, i.e. involving the government in dispensing justice wasn't justified nor efficient. Life is too short.

[1] the day after the event on Dec. 1, 1999 that cost me blindness in my eye, I in my mind and heart forgave the guys who did it to me and decided to moveon. I don't have time for grudges. There is too much stuff I can still do. Until you render me incapable of doing anything, I will try my best to remain positive about life. I did however take up boxing as form of self-defense training and exercise since that event. And I continue to do sprinting training, because the smartest defense is often to run.

cypherdoc, who is paying you now ? (KNC ?)

Perhaps all those who placed preorders with HF.

[solex]

You think that because you are blinded.

My well informed, prescient suggestion is wholeheartedly support the pegged side chain direction; otherwise prepare to exchange your BTC for another monetary unit or accept centralization.

I've got to put this type of comment in the "give-up" file.
It would have merit if the following weren't the case:

x=Software & design optimizations. e.g. BitcoinCore is orders of magnitude faster in many respects than in 2009.
y=Moore's "Law" and Nielsen's "Law" and other relevant descriptions of the rate of improvement in computing technology

x*y > rate-of-growth-of-the-world-economy  (here's a visual)



It is only a matter of time before Bitcoin can handle a large proportion of world transactions. The key thing is not to f*ck up the process in the meantime.

others think we should let destiny take the wheel.

I think this is the only viable option.

[TPTB_need_war]

You think that because you are blinded.

My well informed, prescient suggestion is wholeheartedly support the pegged side chain direction; otherwise prepare to exchange your BTC for another monetary unit or accept centralization.

I've got to put this type of comment in the "give-up" file.

...

It is only a matter of time before Bitcoin can handle a large proportion of world transactions. The key thing is not to f*ck up the process in the meantime.

Give up if you don't care about centralization. I for one think a Rottweiler will bite the arms off those who don't care:

digitalchamber.org/bootcamp.html

And I don't think centralized scaling can attain the network effects of decentralized scaling. Thus I think the heavy end of the scale goes to NewCoin by orders-of-magnitude (note NewCoin might be something created by Blockstream, not limiting it to Trapqoin by Shetty One Eye).

Make your choice do you want NewCoin pegged to BTC or not. That may be the only choice you have and you may not even have that choice as that appears to maybe be inevitable (although I am concerned about TPTB's ability to attack the federated servers if the community is reticient on the necessary OP code soft fork, but the way around that is build the OP code into NewCoin and aim for extracting all the BTC out of Core asap).

[iCEBREAKER]

Rather amazing that he still is posting.  For starters, his counsel should have advised him to stop posting here.

Frap.doc is the world's leading expert on everything; he is the LeBron of Dunning-Kruger.

What makes you think he'd respect some dumb old lawyer's advice, when he refuses to swallow basic technical facts charitably spoon-fed to him by several of the world's top 10 Bitcoin experts?

His saving grace is that the absurd, pro-forma, borderline barratry lawsuit against him is being laughed out of court; the bankruptcy lawyers just did it to look busy and thus justify their fees and the endless process.

[TPTB_need_war]

Certantly the UTXO size is a major concern for the viability of the system, since it sets a lower bound on the resource requirements (amount of online storage) for a full node... but it is not held in memory and has no risk of running hosts out of ram as you claim.

One of the costs of UXTO is it must be downloaded by every ephemeral miner that wants to be a full node (well I guess I am conflating UXTO and the entire block chain). This has been the source of some complaints against CN coins with large block chains. Now granted that given my points recently, solo mining is really pointless especially for Bitcoin where you need to amortize an ASIC purchase. So this is a non-issue for Bitcoin (but only because Bitcoin is already centralized) whereas it is for CPU mine-able coins such as the newer CN coins.

However, I must take issue with your assertion that unbounded UXTO could never cause a full node to run out of RAM. Afaics it is possible to envision a level of swapping that the system can not keep up with the transaction rate. That is however perhaps irrelevant if for example solo mining is not viable due to variance for small hashrate and where bandwidth (or latency) being the predominant bound for miners with less resources (or who wish to remain ephemeral for privacy reasons and any other reasons).

[iCEBREAKER]

my reference to Peters argument above aid nothing about mempool; I was talking  about block verification times. You're obfuscation again.

In your message to me you argued that f2pool was SPV mining becuase "the" mempool was big. I retored that their mempool has nothing to do with it, and besides they can make their mempool as small as they want. You argued that the mempools were the same, I pointed out that they were not. You responded claiming my responses was inconsistent with the points about verification delay; and I then responsed that no-- those comments were about verification delay, not mempool. The two are unrelated.  You seem to have taken as axiomatic that mempool == verification delay, a position which is technically unjustified but supports your preordaned conclusions; then you claim I'm being inconsistent when I simply point out that these things are very different and not generally related.

Yes, that's exactly what happened.  It's very kind of gmax to so diligently care for our enfeebled but honored elder by chewing his food, changing his diapers, wiping the drool from his chin, and forgiving his inability to form new memories (much less understand new concepts).  Gmax is a saint for lovingly patting Dr. Frappe on the head when he gets cranky, instead of reaching for the Thorazine.

It's possible to construct a contrived block today that takes many minutes to verify, even within the 1MB limit; though a miner that did that would mostly be hurting themselve unless they had some arrangement with most of the hashpower to accept their block.

I'm fascinated by the trolling and research possibilities of such 'contrived' blocks.  Generating them seems less expensive than the brute force high-fee crapflood attacks recently used by the Gavinistas, but it isn't in their interest to publicize them as they contraindicate their desired (>)>1MB blocks.

In order to optimize their contrivance, one would have to understand how best to exploit the weaknesses of Createnewblock's existing inefficiencies.  OTOH, being able to generate such extra-obnoxious blocks provides an excellent tool for testing proposed optimizations.

[TPTB_need_war]

IBLT doesn't currently exist, and other mechenisms like the relay network protocol don't care about mempool synchronization levels.

IBLT does exist as it has been prototyped by Kalle and Rusty. It is just nowhere near ready for a pull request.

It has never relayed a _single_ block, not in a lab, not anywhere. It does _not_ exist. It certantly can and will exist-- though it's not yet clear how useful it will be over the relay network-- Gavin, for example, doesn't believe it will be useful "until blocks are hundreds of megabytes".

But don't you think that I'm saying anything bad about it-- I'm not. Cypherdoc was arguing that mempools were (and had) to be the same, and cited IBLT as a reason---- but it cannot currently be a reason, because it doesn't exist.  Be careful about assigning virtue to the common fate aspect of it-- as it can make censorship much worse. (OTOH, rusty's latest optimizations reduce the need for consistency; and my network block coding idea-- which is what insired IBLT, but is more complex-- basically eliminates consistency pressure entirely)

I recall that you had a tepid response summarizing the benefit of IBLT as a x2 improvement.  Of course this is hugely dismissive because it ignores a very important factor in scaling systems: required information density per unit time. Blocks having to carry all the data in 1 second which earlier took 600 seconds is a bottleneck in the critical path.

It depends on what you're talking about, if you're talking about throughput it's at best a 2x improvement, if your'e talking about latency it's more.  But keep in mind that the existing, widely deployed block relay network protocol reduces the data sent per already known transaction _two bytes_.

Compression of exchange of differential sets is interesting. Your extensive real-world experience in codecs is really evident at the page you linked which is outside my current knowledge. I would need to devote some time to fully digest the specifics of your proposal (and perhaps Rusty's optimizations to IBLT). I do understand the concept that error correcting code allows us to reconstruct a signal in a noisy channel.

As you've pointed out today, large hashrate will still have the latency of 0 for all the blocks it mines. So unless these methods can bring latency down to a level which has a holistic (in and out-of-band game theory) stable equilibrium in economic cascade less than oligarchy centralization, then it won't necessarily stop censorship.

Let's assume we can optimize away latency (which is more general to the economics than just orphan rate) in the current design of PoW cryptocurrency, yet we are still faced with the unavoidable centralization due to block reward variance (that I explained today) and that due to transaction rate exceeding the bandwidth of home internet connections which vary widely around the world and between wireless and wired.

And then if you were really serious about Ultimate Decentralization™ (the ™ is a joke), everyone who transacts would be a miner. In that case, the bandwidth of home connections could be a critical consideration depending on the design of the consensus mining system.

[iCEBREAKER]

LN which doesn't exist yet and will arrive far too late to help with scaling when blocks are (nearer to) averaging 1MB.

The 1MB is proving a magnet for spammers as every day the average block size creeps up and makes their job easier. A lot of people have vested interest in seeing Bitcoin crippled. We should not provide them an ever-widening attack vector.

BTC blocks are always going to tend towards full because demand is infinite and supply is not.  Get used to it.

The Lightening Network will exist in due course.  Meanwhile, the Litecoin Network is entirely capable of absorbing excess transactions should BTC be pushed over capacity (whatever that means...where is JR when we need him to jump up and down demanding 'overcapacity' be rigidly defined in objective metrics?).

Don't fear spammers; they provide incentives to optimize.

Stop fretting about people desiring to see Bitcoin crippled; their attacks should be welcomed because antifragile systems grow stronger when, and only when, presented with adversity.

That min fee at 0.0005 is 14 cents, and most users consider this to be way too high, especially if BTC goes back to $1000 and this becomes 50 cents. I kicked off a poll about tx fees and 55% of users don't want to pay more than 1 cent, 80% of users think 5 cents or less is enough of a fee.

Stop the presses, the Free Shit Army wants moar free shit!  So what?

It costs much more than 14 cents worth of electricity/equipment/labor to process a BTC tx.  The block reward has done its job, and it's time to start the process of weaning the ecosystem (IE economy) off such giant subsidies by making each BTC tx do more to earn its keep.

So, in 2010 Satoshi was forward looking, when the 1MB was several orders of magnitude larger than block sizes.. Yet today we are no longer forward-looking or care about peak volumes, and get ready to fiddle while Rome burns.

 

If you could calm down and stop exaggerating, that would be great.