How you store data is completely independent of the consensus rules (and behaviour of other nodes).

You can cut about 25% off the storage space by switching to a more efficient transaction serialization and no other design or layering or consensus changes. You don't need anyone else's permission or coordination, you can just do it.

People have implemented this previously (e.g. blockstream did so) but apparently haven't considered it worth the effort to make it production ready and submit it to the project.  At the end of the day it's only a small constant factor reduction and the blockchain keeps growing at a considerable rate.

Trying to foist those considerations onto consensus rules would both be bad engineering and politically foolish. Concerns which can be separated should be, work on consensus logic is hard enough without hyper-milling the storage and transmission formats.

Consensus only cares about how data is presented to hash functions and hash functions are so fast (like <167 picoseconds per byte for large inputs with sha-ni on a 3GHz quad core) that the serialization into hashes isn't a particularly important optimization target.

If you're getting into implementation details,

It might also be worth mentioning that most of the validation is typically done *before* the block comes in. In Bitcoin a great effort has been made to design things so that most of the validation is a pure function of the transaction-- for almost all rules given transaction is either valid or invalid depending only on its own content and nothing else.  The exceptions to this are checking that the inputs exist and aren't already spent, checking that the transaction nlocktimes are valid according to the block time, and checking that various block-wide global limits are obeyed (primarily that the block weight isn't exceeded).

This means that in the common case of transactions that were received ahead of the block most of their validation (amounts, scripts, signatures, etc) is cached and doesn't need to be done again when the block shows up, greatly reducing latency.

I think it's important to be aware of this because I've seen a number of posts over the years that seem to think that every time a new block is accepted everything in it must be laboriously reprocessed or even that the whole blockchain must be reprocessed. This isn't true, and it gives the wrong impression about where bottlenecks are on average.

This is made more complicated in Bitcoin because since 2010 validity in relay uses somewhat different, more strict, rules than validity in blocks to improve forward-compatibility.

The idea consensus rules can be improved in a way which is completely compatible with old software by strictly reducing the set of things that are permitted and by purposefully setting aside extension space in the protocol to eventually get restricted (such as NOP op-codes), old nodes then know something 'future' is going on and thus won't relay or mine it which would risk including something invalid, but are happy to accept whatever new nodes mine using the extensions.   Along with that, the consensus rules for a given block might be more strict than the rules for the block before it e.g. when a new rule activates.  

So the caching has to handle the rules changing and does that by augmenting the key for the cache with the set of rules, so the cache can only have a hit when the right rules are being used... also when a node receives a transaction it validates it under both the stricter relay rules and the looser block rules (why both when the relay rules are stricter?  because then there would be a risk if there were a bug in the relay rules it could result in accepting an invalid block, which would make it much slower to develop code for relay-rules).

[Aside, I removed a thread-derailing repetitive post by franky; and I'd strongly recommend everyone follow my lead and stick him on ignore to avoid wasting time and energy on his aggressive misinformation and disruption]


Yes, in Bitcoin it is technically possible for the height to go down!  But it's not likely something we'll ever observe on mainnet, at least not near the tip of a synced node.  The reason is that the difficult of blocks only changes (retargets) once every 2016 blocks and reflects the time that the last 2015 blocks took. So for fewer blocks to have more work there has to be a fork around a retarget and the difficulty on the two forks has to be different enough that a branch with N blocks can have more work than a Block with N+1 blocks, and the fork has to be at least N blocks long.  Commonly forks are only 1 block long and for a branch with one block to have more work than a branch with two the difficulty will have had to more than double in that one fork.  It would be really hard for such a large difficulty difference because the two blocks will share most of the same history going into their difficulty calculation.  Having it happen with a longer fork would require less of a difficulty difference but longer forks are themselves more rare.

So the only time with a mainnet node that I would expect to see height go down is during initial sync, if you connected initially to a malicious clown who intentionally forked the blockchain early on and purposefully kept the difficulty down and churned out a lot of blocks at low difficulty, then height might go down temporarily as you switch from his joke chain to the real chain.

Various altcoins have various psycho over-eager difficulty update rules where its very easy for height to go down, either intentionally or by accident. In theory that's okay, but I bet a lot of software won't handle it well.

Fun fact: the Bitcoin whitepaper and the software up until IIRC early 2010 made the best valid chain decision on total blocks and ignored total work. Unfortunately, as I mentioned, it's quite possible for a fork that starts really early in the chain to win by number of blocks but not work-- even though it's something you'll never see in normal operation, so Satoshi's original way of handling it was vulnerable and easily attacked.

Sure they are, they're required to tell two different transactions apart.

When txs are only identified by txids I can take a valid transaction mutate its witness to make it invalid (or just too low a feerate), and it'll have the same txid, so if you fetch by txid you can't avoid fetching the same junk multiple times-- you can't negative cache the invalid values.

wtxid works much better in that respect.  They need to be computed in any case because otherwise block propagation latency would be much higher,  and the computational cost of hashing a single transaction is pretty negligible in any case (and as mentioned, it's already being done). Switching to wtxid for relay has also been in the works for over three years.

Doesn't have the property that an old lost backup that you forgot existed can't be used to steal your money... while the simpler expedient of a wallet that can add an additional seeds and, eventually, forget old ones has both the property you were looking for and this leakage resistance property too.

Separately, the chained computation has a somewhat unfortunate property that it's fragile.  E.g. if a cosmic ray rains on your day when you compute step 37 then all further steps will be irreparably corrupted.    Armory's wallet scheme worked like that while BIP32 intentionally did not, and the chained behavior has lead to at least one major funds loss incident.

Perhaps this last point isn't a big deal because you can and you should use really aggressive double checking of all computation related to generating keys ... but the point that the simpler seed-rotation mechanism gets you both forward and backward security (and bonus, the backward security is better, since if you leave an old backup around it doesn't get all the future keys that you'd already forgotten), I think is perhaps enough of a reason to not use the more complicated thing.

Simulation seems to show that it improves block propagation delay in fact.

Erlay's latency increase for propagation is almost entirely an increase in latency of the first hop.

Overall with the recommended parameters in simulation the result has transactions take a little longer to propagate but there is less variance in their overall propagation time.  This improves the block propagation speed because more often if a miner includes a txn everyone already has it. ... at least in simulation.

I wouldn't promote that as an advantage, because miners adding a small delay before including transactions would likely have a much greater non-additive effect ...  but it doesn't appear the an increase in orphaning is likely, at least.

We're mostly past that point. The same people they could deceive by doing that can be even better deceived by pure jargon-technobabble, and the Bitcoin industry won't even bother to call out the fraud in that case.  If they just copy bitcoin they'll be constrained by the realism of the bitcoin discussion of the copied tech and also risk irritating bitcoin people calling out their garbage.

Doesn't look good for your erlay fuelled scam if the inventors of erlay step up and call it a scam, much safer to promote rolling-hypercube-slasher-coordinator-masternode-unls or whatever where you can dismiss any criticism as pure ignorance and "toxic maximalism".

So weird that the media won't get within 100 feet of pointing out that Wright is an obvious scammer, but seems to have no problem doing so for onecoin.  Onecoin is obviously a scam, but not more obviously than wright...

Funny that after all these years people are still falling for Emin Gun Sirer's market manipulation-- even though his twitter is stuff full of obviously paid altcoin promotion.

Sometimes it makes me feel like people are just dying to do something stupid and only looking for an excuse...

Because mining is a noisy poisson processes it provides a never ending stream of excuses for those who hope to dare to be stupid.

Yep, this could also be accomplished with specialized hardware but the security would be limited by the physical security of the device... which is historically kind of dubious.... esp because there is a tradeoff with device robustness.

E.g. if you build a device that self destructs on tamper maybe it also self destructs if you drop it.

What you're asking for exactly isn't possible because an attacker could isolate the computer or just use different software.

What would be possible is if the decryption required the help of a third party.   For example, it would be technically possible to construct a scheme where you have two pieces of data and a password,  and a third party has a public/private key.    To decrypt your data you need both the data and the password, plus the assistance of the third party.  And the protocol could be setup so that if you provide the correct password the third party learns nothing about your data or the password, but if you provide an incorrect password, the third party learns only the second piece of data.

Then what you want could work (the second piece of data could be a txn sweeping your funds).  However, the reliance on the third party means that if he got hit by a bus your funds would be lost. The other disadvantage of this idea is that a protocol to do it securely would be very complex and errors in its design or implementation might break your security.

If you actually cared about size you'd drop the hash entirely-- the result is much smaller still.  Recovery is pretty slow, unbatchable (which means another 2x slowdown), and has a complicated patent situation.

That project is a malware scam. They promote their activity using jibberish technobabble and have been caught distributing actual backdoored software.

Bitcoin Core's RPC interface is secure by default and at least slightly difficult to make insecure.

Locking this thread so it stops getting bumped and soliciting more suckers.

Using google to search bitcointalk for anything involving the name of any subforum moderator is pretty much useless because *all* the pages have the moderators listed on them.

A simple fix would be to leave these off of the page views rendered for logged out users.

Alternatively, just leaving them off the threads themselves for everone would also work and probably wouldn't reduce access to figuring out who the relevant mods were by much.

The naive way of constructing this makes it then take 20 minutes computation to verify, which isn't that attractive if you imagine the verifier is some lame mobile client and the attacker is a mining farm.    Fortunately you can use a verifiable delay function.


I keep seeing people propose this an their proposals always turn out to be totally broken, but in fact there is a way to do this that I call tapering.

Step 1. A block is 3xVDFed to pick a random value.
Step 2. The next block is 2xVDFed to pick a single bit: if it's one, go back to step 1 for the next block.
Step 3. The next block is VDFed to pick a single bit: if it's one, go back to step 1 for the next block.
Step 4. The last step 1 outcome is accepted.

This will terminate in a couple steps on average.

You can make all kinds of variations on this, e.g. start the reset probability high and slope it down. But the general idea is that while single block may be biased, multiple blocks are at least somewhat mutually independent. Effort you spend grinding out a low block in step 1 is likely to get discarded by step 2/3 resets... but all can do is accept or reset the process, so those blocks doesn't have much influence on the outcome.

I'm sure effort that has gone into research on "randomness extractors" could be used to generalize this to achieve a guaranteed worst case bias achievable under useful assumptions- Essentially that work is trying to address this problem:  You have a function with multiple inputs, the attack has control over some not all of them, can observe at least some of the ones he can't control, and you don't know what inputs are attacker controlled/observed... what function allows you to extract the most uniform number, in spite of the attackers efforts, and how far from uniform can the attacker make it?

Unfortunately, almost all randomness extractor work is highly theoretical, non-constructive, and nearly incomprehensible to me... so I can't even tell if any of the existing approaches is applicable.

In any case, if large amounts of bias is needed by the attacker, I think these techniques can make achieving it much more costly.

Tapering works nice with VDFs, since you can attempt to make the time that the random value will be known long after the decision to use it or not needs to be made.

Another hardening approach you can use that I've described before is bonding:

Block X pays a bitcoin to pubkey A.
Block X+1 picks a random value.
Some expensive work function operates on the random value using the private key for pubkey A.
Block >X+10 spend A's coins off to someplace else.

Now: to rig the random value the miner and the private key holder for A must collude.  "A" can't influence the outcome at all (except say by just never releasing the private key), but the miner cannot grind without their help.  Why a bitcoin and privatekey instead of just a hash commitment in A's transaction?  So that if A tells the miner(s) the secret before X+2, the miners could just steal the coins instead of helping A's randomness.

Another hardening approach you can use that I've described before is multiparty verifiable oracles:

BLS signatures are completely deterministic the signature of message X is always S. BLS signatures can also easily support N of M threshold signing.  So you have published N of M BLS threshold public key...   to produce a final random value, you have a transaction (or OTS timestamp) commit to your BLS pubkey, then get a blockchain produced random value from a later block and sign it. The signature is your final random result.

Now creation of a biased random result requires a collusion of N parties plus the miner, jamming release of the random value requires >M-N parties to fail.

My understanding is that Friedcat's name was Jiang Xinyu.  https://news.8btc.com/tracking-friedcat-after-his-missing-in-2015  (I especially like how Ayn Rand is spelled in it...)

Jihan knew about my private communication with Asicminer.  Unless I misunderstood his explanation, he worked with asicminer before he started bitmain.

I thought this was public knowledge, but searches aren't turning up one thing... unfortunately "asicminer" is essentially ungooglable because it's a generic term.

The above link says about Xie Jian, "He invested more than 1000 bitcoin and bought 11,000 FC shares, which made him one of the board members of Bitfountain." Another article says about Jihan Wu: "Wu Jihan and his friend Crazy Xiaoqiang (web writer, first-generation bitcoin speculator) each bought 15,000 shares and 12,500 shares of the roast cat company's virtual stock.".  So this suggests that Jihan Wu would have been a board member?

Yang Zuoxing, the former bitmain engineer that they were suing for making a rival chip claims to have done design for asicminer. He might be the right person to ask if someone really wants the straight dope.

If Bitcoin as it exists were turing complete, than craig would be able to point to the transactions he claimed he had in the network implementing a turing complete cellular automata.  Yet no such transactions exist.

Craig's paper is mashing together terminology culled from the bitcoin-wizards irc logs and bitcoin-dev (e.g. total language), none of which means turing completeness.

If you can't actually defend the claim you shouldn't be repeating it, unless you want to get downrated for promoting a scam.  Not trying to be mean, but people confirming things they half understand and repeating it is a big part of how he perpetrates his scam.

Transactions are sent to all peers in erlay. The paper even specifically shows that the behaviour holds up under varying tx rates.

I have no idea. I'm not planning on working on it, Gleb or Pieter might have some idea.