The block for error correction  is the entire block, so it is entirely non-local. You normally hear about error coding using small blocks because when you are correcting arbitrary errors and not just erasures the computation for decoding becomes prohibitive for coding large amounts at once. But we only need to handle erasures, so that makes it tractable to code everything at once.

In Fibre the short-ids are somewhat different from compact blocks because they also include transaction lengths. These are needed so it knows how big the holes for the missing transactions are.

Yep. well m' = m: it needs to get as many fragments as are missing.   The block, broken into fragments is n fragments long. The rx is missing m fragments, m <= n (obviously), and can decode once it has received any m correction fragments sent by the sender.

Not quite-- that would require a round trip, which is fatal in terms of delay. Bam several hundred milliseconds out the door for the request to reach the sender and for the sender's reply to get back.

So instead the sender immediately sends encoded data when it gets a block, and the receiver tells it to stop when it has had enough. If the receiver was missing nothing, it'll just get some fragments it doesn't need which were sent during the time it took the sender to hear its stop request. It just throws them out (or relays the on to other peers that haven't said stop, then throws them out).

Otherwise your description seems okay.

TCP is a hell of a slow transport over long distance links.  As I pointed out, on international internet links there is typically 1%+ packet loss (akamai says more, but lets use 1% for discussion).  If TCP were used and even one packet were dropped then WHAM the connection is stalled for hundreds of milliseconds waiting for a retransmission, and further the transmission speed it cut down until the window regrows. Prior measurements before fibre and such showed that the effective block transmission speed between mining pools was about 750kbit/sec.

TCP's reliability only hurts and doesn't help-- the fibre recovery doesn't care which of the sent fragments it gets, it just needs enough of them... so pausing the whole stream just to retransmit a lost one doesn't make much sense.

Missed transactions and lost packets on long links happen at about similar rates-- 1%-10%. But regardless, if TCP were used every time there was a dropped packet the transmission would stall, so they couldn't be disregarded unless they were very rare.

The code we are using was designed specially to be very very fast on general purpose hardware. Almost all its decoding inner work is accomplished via SIMD xors. This was a major barrier in making fibre realistic after I originally proposed the general idea back in 2013-- a naive implementation took too long to decode. Above I say that the decode happens in in the speed of light delays plus 10ms or so, much of that 10ms is the FEC decode (obviously if nothing was missing, no FEC decode was needed). The FEC decode is also faster in the common case that relatively few fragments are missing.

The whole idea essentially!   Don't worry, you're in common company.

Fibre never sends the block data itself. Not a single blinking packet of it.

Many people get stuck thinking that it sends the block and then additional correction data to recover from packet losses, because they've heard some description of how FEC is used elsewhere but Fibre doesn't work like other things.  This misunderstanding gets in the way of actually understanding what it does.

The way fibre works is that it sends only correction data.  The receiver uses the correction data to correct holes in its knowledge. The holes come from transactions in the block that weren't known to the receiver-- not from packet loss. In technical language: we construct a perfectly efficient list reconciliation protocol out of a non-systematic erasure code.

Say a block packetizes out to 1300 packets.  Because of missing transactions in your mempool you only have 1297 packets worth of transaction data from this block. Fibre sends you a compact block and additional packets. As the additional packets come in you also stream them to your fibre peers, and as soon has you have received _three_ of those packets (1300 - 1297 = 3) you will reconstruct the block, you will tell your fibre peers to stop sending data for that block and you will start relaying the block to your non-fibre peers via BIP152.  At the same time, you'll start generating novel correction data on your own, and streaming it to the fibre peers that haven't told you that they had enough.

If two fibre peers get a block at the ~same time, they will both start streaming it at once.  You only need enough packets from all peers combined to recover your block: so in the above, packet one and two might have been generated by peer A, and three might have been originally generated by peer B.  

When a source generates packets it round-robbins them out to its peers, and those peers send them to each other as they come in. So imagine five fibre speaking nodes A, B, C, D, E.  A is originating the block.  B, C, D are missing three packets and E is missing four.  After A has sent three packets out its network interface  B, C, D will all recover the block because they shared the data with each other. E will recover once it gets a forth packet which either comes from A or could be generated by any of B/C/D because having recovered the block they can generate more novel packets on their own. This makes good use of the full bisection bandwidth of the network rather than being bottlenecked by the outbound bandwidth of the source(s).

Even if you knew nothing (your mempool was empty or the block contained no previously relayed transactions)--  you'll recover the block once 1300 packets (again, from any source) come in. It doesn't matter which packets come in: each contributes equally to your efforts to recover the missing data. You never need to make a roundtrip to recover missing data no matter what is missing or how much: this is critical because a link around the earth can have a RTT of hundreds of milliseconds.

In this protocol packet loss is pretty much irrelevant-- it's not that it "repairs it" so much as it never matters to Fibre in the first place.  If you need six packets, you don't care if you get packets 1, 2, 3, 4, 5, 6 or 2, 6, 4, 8, 10, 12 or 4, 8, 15, 16, 23 and 42... you just need six in that case, any six, from any source in any order because that's how much you were didn't know in advance before the block was even created. The only effect packet loss has is that if packet 4 is sent 0.01ms later than packet 3 and you needed  packets to recover and got 1 and 2 but packet 3 got lost along the way your recovery will end up happening 0.01ms later than it would have absent the loss when packet 4 comes in. -- unlike ordinary transmission where it would happen hundreds of milliseconds later after you've requested and received a retransmission. If you only needed two packets due to knowing more of the transactions in advance then you'd finish after 1, 2 came in.

If your link is momentarily interrupted and you miss the first 1400 packets sent, you'll recover from the next N when the link comes back up-- however much you were missing.  Fibre keeps sending until the peer says it has enough-- effectively it's a rateless code (well, technically the implementation defaults to a maximum of eight times the size of the block, just simply to avoid wasting endless amounts of bandwidth on a peer that has gone offline is will never say stop; on blocksat-- where there is no back-channel-- it's configured to send 2x the block size + it repeats data on a ~24 hour loop as excess capacity is available).

I've left out a bunch of fine details to capture the basic understanding (e.g. the correction doesn't really need to work on a whole packet basis, it's just easier to explain that way.)

UDP is used so that TCP's loss correction retransmissions, reordering, and rate control don't get in the way.

Our work is not related to making "mempools synced", though they are naturally similar.

Our work is exclusively related to eliminating the massive overheads from relaying transactions the first time through as they go around the network.
 
That doesn't happen in the Bitcoin protocol, no one has proposed for it to happen, and it isn't needed.

It's really a shame that people are forced to waste their time correcting you simply because you are so persistent and voluminous in your inaccuracies that you manage to confuse many people even though your posts are not very convincing.

Your statement here makes no sense.  Nodes prioritize transactions by feerate, none of that has been removed.  The only "individualizing" in practice is that a low memory host might reduce their mempool size. This is, in any case, totally unrelated to removing the relay inefficiencies.

Again, Bitcoin nodes don't interrogate nodes to list mempools nor pick up transactions again, no one has proposed they do, because there is no reason to do that.

Just pre-empting another confused tangent: There is a "mempool" p2p message which was added to the protocol by bitpay for the purpose of surveilling the network under a dishonest justification, which was later realized to be a privacy problem and the privacy leak was removed (and after that bitpay's staff recommended removing it from the protocol).  Bitcoin Core has no ability to send a mempool p2p request and never has had the ability to do so. It might be interesting to do so at initial startup to quick start the mempool and give miners something to mine after being offline for a while, but at the moment no one is working on that, AFAIK.


The problem we are addressing is that if you have 100 peers, each of your hundred peers will advertise (or have advertised to them) each of those 8 transactions, using 100x the bandwidth on those advertisements as if you had only one peer.

The need for nodes to potentially drop transactions has nothing to do with free market behaviour and everything to do with nodes not having infinite storage to keep the transactions.  But there is, again, no interrogation-- they don't need to go refetch them again.


You've misunderstood what we've accomplished here.

If at some point during the initial relay of transactions,  you receive from your other peers TX  A, B, C, D, E, F  and I get TX B, C, D, E, F In the historical Bitcoin protocol each of those 6 values would be sent across the link between us (potentially twice).

Instead, you could send me the single value X = A xor B xor C xor D xor E xor F,  or I could send you the single value Y = B xor C xor D xor E xor F.  

After the single value is exchanged whomever received it computes  X xor Y = A -- the missing transaction, even though neither of us knew in advance which transaction was missing.

Minisketch generalizes this to support any number of differences.  The data sent is exactly equal to the number of different values, regardless of how big the original sets are. (In fact, the first value in a minisketch is exactly what I described above: the xor of all the elements in your set).

So, if you have received in relay A, B, C, D ... X  and I have  already received B, C, D ... X, Y, Z;   Then I need send you only three values (or you me): The xor of all my values, the xor of all my values cubed, and the xor of all my values to the fifth power... and then you will know that I am missing A from you, and you are missing Y, Z from me.  And by doing this we send only three values on the link between us in the initial relay instead of 26 - 52 (depending on how much duplication there is from concurrent sends).

There has never been and can never been a "consensus priority formula", because priority by its very definition is external to consensus.  But the behaviour of existing nodes is consistent-- they keep and drop the same transactions, subject to having them in the first place, and subject to the restriction that anything configured to use less memory obviously can't keep as much.

There is no re-interrogation, no-rerelay in Bitcoin, nor is any proposed.  It exists only in the imaginary protocol that you spend your days attacking and confusing people with.  The inefficiency in Bitcoin that we're working to resolve exists in the initial relay itself, and would still exists even if nodes had no mempools at all.

On a good day transcontinental internet links lose more than 1% of packets. The way congestion control in TCP works essentially guarantees loss on links unless they are massively over-provisioned/under utilized-- packet loss is the mechenism the network uses to tell senders that the link is full-- especially since ECN is not ubiquitous. Periods of 3% loss are not uncommon on international links, even worse is also common in and out of China due to firewalling and weird national telecom policies.

To give some concrete examples: Akamai claims 1.94% packet loss from Atlanta to Amsterdam, 4.5% packet loss from Seattle to Sydney, 4.12% LA to Tokyo. (Akamai's numbers seem a bit high to me, but only a bit)

Without a roundtrip-less protocol, packet loss during block transmission requires a an additional delay of at least one round-trip (but with TCP the effect is even worse). Wireless wasn't a consideration in the design of Fibre.  Without handling this you cannot get the 99th percentile worldwide block transmission times under 450ms regardless of your link bandwidths. (or 95th percentile, per Akamai's loss figures).

The FEC in Fibre isn't used exclusively, or even, primarily to deal with packet loss-- in any case.  It's used to compensate for missing transactions without the sender knowing which transactions the receiver is missing.  Without doing this you cannot get the ~90th percentile world-wide block transmission times under 450ms and you cannot avoid massive propagation penalties when unexpected transactions are included.

The only reason that use of fibre wouldn't be a several times speedup in the 99th percentile worldwide block propagation is simply because it's already in use, and we gain most of the benefits from it even if only a few nodes in each geographic region use it since they'll blast through the lossy paths and nodes in their regions will simply get the blocks from the places to that have it.

This is important for mitigating harm from mining centralization because a centralized miner can achieve equally fast transmission for their own blocks using simpler protocols because they know what transactions they'll include in advance and can extend their own next block without waiting for propagation.

I know otherwise, since we are working to replace transaction rumouring with a reconciliation protocol, which will cut tx rumouring bandwidth usage on the order of 40x. After that's complete we'll probably work on integrating fibre for the above reasons.

We're also still actively merging improvements that have first been deployed in the fibre implementation, e.g. supporting multiple outstanding concurrent getblocktxn decodes for compact blocks.

It already is, Matt isn't the only user of it. Several mining pools also use it on their own, and Blockstream's satellite uses it (not for high speed, but because it also needed a roundtripless protocol for relaying blocks over a lossy channel).

Sure there is.

Good thing it's not a webservice then.

No it doesn't. It's simply a protocol for relaying blocks that never requires a round trip.

You absolutely do not.  In fact, the text "The FIBRE codebase running on each is optimized to ensure that Compact Blocks for each new block are made available to peers after only SPV validation of new blocks, without blocking for lock acquisition" also describes BIP152 in Bitcoin Core right now.   BIP152 was designed to permit blocks to be relayed after only SPV validation.  This wasn't implemented in the very first release with BIP152 because it was optional required a lot of additional work, but was implemented long ago, and similarly-- the ability to relay a compact block without acquiring cs_main was first done in Fibre but has also been in bitcoin core for more than a year.

So if it's already a standard part of BIP152 deployment, why is it mentioned on that page?  Because the page pre-dates these updates.

No trust is required for full nodes to use this sort of optimization for the reasons explained in BIP152.

So again, what you are saying can't be done not only can be done but has been done for a long time already.


As pointed out above, BIP152 also relays blocks without validating them. To what BIP152 HB modes does-- fibre adds: Unconditionally never using any round trips, not in the protocol, not at the network layer-- even when the block contains transactions that the network has never seen before. It also results in packet loss not causing  any delay (other than the time until the next packet).

Yes, you are confusing the fibre protocol with Matt's public relay network.


You don't get to decide what Fibre is, the people who made it do. Fibre is a protocol for relaying block without ever making even a single round trip, even in the presence of packet loss, which allows the receiver to recover the block as soon as they received only as much data as they were missing.  This is how Fibre has always been described.  Matt's public relay network is a service run by Matt which pre-dated fibre by many years and helped drive its development. We specifically introduced the name Fibre because people were confusing the earlier relay protocol used by Matt's public relay network with the relay network itself because the protocol didn't have a name (for a long time it was just called relay network protocol, though later I started calling it fast-block-relay-protocol). When we created Fibre we decided to fix that problem by making sure that the free standing nature of the protocol was more clear. It seemed to work more or less until recently, when an ICO and its perpetrators started promoting otherwise.

Don't fall for the re-contextualization of an ICO scam that wants to fraudulently call the innovative free and open protocol we invented for relaying blocks as fast as possible a "service" so that they can sell unsophisticated investors on investing in their competing service.

And as far as "no comments to make about its feasibility"-- you just wrote "but there is no way to go further than bip 152".  So, you do have comments-- but you're simply incorrect about it.

Oh man, _please_ go re-read my first message to you in this thread.  You are confusing matt's public systems with the protocol-- linux with amazon.

Fibre is a protocol for distributing blocks very fast.  The protocol is open source and distributed, and I helpfully linked to it above.

Having a great protocol alone isn't enough to get block propagation as fast as a large well run centralized miner could achieve for themselves because a large centralized miner could carefully operate a well run DOS protected network of their own nodes to forward their blocks.  It would be bad if only the largest centralized miners had access to those facilities.  As a result, Matt runs a public network of nodes that provides a collection of geographically distributed, well maintained, dos protected nodes for anyone to connect their ordinary stock bitcoin nodes to who wants to, so that the the advantages of that kind of well maintained infrastructure are available to more parties.

Read the heading of the very page you are quoting.  "Public highly optimized fibre network: As there is significant benefit in having at least one publicly-available high-speed relay network, I maintain one."   Fibre is analogous QUIC-- a free licensed protocol with higher performance than the earlier alternatives-- and Matt's public relay network is analogous to Google's webservers-- a privately operated infrastructure using a high speed protocol and making its services available to the public.  

The experience of running this infrastructure is a lot of what made it possible to design fibre (and BIP152) in the first place, since it created a network in a bottle that could be well measured and characterized, exposing the limits of the legacy protocol and made it easy to deploy experimental adjustments in a rapid fire basis without risking disrupting the rest of the network.

Do you see how you are intermixing two different things?

What you're saying about complementary/supplementary services and their value as an addition applies perfectly to Matt's public relay network (and I agree!).  It just doesn't apply to the protocol-- which is what I was pointing to above, not matt's public relay network.


This is amusing because we already completely replaced how blocks are propagated in Bitcoin with technology from FIBRE (optimized for minimizing bandwidth somewhat more than minimizing latency, which FIBRE optimizes) over two years ago now.

I've heard the old adage that people saying “It can’t be done,” are always being interrupted by somebody doing it. Being corrected by people who have been doing it for years is a bit of a twist...

It would be perfectly possible to use FIBRE for block relay everywhere and abandon BIP152 just like we abandoned the original block relaying mechanism. But the vast majority of nodes presumably don't want to use a couple times the bandwidth in exchange for shaving off the last few milliseconds, and so the benefit of doing hasn't yet been worth the effort to bring it up to that level of maturity (in particular, one has to be pretty sure there are no dangerous bugs for software that will run on all the nodes). This is especially true because most of the latency benefit of FIBRE can be achieved with just a small number of nodes running it, since they do the heavy lifting of carrying blocks over the 100+ms RTT transcontinental links while BIP152 has pretty similar performance to Fibre where the link RTTs are 10ms or less. Block propagation with BIP152 flows along the lowest delay paths, so when some nodes running Fibre manage to wormhole blocks around the globe faster than BIP152 does all the other nodes happily exploit that fact.

Presumably, Fibre protocol will eventually make it into default node software once doing so becomes the lowest hanging fruit-- likely with more knobs and/or automation to set the bandwidth vs latency trade-off more intelligently.   In BIP152 there are already two different trade-offs:  Normal and High Bandwidth mode. HB mode essentially eliminates one round trip time for relay, at the expense of wasting perhaps 13KB of bandwidth per block, nodes figure out which three peers would be most useful to have in HB mode and users it with those... so as to usually achieve the latency advantages of HB mode without much bandwidth cost.  The addition of Fibre into this mix would probably act as a third higher bandwidth lower latency trade-off that nodes would use if either manually configured or if they detected they had over some threshold of available bandwidth, so we could get the X percent fastest nodes on the network using it, thereby capturing most of the latency advantage without most of the bandwidth overhead. Until then, anyone that wants to run it is still free to do so, ... and without ever interacting with Matt's servers, because, again-- matt's servers are just a user of a protocol that is open and useful to anyone.

Because it very clearly is a scam. Just because you've managed to bamboozle yourself doesn't make it right.  Selling some ICO token as a pretext to collect money from unsophisticated investors for some business that makes little sense is flat out unethical, makes everyone in the cryptocurrency space look bad, and the stink of it scares off competent technical contributors because they don't want to get anywhere near the scammyness of it.

Your post really does have nothing to do with the OP's question-- it seems like you didn't even bother to read it, and only read the subject line.  The OP was asking if gossiped messages would reach all nodes or, in other words, "is the communication lossless?". Achow101 responded pretty much completely-- that it can't be guaranteed to reach all (no physically realizable network could make such a guarantee, since nodes can become disconnected). Though in practice almost all txn reach almost all nodes quickly (subject to the intentional delays added to help obscure origins).

That is a fair enough request, but at the same time anyone in this space is utterly flooded with garbage. It's literally impossible for any person to explain why all of it is junk, worse-- in some cases the perpetrators are actively malicious and aggressively retaliate (for ones that are pure marketing they have literally nothing better to do than spend time and money attacking their critics) so people simply don't.

Why is is what you're working on a scam?  Lets start with the fact that it's soliciting investments from the general public (likely in violation of US securities law) for an enterprise which expects to make money ultimately from resources provided by third parties. ... to accomplish a task which is already accomplished by existing distributed software.  Even if the result manages to do something better compared to existing systems, at the end there would be no reason for users to not take the software and rip out the monetization leaving the investing public holding nothing (and using non-free licensing wouldn't help, since few are foolish enough to run closed systems with their cryptocurrency applications)... and I find it personally pretty doubtful that BloxRoute or Marlin will manage to construct something that outperforms the existing free state of the art, considering that both of the whitepapers massively misrepresent the state of the art.

Right now the current state of the art in Bitcoin block propagation is that >99.99% of blocks are transmitted to locations everywhere in the world within the one-way network delay plus less than 10ms while running on commodity hardware and networks.  The one-way network delay is the physical limit, which can only be improved on by putting in networks with more efficient physical paths (which someone can do, e.g. what HFT companies do... but isn't what you're doing [1]). 10ms could be improved on (esp with asic decoders, or creative protocol/software optimization) but it's not clear that going from 10ms to (say) 1ms would actually matter much-- perhaps worth doing, but it isn't something that is going to recoup the investors investment.

Without a clear statement on: How is it going to outperform light+10ms  and by how much  [2], why Bitcoin participants would be willing to _pay_ to gain access to that 0-10ms improvement, and why they wouldn't just cut out the 'middleman' token to do it?-- then the whole thing just looks like something designed to bilk unsophisticated investors which don't know the most basic questions to ask or have been deceived by misrepresentations about the performance of the existing state of the art.

Scamyness bonus round:  Text from the Marlin Protocol whitepaper was clearly plagiarized directly from the bloxroute whitepaper. As an example "The Falcon Network was deployed (by two members of our bloXroute Labs team)" (em mine, but this is not the only material copied outright-- e.g. it wholesale copies the incorrect description of what fibre is).  So not only are you working on a scammy ICO, it's an unoriginal scammy ICO that is ripping off another scammy ICO.

[1] BloXroute claims to be doing that with AWS but anyone can run the existing free and open state of the art software for block relay on AWS and gain whatever advantages locating relay on AWS has... also,  I have measured relay on AWS and it's actually not impressive compared to other hosting options.

[2] To be clear: we already know how to improve on the existing state of the art-- use a more efficient sketch encoding, use a more efficient serialization of encoded transactions, use a faster to decode FEC, optimize the software (and in particular use more precomputation), improve packet scheduling, support using nics with integrated FPGAs for faster decode and packet relay, and get more users using state of the art protocols-- , but even if the improvements achieve the maximum possible gain, it doesn't seem like it'll matter that much, so currently the efforts of open source developers are being spent on other areas.  But since both Marlin and bloxroute misrepresent the state of the art-- and fail to propose any performance advancement beyond it that I'm aware of--, I am doubtful that either group understands it well enough to even match it, much less exceed it.

Sure we do.  Simply just saying that we don't without any reasoning or justification (except perhaps avoiding admitting that you were mistaken before) doesn't make it so.


I'm not sure what you mean by label. Do you mean a link?

Well thats good, because anyone can run anything they want and they don't need your or anyone elses approval and in fact there is nothing you can to to stop them.

No it isn't not at all.

The reason Matt runs a network of well maintained nodes for people to connect to is because doing so is beneficial (because well maintained, well positioned, well networked hosts have better latency than random stuff) and because if someone doesn't do it and allow the public to use it large miners will be the only parties to benefit from that kind of service (because they do it for themselves) and enjoy an advantage against others.

Fibre is just a protocol that renders bitcoin block transmission free of round trips and exploits pre-knoweldge of transaction data, which allows blocks to be transferred in very close to the one way delay 99.9999% of the time.  There is nothing "service", "centeralized", or "data-center" about it. Please discontinue spreading that misinformation.

BloxRoute appears to be a straight up ICO scam as far as I can tell-- It doesn't appear to propose anything new or interesting that would be and advance over what is in use in the network today, but uses a lot of hype and jargon to try to sound new and exciting.

Then again googling around suggests the thing you claim to be working on is a competing ICO scam. ... and presumably you only posted here to hype it-- especially since your post is offtopic from the thread.

Fibre is a protocol, not a service (and certainly not a centralized service). It's like you're calling Linux a centralized service because amazon runs it.

Bitcoin nodes already tell their peers the minimum feerate they'll accept, so their peers don't won't offer anything that they won't accept. Doesn't even need to go into the IDs.   See BIP-133.  It would be possible to put the feerate in the IDs for even more fine grained/realtime filtering but I believe that would be a net waste of bandwidth due to the extra space for that information and the relatively few extra transactions that get sent in between feefilter updates.


We don't plan on changing how the mempool works-- rather, for each peer a sketch would be kept with all the transactions that your node would like to send / expect to receive with that peer. It takes a bit of memory but it's pretty negligible compared to other data kept per peer (such as the filters used to avoid redundant tx relay today).

For computation we've been designing assuming needing to accommodate a rpi3 ... and part of the purpose of building minisketch as a fairly well optimized library was understanding exactly what the performance tradeoffs would be. 

One nice thing is that all the CPU heavy lifting is done by the party that requests reconciliation, so if you are CPU starved you can just reconcile less frequently. Doing so will also further reduce your bandwidth but just has a downside of propagating transactions somewhat slower.
 

Releases like 0.17.0.x are just packaging respins, so they don't get announcements.  Basically nothing was changed in it but something about the OSX installer.

However, 0.17.1 is done now-- and tagged in git for over a day-- holiday travel is just delaying getting the binaries up, which is why it hasn't been announced yet. They should be up by christmas however!

And I'm really sick of your insulting lectures when you can't even bother to keep up on the state of the art in what's already been proposed. Please spare me the excuses about how its other people's fault that you're not doing other things. You seem to have plenty of time to throw mud...

There are already existing designs for an assumevalid 'pruned sync'...  but not enough hours in a day to implement everything immediately, and there are many components that have needed their own research (e.g. rolling hashes like the ecmh, erasure codes to avoid having snapshots multiplying storage requirements, etc.).

If you want to help--great! But no one needs the drama.  It's hard enough balancing all the trade-offs, considerations, and boring engineering without having to worry about someone being bent that other people aren't going out of their way to make them feel important. It's hard even to just understanding all the considerations that other engineers express: So on one has time for people who come in like a bull in a china shop and don't put in that effort towards other people's work. It doesn't matter who you are, no one can guarantee that any engineering effort will work out or that its results will be used even if it does.  The history of Bitcoin (as is the case in all other engineering fields) is littered with ideas that never (yet) made it to widespread use-- vastly more of them from the very people you think aren't listening to you than from you. That's just how engineering goes in the real world. Don't take it personally.

Initial sync is already an enormous problem, and even if you assume computers/bandwidth improve at 18% year over year (which is a big assumption...) any blocksize over ~300kbytes means that the initial sync time will continue to get worse.

It's unclear, we know that the numbers are somewhat distorted by spynodes which commonly claim to be old versions, but we don't know by how much. I know that on nodes where I've aggressively banned spynodes I see a much newer node mix than luke does.

Regardless, in the future nodes could eventually decide to stop relaying unconfirmed transactions to old nodes... it's pretty backwards compatible to do so. But thats getting way ahead of things...

I'm wondering if things went like this? "Problem ECDSA signing isn't derandomized so I know...! I'll invent some novel crypto. Nothing could go wrong."

I'm guessing your central idea isn't to have used completely insecure crypto, thus permitting first third party in their lightcone to steal the funds?

If so you might want to revise your proposal, perhaps minimizing the number of novel cryptographic structures you attempt to construct.

Aside, the response to no pool mining, isn't "I guess I won't pool mine", it's "I guess well fund bob to build us a big centrally controlled mining farm to mine for us and share the income".

No there isn't. The number is more like 60k or so.

It appears to be lower than it was in 2011.


This is nonsense.


All this would do is fund people to pretend to run zillions of nodes on as much address space as they can obtain.

I would be strongly opposed to adding support to secp256r1-ecdsa to Bitcoin, particularly for this rather shallow application.  r1 is slow to work with, now officially recommended against by the NSA, normal ECDSA cannot be batch verified and cannot be easily used as a threshold or adaptor signature.

There are many hardware wallets out there already, and U2F devices do not make for a good hardware wallet because they lack a display so that users can have any idea what they're signing (so they provide limited protection against a hacked computer).

If there is need for a U2F like device that works with bitcoin they could as easily be produced as ones that don't (including dual mode devices) but there just doesn't appear to be enough demand for that... accordingly, there isn't enough demand to add inferior cryptography to Bitcoin.

Achow's post seems to sound like it's saying that there is some kind of danger in random K because of 'getting more information out' -- there isn't in and of itself,  to the extent that different K's "get more information out" so does signing multiple messages.

The reason for 6979 is that reliably generating random numbers is hard and it turns out that over and over and over again applications screw it up.  Worse, you can't easily tell when a random number is screwed up because it's random, one looks as good as any other. ... unless the screwup is so bad that they're just constantly repeating.  But basically any form of predictability of K will break the signature, even ones like being linearly related to the Ks in other signatures.

You still need to generate a secure random number to get your private keys, but you only need to be successful at that _once_.  RFC6979 is really just a way to safely reuse that ONE random number you successfully got (the private key) for all your further transactions... rather than needing a constant influx of new random values, each one a potential point where a software error could cause the loss of your keys.

The fact that the procedure also gives the same signature every time for the same key/message if you don't use the optional extra-data input to 6979 is a bonus that makes software testing a lot easier,  but it is not the source of the advantage of this approach itself.

This distinction is important because multiparty schnorr (or mpc-ecdsa) signing can use RFC6979 but MUST be constructed in a way where the same signature is not repeated even for the same message.

This site appears to connect back to the server every time a new key is entered...


People should NEVER use any key management webpage or webapp.

I posted the standard recovery procedure for the kind of corruption described here.  It seems to be being ignored.

I would take a substantial bet that the OP here is either scamming or is going to get scammed.

As an aside, it is not safe to use potentially malicious wallet.dat files.  Anyone who gets sent a wallet.dat from a third party should take great care in using it. I would not be shocked if it were possible to get arbitrary code execution from a wallet.dat file.  If a bad guy found a way to do that the best way to exploit that discovery would be to pose as someone who corrupted their wallet and encourages people to try to 'scam' them by getting a copy of their wallet or help them with a promise of an outsized reward.