How faster transactions can be implemented

[ziliboba487]

The "difficulty" manages to have one block built not faster than in 10 minutes in average. It helps to avoid collisions. If we decrease the difficulty to have 1 block in 10 seconds instead of current 10 minutes, we would have way more collisions, and the blocks we mine will have higher chance to be discarded.

What can be a solution for a decentralized pool of nodes around the world? May be already invented but I am not aware?

[BlackHatCoiner]

What do you mean a decentralized pool of nodes?

A pool where everyone has the same rights? Where there's no owner? There isn't such thing, but even if you didn't know it, what would be the obvious answer? Why would the miners work on “centralized” pools while there were “decentralized”.

The sooner you understand the purpose of working for a pool, the sooner you'll comprehend the need for this kind of pools we already have. Essentially, the decentralized pool is the network and these pools derive computational power from miners to compete who's going to solve the next block. Then, they reward miners accordingly.

[n0nce]

It's unclear what you actually try to ask; so I will just answer the title of your topic, which is probably your main question - accelerating transactions.
The best way to do so is not to shorten the block time, because scaling in general can't be feasibly done on a large-scale level purely on L1. It's best to keep L1 simple, and easy to run, and use that to secure sidechains and off-chain mechanisms such as Lightning Network.

Finally, I don't get how people always complain about Bitcoin's speed. When I send bank transfers, it often takes hours or until the next day for the money to arrive at the receiver, even though we're in the same country. International takes even longer. Meanwhile, easy pure good old Bitcoin L1 settles your transaction permanently in on average 10 minutes - that should be quick enough for most things.

Again, for micro transactions, 'streaming money' and the like, feel free to try LN, it works pretty well nowadays. And no blockchain can be this fast, even 10s block time would be pathetic compared to off-chain solutions that are virtually as fast as your network allows to send data.

[DannyHamilton]

The "difficulty" manages to have one block built not faster than in 10 minutes

Many blocks are faster than 10 minutes. As a matter of fact, more than half of the blocks solved are solved in less than 10 minutes.

in average.

Correct, the difficulty is adjusted every 2,016 blocks to bring the AVERAGE time between blocks as close to 10 minutes as possible. This is the speed of blocks, not the speed of transactions.  Transactions have many different speeds, depending on what you are trying to accomplish. There is the time to create a transaction, the time to broadcast a transaction, the time for a transaction to be relayed throughout the majority of nodes, the time for the first confirmation, the time until the recipient is willing to release whatever is being exchanged, and the time until most people feel confident that the block containing the transaction won't be replaced.

What can be a solution for a decentralized pool of nodes around the world?

Nodes are already decentralized. That has no effect on the time for blocks to be solved and has minimal effect on the time for transactions to be relayed. Since nodes are already decentralized without a pool, why do you want a pool?  Who would participate in the pool, and why? What benefit would come from a pool of nodes in place of individual nodes?

Perhaps you mean pool of miners instead of pool of nodes?  Those already exist. This allows those supplying hashpower to work together on solving a block, and then share the rewards from the block. That way their is less volatility in frequency and value of earnings from hashing. This has no effect on the frequency of blocks nor the speed of transactions globally.

May be already invented but I am not aware?

Maybe.  Depeends on what you are trying to accomplish, and why you think a pool is the appropriate solution.

[o_e_l_e_o]

Shortening the block time will never be a solution for faster transactions.

The most common scenario to need a fast confirmation is a physical point of sale transaction - where I am in a store in person, paying for goods at a cashier. These sales need to take place within seconds. Even a minute is too long to wait for a busy cashier with a queue of people waiting to pay. To achieve this you would need to drop the block time to well under a minute, which would cause so many stale blocks (as you point out), that you would end up having to wait for more than a minute anyway to be sure the chain of blocks you are viewing isn't about to be discarded for a longer chain.

The solutions to this are for merchants to accept small value zero confirmation transactions, or to use a second layer solution such as Lightning.

Finally, I don't get how people always complain about Bitcoin's speed. When I send bank transfers, it often takes hours or until the next day for the money to arrive at the receiver, even though we're in the same country. International takes even longer. Meanwhile, easy pure good old Bitcoin L1 settles your transaction permanently in on average 10 minutes - that should be quick enough for most things.

Because people don't understand the fiat system, and so compare a transaction being broadcast with fiat to a transaction confirming with bitcoin.

When I swipe my credit card in a store, that is pretty much the same as broadcasting a still unconfirmed bitcoin transaction. I see the transaction on my wallet or app, the merchant can see the incoming transaction, but no money has actually moved. For some reason, people think that the credit card transaction is now complete, and the bitcoin one isn't, whereas in reality the fiat does not actually reach the merchant for 3-5 working days and can be reversed for anywhere between 90-180 days, while the bitcoin reaches the merchant and cannot be reversed after ~10 minutes. They compare broadcasting a credit card transaction to a bitcoin transaction becoming irreversible, which is obviously an unfair comparison, then lambast bitcoin for being so slow, while in reality the bitcoin transaction is orders of magnitude faster to finalize than the credit card transaction is.

[larry_vw_1955]

When I swipe my credit card in a store, that is pretty much the same as broadcasting a still unconfirmed bitcoin transaction. I see the transaction on my wallet or app, the merchant can see the incoming transaction, but no money has actually moved. For some reason, people think that the credit card transaction is now complete, and the bitcoin one isn't, whereas in reality the fiat does not actually reach the merchant for 3-5 working days and can be reversed for anywhere between 90-180 days, while the bitcoin reaches the merchant and cannot be reversed after ~10 minutes. They compare broadcasting a credit card transaction to a bitcoin transaction becoming irreversible, which is obviously an unfair comparison, then lambast bitcoin for being so slow, while in reality the bitcoin transaction is orders of magnitude faster to finalize than the credit card transaction is.

Credit cards take time to settle but debit cards are instant. they take the money out of the account immediately or they decline it immediately.

[Quickseller]

When I swipe my credit card in a store, that is pretty much the same as broadcasting a still unconfirmed bitcoin transaction. I see the transaction on my wallet or app, the merchant can see the incoming transaction, but no money has actually moved. For some reason, people think that the credit card transaction is now complete, and the bitcoin one isn't, whereas in reality the fiat does not actually reach the merchant for 3-5 working days and can be reversed for anywhere between 90-180 days, while the bitcoin reaches the merchant and cannot be reversed after ~10 minutes. They compare broadcasting a credit card transaction to a bitcoin transaction becoming irreversible, which is obviously an unfair comparison, then lambast bitcoin for being so slow, while in reality the bitcoin transaction is orders of magnitude faster to finalize than the credit card transaction is.

Credit cards take time to settle but debit cards are instant. they take the money out of the account immediately or they decline it immediately.

The issuing bank (via the debit card network) can reverse a debit card transaction under certain circumstances. The money being immediately debited from your available balance allows the bank to know the transaction will not be declined due to insufficient funds.

The biggest difference between using a credit (or debit) card and accepting a zero confirmation bitcoin transaction is that the merchant will largely know the identity of the person using a credit card (unless the physical card is stolen), while a bitcoin transaction is, for all intents and purposes anonymous in many cases.

Most merchants do not need the money in their bank account immidiately, they only need a commitment from someone that is reliable (such as a card network) that payment will be received in due time.

I think the most likely long term solution to in-person payments being completed quickly on any kind of scale will be Lightning, or some other similar L2 protocol.   

[d5000]

I fully agree with what among others o_e_l_e_o, DannyHamilton and nonce0 have written. Layer 2 is the way to go for shorter transaction confirmation times.

But @OP, I suspect* that your reasoning goes the following way, which would explain the slightly confusing OP:

"Collisions" between competing blocks, as you call it, could be decreased if all miners were connected better between them - i.e. in some kind of decentralized pool which ends finding all the blocks.

There was a protocol called P2Pool (I think it's this link, but be careful), which was decentrally organized as a P2P network, and - if I remembered well - used an own blockchain or ledger to distribute the rewards to the nodes. I don't know why but at least in recent years it never had big mining shares.

Maybe centralized pools are faster (the website lists also higher CPU/RAM requirements as disadvantages). But having a centralized pool finding all the blocks (or more than 40-50%) would not be desirable, because the pool operator could attack the network then. It is possible that if in the case that if P2Pool was the dominant mining pool attacks were also possible, I don't know the protocol in detail.

You could get similar effects with a faster network which connects only miners, where they can listen to other pools submitting their blocks, so they don't submit a block slightly afterwards which would be orphaned. Such a network does, or at least did, exist between some pools; but I don't remember its name.

There is also Ethereum's "uncles" concept, where some orphans get rewards, but this only solves the problem for miners, the efficiency of the network is (as far as I understand) not affected and lower as in "slower" chains like BTC.

*in the case this is a serious technical question - briefly looked in the user's contributions and they don't give me too much hope

[o_e_l_e_o]

Credit cards take time to settle but debit cards are instant. they take the money out of the account immediately or they decline it immediately.

They might debit your account immediately, but the merchant does not receive the money immediately and still has to wait a few business days. And as has been said above, these transactions are still reversible for weeks or even months after they are made. The same is true for things like PayPal, Google Pay, Apple Pay, and so on.

The only fiat method which is faster and more final than bitcoin is physical cash.

[odolvlobo]

Shortening the block time will never be a solution for faster transactions.

The most common scenario to need a fast confirmation is a physical point of sale transaction - where I am in a store in person, paying for goods at a cashier. These sales need to take place within seconds. Even a minute is too long to wait for a busy cashier with a queue of people waiting to pay.

I agree that even a 1 minute block time is not short enough in your particular scenario, but certainly a shorter block time is better in general, even if it isn't a "solution".

I don't know what the optimal block time is when considering downside of stale blocks, but I bet it is much less than ten minutes. And with improved communications in the future, the optimal time could become very short -- less than a minute. Maybe we could shorten the block time now to 1 minute in order to prepare for the future.

[d5000]

I don't know what the optimal block time is when considering downside of stale blocks, but I bet it is much less than ten minutes. And with improved communications in the future, the optimal time could become very short -- less than a minute. Maybe we could shorten the block time now to 1 minute in order to prepare for the future.

I've found this formula at reddit (in a thread where folks compare LTC, BTC and Doge stale block (orphan) rates):

orphan rate ~= t/T

where T is the mean block time and t is the average block propagation time to the network's hash power.

According to that formula, if the block propagation time is 6 seconds (like the Reddit user assumes), BTC has a "theoretic" orphan rate of 1%, LTC, with a 2.5 minute block interval (thus 4 times shorter than BTC), of 4% (the thread wrongly states 2,5%, but LTC has 150 seconds, so it's 6/150 which is 0.04).

Some people in the thread however complain about LTC orphan rates of around 10%, but they may have bad network connections. And I've seen "real"-data based estimations for BTC of 0,5%. Blockchain.com used to have a stale block chart but it only covers years between 2014 and 2017. It seemed to show rates of 0,1 to 2 blocks per day, with a decreasing trend since 2015.

The difference between the "lower-than-expected" Bitcoin stale block rates and the "higher-than-expected" LTC rates may have to do with the greater professionalism in BTC mining (also take into account the Reddit thread is from 2016 or so), which led to the specific network protocol which connects mining pools. I now remembered its name, it's called Fibre Network.

Now the question is: what is the "optimal" stale block rate? Is an increase from 1% to 4% still acceptable? An 1-minute-blockchain would have 10%. Ethereum, like I wrote above, with its 15 sec block interval has 40% of theoretic orphan rate and thus already needs a compensation mechanism (uncles) to avoid an excessive mining centralization, but one must say that a part of their hashpower is basically wasted.

Here is another thread about stale blocks, with input from gmaxwell.

[larry_vw_1955]

They might debit your account immediately, but the merchant does not receive the money immediately and still has to wait a few business days. And as has been said above, these transactions are still reversible for weeks or even months after they are made. The same is true for things like PayPal, Google Pay, Apple Pay, and so on.

The only fiat method which is faster and more final than bitcoin is physical cash.

Well, i get what you're saying but certainly there's other cryptocurrencies that have faster confirmation times than bitcoin and they are just as final as bitcoin. But in general I think I agree. physical cash would include things like western union right?

[Quickseller]

A business receiving cash cannot effectively spend it instantly because most of their bills are those that require payments being sent to locations far away. Also, payment due dates are not immediate, so this is generally an issue. Most businesses will have some kind of armored car service pick up cash deposits, which often will not happen daily and takes time for the bank to process.

The underlying issue is not when a business actually receives a payment, the issue is when a business knows with x amount of certainty that they will receive a payment. With bitcoin, this can be as quickly as a few seconds, even with a 10 minute block time, as long as certain criteria is met with the transaction (RBF being set to False, no unconfirmed inputs, sufficiently high tx fee, etc). It is difficult for a business to ensue a transaction has a high probability of quick confirmation because the sender has so much control over the transaction.

The above would again make me want to circle back (please don’t discredit me for using this term), to L2 solutions. LN will ensure that a business will receive a payment nearly immediately, and if the transaction fails, both the buyer and seller will know immediately. 

[pooya87]

Some people in the thread however complain about LTC orphan rates of around 10%, but they may have bad network connections.

Orphan rate (or I believe you mean stale block rate) is a universal thing that depends on miners not something that individual nodes would experience based on their connectivity. If a shorter chain is created then all nodes would technically find out about it, a node with bad connection finds it later.

[d5000]

Some people in the thread however complain about LTC orphan rates of around 10%, but they may have bad network connections.

Orphan rate (or I believe you mean stale block rate) is a universal thing that depends on miners not something that individual nodes would experience based on their connectivity. If a shorter chain is created then all nodes would technically find out about it, a node with bad connection finds it later.

My post was probably not clear enough. These people complaining about a high orphan (=stale block) rate were miners talking about their own stale block rate (=number of blocks they found, but were discarded by the "longest chain", versus the total number of blocks they found).

I for myself can only imagine that their connections weren't ideal. Maybe they also had simply bad luck and their "sample" was too small (if they were low-hashrate miners and do not find that many blocks at all). Maybe also the 6 seconds the reddit user mentioned was too optimistic in the LTC network (where afaik there are still many smaller miners and I don't know if they use Fibre or a similar protocol).

[pooya87]

My post was probably not clear enough. These people complaining about a high orphan (=stale block) rate were miners talking about their own stale block rate (=number of blocks they found, but were discarded by the "longest chain", versus the total number of blocks they found).

I for myself can only imagine that their connections weren't ideal. Maybe they also had simply bad luck and their "sample" was too small (if they were low-hashrate miners and do not find that many blocks at all). Maybe also the 6 seconds the reddit user mentioned was too optimistic in the LTC network (where afaik there are still many smaller miners and I don't know if they use Fibre or a similar protocol).

That makes more sense but only if these litecoin miners were actually solo miners running their own node instead of connecting to a mining pool (similar to bitcoin miners). I don't really follow LTC scene to know how things were back in 2016 but since LTC follows bitcoin in most things, pools might have been what everyone connected to, in which case the rate depends on the pool's connection quality not the miners.

[n0nce]

I agree that even a 1 minute block time is not short enough in your particular scenario, but certainly a shorter block time is better in general, even if it isn't a "solution".

I would disagree with this. If you reduce block time, you need to proportionally increase the number of confirmations for the same level of security. For example, 1 confirmation now would be 10 confirmations @ 1 minute block time. Since the same resources to reverse 1 block now could reverse ten 1 minute blocks. Makes sense? So the same 'amount' of transaction finality & security will always take the same amount of time, no matter how fast your blocks come in. If you have a difficulty target & block time so high that a block only gets mined once per hour, 1 conf would be equally as secure as 6 confirmations in today's Bitcoin. Just to show the other extreme..

If you assume that customers at a brick-and-mortar store, spend amounts way too small to make an attack feasible, you can either accept 0 confirmation transactions. As far as I know, that's how BTC-accepting stores did it so far. Or just use the Lightning Network or a future, maybe better, L2 alternative.

[odolvlobo]

I agree that even a 1 minute block time is not short enough in your particular scenario, but certainly a shorter block time is better in general, even if it isn't a "solution".

I would disagree with this. If you reduce block time, you need to proportionally increase the number of confirmations for the same level of security. For example, 1 confirmation now would be 10 confirmations @ 1 minute block time...

That's true, but the key is that there is a huge difference between no confirmation (no security) and 1 confirmation (some security), especially with RBF, and even with a shorter block time.

[d5000]

I agree both with n0nce and odolvlobo. The security of a 10-minute confirmation is equivalent to 10 1-minute confs, but in Bitcoin you don't have anything close to a 1-minute-conf (zero-conf or nothing).

The 10-minute interval, according to a stockexchange post, was chosen because Satoshi believed that block propagation would be slower (1 minute), but things have improved greatly since then. The Bitcoin whitepaper also cites indirectly the advantage of a low requirement for space for block headers if the nodes do pruning (~4,2 MB/year, in the case of LTC you have already ~17 MB for headers alone, an 1-minute Bitcoin would have 42, and in the case of a "15-second-Bitcoin" about 160. These numbers look small when we look at current HD and RAM capacity, but it still is an advantage of the "slow" Bitcoin.

The most voted Stackexchange answer cite also other aspects of this "tradeoff":

Shorter block time:

    PRO - Faster 1 confirmation time (to protect from 0-confirm double spend)
    PRO - Less payout variance for miners (less reliance on large pools)
    CON - Requires increased bandwidth (inter node communication)
    CON - More forks, longer forks, and longer re-org time
    CON - A greater portion of the raw hashpower is wasted, resulting in lower effective security.

The point with the "wasted" hashrate is maybe somewhat debatable. The security of a cryptocurrency depends on the money being wasted by an attacker to get 50%+ of the hashrate, not on its "raw" hashrate. Thus, an attacker also has to take into account the stale block/fork rate while he tries to make his attack chain the canonical chain. However, as attack forks are typically short, and he can mine a large part in secret (without having to fear stale blocks) they're may be less relevant to them.

PS: I think Bitcoin made a pretty good tradeoff with its 10-min blocks, although if sidechain mechanisms like Drivechain became reality, I would bet on them choosing an interval more close to 1 minute, as some of the tradeoffs aren't that important for them.

@pooya87: Yes, they seemed to be solo miners. 2016 was still in crypto winter, and I mean to remember that altcoins were even deeper immersed in it (had 90%+ losses compared to their previous ATH), so solo mining should have been normal then.

[Quickseller]

I agree that even a 1 minute block time is not short enough in your particular scenario, but certainly a shorter block time is better in general, even if it isn't a "solution".

I would disagree with this. If you reduce block time, you need to proportionally increase the number of confirmations for the same level of security. For example, 1 confirmation now would be 10 confirmations @ 1 minute block time...

That's true, but the key is that there is a huge difference between no confirmation (no security) and 1 confirmation (some security), especially with RBF, and even with a shorter block time.

If a transaction is not opting in to RBF, I would argue that a zero confirmation transaction has modest amounts of security. Obviously a single confirmation has much more security than zero confirmations. Once a transaction has a single confirmation (with a 10 minute block time), the chance of loss goes down to close to zero, and will incrementally decline until 6 confirmations, when the risk is still something very close to zero, but not zero, and will not decline further as the transaction receives additional confirmations.

Someone accepting (and exchanging value for) an unlimited amount of bitcoin will want to wait for 6 confirmations, especially if they receive large amounts of bitcoin. However this is not true for retail establishments, as they are limited by their available inventory, are further limited by bottlenecks at the checkout process, and are further limited by the fact that many retail establishments will require some kind of a delay when selling very large amount of inventory to a single customer.

While it is not especially uncommon to see an orphan bitcoin block (d500 cites data suggesting about 0.5% of blocks are orphaned, which works out to about 2 every 3 days), it is uncommon for a transaction confirmed in an orphaned block to not get confirmed shortly thereafter. This implies the risk of accepting a 1 confirmation transaction is very low. Double spending a transaction that received one confirmation generally requires both mining hardware and multiple tries at double spending (or without mining hardware, MANY tries, even with RBF enabled).

I speculate at least part of the reason for the above is that most miners (the entities that actually propagate found blocks -- eg pools and solo miners) are well connected to the network, and follow similar rules to maximize tx fee revenue. Having a 1 minute average block time would obviously allow for more entitles to enter the mining world, and the rate at which a particular mining entity has not yet received a particular transaction as of when a block is found would probably increase.

I am curious if anyone has run any simulations on the effect shortening the average block time has on the percentage of transactions getting double spent that were included in an orphaned block. It shouldn't be especially difficult to figure out a way to compensate miners for orphaned blocks (such as eth's 'uncle' system). The more important issue is the risk of loss in accepting n confirmations when the average block time is changed.

[n0nce]

I agree that even a 1 minute block time is not short enough in your particular scenario, but certainly a shorter block time is better in general, even if it isn't a "solution".

I would disagree with this. If you reduce block time, you need to proportionally increase the number of confirmations for the same level of security. For example, 1 confirmation now would be 10 confirmations @ 1 minute block time...

That's true, but the key is that there is a huge difference between no confirmation (no security) and 1 confirmation (some security), especially with RBF, and even with a shorter block time.

Yeah; correct - there is no way to get like '1 tenth security' of a single confirmation in Bitcoin, that's right. But I don't understand the application / scenario.

Waiting for 1 minute (on average! It can be significantly longer - we sometimes see 20 - 30 minute block intervals in Bitcoin..), is not feasible at the counter of a brick-and-mortar store; it's way too long. And for online purchases, waiting 1 or 10 minutes makes no difference in terms of when the item arrives. It's not like the processing takes days.. So I see no added benefit but many potential and real issues.

[o_e_l_e_o]

-snip-

Also worth noting that Monero initially had a block time of 1 minute, but increased it to 2 minutes in 2016, partly due to an excessive number of stale blocks.

Waiting for 1 minute (on average! It can be significantly longer - we sometimes see 20 - 30 minute block intervals in Bitcoin..), is not feasible at the counter of a brick-and-mortar store; it's way too long. And for online purchases, waiting 1 or 10 minutes makes no difference in terms of when the item arrives. It's not like the processing takes days.. So I see no added benefit but many potential and real issues.

This is essentially the point I was making above. The probability of not mining a block within the next x seconds, when the expected block time is y seconds, is given by the following equation:

e^(-x/y)

If you drop the block time to 1 minute, then the probability of not mining a block within a minute is

e^(-60/60) = 36.8%

13.5% of blocks will still take longer than 2 minutes, and 1% of blocks will still take longer than almost 5 minutes. And this is all assuming that your transaction makes it in to the very next block, and the merchant is happy with only one confirmation despite the increased risk of stale blocks.

None of this is acceptable for a point of sale transaction. You are either going to have to wait for confirmations anyway, which you would be doing already if you were buying online or making a large purchase such as a car in person, or you are going to have to accept zero confirmation transactions or use Lightning. No one is going to hang around waiting for a confirmation while their coffee gets cold, regardless of if the block time is 10 minutes or 1 minute.

If you want to argue for a shorter block time for other reasons, by all means, and I do think there are some good arguments for doing so, but it will never solve point of sale transactions.

[Quickseller]

As has been mentioned by multiple people (including myself), LN is superior for retail transactions.

However when determining if the amount of time is acceptable for a retail establishment in terms of payment confirmation, it is more complex than the worse case scenario for when the next block will be found.

It is important to note that the average block time is going to be twice the average amount of time until the next block at any given time. So on average, over many transactions at an establishment, if the average block interval is 1 minute, the average amount of time until the next block is found will be 30 seconds -- about half the time it will be less than 30 seconds, and about 1/2 the time it will be longer.

From an operational standpoint, it is important to understand where the bottlenecks are, and how long they are. At a coffee shop for example, there are bottlencks at the line to place an order, and at the barista who is making various orders. In my experience, there is a longer bottleneck at the barista than there is to place an order. This means if there is a 1 minute block interval, the coffee shop could direct the customer to proceed to the waiting area after 5 seconds when the coffee shop is confident the transaction will confirm in the next two blocks. While the coffee shop is waiting for a confirmation, the order can be placed in the queue (as is the case today with cash/credit card txns), and once the order reaches the front of the queue, if the txn has not yet confirmed, it can be moved back one place in favor of another order (or the coffee shop can elect to accept the unconfirmed txn provided certain criteria is met, such as no blocks being found, and no recent double spends against them).

There is a similar bottleneck at fast food restaurants, and a similar procedure can be implemented. At dine-in restaurants where payment is typically made after the meal is consumed, waiting 5 or 10 minutes for a confirmation should not be a major concern.

A retail establishment is more complex. There is currently a single bottleneck and that is waiting in line for the clerk to "ring up" the items a customer is buying. In addition to "ringing up" items, clerks will also place purchased items in bags. One solution might be for the customer to "scan" the items being purchased prior to interacting with the clerk, paying for the items via a txn, and by the time the clerk has finished placing all items in bags (which will not happen until the clerk has placed all previous customers' items in bags), hopefully the txn will have confirmed.

[BlackHatCoiner]

The conclusion is that the Lightning Network is much better for retailers and customers, because not only they get their transaction confirmed instantly, but they also, both, retain their privacy and pay less in fees.

The probability of not mining a block within the next x seconds, when the expected block time is y seconds, is given by the following equation:

e^(-x/y)

How's that true? Is it a mathematical parity in probability theory that I'm not aware? I would really be interested to know more.

However when determining if the amount of time is acceptable for a retail establishment in terms of payment confirmation, it is more complex than the worse case scenario for when the next block will be found.

I don't understand this. Do you want to belie the Lightning Network? Which part of it is more complex and by who?

[o_e_l_e_o]

It is important to note that the average block time is going to be twice the average amount of time until the next block at any given time. So on average, over many transactions at an establishment, if the average block interval is 1 minute, the average amount of time until the next block is found will be 30 seconds -- about half the time it will be less than 30 seconds, and about 1/2 the time it will be longer.

This is not accurate. Mining is a Poisson process, which is memoryless. It doesn't matter if it has been 1 second or 1 hour since the last block; the average amount of time to the next block is always the same (1 minute in your example). With a 1 minute block time, the chance of mining a block in the next 30 seconds is 39.3%, not 50%.

How's that true? Is it a mathematical parity in probability theory that I'm not aware? I would really be interested to know more.

As I've mentioned, bitcoin mining is a Poisson process, which can be modeled mathematically. See my previous post regarding this here:

As I stated above, bitcoin mining is a Poisson process. You can read about this here: https://en.wikipedia.org/wiki/Poisson_point_process

You can model the distribution using the equation (as given on the linked page above):

P{N = n} = Λⁿ * e^-Λ / n!

If you take n to equal 0 (i.e. you won't find a block), and take lambda (Λ) to equal the number of blocks you would expect to find in a given time frame, then you can simplify that equation to

P = Λ⁰ * e^-Λ / 0!
P = 1 * e^-Λ / 1
P = e^-Λ

Therefore, the probability of not finding a block when you would expect to find Λ blocks is equal to e^-Λ. For example, the probability of not finding a block in 10 minutes, when you would ordinarily expect to find 1 block in 10 minutes, is e^-1 = 36.8%

Given the equation P = e^-Λ, then we can take the inverse to find the probability of finding a block: 1-P = 1-e^-Λ. So the probability of finding a block in 10 minutes is 1-e^-1 = 63.2%.

As I've done here, instead of using lambda to equal the expected number of blocks you would find, you turn it in to the time frame you are interested in divided by the average block time, which gives you the same result. If I'm interested in not finding a block in 10 minutes, then the equation is e^(-600/600), which is e^-1. If I'm interested in not finding a block in 30 seconds when the block time is one minute, the equation is e^(-30/60), which is e^-0.5.

[BlackHatCoiner]

Oh, so that's what the seventh page of the whitepaper is about... Finally.

In Bitcoin, the block interval is 10 minutes. This means that the probability for a block to be found within the next 10 minutes is 1 - e^-1 = 63.2%. I'm trying to understand why the 6 confirmations are considered safe as long as honest nodes control most of the computational power, but here's something I don't get.

Given that Λ is the number of blocks and 1 - e^-Λ the probability of finding Λ blocks within the next 10 minutes, don't we increase our chances as long as Λ increases?

For instance, generating 2 blocks within the next 10 minutes would have probability of 1 - e^-Λ = 86.5% which is greater than 63.2%. Shouldn't it drop off exponentially since generating 1 block instead of 2 blocks within the next 10 minutes is more probable?

[o_e_l_e_o]

I'm trying to understand why the 6 confirmations are considered safe as long as honest nodes control most of the computational power, but here's something I don't get.

You can play around with the numbers here: https://web.archive.org/web/20181231045818/https://people.xiph.org/~greg/attack_success.html. This page is essentially the equations from the whitepaper put in to action.

6 was chosen as a trade off between speed and security. In reality it's a continuum of how likely an attacker might be able to reverse x confirmations, not a case of "5 is unsafe, but 6 is safe". As you can see from the whitepaper and the calculator I linked, if someone has a significant (but less than 51%) percentage of the hashrate, then even 6 confirmations might not be safe. If you want an attacker with 40% of the hashrate to have a less than 0.1% chance of reversing a transaction, then you need to wait for 89 confirmations, for example. However, given the huge amount of hash power the bitcoin network now has, I would accept 3 for a lot of things, as do many exchanges and services.

For instance, generating 2 blocks within the next 10 minutes would have probability of 1 - e^-Λ = 86.5% which is greater than 63.2%. Shouldn't it drop off exponentially since generating 1 block instead of 2 blocks within the next 10 minutes is more probable?

No, it's not that simple. The simplified equation I gave only holds when you are looking at not finding a block at all, because in that case n = 0 and you can cancel out most of the terms. To work out any other number of blocks, you would need to go back to the original equation that I have quoted from myself above, and make n = 2. So if you wanted to see how likely it would be to find 2 blocks in 10 minutes, given that you would ordinarily expect to find 1 block in 10 minutes, you would set n = 2 and Λ = 1.

P = Λⁿ * e^-Λ / n!
P = 1² * e^-1 / 2!
P = 1 * e^-1 / 2
P = e^-1 / 2
P = 18.39%

Note that this is the value to find exactly 2 blocks in 10 minutes. If you wanted to find 2 or more blocks in 10 minutes, then the math becomes much more complicated, but would give a probability of 26.42%.

[Quickseller]

However when determining if the amount of time is acceptable for a retail establishment in terms of payment confirmation, it is more complex than the worse case scenario for when the next block will be found.

I don't understand this. Do you want to belie the Lightning Network? Which part of it is more complex and by who?

LN is far superior to a hypothetical coin with a one-minute block time that has the security of bitcoin. Provided there is an available route, a transaction will be completed nearly instantly. Further, a transaction completed on LN will have the security of 6+ confirmations as of the second it is completed.

If there is some reason why using LN is not feasible, there are ways for a business to make waiting potentially 3-5 minutes not a negative customer experience, while not giving up security. A business can do this by engineering their process such that the customer will send a payment with sufficient time before the business would be able to provide the product to the customer.

It is important to note that the average block time is going to be twice the average amount of time until the next block at any given time. So on average, over many transactions at an establishment, if the average block interval is 1 minute, the average amount of time until the next block is found will be 30 seconds -- about half the time it will be less than 30 seconds, and about 1/2 the time it will be longer.

This is not accurate. Mining is a Poisson process, which is memoryless. It doesn't matter if it has been 1 second or 1 hour since the last block; the average amount of time to the next block is always the same (1 minute in your example). With a 1 minute block time, the chance of mining a block in the next 30 seconds is 39.3%, not 50%.

I ran some tests, and you're right. I stand corrected.

[BlackHatCoiner]

No, it's not that simple.

Oh, there's an n that defines the exact number of blocks you may find while you expect to find Λ number of blocks in 10 minutes. Question: Does that apply to any block interval? Would it be the same whether it was 10 or 20 minutes? I don't see it being part of the equation anywhere.

So, let's play with it. If I wanted to see my result, I'd take:
P = Λⁿ * e^-Λ / n!   --->  P = 1² * e^-1 / 2   --->  P = 18.39%

For 3 blocks within 10 minutes, it'd be:
P = Λⁿ * e^-Λ / n!   --->  P = 1³ * e^-1 / 6   --->  P = 6.13%

For 4 blocks within 10 minutes, it'd be:
P = Λⁿ * e^-Λ / n!   --->  P = 1⁴ * e^-1 / 24   --->  P = 1.53%

For 5 blocks:
P = Λⁿ * e^-Λ / n!   --->  P = 1⁴ * e^-1 / 120   --->  P = 0.306%

And finally for 6 blocks:
P = Λⁿ * e^-Λ / n!   --->  P = 1⁴ * e^-1 / 720   --->  P = 0.051%

So, 0.051%. That's what is considered improbable to happen.


This is truly interesting, I concede. I'd like to make the probability viewable graphically.

f(n) = e^-1 / !n

Skipping anything prior 0, we can observe how abruptly improbable it becomes for the network to generate stale blocks. Question, is this true only if we assume there are no attackers and everyone mines honestly?

[o_e_l_e_o]

Question: Does that apply to any block interval? Would it be the same whether it was 10 or 20 minutes? I don't see it being part of the equation anywhere.

The exact block interval itself isn't relevant. What is relevant is how many blocks you would expect to find within the time frame you are interested in, which is what Λ is. So if you are considering blocks found within 10 minutes, then Λ = 1 for bitcoin since you would expect to find 1 block within those 10 minutes on average. If you were considering monero, which has a block time of 2 minutes, then for a 10 minute interval Λ would equal 5.

If you wanted to examine bitcoin over an hour, then Λ = 6. Over a day, Λ = 144. Over a minute, Λ = 0.1.

So, let's play with it.

Yes, your results are all correct. You've entered the incorrect exponent of 1 in your last two equations, but it is irrelevant to the final result. Note that as above, this equation is for finding that exact number of blocks in the time frame specified, and not at least that number of blocks.

I'd like to make the probability viewable graphically.

Graphing negative numbers on the x axis here makes no sense, since n! is not defined for negative numbers. Here is the same graph over a much more zoomed in and relevant axis:

[BlackHatCoiner]

Graphing negative numbers on the x axis here makes no sense, since n! is not defined for negative numbers.

If it isn't defined, then why did Desmos draw it? 

So if you are considering blocks found within 10 minutes, then Λ = 1 for bitcoin since you would expect to find 1 block within those 10 minutes on average. If you were considering monero, which has a block time of 2 minutes, then for a 10 minute interval Λ would equal 5.

So, it's all a matter of assumption. We just compare the final result; when we divide Λ with 5 to understand the probability in monero, what we actually do is divide the presumed 10 with 5 to get 2.

Note that as above, this equation is for finding that exact number of blocks in the time frame specified, and not at least that number of blocks.

May I conclude that the P of the exact number of blocks is not greater than the P of the at least number of blocks?

[o_e_l_e_o]

If it isn't defined, then why did Desmos draw it?

It is drawing the plot of the gamma function, which is undefined for non-positive integers and tends to infinity for these values, and so the graph of e^-1 / n! is zero at these values (since the infinity term is the denominator).

May I conclude that the P of the exact number of blocks is not greater than the P of the at least number of blocks?

Correct. You will always have a higher chance of finding "x or more" blocks than you will of finding "exactly x" blocks. 2 blocks in 10 minutes is 18.39%, but 2 or more blocks in 10 minutes is 26.42%. 3 blocks in 10 minutes is 6.13%, but 3 or more blocks in 10 minutes in 8.03%.

[Quickseller]

If I am not mistaken, to calculate the chances of x or more events over an interval when y events are expected, you can do one of two things:
If x is sufficiently small, you can plug (x - 1) and y into the Poisson formula, note the resulting probability, repeat with x being decreased by an additional 1, noting the result, and repeating until you note the result after x = 0. The sum of the probabilities subtracted from one is the resulting probability.

For larger x values, you can plug x and y into the Poisson formula, note the resulting probability, repeat with x being increased until x is 4 standard deviations away from y (for Poisson distributions, one standard deviation is the square root of the expected value, or y in my example), and the sum will be within 0.1% of the actual probability, or you can continue until 5 standard deviations from y, and the sum of the resulting probabilities will be within 0.00006% of the actual probability.

[o_e_l_e_o]

-snip-

Essentially yes, if you want to do it manually.

What you really want to do is to use the summation function, denoted by capital sigma (Σ). Difficult to print it out properly in BBCode, but essentially the formula would be:

f(n,Λ) = Σ (Λⁿ * e^-Λ / n!)

Your bounds would be between 0 and n for the lower distribution (so finding n or fewer blocks), and between n and infinity for the upper distribution (so finding n or more blocks). Or alternatively change the bounds to any arbitrary numbers to find the chance of finding that number of blocks. For example, the chance of finding either 1 or 2 blocks in 10 minutes is 55.18%.

[Quickseller]

-snip-

Essentially yes, if you want to do it manually.

What you really want to do is to use the summation function, denoted by capital sigma (Σ). Difficult to print it out properly in BBCode, but essentially the formula would be:

f(n,Λ) = Σ (Λⁿ * e^-Λ / n!)

Your bounds would be between 0 and n for the lower distribution (so finding n or fewer blocks), and between n and infinity for the upper distribution (so finding n or more blocks). Or alternatively change the bounds to any arbitrary numbers to find the chance of finding that number of blocks. For example, the chance of finding either 1 or 2 blocks in 10 minutes is 55.18%.

The chances of not finding n blocks within one block interval is the same as 1 - chances of finding n or more blocks within one block interval.

If you need a precise chance of 'x or more' occurrences happening, it is probably better to calculate the chances of x -1 or less occurrences not happening. 

[o_e_l_e_o]

If you need a precise chance of 'x or more' occurrences happening, it is probably better to calculate the chances of x -1 or less occurrences not happening.

If you are doing it by hand, then sure, that way is easier, but working out the upper distribution to infinity is a trivial task for any calculator or piece of software. Here, I've put the necessary equation in to Wolfram Alpha so you can play around with numbers yourself. Just adjust the upper and lower bounds (the ∞ and 0 above and below the capital sigma respectively) as desired. The output is a percentage chance.

https://www.wolframalpha.com/input/?i2d=true&i=Sum%5BDivide%5BPower%5Be%2C-1%5D%2Cn%21%5D%2C%7Bn%2C0%2C%E2%88%9E%7D%5D*100

[Quickseller]

If you need a precise chance of 'x or more' occurrences happening, it is probably better to calculate the chances of x -1 or less occurrences not happening.

If you are doing it by hand, then sure, that way is easier, but working out the upper distribution to infinity is a trivial task for any calculator or piece of software. Here, I've put the necessary equation in to Wolfram Alpha so you can play around with numbers yourself. Just adjust the upper and lower bounds (the ∞ and 0 above and below the capital sigma respectively) as desired. The output is a percentage chance.

https://www.wolframalpha.com/input/?i2d=true&i=Sum%5BDivide%5BPower%5Be%2C-1%5D%2Cn%21%5D%2C%7Bn%2C0%2C%E2%88%9E%7D%5D*100

It might be trivial for a software program to perform the calculation, but the runtime will be slower calculating to infinity versus calculating the chances of x -1 not happening.

To demonstrate, I created two simple functions:
def to_inf():
    count = 0
    for x in range(100):
        count +=1
def sub_3x():
    count = 0
    for x in range(3):
        count -=1

I ran some test on both to see how fast they execute:
%timeit to_inf()
4.64 µs ± 67.7 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
%timeit sub_3x()
512 ns ± 4.11 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

I could create functions that calculate a poisson probability until the probability is zero, and that calculates all probabilities of x-1 down to 0, and the outcome would be the same (the later would have a quicker runtime)

[thecodebear]

I agree that even a 1 minute block time is not short enough in your particular scenario, but certainly a shorter block time is better in general, even if it isn't a "solution".

I would disagree with this. If you reduce block time, you need to proportionally increase the number of confirmations for the same level of security. For example, 1 confirmation now would be 10 confirmations @ 1 minute block time...

That's true, but the key is that there is a huge difference between no confirmation (no security) and 1 confirmation (some security), especially with RBF, and even with a shorter block time.

Yeah; correct - there is no way to get like '1 tenth security' of a single confirmation in Bitcoin, that's right. But I don't understand the application / scenario.

Waiting for 1 minute (on average! It can be significantly longer - we sometimes see 20 - 30 minute block intervals in Bitcoin..), is not feasible at the counter of a brick-and-mortar store; it's way too long. And for online purchases, waiting 1 or 10 minutes makes no difference in terms of when the item arrives. It's not like the processing takes days.. So I see no added benefit but many potential and real issues.

Yeah. The reason for short block times would be to pay for stuff in person. If you're paying online you're not gonna care if it takes 10 seconds or 1 hour to confirm. But in person, anything more than a handful of seconds is too long, so even if the blocktime were 20 seconds that'd be a very long time to wait jus to get the first confirmation.

So reducing blocktime for this purpose is basically pointless. For in person transactions you need something that has finality within seconds. That's where L2 solutions like LN come in. There's no point in using L1 for that sort of stuff. Just like you're not gonna wire money or even write a check these days while in the line at the grocery store, you're gonna use one of the higher up networks like a credit card.

Bitcoin has exactly the tradeoff we need for a robust monetary system. Security on L1, that is still useful for moving money around and buying things online (assuming the fee isn't too high), and efficiency on L2 for buying stuff in person or just making every day transactions.

The fact that usually you can see a transaction hit the mempool in a few seconds or so is good enough for getting confirmation that your transaction is being processed, if we're talking about just making sure it went through (sort of like when a credit card tx says confirmed).

[Wind_FURY]

The "difficulty" manages to have one block built not faster than in 10 minutes in average. It helps to avoid collisions. If we decrease the difficulty to have 1 block in 10 seconds instead of current 10 minutes, we would have way more collisions, and the blocks we mine will have higher chance to be discarded.

What can be a solution for a decentralized pool of nodes around the world? May be already invented but I am not aware?

The solution? A low latency cryptocurrency network, which is only possible through a gigameg internet network that could download and upload gigamegs of data within seconds that’s available for everyone. If Bitcoin can’t have something like this, the Core Developers must adapt with what’s available, and they are doing a very good job.

[pooya87]

The solution? A low latency cryptocurrency network, which is only possible through a gigameg internet network that could download and upload gigamegs of data within seconds that’s available for everyone.

I'm afraid that won't solve the real issue either. Bitcoin blocks are not like a video that you just stream, they have to be validated which requires expensive computation work followed by database updates, which is the actual bottlenecks.

[Wind_FURY]

The solution? A low latency cryptocurrency network, which is only possible through a gigameg internet network that could download and upload gigamegs of data within seconds that’s available for everyone.

I'm afraid that won't solve the real issue either. Bitcoin blocks are not like a video that you just stream, they have to be validated which requires expensive computation work followed by database updates, which is the actual bottlenecks.

Then besides a very low latency network, thanks to our fictitious, perfect gigameg internet available to all, add our fictitious, cheap, high-grade institutional hardware network of full nodes. All post-IBD validation is processed within milli-seconds.

[Storytellers]

There are several ways to speed up the bitcoin transaction and i will list them below:

- Replace-by-Fee (RBF (for transactions sent by you)To implement RBF you need to use a wallet thats upports this functionality
- Child Pays for Parent (CPFP)
- Transaction accelerators

if you need more info be welcome to reply to me to try to help you little bit more.

[ABCbits]

All post-IBD validation is processed within milli-seconds.

All? milliseconds? It's unlikely since there are many factor which affect verification time. Check this snipped answer by Pieter Wuille.

• Signature cache size and CPU speed. The larger the cache, the more signature validations can be avoided. These validations - depending on the software version and hardware van vary from 0.01ms to 0.6ms per signature (45ms to 2.7s for a block).
• Correlation between the memory pool and the new block. If a block contains transactions that a node hasn't seen before, its inputs and signatures are less likely to be cached before.

There are several ways to speed up the bitcoin transaction and i will list them below:

- Replace-by-Fee (RBF (for transactions sent by you)To implement RBF you need to use a wallet thats upports this functionality
- Child Pays for Parent (CPFP)
- Transaction accelerators

if you need more info be welcome to reply to me to try to help you little bit more.

It's not wrong answer, but what OP means is increasing transaction speed (as in getting confirmed) by modifying Bitcoin protocol/node implementation.

[n0nce]

The solution? A low latency cryptocurrency network, which is only possible through a gigameg internet network that could download and upload gigamegs of data within seconds that’s available for everyone. If Bitcoin can’t have something like this, the Core Developers must adapt with what’s available, and they are doing a very good job.

Besides the fact that this claim is false, a 'gigameg' doesn't mean anything regarding an internet network. It literally means 'billion million'. You're saying: which is only possible through a billion million internet network.