How faster transactions can be implemented

[n0nce]

I agree that even a 1 minute block time is not short enough in your particular scenario, but certainly a shorter block time is better in general, even if it isn't a "solution".

I would disagree with this. If you reduce block time, you need to proportionally increase the number of confirmations for the same level of security. For example, 1 confirmation now would be 10 confirmations @ 1 minute block time...

That's true, but the key is that there is a huge difference between no confirmation (no security) and 1 confirmation (some security), especially with RBF, and even with a shorter block time.

Yeah; correct - there is no way to get like '1 tenth security' of a single confirmation in Bitcoin, that's right. But I don't understand the application / scenario.

Waiting for 1 minute (on average! It can be significantly longer - we sometimes see 20 - 30 minute block intervals in Bitcoin..), is not feasible at the counter of a brick-and-mortar store; it's way too long. And for online purchases, waiting 1 or 10 minutes makes no difference in terms of when the item arrives. It's not like the processing takes days.. So I see no added benefit but many potential and real issues.

[o_e_l_e_o]

-snip-

Also worth noting that Monero initially had a block time of 1 minute, but increased it to 2 minutes in 2016, partly due to an excessive number of stale blocks.

Waiting for 1 minute (on average! It can be significantly longer - we sometimes see 20 - 30 minute block intervals in Bitcoin..), is not feasible at the counter of a brick-and-mortar store; it's way too long. And for online purchases, waiting 1 or 10 minutes makes no difference in terms of when the item arrives. It's not like the processing takes days.. So I see no added benefit but many potential and real issues.

This is essentially the point I was making above. The probability of not mining a block within the next x seconds, when the expected block time is y seconds, is given by the following equation:

e^(-x/y)

If you drop the block time to 1 minute, then the probability of not mining a block within a minute is

e^(-60/60) = 36.8%

13.5% of blocks will still take longer than 2 minutes, and 1% of blocks will still take longer than almost 5 minutes. And this is all assuming that your transaction makes it in to the very next block, and the merchant is happy with only one confirmation despite the increased risk of stale blocks.

None of this is acceptable for a point of sale transaction. You are either going to have to wait for confirmations anyway, which you would be doing already if you were buying online or making a large purchase such as a car in person, or you are going to have to accept zero confirmation transactions or use Lightning. No one is going to hang around waiting for a confirmation while their coffee gets cold, regardless of if the block time is 10 minutes or 1 minute.

If you want to argue for a shorter block time for other reasons, by all means, and I do think there are some good arguments for doing so, but it will never solve point of sale transactions.

[Quickseller]

As has been mentioned by multiple people (including myself), LN is superior for retail transactions.

However when determining if the amount of time is acceptable for a retail establishment in terms of payment confirmation, it is more complex than the worse case scenario for when the next block will be found.

It is important to note that the average block time is going to be twice the average amount of time until the next block at any given time. So on average, over many transactions at an establishment, if the average block interval is 1 minute, the average amount of time until the next block is found will be 30 seconds -- about half the time it will be less than 30 seconds, and about 1/2 the time it will be longer.

From an operational standpoint, it is important to understand where the bottlenecks are, and how long they are. At a coffee shop for example, there are bottlencks at the line to place an order, and at the barista who is making various orders. In my experience, there is a longer bottleneck at the barista than there is to place an order. This means if there is a 1 minute block interval, the coffee shop could direct the customer to proceed to the waiting area after 5 seconds when the coffee shop is confident the transaction will confirm in the next two blocks. While the coffee shop is waiting for a confirmation, the order can be placed in the queue (as is the case today with cash/credit card txns), and once the order reaches the front of the queue, if the txn has not yet confirmed, it can be moved back one place in favor of another order (or the coffee shop can elect to accept the unconfirmed txn provided certain criteria is met, such as no blocks being found, and no recent double spends against them).

There is a similar bottleneck at fast food restaurants, and a similar procedure can be implemented. At dine-in restaurants where payment is typically made after the meal is consumed, waiting 5 or 10 minutes for a confirmation should not be a major concern.

A retail establishment is more complex. There is currently a single bottleneck and that is waiting in line for the clerk to "ring up" the items a customer is buying. In addition to "ringing up" items, clerks will also place purchased items in bags. One solution might be for the customer to "scan" the items being purchased prior to interacting with the clerk, paying for the items via a txn, and by the time the clerk has finished placing all items in bags (which will not happen until the clerk has placed all previous customers' items in bags), hopefully the txn will have confirmed.

[BlackHatCoiner]

The conclusion is that the Lightning Network is much better for retailers and customers, because not only they get their transaction confirmed instantly, but they also, both, retain their privacy and pay less in fees.

The probability of not mining a block within the next x seconds, when the expected block time is y seconds, is given by the following equation:

e^(-x/y)

How's that true? Is it a mathematical parity in probability theory that I'm not aware? I would really be interested to know more.

However when determining if the amount of time is acceptable for a retail establishment in terms of payment confirmation, it is more complex than the worse case scenario for when the next block will be found.

I don't understand this. Do you want to belie the Lightning Network? Which part of it is more complex and by who?

[o_e_l_e_o]

It is important to note that the average block time is going to be twice the average amount of time until the next block at any given time. So on average, over many transactions at an establishment, if the average block interval is 1 minute, the average amount of time until the next block is found will be 30 seconds -- about half the time it will be less than 30 seconds, and about 1/2 the time it will be longer.

This is not accurate. Mining is a Poisson process, which is memoryless. It doesn't matter if it has been 1 second or 1 hour since the last block; the average amount of time to the next block is always the same (1 minute in your example). With a 1 minute block time, the chance of mining a block in the next 30 seconds is 39.3%, not 50%.

How's that true? Is it a mathematical parity in probability theory that I'm not aware? I would really be interested to know more.

As I've mentioned, bitcoin mining is a Poisson process, which can be modeled mathematically. See my previous post regarding this here:

As I stated above, bitcoin mining is a Poisson process. You can read about this here: https://en.wikipedia.org/wiki/Poisson_point_process

You can model the distribution using the equation (as given on the linked page above):

P{N = n} = Λⁿ * e^-Λ / n!

If you take n to equal 0 (i.e. you won't find a block), and take lambda (Λ) to equal the number of blocks you would expect to find in a given time frame, then you can simplify that equation to

P = Λ⁰ * e^-Λ / 0!
P = 1 * e^-Λ / 1
P = e^-Λ

Therefore, the probability of not finding a block when you would expect to find Λ blocks is equal to e^-Λ. For example, the probability of not finding a block in 10 minutes, when you would ordinarily expect to find 1 block in 10 minutes, is e^-1 = 36.8%

Given the equation P = e^-Λ, then we can take the inverse to find the probability of finding a block: 1-P = 1-e^-Λ. So the probability of finding a block in 10 minutes is 1-e^-1 = 63.2%.

As I've done here, instead of using lambda to equal the expected number of blocks you would find, you turn it in to the time frame you are interested in divided by the average block time, which gives you the same result. If I'm interested in not finding a block in 10 minutes, then the equation is e^(-600/600), which is e^-1. If I'm interested in not finding a block in 30 seconds when the block time is one minute, the equation is e^(-30/60), which is e^-0.5.

[BlackHatCoiner]

Oh, so that's what the seventh page of the whitepaper is about... Finally.

In Bitcoin, the block interval is 10 minutes. This means that the probability for a block to be found within the next 10 minutes is 1 - e^-1 = 63.2%. I'm trying to understand why the 6 confirmations are considered safe as long as honest nodes control most of the computational power, but here's something I don't get.

Given that Λ is the number of blocks and 1 - e^-Λ the probability of finding Λ blocks within the next 10 minutes, don't we increase our chances as long as Λ increases?

For instance, generating 2 blocks within the next 10 minutes would have probability of 1 - e^-Λ = 86.5% which is greater than 63.2%. Shouldn't it drop off exponentially since generating 1 block instead of 2 blocks within the next 10 minutes is more probable?

[o_e_l_e_o]

I'm trying to understand why the 6 confirmations are considered safe as long as honest nodes control most of the computational power, but here's something I don't get.

You can play around with the numbers here: https://web.archive.org/web/20181231045818/https://people.xiph.org/~greg/attack_success.html. This page is essentially the equations from the whitepaper put in to action.

6 was chosen as a trade off between speed and security. In reality it's a continuum of how likely an attacker might be able to reverse x confirmations, not a case of "5 is unsafe, but 6 is safe". As you can see from the whitepaper and the calculator I linked, if someone has a significant (but less than 51%) percentage of the hashrate, then even 6 confirmations might not be safe. If you want an attacker with 40% of the hashrate to have a less than 0.1% chance of reversing a transaction, then you need to wait for 89 confirmations, for example. However, given the huge amount of hash power the bitcoin network now has, I would accept 3 for a lot of things, as do many exchanges and services.

For instance, generating 2 blocks within the next 10 minutes would have probability of 1 - e^-Λ = 86.5% which is greater than 63.2%. Shouldn't it drop off exponentially since generating 1 block instead of 2 blocks within the next 10 minutes is more probable?

No, it's not that simple. The simplified equation I gave only holds when you are looking at not finding a block at all, because in that case n = 0 and you can cancel out most of the terms. To work out any other number of blocks, you would need to go back to the original equation that I have quoted from myself above, and make n = 2. So if you wanted to see how likely it would be to find 2 blocks in 10 minutes, given that you would ordinarily expect to find 1 block in 10 minutes, you would set n = 2 and Λ = 1.

P = Λⁿ * e^-Λ / n!
P = 1² * e^-1 / 2!
P = 1 * e^-1 / 2
P = e^-1 / 2
P = 18.39%

Note that this is the value to find exactly 2 blocks in 10 minutes. If you wanted to find 2 or more blocks in 10 minutes, then the math becomes much more complicated, but would give a probability of 26.42%.

[Quickseller]

However when determining if the amount of time is acceptable for a retail establishment in terms of payment confirmation, it is more complex than the worse case scenario for when the next block will be found.

I don't understand this. Do you want to belie the Lightning Network? Which part of it is more complex and by who?

LN is far superior to a hypothetical coin with a one-minute block time that has the security of bitcoin. Provided there is an available route, a transaction will be completed nearly instantly. Further, a transaction completed on LN will have the security of 6+ confirmations as of the second it is completed.

If there is some reason why using LN is not feasible, there are ways for a business to make waiting potentially 3-5 minutes not a negative customer experience, while not giving up security. A business can do this by engineering their process such that the customer will send a payment with sufficient time before the business would be able to provide the product to the customer.

It is important to note that the average block time is going to be twice the average amount of time until the next block at any given time. So on average, over many transactions at an establishment, if the average block interval is 1 minute, the average amount of time until the next block is found will be 30 seconds -- about half the time it will be less than 30 seconds, and about 1/2 the time it will be longer.

This is not accurate. Mining is a Poisson process, which is memoryless. It doesn't matter if it has been 1 second or 1 hour since the last block; the average amount of time to the next block is always the same (1 minute in your example). With a 1 minute block time, the chance of mining a block in the next 30 seconds is 39.3%, not 50%.

I ran some tests, and you're right. I stand corrected.

[BlackHatCoiner]

No, it's not that simple.

Oh, there's an n that defines the exact number of blocks you may find while you expect to find Λ number of blocks in 10 minutes. Question: Does that apply to any block interval? Would it be the same whether it was 10 or 20 minutes? I don't see it being part of the equation anywhere.

So, let's play with it. If I wanted to see my result, I'd take:
P = Λⁿ * e^-Λ / n!   --->  P = 1² * e^-1 / 2   --->  P = 18.39%

For 3 blocks within 10 minutes, it'd be:
P = Λⁿ * e^-Λ / n!   --->  P = 1³ * e^-1 / 6   --->  P = 6.13%

For 4 blocks within 10 minutes, it'd be:
P = Λⁿ * e^-Λ / n!   --->  P = 1⁴ * e^-1 / 24   --->  P = 1.53%

For 5 blocks:
P = Λⁿ * e^-Λ / n!   --->  P = 1⁴ * e^-1 / 120   --->  P = 0.306%

And finally for 6 blocks:
P = Λⁿ * e^-Λ / n!   --->  P = 1⁴ * e^-1 / 720   --->  P = 0.051%

So, 0.051%. That's what is considered improbable to happen.


This is truly interesting, I concede. I'd like to make the probability viewable graphically.

f(n) = e^-1 / !n

Skipping anything prior 0, we can observe how abruptly improbable it becomes for the network to generate stale blocks. Question, is this true only if we assume there are no attackers and everyone mines honestly?

[o_e_l_e_o]

Question: Does that apply to any block interval? Would it be the same whether it was 10 or 20 minutes? I don't see it being part of the equation anywhere.

The exact block interval itself isn't relevant. What is relevant is how many blocks you would expect to find within the time frame you are interested in, which is what Λ is. So if you are considering blocks found within 10 minutes, then Λ = 1 for bitcoin since you would expect to find 1 block within those 10 minutes on average. If you were considering monero, which has a block time of 2 minutes, then for a 10 minute interval Λ would equal 5.

If you wanted to examine bitcoin over an hour, then Λ = 6. Over a day, Λ = 144. Over a minute, Λ = 0.1.

So, let's play with it.

Yes, your results are all correct. You've entered the incorrect exponent of 1 in your last two equations, but it is irrelevant to the final result. Note that as above, this equation is for finding that exact number of blocks in the time frame specified, and not at least that number of blocks.

I'd like to make the probability viewable graphically.

Graphing negative numbers on the x axis here makes no sense, since n! is not defined for negative numbers. Here is the same graph over a much more zoomed in and relevant axis:

[BlackHatCoiner]

Graphing negative numbers on the x axis here makes no sense, since n! is not defined for negative numbers.

If it isn't defined, then why did Desmos draw it? 

So if you are considering blocks found within 10 minutes, then Λ = 1 for bitcoin since you would expect to find 1 block within those 10 minutes on average. If you were considering monero, which has a block time of 2 minutes, then for a 10 minute interval Λ would equal 5.

So, it's all a matter of assumption. We just compare the final result; when we divide Λ with 5 to understand the probability in monero, what we actually do is divide the presumed 10 with 5 to get 2.

Note that as above, this equation is for finding that exact number of blocks in the time frame specified, and not at least that number of blocks.

May I conclude that the P of the exact number of blocks is not greater than the P of the at least number of blocks?

[o_e_l_e_o]

If it isn't defined, then why did Desmos draw it?

It is drawing the plot of the gamma function, which is undefined for non-positive integers and tends to infinity for these values, and so the graph of e^-1 / n! is zero at these values (since the infinity term is the denominator).

May I conclude that the P of the exact number of blocks is not greater than the P of the at least number of blocks?

Correct. You will always have a higher chance of finding "x or more" blocks than you will of finding "exactly x" blocks. 2 blocks in 10 minutes is 18.39%, but 2 or more blocks in 10 minutes is 26.42%. 3 blocks in 10 minutes is 6.13%, but 3 or more blocks in 10 minutes in 8.03%.

[Quickseller]

If I am not mistaken, to calculate the chances of x or more events over an interval when y events are expected, you can do one of two things:
If x is sufficiently small, you can plug (x - 1) and y into the Poisson formula, note the resulting probability, repeat with x being decreased by an additional 1, noting the result, and repeating until you note the result after x = 0. The sum of the probabilities subtracted from one is the resulting probability.

For larger x values, you can plug x and y into the Poisson formula, note the resulting probability, repeat with x being increased until x is 4 standard deviations away from y (for Poisson distributions, one standard deviation is the square root of the expected value, or y in my example), and the sum will be within 0.1% of the actual probability, or you can continue until 5 standard deviations from y, and the sum of the resulting probabilities will be within 0.00006% of the actual probability.

[o_e_l_e_o]

-snip-

Essentially yes, if you want to do it manually.

What you really want to do is to use the summation function, denoted by capital sigma (Σ). Difficult to print it out properly in BBCode, but essentially the formula would be:

f(n,Λ) = Σ (Λⁿ * e^-Λ / n!)

Your bounds would be between 0 and n for the lower distribution (so finding n or fewer blocks), and between n and infinity for the upper distribution (so finding n or more blocks). Or alternatively change the bounds to any arbitrary numbers to find the chance of finding that number of blocks. For example, the chance of finding either 1 or 2 blocks in 10 minutes is 55.18%.

[Quickseller]

-snip-

Essentially yes, if you want to do it manually.

What you really want to do is to use the summation function, denoted by capital sigma (Σ). Difficult to print it out properly in BBCode, but essentially the formula would be:

f(n,Λ) = Σ (Λⁿ * e^-Λ / n!)

Your bounds would be between 0 and n for the lower distribution (so finding n or fewer blocks), and between n and infinity for the upper distribution (so finding n or more blocks). Or alternatively change the bounds to any arbitrary numbers to find the chance of finding that number of blocks. For example, the chance of finding either 1 or 2 blocks in 10 minutes is 55.18%.

The chances of not finding n blocks within one block interval is the same as 1 - chances of finding n or more blocks within one block interval.

If you need a precise chance of 'x or more' occurrences happening, it is probably better to calculate the chances of x -1 or less occurrences not happening. 

[o_e_l_e_o]

If you need a precise chance of 'x or more' occurrences happening, it is probably better to calculate the chances of x -1 or less occurrences not happening.

If you are doing it by hand, then sure, that way is easier, but working out the upper distribution to infinity is a trivial task for any calculator or piece of software. Here, I've put the necessary equation in to Wolfram Alpha so you can play around with numbers yourself. Just adjust the upper and lower bounds (the ∞ and 0 above and below the capital sigma respectively) as desired. The output is a percentage chance.

https://www.wolframalpha.com/input/?i2d=true&i=Sum%5BDivide%5BPower%5Be%2C-1%5D%2Cn%21%5D%2C%7Bn%2C0%2C%E2%88%9E%7D%5D*100

[Quickseller]

If you need a precise chance of 'x or more' occurrences happening, it is probably better to calculate the chances of x -1 or less occurrences not happening.

If you are doing it by hand, then sure, that way is easier, but working out the upper distribution to infinity is a trivial task for any calculator or piece of software. Here, I've put the necessary equation in to Wolfram Alpha so you can play around with numbers yourself. Just adjust the upper and lower bounds (the ∞ and 0 above and below the capital sigma respectively) as desired. The output is a percentage chance.

https://www.wolframalpha.com/input/?i2d=true&i=Sum%5BDivide%5BPower%5Be%2C-1%5D%2Cn%21%5D%2C%7Bn%2C0%2C%E2%88%9E%7D%5D*100

It might be trivial for a software program to perform the calculation, but the runtime will be slower calculating to infinity versus calculating the chances of x -1 not happening.

To demonstrate, I created two simple functions:
def to_inf():
    count = 0
    for x in range(100):
        count +=1
def sub_3x():
    count = 0
    for x in range(3):
        count -=1

I ran some test on both to see how fast they execute:
%timeit to_inf()
4.64 µs ± 67.7 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
%timeit sub_3x()
512 ns ± 4.11 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

I could create functions that calculate a poisson probability until the probability is zero, and that calculates all probabilities of x-1 down to 0, and the outcome would be the same (the later would have a quicker runtime)

[thecodebear]

I agree that even a 1 minute block time is not short enough in your particular scenario, but certainly a shorter block time is better in general, even if it isn't a "solution".

I would disagree with this. If you reduce block time, you need to proportionally increase the number of confirmations for the same level of security. For example, 1 confirmation now would be 10 confirmations @ 1 minute block time...

That's true, but the key is that there is a huge difference between no confirmation (no security) and 1 confirmation (some security), especially with RBF, and even with a shorter block time.

Yeah; correct - there is no way to get like '1 tenth security' of a single confirmation in Bitcoin, that's right. But I don't understand the application / scenario.

Waiting for 1 minute (on average! It can be significantly longer - we sometimes see 20 - 30 minute block intervals in Bitcoin..), is not feasible at the counter of a brick-and-mortar store; it's way too long. And for online purchases, waiting 1 or 10 minutes makes no difference in terms of when the item arrives. It's not like the processing takes days.. So I see no added benefit but many potential and real issues.

Yeah. The reason for short block times would be to pay for stuff in person. If you're paying online you're not gonna care if it takes 10 seconds or 1 hour to confirm. But in person, anything more than a handful of seconds is too long, so even if the blocktime were 20 seconds that'd be a very long time to wait jus to get the first confirmation.

So reducing blocktime for this purpose is basically pointless. For in person transactions you need something that has finality within seconds. That's where L2 solutions like LN come in. There's no point in using L1 for that sort of stuff. Just like you're not gonna wire money or even write a check these days while in the line at the grocery store, you're gonna use one of the higher up networks like a credit card.

Bitcoin has exactly the tradeoff we need for a robust monetary system. Security on L1, that is still useful for moving money around and buying things online (assuming the fee isn't too high), and efficiency on L2 for buying stuff in person or just making every day transactions.

The fact that usually you can see a transaction hit the mempool in a few seconds or so is good enough for getting confirmation that your transaction is being processed, if we're talking about just making sure it went through (sort of like when a credit card tx says confirmed).

[Wind_FURY]

The "difficulty" manages to have one block built not faster than in 10 minutes in average. It helps to avoid collisions. If we decrease the difficulty to have 1 block in 10 seconds instead of current 10 minutes, we would have way more collisions, and the blocks we mine will have higher chance to be discarded.

What can be a solution for a decentralized pool of nodes around the world? May be already invented but I am not aware?

The solution? A low latency cryptocurrency network, which is only possible through a gigameg internet network that could download and upload gigamegs of data within seconds that’s available for everyone. If Bitcoin can’t have something like this, the Core Developers must adapt with what’s available, and they are doing a very good job.

[pooya87]

The solution? A low latency cryptocurrency network, which is only possible through a gigameg internet network that could download and upload gigamegs of data within seconds that’s available for everyone.

I'm afraid that won't solve the real issue either. Bitcoin blocks are not like a video that you just stream, they have to be validated which requires expensive computation work followed by database updates, which is the actual bottlenecks.