A block chain for real-time confirmations

[MoonShadow]

Can floating transactions be made low-risk enough that it doesn't matter if they aren't included in a block right away?

This is starting to feel familiar...

http://bitcointalk.org/index.php?topic=723.msg8019#msg8019

http://bitcointalk.org/index.php?topic=691.msg7283#msg7283

[Cdecker]

I think the best way to make progress with this issue would be for somebody to look at the impact of using much faster block generation times.

10 minutes was chosen on the assumption of really large networks, really large blocks and thus potentially large propagation times. It's a tradeoff between convenience and avoiding wastage of mining effort.

But 10 minutes for block propagation is pretty huge. BGP updates propagate across the whole internet in less than one minute, as far as I can tell. And BitCoin is nowhere near large enough to see 10 minute propagation times today. It'd probably be possible to have one minute confirmations on todays network.

The questions to ask are:

• Is 1 minute really better than 10? I think suspect there's really only two speeds that matter, "<5 seconds" and ">5 seconds".
• Can floating transactions be made low-risk enough that it doesn't matter if they aren't included in a block right away?

I think we can actually have a tradeoff between fast confirmations and network size. Right now every single node gets all transactions, but what if we segment the network? The network would be segmented into a hypercube with variable degree. Each vertex is a cluster of nodes (mini-networks) which gets assigned there by hashing their IP (forcing nodes into a specific position of the overall network) and they'd take care of small amounts of outgoing addresses (to mark spent inputs), achieving consensus with a much faster block chain construction (micro-tick if you will). The miners still compute the usual proof of work, but if they find a lower difficulty block, they report it to their cluster to advance the local block chain. If a miner finds a global block it starts collecting transactions from all other vertices and announces the block.

I proposed this first way back while arguing that it would allow the network to scale better, but the micro-ticks look a lot like the block creation speedup you guys are looking for

[Maged]

On a different note, other than my "hyperchain" extension (or maybe even Cdecker's, but it's too late for me to put thought into that one), to combat the finney attack, we could do something pretty scary: (temporarily) reject blocks that contain a different transaction for a coin than the client initially received. Yes, this would open the network up to a potential attack. It'd be interesting to look into, though.

[Steve]

So I have to ask, what is the problem trying to be solved?

I think the problem that this system tries to solve is in-place transactions without the need of a trusted third party.

Exactly, and it's a problem that is better solved with a trusted third party IMO.  I think all of the proposals on this thread still have an element of time that must pass before you can really consider a transaction to be firmly embedded in history.  And, what is interesting is that the very nature of double spending involves the passage of time.  The only way to ensure that a double spend has not occurred is to wait some period of time until you are confident that you are still well connected to the real network and that no double spends have been attempted.  You can gain a degree of such confidence with the existing network by staying well connected and listening for double spends even before a new block has been generated.

I think what it comes down to are a few simple choices:
a) accept a transaction immediately and not worry about the potential for a double spend
b) wait a period of time for confirmation blocks
c) use a trusted third party to clear transactions instantly

For transactions of relatively small amounts, the effort required to double spend isn't worth it hence the occurrence should be small to non-existent.  Even accounting for some amount of fraud, this cost should be far cheaper than the transaction fees currently imposed by payment processors.

[Maged]

I just realized something... If "instant" transactions were made too safe/secure, why would a person pay a transaction fee? Sure, each second that goes by after the transaction would increase the amount of mining nodes that weren't around when the transaction was first submitted, but that would add little additional risk. Since we need transaction fees to maintain the block chain in the future anyway, and double-spend protection could be used to attack the network...

• Can floating transactions be made low-risk enough that it doesn't matter if they aren't included in a block right away?

I think floating transactions can easily be made very low-risk.  Remember that if your client sees a new tx, it forwards it to all nodes around it.  If you are connected directly to some of the large miners, the probability that your client would not be notified of a double tx (assuming the attacker specifically sends conflicting txes to you and large miners at the same time to increase their chance) is very low.  Thus if the client informed the user if it received a conflicting tx for a tx which is to/from the user, the user could be easily notified within seconds of the original tx. 
Alternatively, each node could pass the first conflicting tx it sees or some kind of "Ive seen a tx which conflicts with this tx" message.  However, this increases the possibility for DDoS per coin (roughly x2 in terns of txes which need transferred through the network) so this is probably not a good idea. 

...why not require a certain transaction fee to enable advanced double-spend protection (basic ds protection being what we have now) for a transaction? Such a fee would make both practical AND technical sense.

Practically, if a person doesn't want to wait when they buy something with BTC from an untrusted wallet (trusted wallets being banks that the seller trusts), they could pay the fee and the seller could feel much safer sending the order to the customer right away because double spends would be seen very quickly, the transaction would be significantly more likely be added to the next block, and, assuming we make the risky choice of rejecting blocks that clients think contain a conflicting transaction, they would also be fairly protected from the 'finney attack'.

Technically, a minimum fee to get advanced double-spend protection would mean that spamming the network by using double-spends to send twice the data per transaction would cost money, just as other types of spam currently do. Additionally, again also assuming my risky defense to the 'finney attack' doesn't have holes in its logic, it also means you would have to spend money just to have the chance of splitting the network with a 'finney attack', since this would make your block invalid and would give the fee to whoever generates a legitimate block.

As an amendment to my 'finney attack' suggestion, make clients accept blocks with the "wrong" transaction if the 'wrong' transaction initially arrived within, say, 5 seconds of the first one. That way you would have lower success of purposefully splitting the network if the transactions were sent from 2 sides of the network at once. In such a case, the seller would hear of the double-spend attempt fast enough anyway.

[BitterTea]

c) use a trusted third party to clear transactions instantly

Isn't that what ClearCoin is?

I don't think so. I think ClearCoin waits 6 confirmations before a balance shows up in the account. I assumed he was talking about something more like a Bitcoin "bank".

[jtimon]

c) use a trusted third party to clear transactions instantly

Isn't that what ClearCoin is?

I don't think so. I think ClearCoin waits 6 confirmations before a balance shows up in the account. I assumed he was talking about something more like a Bitcoin "bank".

Yes I think he means a Bitcoin bank, payment processor, account hub...I've heard a few names for this. The proposal I like most for third parties is thi one (Bitcoin AH):

http://bitcointalk.org/index.php?topic=2628.msg37261#msg37261

Maybe third parties is the only solution, but very fast transaction (seconds) without them would be definitely better.

[jtimon]

Maybe it just can't be done. Third parties wouldn't be bad, just worse than a p2p solution.

[Steve]

ClearCoin with some modifications could provide a service such as this...it could do it multiple ways...if ClearCoin doesn't trust the payer, then they must first have a clear coin account with confirmed funds in it...if ClearCoin does trust you, then ClearCoin is taking on the double spend risk any time you want to buy something.  The payee trusts that ClearCoin won't double spend and accepts its payments instantly.  I think this solution is far better than complicating (and possibly compromising) the underlying bitcoin protocol.  For many people, the risk of a double spend will still be so low that they won't even use the trusted third party service.  They'll just accept the transactions immediately upon broadcast. 

I guess I just don't see a big issue with this in the current system.

[jtimon]

Yes, there are lots of things that could be done with a service like this to mitigate risk, even without a registration process. You could, for instance, do [transaction limit]+[captcha solve]+[ip ban] for small transactions.

That sounds good, but I'm sorry, I don't understand it.

[Jim Hyslop]

There is a big misconception I'd like to clear up before I go back and examine the details of the discussion that I've missed over the last few days.

The 10 minute interval has no connection at all to technical issues such as network latency, bitcoin creation, etc. etc. The sole purpose of the ten minute interval is to ensure that the rate of Bitcoin creation is reasonably constant and predictable. It is an economic constraint, not a technical one. If there were no constraint on Bitcoin creation, then the market would very quickly get flooded, and the value of BTC would be extremely volatile.

In a similar vein, difficulty has exactly one purpose: to maintain the constant rate of Bitcoin creation. Every 2016 blocks, the software checks how long it took to create the last 2016 blocks. The goal is exactly two weeks (do the math). If it took longer than two weeks to create 2016 blocks, then the difficulty is reduced. If it took less than two weeks, then the difficulty is increased. For the purposes of this discussion, difficulty can be ignored, and we can assume a rate of 10 minutes per block.

[theymos]

The 10 minute interval has no connection at all to technical issues such as network latency, bitcoin creation, etc. etc. The sole purpose of the ten minute interval is to ensure that the rate of Bitcoin creation is reasonably constant and predictable. It is an economic constraint, not a technical one. If there were no constraint on Bitcoin creation, then the market would very quickly get flooded, and the value of BTC would be extremely volatile.

That's totally wrong. Maybe latency was one of a few factors in determining the 10-minute target, but it definitely was a factor. From the source:

Code:

        // It is not an error when generated blocks are not accepted.  By design,
        // some percentage of blocks, like 10% or more, will end up not accepted.
        // This is the normal mechanism by which the network copes with latency.

10% block loss was anticipated with even 10-minute target times. This is made worse when you reduce the target time.

Generation rate would not have been a factor in determining target time, since you can easily reduce the reward per block to get the same result.

[Jim Hyslop]

I'm going to combine all my responses into a single reply. I hope this doesn't confuse anyone ('specially me

There are many issues with this idea, most notably the lack of miners, the continued vulnerability to the 'finney attack', and the added bloat.

Lack of miners??? The difficulty increased by 37% a few days ago because a lot of miners have jumped on board.

I've seen a few references to "the finney attack" in these forums, but I can't find a description of what it is.  Can someone point me to a discussion of the attack, please?

There are some serious issues with such an idea[...].  Another is the complexity added to the protocol.

Extending the blocks to add a new type is not a big extension to the protocol. Unless we are using different definitions of the word - to me "protocol" is the definition of how data is passed from one node to another. In any case, the protocol must be extensible or sooner or later the whole system will collapse because of its inflexibility.

Transactions are complete in milliseconds, it's the confirmations that take ten minutes, which are way faster than how this is done by just about any other online method.  Credit card transactions are, likewise, nearly instant; but it can take up to 60 days before they are final.  So I have to ask, what is the problem trying to be solved?

I thought the subject line summed it up nicely: real-time confirmations.

Your comparison is flawed, because credit card transactions are reversible. Bitcoin transactions are not. Double-spent transactions can be rejected, but not reversed. That's why a (near) real-time confirmation is desirable. Before I let the guy who showed up at my place of business walk out the door with my merchandise, I want a reasonable assurance that the Bitcoins he used are valid.

Anyone making a large transaction surely will have the patience to wait an hour or so for 6 confirmations.

Maybe for an online transaction, but I'm thinking about a face-to-face transaction. I am not going to hang around some merchant's store for an hour just to have my transaction confirmed.

I could see a separate chain running with a target time of perhaps as low as one minute, but a target time of seconds will be impossible due to latency. I don't think a separate chain is the answer, though: there would be no incentive to create blocks on it, and there are probably other methods that will work even better.

Well, until someone tosses in an idea that works better, why not continue discussing this approach? There's no rule that says we have to actually do anything with these ideas. Maybe this discussion will spark someone thinking of a better solution.

On the topic of latency - has anyone attempted to measure how long it takes for a transaction to propagate through the P2P network? Is it even possible to measure? I can think of a couple of ways to measure the propagation, but they all involve changing the client so that it reports back to a single monitoring site - a proposition which I doubt very many users would approve of.

Can floating transactions be made low-risk enough that it doesn't matter if they aren't included in a block right away?[/li][/list]

What do you mean by "floating transactions"?

The network might never be large enough to support this, though, and no changes are necessary to Bitcoin now, so it's not worth working on. By the time it becomes useful, solutions for dealing with fast transaction acceptance will already have been established.

Sorry, I cannot agree with that sentiment. That's the kind of thinking that led to the Y2K problem. People kept putting off looking at their systems, so that in the end some of the solutions that went in were poorly thought out and ill-considered, or just plain abandoned.

Also, anyone remember Lotus 1-2-3? It was one of the first spreadsheets. It quickly took a lion's share of the marketplace, but because nobody at Lotus thought ahead and tried to put in new features or otherwise improved the software, other spreadsheets were able to surpass Lotus 1-2-3's capabilities. By the time Lotus figured out what was happening, they had lost any possible advantage they may have had, and 1-2-3 died. (and then they foisted Lotus Notes on us, but that's a whole other rant )

The time to think about these issues is now, while we have the luxury of tossing around ideas and exploring a lot of "what if" scenarios, and while the software is still in beta and changes are expected.

That's why I'm bringing up topics like the face-to-face exchange, which (to my knowledge) aren't currently supported by any clients. Someday, they may very well be.

[Jim Hyslop]

(Whew, caught up )

The 10 minute interval has no connection at all to technical issues such as network latency, bitcoin creation, etc. etc. The sole purpose of the ten minute interval is to ensure that the rate of Bitcoin creation is reasonably constant and predictable. It is an economic constraint, not a technical one. If there were no constraint on Bitcoin creation, then the market would very quickly get flooded, and the value of BTC would be extremely volatile.

That's totally wrong. Maybe latency was one of a few factors in determining the 10-minute target, but it definitely was a factor. From the source:

Code:

        // It is not an error when generated blocks are not accepted.  By design,
        // some percentage of blocks, like 10% or more, will end up not accepted.
        // This is the normal mechanism by which the network copes with latency.

10% block loss was anticipated with even 10-minute target times. This is made worse when you reduce the target time.

Generation rate would not have been a factor in determining target time, since you can easily reduce the reward per block to get the same result.

Well, I wouldn't say totally wrong. I interpret that comment slightly differently. My interpretation is: "because of network latency, I estimate about 10% of blocks will be rejected. That is perfectly normal and nothing to be concerned about." There's no indication in that comment whether 10% was a goal, or a byproduct of the 10 minute spacing.

At this point, that comment is the only thing I've read which suggested there were any technical considerations for choosing 10 minutes. In fact, what I have read said that if there was a reason for choosing 10 minutes, Satoshi never communicated it to anyone. Unless you have another source which confirms that network latency was a factor in choosing 10 minutes, I think we'll have to leave the question open.

And this is probably a good place to repeat the question I asked in my last post (which will probably get glossed over because I crammed so much into a single post): Has anyone measured, or calculated, how long it would take for a transaction to propagate through the network?

[Maged]

There are many issues with this idea, most notably the lack of miners, the continued vulnerability to the 'finney attack', and the added bloat.

Lack of miners??? The difficulty increased by 37% a few days ago because a lot of miners have jumped on board.

That statement was in regards to an independently mined block chain that was being suggested, hence my suggestion to combine the two chains so that the same miners would mine both at the same time, as a result of mining just one.

I've seen a few references to "the finney attack" in these forums, but I can't find a description of what it is.  Can someone point me to a discussion of the attack, please?

Finney Attack, as defined by Hal:
Suppose the attacker is generating blocks occasionally. in each block he generates, he includes a transfer from address A to address B, both of which he controls.

To cheat you, when he generates a block, he doesn't broadcast it. Instead, he runs down to your store and makes a payment to your address C with his address A. You wait a few seconds, don't hear anything, and transfer the goods. He broadcasts his block now, and his transaction will take precedence over yours.

----
Also, I just realized that the risky method of rejecting suspicious blocks that I mentioned later in this thread was already discussed:
http://bitcointalk.org/index.php?topic=3441.0

Finally, I really would like to see that question answered too.

[MoonShadow]

Transactions are complete in milliseconds, it's the confirmations that take ten minutes, which are way faster than how this is done by just about any other online method.  Credit card transactions are, likewise, nearly instant; but it can take up to 60 days before they are final.  So I have to ask, what is the problem trying to be solved?

I thought the subject line summed it up nicely: real-time confirmations.

Your comparison is flawed, because credit card transactions are reversible. Bitcoin transactions are not. Double-spent transactions can be rejected, but not reversed. That's why a (near) real-time confirmation is desirable. Before I let the guy who showed up at my place of business walk out the door with my merchandise, I want a reasonable assurance that the Bitcoins he used are valid.

Well, let's examine that situation.  Could a convenience store accept bitcoin in real time?  I say certainly.  You walk in to the store to buy a candy bar, if you pay cash the attendant is likely to use a cash marker as a quick check for counterfit cash, but this is only a relatively minor means of avoiding such fraud.  In a similar fashion, a POS bitcoin client can check it's (presumedly recent) copy of the blockchain against the transaction that you produce for bitcoin.  If this check says that you honestly owned those coins up to the last update, it is unlikely that you are trying to double spend.  If we are talking about something a bit pricer, such as a grocery shopping trip in the 100's-of-dollars-range or so, the POS client might also be connected to a private VPN that exists to monitor for double spends in real time, (does your credit card transaction get approved in less than three seconds?  Even if it did, would it be unreasonable for the POS system to wait three seconds, looking for a competing transaction?  Longer than three seconds, and it likley doesn't even matter if a double spend is attempted, because the odds are extremely high that this transaction would be the one to get into the blockchain anyway.) signs of network divisions, or other possible frauds.  Some store chains, such as wal-mart would be likely to do this for themselves; but smaller stores and chains could join a VPN sponsored by their association.  Anything bigger, such as buying a new car, is likely to justify the increasing certainty of one or more blockchain confirmations.  Have you ever bought a car wherein the negotiations & credit process took less than a half-hour?

My point here is that it is possible, and likely, for a vendor to have a reasonable degree of trust that a valid transaction is legit without any confirmations (depending upon the relative value of the sale) and that increasing confirmations only add to the certainty.

[Matt Corallo]

I just realized something... If "instant" transactions were made too safe/secure, why would a person pay a transaction fee? Sure, each second that goes by after the transaction would increase the amount of mining nodes that weren't around when the transaction was first submitted, but that would add little additional risk. Since we need transaction fees to maintain the block chain in the future anyway, and double-spend protection could be used to attack the network...

• Can floating transactions be made low-risk enough that it doesn't matter if they aren't included in a block right away?

I think floating transactions can easily be made very low-risk.  Remember that if your client sees a new tx, it forwards it to all nodes around it.  If you are connected directly to some of the large miners, the probability that your client would not be notified of a double tx (assuming the attacker specifically sends conflicting txes to you and large miners at the same time to increase their chance) is very low.  Thus if the client informed the user if it received a conflicting tx for a tx which is to/from the user, the user could be easily notified within seconds of the original tx. 
Alternatively, each node could pass the first conflicting tx it sees or some kind of "Ive seen a tx which conflicts with this tx" message.  However, this increases the possibility for DDoS per coin (roughly x2 in terns of txes which need transferred through the network) so this is probably not a good idea. 

...why not require a certain transaction fee to enable advanced double-spend protection (basic ds protection being what we have now) for a transaction? Such a fee would make both practical AND technical sense.

Practically, if a person doesn't want to wait when they buy something with BTC from an untrusted wallet (trusted wallets being banks that the seller trusts), they could pay the fee and the seller could feel much safer sending the order to the customer right away because double spends would be seen very quickly, the transaction would be significantly more likely be added to the next block, and, assuming we make the risky choice of rejecting blocks that clients think contain a conflicting transaction, they would also be fairly protected from the 'finney attack'.

Technically, a minimum fee to get advanced double-spend protection would mean that spamming the network by using double-spends to send twice the data per transaction would cost money, just as other types of spam currently do. Additionally, again also assuming my risky defense to the 'finney attack' doesn't have holes in its logic, it also means you would have to spend money just to have the chance of splitting the network with a 'finney attack', since this would make your block invalid and would give the fee to whoever generates a legitimate block.

As an amendment to my 'finney attack' suggestion, make clients accept blocks with the "wrong" transaction if the 'wrong' transaction initially arrived within, say, 5 seconds of the first one. That way you would have lower success of purposefully splitting the network if the transactions were sent from 2 sides of the network at once. In such a case, the seller would hear of the double-spend attempt fast enough anyway.

Not at all true.  The point of a double spend attack is that although you put out two transactions, you don't pay for one of them.  Thus, charging people to have "double spend protection" wouldn't work as only one of the two transactions would be considered "valid" and thus only one would make it into the block chain.  The other transaction wouldn't be paid by or to anyone.  So if nodes relayed more blocks an attacker wouldn't pay any more to get more txes being pushed through the network (hence DDoS potential).

[Maged]

I just realized something... If "instant" transactions were made too safe/secure, why would a person pay a transaction fee? Sure, each second that goes by after the transaction would increase the amount of mining nodes that weren't around when the transaction was first submitted, but that would add little additional risk. Since we need transaction fees to maintain the block chain in the future anyway, and double-spend protection could be used to attack the network...

• Can floating transactions be made low-risk enough that it doesn't matter if they aren't included in a block right away?

I think floating transactions can easily be made very low-risk.  Remember that if your client sees a new tx, it forwards it to all nodes around it.  If you are connected directly to some of the large miners, the probability that your client would not be notified of a double tx (assuming the attacker specifically sends conflicting txes to you and large miners at the same time to increase their chance) is very low.  Thus if the client informed the user if it received a conflicting tx for a tx which is to/from the user, the user could be easily notified within seconds of the original tx. 
Alternatively, each node could pass the first conflicting tx it sees or some kind of "Ive seen a tx which conflicts with this tx" message.  However, this increases the possibility for DDoS per coin (roughly x2 in terns of txes which need transferred through the network) so this is probably not a good idea. 

...why not require a certain transaction fee to enable advanced double-spend protection (basic ds protection being what we have now) for a transaction? Such a fee would make both practical AND technical sense.

Practically, if a person doesn't want to wait when they buy something with BTC from an untrusted wallet (trusted wallets being banks that the seller trusts), they could pay the fee and the seller could feel much safer sending the order to the customer right away because double spends would be seen very quickly, the transaction would be significantly more likely be added to the next block, and, assuming we make the risky choice of rejecting blocks that clients think contain a conflicting transaction, they would also be fairly protected from the 'finney attack'.

Technically, a minimum fee to get advanced double-spend protection would mean that spamming the network by using double-spends to send twice the data per transaction would cost money, just as other types of spam currently do. Additionally, again also assuming my risky defense to the 'finney attack' doesn't have holes in its logic, it also means you would have to spend money just to have the chance of splitting the network with a 'finney attack', since this would make your block invalid and would give the fee to whoever generates a legitimate block.

As an amendment to my 'finney attack' suggestion, make clients accept blocks with the "wrong" transaction if the 'wrong' transaction initially arrived within, say, 5 seconds of the first one. That way you would have lower success of purposefully splitting the network if the transactions were sent from 2 sides of the network at once. In such a case, the seller would hear of the double-spend attempt fast enough anyway.

Not at all true.  The point of a double spend attack is that although you put out two transactions, you don't pay for one of them.  Thus, charging people to have "double spend protection" wouldn't work as only one of the two transactions would be considered "valid" and thus only one would make it into the block chain.  The other transaction wouldn't be paid by or to anyone.  So if nodes relayed more blocks an attacker wouldn't pay any more to get more txes being pushed through the network (hence DDoS potential).

Oh, I think I get what you are saying: Although the original transaction would have the fee added, there's nothing that would require the double-spent coin to have the fee. However, it doesn't really matter. As long as one of the transactions (the real one) pays the minimum required fee (let's assume that it is .01 BTC), you would still need a minimum of .02 BTC to transverse the network (.01 output, .01 fee), which is a code change that would need to be made (nobody sends real transactions with a .01 fee that output less than .01 BTC anyway). That would result in a .01 BTC/tx average for the network. Hell, we're already getting DDoSd with .01 BTC anyway.

[Mike Hearn]

So to clear this up, the 10 minute interval is to do with latency and Satoshi has confirmed it to me in the past via email. Let me quote from some of our conversations:

Another is the 10 minute block target. I understand this was chosen to
allow transactions to propagate through the network. However existing
large P2P networks like BGP can propagate new data worldwide in <1
minute.

If propagation is 1 minute, then 10 minutes was a good guess.  Then nodes are only losing 10% of their work (1 minute/10 minutes).  If the CPU time wasted by latency was a more significant share, there may be weaknesses I haven't thought of.  An attacker would not be affected by latency, since he's chaining his own blocks, so he would have an advantage.  The chain would temporarily fork more often due to latency.

Though Satoshi is not posting to the forums at the moment, he is still paying attention to email, at least I get answers to my questions that way.