latency and locality

[mkrogh]

Hi bitcoin users

I found bitcoin yesterday, and I am really impressed by the idea of avoiding issuers/banks in a digital cash system.

However, I have one objection that I would like your view on.

That is the latency, or delay in a local transaction. Let us say that a payer and payee are located in the same city for example. I will claim, that they want a payment to go through
with a delay not much slower than the latency of network connections. With an issuer and using local coins (a prefix, say, can be used to localize a server of the issuer), the transaction should be done and verified in 100 microseconds or less. With a global p2p network, it is necessary to have all nodes receive the transaction, do some calculations and send results back.
It is not feasible,  with the speed of light as an upper limit, to do this faster than a second. As I understand it you actually use something like 10 minutes for one block, or even more for several block verifications. This means that verification of the absence of double spending is 10,000 or even tens of millions times too slow. This would get even worse in interplanetary trade of course.

The ideal payment system should be decentralized but also local, i.e., the verification of a local transaction must not have to wait for a light signal to travel to the other end of the "universe" and come back again.

Locality, in the sense describe here, is crucial, I will claim.

Sincerely,

Morten Krogh.

[FreeMoney]

Why is it crucial?

One system doesn't have to be used for all needs. If interstellar money is needed people will choose whatever is best for that, same if people need their money in less than a second.

[knightmb]

It is not feasible,  with the speed of light as an upper limit, to do this faster than a second. As I understand it you actually use something like 10 minutes for one block, or even more for several block verifications. This means that verification of the absence of double spending is 10,000 or even tens of millions times too slow. This would get even worse in interplanetary trade of course.

It depends on your network layout. If you are using the entire public network, then yes you are depending on that time for the data to be processed by everyone else that is running a node. But, you don't have to do it that way. You can build a trusted node (like a server) and have your clients direct connect to it and let it do the fast verification for you.

It won't net you 20 confirmations in a few milliseconds, but it will help stop double-spending as long as you remain in control of that node because it knows what is being spent. The whole double-spending issue arises from trust. When you are comparing anonymous seller to anonymous buyer, the only "middle man" for trust is the entire swarm (hence confirmations that neither is cheating)

If you know you can trust your own node(s), then it reduces the likely-hood that double-spending will occur within your own trusted network. So if your example applied to an ATM like card for example, the bank (or whoever) would have all of those ATM machines networked to their own private node (or server) which would then further connect to the outside (Internet) for the rest of the transactions. So if you went to an ATM, withdrew 100 BTC (say the entire account only had that much), then moments later went to another ATM elsewhere and tried to withdraw that amount again, somewhere along that trusted chain would spot a double-spend attempt and basically report back no BTC left for that address.

The whole verification process is how you put the swarm to work, all those clients are examining if your transaction was valid or not, but you don't have to wait for them to transfer the coin around. You can spend the coin around in circles all day without a single transaction verification. It's only when double-spending occurs does the network correct for itself and fix the balances.

Banks already have this exact same problem. They all have their own networks that connect to other bank networks and it's just as easy to double-spend on their ATM as has been demonstrated at black-hat hacker conventions for probably 10+ years now. I remember reading about the issue back in 1997 because a lot of networks will still on dial-up then for banks and the latency issue was part of the attack because everyone had their own *database* that all had to inter-connect.

With bitcoin, everyone is the database and banker and guard at the same time, so while it's not perfect, it's actually a lot better than what banks are using. Have you ever noticed at an ATM they have about 20 "network" stickers for all the different banks and setups that the ATM has to be compatible with?

[Insti]

OP, have you read the Bitcoin snack machine thread?

[mkrogh]

FreeMoney, I think it is crucial to have fast local transactions. It is needed for many businesses and person to person payments.
10 minutes or even 10 seconds is too slow. An extraterrestrial system was just an extreme example.

The most pressing problem, as I see it, is to get a digital cash system that makes it possible for people and small businesses to pay each other fast and without
registration and supervision from a big company/government.

I read the thread about vending machines. There were no real solutions except companies that would monitor for double checking in a few seconds. I agree that would be possible,
but we would lose the whole point of not having an issuer. This company needs to be trusted, it will break anonymity, and people need an account with that company.

What is wrong with the follwing "standard" system. A central bank/government issues a currency like today. It sets up servers around the world to validate transactions.
People pay each other by means of a single verification with one of the centeal bank's servers. The speed issue is solved by coins having the location of the relevant server embedded in them. In worst case, the server would need to make long round trips to other severs, but in most cases the first server would know the answer already. The cental bank could have an algorithm for distributing coins around the network to optimise latency and server load. The wallet would have an algorithm for choosing the most local coins when making a payment.
The wallet could also prepare by exchanging coins with the server. On your way to Europe, the wallet replaces "American coins" with European coins, such that payments will be fast in Europe. For the user, this is incredibly easy. It is anonymous, and it requires no agreements with any processing companies.
Also, there could be many issuers of the same currency if so desired.

Bitcoin gets rid of the issuers, but instead introduces unacceptable slowness or a lot of middlemen like credit card companies/debit accounts etc. Satoshi's solution, on the vending machine thread, with a double spending checking company is still too slow, even though faster than a block, and this company needs an omnipresence on the network almost like
an issuer.

But discarding speed, bitcoin seems extremely elegant. And of course there are uses for this kind of system. One could imagine many local bitcoin systems with an exchange between
different currencies. That would work except that an attacker can take over a small currency.

[Insti]

The most pressing problem, as I see it, is to get a digital cash system that makes it possible for people and small businesses to pay each other fast and without
registration and supervision from a big company/government.

...

What is wrong with the follwing "standard" system. A central bank/government issues a currency like today. It sets up servers around the world to validate transactions.

So the solution to avoiding a big company/government regulation is a big company/government?

It is currently possible with Bitcoin to do transactions which are practically instant particularly if you do not fear some vast conspiracy to defraud you.
If you really do have need of transactions faster than Bitcoin can offer, use a different system, but you will probably have to forgo some of the benefits that Bitcoin currently offers.

[mkrogh]

No, my goal is not to avoid government involvement in the payment system, but to not have the government monitor every transaction. There is a difference.

The current bill/coin system works like that. The government/central bank issues the currency but they don't monitor individual cash payments.
I think that is okay.

You claim that is possible to make almost instantaneous payments with bitcoin. How? That is what I thought everybody agreed upon was not possible. You need to wait for
one or more blocks to be solved before you can trust a coin. That takes minutes. Even in the best case it is limited by propagation, with the speed of light, around the network which is slow. The only solution is to create another banking system on top of this. Bitcoin then becomes the backbone of an intebank transfer system, not what the population at large is using.
   
It is not just me that needs fast transactions. There is giant demand for fast transfers of digital cash from cell phone to cell phone, web shopping, stock trading etc. 

But, we might just have different expectations from a digital cash system.

Has anyone ever stated clearly what problem bitcoin is solving? It seems to solve the problem of payments that don't need to be fast. But who are the potential users of that beyond people who thinks it is theoretically interesting like people on this board.

Do people agree with me on this point: We still need another digital cash system, and that probably needs to be issuer based.

[Insti]

No, my goal is not to avoid government involvement in the payment system, but to not have the government monitor every transaction. There is a difference.

The current bill/coin system works like that. The government/central bank issues the currency but they don't monitor individual cash payments.
I think that is okay.

Every transaction is monitored in the bitcoin system.

You claim that is possible to make almost instantaneous payments with bitcoin. How? That is what I thought everybody agreed upon was not possible. You need to wait for
one or more blocks to be solved before you can trust a coin.

That is your requirement for trust, not mine. You can trust a transaction as soon as you see it if you like.
And people are going to shop with those who will give them speedy transactions.

That takes minutes. Even in the best case it is limited by propagation, with the speed of light, around the network which is slow. The only solution is to create another banking system on top of this. Bitcoin then becomes the backbone of an intebank transfer system, not what the population at large is using.
   
It is not just me that needs fast transactions. There is giant demand for fast transfers of digital cash from cell phone to cell phone, web shopping, stock trading etc. 

But, we might just have different expectations from a digital cash system.

Clearly.

Has anyone ever stated clearly what problem bitcoin is solving? It seems to solve the problem of payments that don't need to be fast. But who are the potential users of that beyond people who thinks it is theoretically interesting like people on this board.

Anyone who currently uses PayPal for a start.

Do people agree with me on this point: We still need another digital cash system, and that probably needs to be issuer based.

I don't. But I'm just one person.

[mkrogh]

Every transaction is monitored in the bitcoin system.

But you can make it anonymous. Why are you bringing this up?This is a point, were I thought we agreed.
When I said monitoring, I meant including identification of the payer and payee.

That is your requirement for trust, not mine. You can trust a transaction as soon as you see it if you like.
And people are going to shop with those who will give them speedy transactions.

Yes, that is my requirement, and will be most others as well. You will see almost no companies that sell without a guarantee against double spending.

Anyone who currently uses PayPal for a start.

Most of those will prefer a fast issuer based system over a slow decentralized system, I will claim.
But they might prefer bitcoin over paypal.

[Insti]

Every transaction is monitored in the bitcoin system.

But you can make it anonymous. Why are you bringing this up?This is a point, were I thought we agreed.
When I said monitoring, I meant including identification of the payer and payee.

Ok, when you said monitoring I thought you meant monitoring, so I was just pointing out that all the transactions were trackable in case you did not already understand this. Since it seems you do it's not a problem

That is your requirement for trust, not mine. You can trust a transaction as soon as you see it if you like.
And people are going to shop with those who will give them speedy transactions.

Yes, that is my requirement, and will be most others as well. You will see almost no companies that sell without a guarantee against double spending.

Anyone who currently uses PayPal for a start.

Most of those will prefer a fast issuer based system over a slow decentralized system, I will claim.
But they might prefer bitcoin over paypal.

It seems we have differing opinions on the way future will develop. Which is fine.
It'll probably turn out nothing like either of us expect anyway.

[mkrogh]

Yes, it might turn out very differently. No digital cash for the masses for instance. A giant banking cartel that bribes politicians to keep the current system.

[Red]

mkrogh: Yes, you are correct.

There is absolutely no requirement for there to be such latency in the bitcoin system. Nothing fundamental would change if the system were implemented with a block list that updated every 10 seconds, instead of every 10 minutes. It was just a design decision that could be changed without effecting monetary policy a bit.

In fact, there is no requirement their be a block list that updates in increments at all. The system would work fine if each transaction were validated one-by-one. In effect one transaction per block. That was a design decision to make networked consensus building easier.

However, when you say "local transaction validation" what you are grasping for is a truly distributed system, rather than a monolithic system made redundant. The current version of bitcoin is the latter. Each node does exactly the same work as every other node. Each redundantly checking every transaction. It is a system horribly wasteful of resources. And because it is so wasteful, generates latency as a waste product.

Bitcoin could be made into a truly distributed system by storing the transaction graph in a distributed hash table. This is the kind of magic that is behind most other P2P systems. In effect, each bitcoin address would be arbitrarily mapped to a smaller set of nodes that checked its transactions. In such a system, there is no need to broadcast global state to every machine. This takes huge amount of latency out without needing to change any of the desired behavior.

[FreeMoney]

Yes, that is my requirement, and will be most others as well. You will see almost no companies that sell without a guarantee against double spending.

Did you know about this? You don't need to wait for it to get in a block to have way better certainty than PP or checks.

I believe it'll be possible for a payment processing company to provide as a service the rapid distribution of transactions with good-enough checking in something like 10 seconds or less.

The network nodes only accept the first version of a transaction they receive to incorporate into the block they're trying to generate.  When you broadcast a transaction, if someone else broadcasts a double-spend at the same time, it's a race to propagate to the most nodes first.  If one has a slight head start, it'll geometrically spread through the network faster and get most of the nodes.

A rough back-of-the-envelope example:
1         0
4         1
16        4
64        16
80%      20%

So if a double-spend has to wait even a second, it has a huge disadvantage.

The payment processor has connections with many nodes.  When it gets a transaction, it blasts it out, and at the same time monitors the network for double-spends.  If it receives a double-spend on any of its many listening nodes, then it alerts that the transaction is bad.  A double-spent transaction wouldn't get very far without one of the listeners hearing it.  The double-spender would have to wait until the listening phase is over, but by then, the payment processor's broadcast has reached most nodes, or is so far ahead in propagating that the double-spender has no hope of grabbing a significant percentage of the remaining nodes.

[mkrogh]

FreeMoney, yes I knew about that.

Red, very interesting. So, a set of transactions in bitcoin can actually be made local. But it must be possible to control where a transaction should be validated. That must be coin
dependent because of double spending. So, if I understand, it must be possible, for any transaction, to choose which location the resulting coin should be validated in when that
coin is used next time. A random choice would not be good enough. But, are there some problems with an attacker taking over a certain location. The global system had the strength that an attacker would be outworked by the sum of all other CPUs.

Red, can you elaborate more on such a distributed version of bitcoin. Is it described anywhere?

Cheers.

[Gavin Andresen]

Bitcoin could be made into a truly distributed system by storing the transaction graph in a distributed hash table. This is the kind of magic that is behind most other P2P systems. In effect, each bitcoin address would be arbitrarily mapped to a smaller set of nodes that checked its transactions. In such a system, there is no need to broadcast global state to every machine. This takes huge amount of latency out without needing to change any of the desired behavior.

Interesting idea.

So, lets see, I create a transaction to pay you (say) 100 of my newly minted bitcoins.

That'll be a transaction with two 50BTC TxIns (signed by me, pointing to two mature GENERATE transactions somewhere in the block chain) and one 100BTC TxOuts.

You want to make sure I haven't double-spent those TxIns, so instead of flooding the network with that transaction you find the hash of the two GENERATE transactions and send two queries down into the DHT network:  "Hey, here's a transaction, tell me if it is valid."  They say "yup", and then... what?  Include it in any blocks they're lucky enough to generate?  Broadcast it to everybody (which'd be no better than the current scheme) or some subset of the DHT network (what subset?)?

How do you know that you won't get a different answer to "is this transaction valid" if you ask again in 10 minutes when the network topology might have changed?

I don't know much about DHT networks and how they manage to keep reliable information when nodes may be coming and going (or buggy or malicious).  How would it work?

[MoonShadow]

Red, very interesting. So, a set of transactions in bitcoin can actually be made local. But it must be possible to control where a transaction should be validated.

I think that you are getting lost in the choice of wording on this list.  Every bitcoin client, whether or not it is 'generating' or not, has a very recent copy of the entire block chain, and can check any new transactions against that block chain to see if those bitcoins were owned by the *address* that the other client claims he owns, at least up until the last block update.  I'm not saying local verification is implimented in the current client, but it's possible.  But that doesn't protect you from a concurrent double spend.  The distributed verification will add confidence in the validity of the transaction with each passing block, but it's not neccessary to perform a transfer.  Cash in person has similar problems, as the vendor can never be *certain* that the customer isn't passing off counterfit currency.  The risks of fraud are included in the costs of retail business, and if speed of transactions are paramount, the vendor can simply accept the results of his own client and take the risks that he might get burned.  Waiting for confirmation would help protect against fraud, but it's not a *requirement* for completion of a trade.  I would expect that smartphone clients would do this quick local verify by default, particularly when communicating with each other directly over ad-hoc wireless.

Also, the system doesn't even require that the receiving client be on the network at the time of the transfer.  You can send money to any address at any time, and their client could find out about it long after the transfer had been verified by the bitcoin network.

[NewLibertyStandard]

Just agree to both use the same free online wallet before you make your trade. MyBitcoin.com offers instant transfers.

[mkrogh]

creighto, I am only talking about verifying the absence of double spending. Of course, you can always receive a coin and just hope. That is clear.

I don't think that physical coins have this issue. Or gold. Firstly counterfeiting is really difficult, secondly "bill verifiers or gold verifiers" are local machines. If someone could
counterfeit gold or coins in a way that no local machine could detect, then it would be the same. But no one can do that.

With bitcoin in the version I have seen (not red's), you cannot build a local machine that verifies it. I don't understand what red exactly is proposing.


NewLibertyStandard, your suggestion is a bank, that the payer and the payee need an account with. I don't mind, but I don't think it is an answer to the issue of this thread.

What I see as the ideal solution has these properties.

1. You can participate by just having some software on your computer/phone.
2. The coins are stored by yourself. A coin is a string of characters, no physical gold or so is involved. Encryption keys are fine if you can generate them yourself.
3. No accounts or registrations with banks or governments are needed.
4. Payments can be totally verified including double spending checks. Tiny probabilites can be ignored. But I mean tiny, not just reasonably small.
5. If a person is traveling to a city, he (she) can prepare for the visit by exchanging coins. This exchange can be slow. After arriving to the city, the person can make fast payments
to a person residing in that city. By fast payments, I mean payments that are completely validated in a time frame that does not depend on the total size of the coin system. In other words local trade should be possible, even if the digital cash system is used all over the world (or galaxy). The preparation before arrival is important here. Of course, many coins can not be verified, fast, against double spending by the payee, but there should exist some coins that can, given the location. And the payer can prepare before arrival by getting such coins.
6. No issuer on whose honesty the system relies.   


1-5 is possible with an issuer. The coins have a tag that denotes where double spending will be verified. The issuer keeps the information about those coins on their servers in that location. In other words, the issuer's distributed hash table follows some rules for what coins are stored where. And those rules are publicly available.

1,2,3,4,6 seems to be bitcoin if I understand correctly.

Does 1-6 exist. Is that what red is saying?

[NewLibertyStandard]

In real life, bank wires take much much longer than ten minutes. Most traditional services which you see as instantaneous are actually not instantaneous. Credit card charges don't go through immediately, the funds are just reserved immediately and the retailer trusts the credit card company. Not to mention that charges can be challenged, which means the final transaction could take weeks. Real bitcoin transactions occur much more quickly than real traditional electronic money transfers. If you need instant transfers, then you just have to set up the same tricks which the traditional electronic currency market uses, and that is basically what I suggested.

[MoonShadow]

creighto, I am only talking about verifying the absence of double spending. Of course, you can always receive a coin and just hope. That is clear.

There is more to it than just hoping.  For starters, you could already have an existing business relationship, and trust built between two parties due to honest prior trading. 

I don't think that physical coins have this issue. Or gold. Firstly counterfeiting is really difficult, secondly "bill verifiers or gold verifiers" are local machines. If someone could
counterfeit gold or coins in a way that no local machine could detect, then it would be the same. But no one can do that.

Counterfeiting bitcoins is really difficult as well, and that is what a local blockchain check would protect you against.  It would also protect you against a double spend that wasn't concurrent, as if one guy backs up his wallet file and then spends his coins the night before and attempts to yank your chain.  It does not protect you against a professional criminal, but against the petty attempt.


[/quote]
With bitcoin in the version I have seen (not red's), you cannot build a local machine that verifies it.
[/quote]


I know from other threads that Red has a different understanding than I about how the system works.  Either or both of us could be wrong.  My understanding is that, at present, the clients attempt to announce a transfer and then wait for the collective to confirm the transfer via accepting the claim as valid in at least two blocks, after which point the network believes that you have valid bitcoins, therefore you do.

However, two clients can interact directly, in theory.

Imagine this picture...

You have a full client on your Iphone running in the background, and then there is a power failure.  You head down to the corner store, and find that the shopkeeper has put everything in the cooler on sale half price, cash or bitcoins only.  Your cellphone client connects with the shopkeeper's cell phone client over ad-hoc bluetooth.  Signs the transfer accouncement (there are no actual cryptocoins in your wallet, they exist only as a series of entries into an encrypted ledger we call the blockchain, more like writing a check than actual coins) over to the shopkeeper's address.  Shopkeeper's client can then (but does not have to) check his copy of the blockchain to verify that you actually owned said bitcoins at the time of his last blockchain update.  If it's good locally, he can assume that you are not trying to cheat him and accept the trade and you leave with your half priced milk.  This does not protect the shopkeeper from an intentional double spend, but you still had to have honestly owned the coins at one time in order to do this.  If you did not own the coins at the time the power went out, his client would have rejected the transfer.

This also means that the shopkeeper can only spend bitcoins that he had at the time of the power failure himself, since any new bitcoins that he receives will not yet be included in the blockchains of any other vendor, and any attempts by his client to transfer his unannounced coins will be rejected by the other clients.  All transfers will either be verified or rejected by the network within 30 minutes of network reconnection.