what can Bitcoin learn from high frequency trading regarding the block size?

[Christian Kammler]

I was just asked at the Bitcoin Foundation forum "What is your plan regarding the question of the block size limit?" and "What option do you support, if any?"

I would like to share my answer in this forum, too.


Never change a running system and I agree, that currently there is no need to change the block size (http://bit.ly/17anoxZ).

If this threshold is reached, this will lead to innovations like off-chain transactions or higher fees. And this is highly recommended, because Bitcoin isn't that anonymous many people do think off (http://bit.ly/1cJWkYx, http://bit.ly/BTCFI).

But there is another good argument to keep the block size low, beside of wasting disk space.

In 2010, Spread Networks completed construction of a new high-speed fiber optic cable connecting financial markets in New York and Chicago. Whereas previous connections between the two financial centers zigzagged along railroad tracks, around mountains, etc., Spread Networks’ cable was dug in a nearly straight line. Construction costs were estimated at $300 million. The result of this investment? Round-trip communication time between New York and Chicago was reduced . . . from 16 milliseconds to 13 milliseconds. 3 milliseconds may not seem like much, especially relative to the speed at which fundamental information about companies and the economy evolves. (The blink of a human eye lasts 400 millisec- onds; reading this parenthetical took roughly 3000 milliseconds). But industry observers remarked that 3 milliseconds is an “eternity” to high-frequency trading (HFT) firms, and that “anybody pinging both markets has to be on this line, or they’re dead.” (http://bit.ly/HFQALGT)

This battle of arms is going to happen between the miners, too. I am following the argumentation of Peter Todd at the bitcointalk.org forum (http://bit.ly/14tBOWL) which is summed up for non IT people here: http://keepbitcoinfree.org.

Larger block sizes can ultimately lead to centralization.

[Come-from-Beyond]

Larger block sizes can ultimately lead to centralization.

Off-chain transactions can ultimately lead to centralization too. It seems the best solution somewhere in the middle.

Btw, higher fees is not an innovation and they don't solve the problem "how to put a million transactions into one block".

[Cryddit]

I have thought about this. 

One answer is don't do high frequency trading.  Is just parasites anyway.  Money 'invested' in a company for fraction of second not do company any good, not contribute wisdom to board, not vote stock even once.  Useless. 

Simply let bids, asks, all limit orders, accumulate and, when low enough hash generated, bam, all trade at once, same price.  One transaction, but big.  Algorithm pick price where greatest amount trade can happen, where bid line cross ask line.  If everybody can see bids & asks, even post facto, can verify own order is present and can verify price-pick algorithm did what supposed to.  Block chain expand by one transaction only.  Transaction has many txins and txouts.

Aim for hash periods of a minute or so.  Variability means nobody know exactly when transaction close.  High volume per round Inhibit volatility caused by high-frequency trading, means larger orders can go through without much affect on market.  No high frequency traders, no wild swings too fast for human to react, fair prices, nobody get cheated by getting fraction of second wrong, everybody trade same price each round.  No effect on real investor except prevent parasite sucking investor money away on trading system.  Price move only in jumps at intervals poisson distribution average around one minute.

[Gavin Andresen]

There was an interesting research paper (which I can't find) that claimed that high frequency trading is largely driven by the artifically high limit on bid/ask spreads. The claim is that if trading was allowed in much smaller fractions of a dollar traders would compete by underbidding/overoffering instead of competing to be first in the trading queue.

That seems plausible to me-- set an arbitrary constraint ("bids must be in multiples of a tenth of a penny" -- or whatever it is these days) and you often get unintended consequences.

I still plan on writing up why I disagree with the idea that a larger block size will lead to centralization, but I'm working on some higher priority things first. I really don't understand why you would think that our current, arbitrary 1MB limit is the Best Size For All Time. I think that arbitrary constraint will have bad unintended consequences.

[markm]

Well with luck, for now, satoshis might be a small enough denomination?

If not then maybe people will bid in DeVCoins, or even more than eight decimals could be used, living in players' trading accounts until they accumulate up to at least a whole satoshi.

I think a lot of people accept that eventually blocks larger than one megabyte will end up being allowed, but hopefully not until upgrading their hardware and bandwidth and so on to accomodate a larger size has been profitably accomplished.

(So those who already accomplished it might try to rush things along, but those still trying to accomplish the move to ASICs might prefer to wait for the whole move-to-ASICs wave of upgrades to be basically pretty much over before starting to budget a move-to-larger-blocks wave of upgrades.)

-MarkM-

[solex]

High-frequency trading is a parasitical activity which harms markets in multiple ways. Zerohedge have written many articles detailing exactly why.

The simplest solution is regulatory, which is that all exchange orders should remain active for at least 1 or 2 seconds so that a human trader can react and execute. Instead the regulators permit HFT as it benefits all the major banks who excel at it. When regulators moan about lack of regulation over Bitcoin they ignore the elephant in the front room which is that HFT is making a wild west of the equities and futures markets.

The Bitcoin block size limit is only indirectly analogous as yes, there is a technological arms-race in progress, but this is for hashing speed, and block size is a secondary consideration for miners. Non-technical people need to be aware that the current limit is just too small for Bitcoin to do anything except remain a niche currency/payments system. Decentralization is extremely important, so it is necessary to find the optimum block size which allows for growth as well as a healthy network. This optimum will move with time as bandwidth and software optimizations improve.

[btcmind]

There are many myths surrounding HF trading. If you impose a time limit the market is going to be terribly inefficient. Actually the amount of volume an exchange is a direct function of time, which should be easy to see I hope. I'm amazed that anyone sympathetic to bitcoin would condemn computer trading. Having said that, there are probably better good rules one can come up in a bitcoin driven system. Btw, the exchange outages of NASDAQ have to do with the fact, that they use people for trading.

[markm]

I guess there are various different kinds of efficiency.

Does HFT prevent bubbles, by making it so clear so fast that a bubble is in progress that, faster than humans could react, the computers have figured out it is a bubble and taken appropriate steps to discover some kind of actual or realistic price as distinct from some kind of bubble/illusion/runaway-algorithm price?

I have long been mulling over the idea of using batches, like whole minutes or whole block-periods or whole hours or quarter hours or half days or whole days or whatever size/duration batch might be convenient or useful. (Maybe X number of bids/offers even instead of X amount of time), in order to be able to compute an average price per unit of whatever is being bid on and offered on that market. Part of the objective of such an idea is to make more-efficient use of bandwidth, but I am wondering if maybe it might also be more resistant to feedback-loop bubbles, especially ones that run out of control rapidly.

The batches would be in effect some kind of auction, I think.

Maybe in effect the trading house would be saying "this is the average or mean or mode (or somesuch) price, based on the batch of bids and offers we have on hand right now" and presumably clearing any bids/offers that match it.

The idea evolved from what players in certain games actually wanted, which was to set themselves up as "trading houses" that would buy the cheapest offers to sell to the highest bids, taking the difference as their operating budget and (they hoped very lucrative) profits.

Presumably it might be more efficient than such "robber baron" trading houses, and competition between trading houses might lead to increases in efficiency of markets.

If traders need expensive data pipelines driven through expensive parts of town, surely the costs of all that must get passed on to customers? Don't such costs count as lowered efficiency of the transport-and-trade network whereby things get delivered to end-users?

Is it efficient, really, to saddle the end-users of soybeans or APPL shares or any other arbitrary commodity with ridiculous infrastructure costs that serve maybe mostly to fritter away value in a maybe even less useful (to the end users who consume the commodities or who consume the products produced by the corporations whose shares are being traded) way than hashing fritters away resources in the bitcoin network?

-MarkM-

[btcmind]

With regards to the point you making about costs. Spreads have come way down in the last 10-20 years due to computers. I mean the whole point is that computers are more efficient than people in some regard. The bandwith HF trader is really not that much compare to the volume they trade. And they pay for the bandwith.

Sure, batches are interupt periods and standard in good exchange markets. You can even trigger them based on volatility measures.

The bitcoin network is absolutely not the same as an exchange. There is a lot of potential in this direction, but in quite different ways. You want transactions to be very, very fast. For example in the first of trading this morning the SPY ETF was traded in cumulative volume of 600M$. That's one product in one hour. Total stock market volume is in the trillions of dollars. You need a lot of speed and efficiency to trade that, and that's what (computer) traders provide. At the point almost all short-term trading is done by computers and there is very good reason for that. Markets work pretty well compare to other institutions (banks, states).

[Cryddit]

Cumulative volume $6e8 in one hour, yes, but how much is actual investment?  Take number of people intend to hold stock even one voting period, even one dividend period, subtract all others, what left?  One one hundredth the bandwidth, one tenth the volume, maybe. 

Fractional-second traders are just parasites living off investor money.  Bad for investors, bad for business. 

Stupid trading model anyway, have "bid ask spread."  Just support more parasites.  Go to batches, bid ask spread is zero.  Everybody trade same price each batch.  Investors pure P2P, no need to support "exchange" or "brokerage" or anything else.  No symbiotic role anymore for such.  No need for bid ask spread to support them, no need for market supports for buggy whip makers either. 

Fuck obsolete inefficient trading model.  Fuck parasites living off inefficiency.  Time for new answer.

[markm]

Wait, no, not the buggy-whip makers! How are we going to have fully detailed episodes of Sherlock without buggy whips?

Is this why recent Sherlock series' are set in a time when buggies are no longer in use?!?

-MarkM-

[btcmind]

Edward, I have no idea what you are talking about. How can you have a market without orders - make no sense whatsoever.

[Cryddit]

I'm not talking about "no orders."  I'm talking about "no bid ask spread."

Let people make bids and offers while market is open.  All these are limit orders.  Then, at end of 'round', market closes for an instant, algorithm finds fair price at which greatest amount can be traded, and executes all orders allowing fair price, all at same price.  Then market opens again for next period. 

'Continuous auction' as traditional stock trading maximizes number of trades, earns much commissions for brokers and trading houses.  Efficient in time only to maximum speed of market's ability to process orders and inefficient in price.  So called high volume trading is just gaming time efficiency against price inefficiency for profit.  'Continuous auction' also maintains existence of bid ask spread, which allows market makers and brokerages to live off price inefficiency.  But price inefficiency is investors money.  Investors would rather keep it.  If investors are P2P, brokers and market makers are not needed, so no reason investors can't keep their money.

Different model here, call it 'periodic fair pricing' or something, allows existence of bids greater than some asks until market period ends but then executes all bids and asks at same price.  Some time inefficiency, but absolutely no price inefficiency.  No bid ask spread, one massive transaction per trading period.  Each period, all orders execute at same price, so no price inefficiency to exploit.  Also no price inefficiency to support profits for brokers, market makers, brokerages, or fractional-second traders  -- all are unnecessary in new model. First few because nothing for them to do anymore, last because never had any useful role to start with.  If close of trading period is unpredictable (when block closes on 'market' subchain because hash is found)  also no way to exploit time inefficiency by placing massive orders in last fractional second before period close, either.

I hope it is not considered rude to post code here, but here is C code implements pricing algorithm.  I encourage you to play with it and see how it works.

/*  This code implements a "fair market" algorithm for trading.
    Instead of matching individual buy/sell orders for a particular
    issue one at a time as they impinge on a bid/ask spread, the "fair
    market" algorithm consolidates all buy and sell orders for an
    issue over some interval and then finds a single price which
    maximizes the number of shares traded.  For that interval, ALL
    purchases/sales which can be made are made at that price.

    Using a fair market algorithm, it is possible to proceed on the
    basis that the market opens every hour (or every ten minutes, or
    whatever), determines a "best price" for all buy/sell orders
    received during that hour, executes all trades that can be
    executed at that price, and immediately closes again.

*/

#include<malloc.h>
#include<stdlib.h>

struct order{
  int minprice;
  int maxprice;
  int amount; 
  int buyorsell;
};
#define SELL 1
#define BUY 0

struct sortrec{
  int keyprice;
  struct order *ref;  
};

int sortrecorder(const void *srpt1, const void *srpt2 ){
  return(((struct sortrec*)srpt1)->keyprice - 
         ((struct sortrec*)srpt2)->keyprice);
}


/* Sets *validresult to 0 for a malloc error or when finding that
   there is no price at which trades can be made.  Returns -1 for 
   a malloc error and 0 for  no-acceptable-price.  

   Otherwise accepts a vector of orders and the length of that vector,
   and returns the best price (ie, the price at which the greatest
   amount of the issue can be traded) for that vector.  If there is a
   range of such prices, it returns the highest. */

int findprice(struct order *orders, int ordercount, int *validresult){

  struct sortrec *ordervec;
  struct order* item;
  int index;

  *validresult = 0;
  /* if there are zero orders, or one order, then there cannot be any
     price at which a valid trade can be made. */
  if (ordercount <= 1)
    return(0);

  sortrec = calloc(ordercount * 2, sizeof(struct sortrec));

  if (ordervec == NULL) 
    return (-1);

  for (index = 0; index < ordercount; index++){
    ordervec[index].keyprice = orders[index].minprice;
    ordervec[index + ordercount].keyprice = orders[index].maxprice;
    ordervec[index].ref = &(orders[index]);
    ordervec[index + ordercount].ref = &(orders[index]);
  }

  qsort((void *)ordervec,  2 * ordercount, sizeof(struct sortrec), sortrecorder);

  int loindex;
  int hiindex = -1;
  int buys;
  int sells;
  int trades;
  int pricechanged;

  int vlength = 2 * ordercount;
  int lastprice;
  int bestprice = lastprice;
  int lastindex = -1;
  int besttrades = 0;
  int priceguess = ordervec[0].keyprice -1;

  while (hiindex < vlength) {
    pricechanged = 1;
    if (lastprice + 1 == ordervec[hiindex+1].keyprice ){
      lastprice = priceguess;
      loindex = hiindex+1;
      priceguess = ordervec[loindex].keyprice;
      for (hiindex = loindex; 
           hiindex < vlength && ordervec[hiindex].keyprice == priceguess;
           hiindex++);
    }
    else if (hiindex + 1 < vlength){
      lastprice = priceguess;
      priceguess = ordervec[hiindex+1].keyprice -1;
    }
    else pricechanged = 0;

    if (pricechanged){
      for (index = loindex; index <= hiindex; index++){
        item = ordervec[index].ref;
        if (item->buyorsell == BUY){
          if (item->minprice == priceguess)
            buys += item->amount;
          else if ( priceguess > item->maxprice 
                    && lastprice <= item->maxprice)
            buys -= item->amount;
        }
        else{
          if (item->minprice == priceguess)
            sells += item->amount;
          else if (priceguess > item->maxprice
                   && lastprice <= item->maxprice)
            sells -= item->amount;      
        }
      }
      trades = buys > sells ? sells : buys;
      if (trades >= besttrades){
        bestprice = priceguess;
        besttrades = trades;
      }
    }
  }
  if (besttrades == 0) 
    /* no trades can be made at any one price. */ 
    return(0);
  *validresult = 1;
  return(bestprice);
}

[markm]

Yes, that is what I meant by batches. Thanks for the code.

But, after clearing all the orders possible using that approach, the residue, the trades that cannot happen at that "fair price", do still exhibit a bid/ask spread presumably.

-MarkM-

[nomailing]

Interesting discussion here. I completely agree with Edward. HFT just serves the brokers and trading houses. For real investors it is not important that their trade executes within seconds. They could wait for the end of a round (i.e. until a new blockchain block is found), which will match all trades. This will give all real investors an effective spread of 0. This will serve the businesses and investors and not some parasites in between.

But, after clearing all the orders possible using that approach, the residue, the trades that cannot happen at that "fair price", do still exhibit a bid/ask spread presumably.

This wouldn't be a spread. They just placed orders which nobody wants to fill.
If AAPL is worth 500 USD and I place an order at NYSE to buy AAPL for 450 USD, then the order is not filled. Would you then say that the spread is 50 USD, just because there is no counter party??

With regards to the point you making about costs. Spreads have come way down in the last 10-20 years due to computers. I mean the whole point is that computers are more efficient than people in some regard. The bandwith HF trader is really not that much compare to the volume they trade. And they pay for the bandwith.

As said, the spread would be zero in a round based system. All this talk, about the benefits of HFT is just a baseless rumor that is spread by large banks, because it is a way for them to make money.
If you are talking about the efficiency of computer trading you should distinguish between computer algorithms that detect bubbles and predict prices (which is a good thing) and HFT (parasites).

[markm]

Yes I think it would be a spread.

Couldn't you even, as, say, a broker, run a batch on your own customer's offers and bids then ship off the residue to a normal market, where one would clearly see that that residue does in effect have a "spread"?

-MarkM-

[nomailing]

Yes I think it would be a spread.

Couldn't you even, as, say, a broker, run a batch on your own customer's offers and bids then ship off the residue to a normal market, where one would clearly see that that residue does in effect have a "spread"?

-MarkM-

I am not sure if I understand your argument... I assume it depends on how you define "spread". I think in a round based system it makes only sense to define the spread at the end of the rounds and not in-between rounds, because trades are only executed at the end of the rounds.
Now per round there are two possibilities:
1) Some trades were matched: then per the definition of "spread", the spread is 0 (although there might be some unfilled trades left)
2) No orders could be matched: Then the spread would be exactly the same as in a market with HFT

Thereby, you see, that the spread will be either the same as with HFT, or smaller.
In this round based market, the orders of "real" investors are more likely to be matched to orders of other "real" investors, without any parasites that make money using some spread. So no drawbacks for "real" investors in comparison to a HFT market. The HFT market only serves some parasites that make money in between.

Of course in the situation of real world stock markets HFT reduces the spread between different exchanges by arbitrage trading and thereby reduces the costs for investors. BUT if you assume a round based market, then you can implement the market globally, such that every investor can participate directly in one global market and no party is favored by better bandwidth. Thereby you make HFT arbitrage trading obsolete.
So considering one single market (similar to one single P2P blockchain) there is no reason to believe that HFT will somehow reduce the costs for "real" investors. It would only increase the costs for real investors because they would have to pay the party in between.

[markm]

Ah maybe you are using different part of the meaning of spread than I am.

I merely meant there is a gap between the lowest ask and the highest bid. (That gap being what I thought was meant by the word spread.)

The wiki page though also goes on to say the liquidity provider gets the spread as their profit/fee for providing liquidity.

I had merely meant there is a gap, not that someone would somehow profit from the gap, albeit if someone jumps the gap by placing a market orderto buy  they are paying more than the hitherto highest bid and the seller does get paid more than if they had been the one to jump the gap by putting in a market order to sell.

-MarkM-

[Cryddit]

Only definition of 'spread' worthwhile to me is difference in price between buying and selling. If no buying and no selling, or buyer/seller not have to pay extra or take cheaper price because of crossing 'gap' in market, then no spread at all as far as I'm concerned. 

Market spreads in 'continuous auction' have come way down recently, but is due mainly to HFT getting more efficient/faster because market speed has gone up.

My point is we can do better yet. Bids & asks execute at same price means no spread at all. 

[nomailing]

Only definition of 'spread' worthwhile to me is difference in price between buying and selling. If no buying and no selling, or buyer/seller not have to pay extra or take cheaper price because of crossing 'gap' in market, then no spread at all as far as I'm concerned. 

Market spreads in 'continuous auction' have come way down recently, but is due mainly to HFT getting more efficient/faster because market speed has gone up.

My point is we can do better yet. Bids & asks execute at same price means no spread at all. 

That's an interesting perspective especially regarding all the recent interest in P2P exchanges. If these are implemented in a blockchain-type-order-book, then you would basically have a round based market.
And with this perspective in mind, you could argue that the bandwidth of most real stock exchanges is not necessary for an efficient market. You could even argue that a round based P2P exchange would be much better for investors and businesses because they would find a "fair market value" without any spread.

I always saw it as a problem that some of these P2P exchange proposals want to match orders using a blockchain. But now I think this could even be an advantage, because the market will have time to find the "fair market value" of the underlying stock.