[WHITEPAPER] Decentralized Bitcoin Prediction Markets

[AsymmetricInformation]

Prediction markets have been around for half a century.  They are known to work.

Year 1907:

"A weight-judging competition was carried on at the annual show of the West of England Fat Stock and Poultry Exhibition recently held at Plymouth, A fat ox having been selected, competitors bought stamped and numbered cards, for 6d. each, on which to inscribe their respective names, addresses, and estimates of what the ox would weigh after it had been slaughtered and " dressed." Those who guessed most successfully received prizes. About 8oo tickets were issued, which were kindly lent me for examination after they had fulfilled their immediate purpose." (emphasis added)

Year 1651:

We need only revive and embellish a suggestion made back during the utopian scientific revolution. Chemical physicians, excluded by the standard physicians from teaching in the British schools, repeatedly offered challenges like the following (circa 1651):

"Oh ye Schooles. ... Let us take out of the hospitals, out of the Camps, or from elsewhere, 200, or 500 poor People, that have Fevers, Pleurisies, etc. Let us divide them into halfes, let us cast lots, that one halfe of them may fall to my share, and the other to yours; ... we shall see how many Funerals both of us shall have: But let the reward of the contention or wager, be 300 Florens, deposited on both sides: Here your business is decided." (emphasis added)
-Debus, A. (1970) Science and Education in the Seventeenth Century, MacDonald, London.

But my feeling is that we do not need to rely on experience to assert that "PMs work". I believe that, actually, it is a logical requirement that PMs are the optimal predictive institution. I wonder if you'll read my draft "PM_Misunderstandings.pdf" in the /docs folder. I'm not completely happy with the clarity of that document so you might be able to offer suggestions (in addition to finding it interesting).

[sportscliche]

OK Paul, if this is directed at me I will take a look at your writeup.  Understand that I'm not a finance/economics professional.  I'm in the physical sciences. 

I do want to comment on the valuable guidance that can be provided by experience.  I accept your claim that predictive markets should work based on pure logic.  But it doesn't hurt to have that logic backed by some experience -- you've done this by citing examples of predictive markets that are more than a century old!

Over the years of reviewing grant proposals and refereeing papers, I've learned to recognize some red flags.  One of them is: "This idea is simple enough that somebody should have thought of it and tried it by now."  It's extraordinarily rare to encounter a situation where a lot of smart people are pondering a problem and have something straightforward get overlooked.  This is especially true in business and finance, where fortunes -- not just scientific prestige -- are at stake.

Innovators will often attempt to make an extension of an established, working concept.  This is how I perceive Bitshares.  I don't need to understand the nuances of predictive markets, but if I accept that they work in the classic (binary) formulation it seems obvious to look for extensions and extrapolations.  Relax a parameter or two.  An exchange that doesn't require price feeds or deliverables is easier to implement than one that does.  Such an exchange provides advantages that should have made it spectacularly successful even if run by a central authority.  Experience tells me that since nothing like this already exists, the likelihood of it working on a decentralized blockchain is, well, open to speculation.

[AsymmetricInformation]

OK Paul, if this is directed at me I will take a look at your writeup.

Yes, PM_Misunderstandings.pdf (not the whitepaper), where I tackle the effectiveness of PMs head on. I kind of rushed it and the formatting isn't great but let me know. Its very short but there I believe PMs are extraordinarily misunderstood. That paper was written for a general audience, specifically people who have heard of PMs but aren't yet converts.

I do want to comment on the valuable guidance that can be provided by experience.  I accept your claim that predictive markets should work based on pure logic.  But it doesn't hurt to have that logic backed by some experience

Of course you are right. All theories should survive the gauntlet of experimentation; this is the insight of the scientific revolution. I am trying push the envelope.

Over the years of reviewing grant proposals and refereeing papers, I've learned to recognize some red flags.  One of them is: "This idea is simple enough that somebody should have thought of it and tried it by now."

That is true, but people have been using bets to argue for centuries, as I just pointed out. Today, several people have standing challenges with no odds against, such as James Randi's prize (essentially a price floor at 100%). This seems to be persuasive.
Mitt Romney, barely a year ago, wagered 10 grand on live international television to rebut a comment from Rick Perry. This was less persuasive. Why might that be? If you aren't aware of this post, I found it so inspiring I committed the list to memory: http://www.overcomingbias.com/2013/07/why-do-bets-look-bad.html

[elux]

Enormously interesting. Top of my reading list.

[AsymmetricInformation]

This event was brought to my attention: http://blog.predictious.com/2014/03/01/we-wont-be-adding-a-contract-on-the-assassination-of-mt-gox-ceo/

This so-called contract is a flavor of "Public Good", which is called a Public Bad by those (such as myself) who feel society would be worse off if the contract helped enable the activity it describes (as crimes of all kind are an attack on the public).

As an aside, the version proposed on that website presumably did not include the LMSR or Schelling-Partition features that I describe to ensure that the provider of the good would actually be the unique recipient of the required money.

Of course the Truthcoin version could-and-would have those things, but I've thought about this in the past and never considered it to be realistic problem for the following reasons:

1] Firstly, the branches can claim in advance that they will vote .5 on anything they find to be violent or immoral, meaning the market fails to resolve into a State, and anyone who paid to list the Decision/Market gets less (probably zero) money. The branch has effectively 'specialized' into 'non-violent markets'.

2] Assuming a crazy branch, the contract needs to specify a time horizon "...killed before 2015". Too long (10 years) and the assassin will go unpaid for an inconvenient amount of time, and too short and the target can bet on surviving and "outlast" the time period, becoming richer. Incidentally anyone on a 'crazy branch' (Owners, Authors, Traders) would almost certainly be tracked by the NSA/FBI/Military and would (and should) be prosecuted for breaking numerous existing laws and disturbing the peace of society.

3] If those ideas don't appeal to you, you can simply bet on your own death, then fake your own death. It's inconvenient (and possibly traumatic to friends/family), but you get paid pretty well (money from your enemies, no less) and it beats dying. Modern law enforcement will even help you do this (so I read).

4] I have even more ideas, involving life insurance, buying up the branch (easier than it sounds), counter-exploitation of crazy branch, etc. Many roads lead to failure.

I don't really believe that people want this man dead, its normal to say (even believe) extreme things when you are angry. The site probably thought it would make "good press", as it is quite dramatic. I just don't see PMs as encouraging this behavior, as most murders are for extremely personal reasons (sexual jealousy, religion, political extremism) and not economic ones (rich people have way more to lose by going to prison).

PMs are new, and people don't understand them, which leads to fear.

Other thoughts: http://www.sirc.org/articles/policy_analysis.shtml

[AsymmetricInformation]

Enormously interesting. Top of my reading list.

Cool, please let me know if you have comments/questions.

[sportscliche]

OK, I took a look at the 'Misunderstandings' essay and have the following comments:

-- What is meant by a meta-tool?  This may mean different things in different contexts, so it would probably be a good idea to define it here.  Is it understood to be an amalgamation of all the available tools?

--  I don't think the amoeba analogy is necessary.  I believe what you are trying to say is that a prediction market tends to weight more accurate sources of information (what you call tools) higher than less reliable ones.

-- Myth 2 appears to be debunking the idea that PMs are inaccurate.  Fig 4 seems to show that it reflects truth better than a naive expectation.  But is a graphic needed here?  Aren't you simply saying that PMs work?

-- I can't understand what Fig 5 is conveying or the myth it is supposed to debunk.  It seems to be trying to explain whether or not an individual with certain characteristics (expert, student, loser, fearful) will take action.  While this may be an attempt to classify behaviour as it relates to certain traits, I don't see how this has any immediate consequences or implications for a PM.  I also don't understand what is meant by 'Talks'.

[AsymmetricInformation]

OK, I took a look at the 'Misunderstandings' essay and have the following comments:

-- What is meant by a meta-tool?  This may mean different things in different contexts, so it would probably be a good idea to define it here.  Is it understood to be an amalgamation of all the available tools?

--  I don't think the amoeba analogy is necessary.  I believe what you are trying to say is that a prediction market tends to weight more accurate sources of information (what you call tools) higher than less reliable ones.

-- Myth 2 appears to be debunking the idea that PMs are inaccurate.  Fig 4 seems to show that it reflects truth better than a naive expectation.  But is a graphic needed here?  Aren't you simply saying that PMs work?

-- I can't understand what Fig 5 is conveying or the myth it is supposed to debunk.  It seems to be trying to explain whether or not an individual with certain characteristics (expert, student, loser, fearful) will take action.  While this may be an attempt to classify behaviour as it relates to certain traits, I don't see how this has any immediate consequences or implications for a PM.  I also don't understand what is meant by 'Talks'.

Ok, thanks. This helps a lot.

I know I rushed that essay so its very help to see its weakest parts from a new perspective (many of your questions are actually not where I expected the weak points to be).

To answer your questions: yes meta-tool is a tool which "works on" (uses) a set of tools (so the PM 'includes' all of the predictive tools we use today). Amoeba is necessary I think, because if a new method is proposed it is 'absorbed' by the PM (I want to distinguish the PM (which is the blob) from the individual knowledge-sources (circles)). I'm surprised this didn't work, I think I will double that figure for two different questions to show the flexibility concept I was after. Fig 4 is attempting to indicate the proper interpretation of a PM price (the 'naive expectation' is someone's interpretation of a PM, for example : http://blog.foreignpolicy.com/posts/2012/06/28/can_we_stop_paying_attention_to_intrade_now , where the author apparently feels that 73% is magically 100%, but I thought that was clear so it is actually very helpful to learn that you did not take away that message).

I knew Fig 5 didn't really work when I made it. Basically it concerns people who "talk", or complain about, the current price. If these people truly disagreed (had better knowledge) they could trade on this information, but I argue that instead these individuals lack awareness of their own un-knowledge. This part will need a tune up.

The first Myth was directly to address that conversation we had about why exactly PMs work so well (because they integrate everything that people believe to work), and if we can continue to expect superior performance from PMs.

[gwern]

It might be better to use more standard terms... You can see a prediction market as a ensemble method https://en.wikipedia.org/wiki/Ensemble_learning which weights experts (users) by their cumulative score (money) on a series of predictions using a proper scoring rule of some sort. This interpretation has been explored by a number of papers: http://scholar.google.com/scholar?q=%22prediction%20market%22%20ensemble

[AsymmetricInformation]

It might be better to use more standard terms... You can see a prediction market as a ensemble method https://en.wikipedia.org/wiki/Ensemble_learning which weights experts (users) by their cumulative score (money) on a series of predictions using a proper scoring rule of some sort. This interpretation has been explored by a number of papers: http://scholar.google.com/scholar?q=%22prediction%20market%22%20ensemble

That's a good idea. I'm intrigued by your link because although the content is very familiar to me I've never heard that term before. But clearly from your scholar search the fault is mine.

[clout]

Are you Robert Shiller?

[AsymmetricInformation]

Are you Robert Shiller?

Nope. He's a True Empiricist and pretty cool guy though. Why do you ask?

(I signed my name "Paul", but of course if I were lying about that I wouldn't change my answer just because you've asked a second time).

[AsymmetricInformation]

I've made a number of updates to the paper for clarity and accuracy. In particular, the Implementation Details section is much better and actually informs developers on how to implement the protocol (thanks to the individuals who wrote in to offer their comments).

I also "finished" the python version of the code for the consensus vote and trading, although there are almost certainly many bugs. The R code is more stable. All code is intended as proof-of-concept.

[tonyk]

Read the white paper! (several times) 
Great Idea! Well almost…

Let me get some things straight.
So the owners/voters try to guess what the others vote will be, and not what the actual outcome would be! Their incentive is vote with the majority not to guess correctly! If they want to gain by correctly predicting the outcome they must become traders! I.e. bet correctly not vote for the correct outcome.

The consequence of this is that their voting (as a whole, as a group) is completely unrelated to how well the market their supposed to care about performs. If they (as a majority) always get the prediction correctly or they are always wrong the actual (trading) market will perform as it wants- i.e. successfully or unsuccessfully based on totally unrelated factors (advertising, competition, fees etc.) factors pertaining to the trading market itself.
Which leads to the logical question – Why they are needed at all?
I have more points but let’s see if you will answer  this one first.

[gwern]

Read the white paper! (several times) 
Great Idea! Well almost…

Let me get some things straight.
So the owners/voters try to guess what the others vote will be, and not what the actual outcome would be! Their incentive is vote with the majority not to guess correctly! If they want to gain by correctly predicting the outcome they must become traders! I.e. bet correctly not vote for the correct outcome.

That seems like a severe oversimplification, akin to describing the stock market as nothing but 'buying based on how you think others will buy'. Yes, there's elements of a Keynesian beauty contest, but the fundamentals still exist: dividends are paid out or not, companies go bankrupt or not, and any traders who are totally unmoored from reality will discover that the hard way.

The easiest way to predict the resolution the majority will choose is to simply look at what the reality is. *Was* Obama elected? *Did* Putin invade Russia? etc. The truth is the Schelling point for all voters; how do conspirators know which way to vote on what contracts? Their communication will be unreliable and harder than the truthful voters, who merely have to look at a data source to decide; combined with the incentive for each of them to defect and screw over their co-conspirators, this produces a fundamental bias towards the majority voting for the true outcome, which produces a fundamental that voters must acknowledge or lose money, which anchors the prediction markets & prevents them from spinning off into navel-gazing.

Just like Bitcoin: if you can trust the majority of hash power (vote power), you're fine. If you can't, you're not.

[tonyk]

That seems like a severe oversimplification, akin to describing the stock market as nothing but 'buying based on how you think others will buy'. Yes, there's elements of a Keynesian beauty contest, but the fundamentals still exist: dividends are paid out or not, companies go bankrupt or not, and any traders who are totally unmoored from reality will discover that the hard way.

The easiest way to predict the resolution the majority will choose is to simply look at what the reality is. *Was* Obama elected? *Did* Putin invade Russia? etc. The truth is the Schelling point for all voters; how do conspirators know which way to vote on what contracts? Their communication will be unreliable and harder than the truthful voters, who merely have to look at a data source to decide; combined with the incentive for each of them to defect and screw over their co-conspirators, this produces a fundamental bias towards the majority voting for the true outcome, which produces a fundamental that voters must acknowledge or lose money, which anchors the prediction markets & prevents them from spinning off into navel-gazing.

Just like Bitcoin: if you can trust the majority of hash power (vote power), you're fine. If you can't, you're not.

BUT  this is not the issue I am talking about here.
The Truthcoin layer is utterly unnecessary for the PM! The prediction market (PM) exists in its own layer, following its own rules and inputs… The TRC layer is added below it only to take 50% of the trading fee. TRC does not provide backing or input feed to the PM layer.(or any other usefulness to the PM layer that I can think of) It exists to serve its own existence and follows its own rules for shares redistribution -rules that decide who will get more of the next round commissions from the PM’s trades…
Let me put in different way being right or wrong how the others will vote determines how much you will get from the next trading fees on the PM. I still do not get what utility of truthcoin entitles it (its holders) to those fees in the first place.

[gwern]

BUT  this is not the issue I am talking about here.
The Truthcoin layer is utterly unnecessary for the PM! The prediction market (PM) exists in its own layer, following its own rules and inputs… The TRC layer is added below it only to take 50% of the trading fee. TRC does not provide backing or input feed to the PM layer.(or any other usefulness to the PM layer that I can think of) It exists to serve its own existence and follows its own rules for shares redistribution -rules that decide who will get more of the next round commissions from the PM’s trades…
Let me put in different way being right or wrong how the others will vote determines how much you will get from the next trading fees on the PM. I still do not get what utility of truthcoin entitles it (its holders) to those fees in the first place.

I don't understand what you're trying to say. Let me try to explain the chain of reasoning and maybe you can specify where you object.

Prediction markets are obviously useful, so that's not your problem.

Centralized prediction markets are useful, but not as useful as they could be - since just like all centralized services, they can be attacked, self-regulate into uselessness, or corrupt: IEM survives in the USA only by being useless, and Intrade illustrate both the weakness to attack (by the CFTC and US government in general) and corruption (embezzlement by employees drove it into insolvency).

Hence, a decentralized prediction market, where embezzlement is impossible, where USG cannot regulate it to death, is highly desirable and likely necessary if we are ever going to get very large liquid prediction markets with contracts on many things of importance to the world.

Now, it's relatively obvious how to implement most of a decentralized prediction market on a blockchain: the currency part is like Bitcoin, the contracts and purchases are doable with public keys. The one part which is not obvious is: how are contracts judged? This is very important since it's where the rubber hits the road and the prediction market is forced to mirror the real world and keep being anchored to facts. If the judging isn't done well, the prediction market will be completely useless.

You could have one person appointed to judge. They sign a message etc. But now you've reintroduced centralization and allowed for corruption or coercion: the judge can simply buy up a bunch of shares on the losing side, which will be cheap because they're going to lose, and judge the opposite of truth, and make a ton of money. If they've become trusted enough to judge many contracts, they get a nice big 'exit scam'. As the black-markets show with centralized escrow, this can sort of work, but it still isn't great, and it limits the growth of prediction markets: no one is going to invest, say, $10m of capital if it can all be stolen by one judge pulling an exit scam.

This sort of partially-decentralized prediction market might work, but it's not a very compelling vision and that may be part of why no one has done it. If you're willing to trust the judge not to scam you, you might as well go use Bets of Bitcoin or another Bitcoin betting service: they'll scam you at some point, but at least it'll be a conveniently-done scam.

How could you improve on this? Well, what if there were some way to 'spread out' judging across a lot of people? And incentivize them to judge honestly? And to defect against any conspiracy to manipulate votes? *That* is what Truthcoin tries to do. And that's the value that the SVD part of Truthcoin delivers: it removes the last and large part of centralization, producing a prediction market system you can genuinely rely on and not worry about being attacked by the government, embezzling employees, corrupt judges, etc.

[tonyk]

I understood my mistake.
 Thanks gwern!

[fairlay]

Great thread for all PM enthusiasts (like us www.fairlay.com)!

First of all I wan't to point out that even if we run a centralized PM right now we believe in a decentralized future.

We see our core tasks in:

• Creating and promoting interesting and well defined events
• being a place to discuss events
• Providing a convenient user interface to market data and placing new predictions
• being a trusted source with a good reputation in resolving events
  (In the decentralized version that would be a "stream")
  

All this task are required in a decentralized system as well.

At the moment as a centralized PM we also have the roll of a bank for storing funds.
Since we do not intend to run away with the coins this is only an additional risk for us and thus we would prefer other solutions.

Some ideas we had to at least kind of decentralize it:
1. We will publish a hash of every event description combined with the outcome "true" or "false" and sign the real outcome. Thus we can build up reputations and predictions can be placed completely independent.
2. With No. 1 bet amounts can be stored in a 2 out of 3 multi sign address. That would make it impossible for the central PM to run away with the money - just to choose the winning side. (of course this is only a little bit better, since the central PM could act as a bettor as well)
3. We though as well about finding a consensus by the user (to have less work by resolving events)
One solution would be that both sides put 10% more money in a multi-sign address. No the person that looses the bet can admit the loss and sign the transaction to the winner but he gets back the 10%. If none of both admits to be the looser we sign a transaction to the winner but take the 10% of the looser.


We think combining 1. and 2. and having multiple independent entities as described in 1. with a good track recored and betting on a consensus of these would be a good solution as well?

Or, where to you see advantages over this of Truthcoin?



Plz. allow me to answer some questions that came up in this thread (little bit off-topic):

On InTrade and predictious, you cannot create a new prediction market (although you can request one), and in fact I believe they are very slow and un-entrepeneurial in providing what people want.

Try fairlay: https://www.fairlay.com/event/new/
We promise a quick response!

I would love to see a good implementation of a prediction market in bitcoin. I think it could serve a use beyond speculation. It could be a new important media source, even an everyday app for making decisions.

That is our goal - and we are open very open for feedback.

[ThePurplePlanet]

A lot of solutions are proposed for PMs but all lack volume. Why no solution is proposed using bitcoin which has already a lot of volume and users? Will a solution like side-chains help to go in and out without liquidity concerns?