Regarding oracles there is a much simpler way to implement them that doesn't even need to require interaction with the oracle or signatures.

Suppose I'm an oracle who will attest to whether or not the Steelers won the 2014 Superbowl. I say that if they do win, I'll make public some nonce, nSteelersLose, and if they win, I'll provide another nonce, nSteelersWin. In the meantime I make public the values of H(nSteelersLose) and H(nSteelersWin) upfront. (where H(d) is some cryptographic hash function)

Now Alice can make a bet with her friend Bob as to whether or not the Steelers win. (Alice and Bob are such committed Steelers fan's that they are sure they'll make it to the playoffs) They jointly author a transaction with two inputs, one from Alice, and one from Bob, and a single output. That output has the following scriptPubKey:

HASH160 H(nSteelersWin) EQUAL
IF
    <alice-pubkey> CHECKSIG
ELSE
    HASH160 <nSteelersLose> EQUALVERIFY
    <bob-pubkey> CHECKSIG
ENDIF

If the Steelers win, the oracle will publish nSteelersWin and Alice can spend the output with the following scriptSig:

signature nSteelersWin

If the Steelers lose on the other hand, only Bob can spend the output:

signature nSteelersLose 0

What's really nice about using hashes rather than signatures is that it's scalable: there could be a million Alice's and Bob's making these bets, and the oracle doesn't have to even know. The oracle can also attest to as many different teams or conditions as required, or there could even be multiple oracles. For instance I could also bet that the Steelers will win and the Democrats win the next election and Canada is invaded by the United States:

HASH160 H(nSteelersWin) EQUAL
HASH160 H(nDemocratsWin) EQUAL
AND
HASH160 H(nCanadaInvaded) EQUAL
AND
HASH160 H(nSteelersLose) EQUAL
OR
HASH160 H(nDemocratsLose) EQUAL
OR
HASH160 H(nCanadaNotInvaded) EQUAL
OR

Of course, Alice and Bob might also want to be protected in case the oracle fails. For instance the oracle might go out of business and stop publishing. But they can modify their script so that Alice and Bob can also mutually agree to a different set of conditions at any time:

HASH160 H(nSteelersWin) EQUAL
IF
    <alice-pubkey> CHECKSIG
ELSE
    HASH160 <nSteelersLose> EQUAL
    IF
        <bob-pubkey> CHECKSIG
    ELSE
        0
    ENDIF
ENDIF
IF
    1
ELSE
    2 <alice-pubkey> <bob-pubkey> 2 CHECKMULTISIG
ENDIF

(FWIW there's lots of ways to make these scripts shorter)

They can sign a transaction in advance that is timelocked so that even if one or the other party is unavailable, if a month after the Superbowl is supposed to happen the oracle still hasn't published either one can trigger the backup condition and cancel the bet and return the money. It's even possible to handle the oracle revealing both values, either accidentally or maliciously, by setting up inputs spent by timelocked transactions to effectively cancel the bet, but that gets complex...

Finally if the oracle can be directly involved in your transaction you can simplify all of this by asking the oracle to commit to a contract. Basically the contract just outlines exactly what the oracle is attesting too, whatever that is, and the oracle promises to reveal specific nonces for specific outcomes. The oracle does not need to sign transactions and the oracle can simple publish the correct nonces in a public place when the event is triggered.

This also means that the problem of making sure the oracle is honest with challenges is irrelevant. Honest is just a matter of what values the oracle publishes and has nothing to do with transactions at all. For that matter, the oracle doesn't even have to know anything about Bitcoin at all: all the oracle does is make promises to make public certain short bits of data if certain events happen.

It's possible to create probabilistic payments in Bitcoin. The big picture idea is that you make a transaction, and 99.99% of the time it won't actually go through and the receiver doesn't get anything; 0.01% of the time it does for 1000x of the amount, enough to be useful. If the scripting language was more useful you could do this easily, but it's kinda useless right now so...

However there's a neat trick: you can prove to someone else that you tried to mine some coins. Now had you been successful, that share would have made it as a block, and the receiver would have got some Bitcoins paid out as part of the block reward. One really cool thing about that is these probabalistic payments can be anonymous - they aren't linked to any other coins.

Problem is, you need hashing power to mine. However you can pay someone else to mine for you, and to pay them you can use a trust-free micropayment channel like Mike's bitcoinj implementation. (remember that a micropayment channel only lets you combine multiple transactions to a single person into one, it doesn't let you make multiple small payments to multiple people)

Having said that, all this stuff is fairly complex, and none of it is implemented yet. Right now my advice is to use a off-chain service like inputs.io or easywallet and use their API. I haven't written any apps for either myself, but the API looks pretty simple to use. Sure they can go out of business and take the funds, but we're talking about tip sites and stuff - there's not a lot of money involved for any one person to lose. Myself I use both of those services all the time for day-to-day transactions, and Armory to keep my savings secure.

Bitcoin-QT doesn't do that unless the output is one of your own.

The OP is using a different wallet - you can tell because the keys aren't compressed.

Luke-Jr's pull-req is the completed one: https://github.com/bitcoin/bitcoin/pull/1647 My code is just taking his idea and applying it in a way that is more general and reliable. (Luke's patch is good, it's just you can extend the idea even further with P2P protocol changes so nodes can know how to retrieve parent transactions if required)

Eligius and another pool uses Luke's patch already, so re-spending your stuck transaction with a fee will help. Unfortunately I don't think any Bitcoin wallets have an easy way to do this; it'd be really good if someone writes that code. The user should be able to click on a payment, either to you or from you to someone else, and say "Increase fee" and have the wallet software take the correct steps.

Successful people are willing to cut their losses and refocus their efforts. You can always retrieve the code from revision control history later when the testing infrastructure improves; without good testing capabilities you're wasting your time.

Working on it.


FWIW I'd suggest you consider removing script evaluation from your library entirely for now. Keep the rest of the scripting code - you need it to create transactions after all - but make it clear that the only way you know that a transaction is valid is if a miner mines it. This is more in-line with the guarantees SPV clients can provide anyway, and will save you an enormous amount of work that is probably wasted until we have better testing infrastructure for third party clients to use.

We do that already. This issue with SIGHASH_SINGLE was documented in the unittests here: https://github.com/bitcoin/bitcoin/blob/master/src/test/data/tx_valid.json#L39 (I did not find it first)

The other main source of test cases is Matt Corallo's block tester: https://github.com/TheBlueMatt/test-scripts

You're welcome to go through those test cases and summarize the more subtle ones, but frankly I don't see the point: any change, no matter how small, triggers a hard fork. Perfect compliance with those tests is a necessary but far from sufficient requirement.

You may find this code interesting, specifically find_digest_in_block(): https://github.com/opentimestamps/opentimestamps-server/blob/master/otsserver/bitcoin.py#L93

Basically it first finds a specific digest in a block, checking every tx, and then rebuilds the merkle tree to extract a merkle path from that transaction to the block header.

Try writing up a quick implementation, even of a cut-down version missing most of the features you want. You need to get a sense for how much complexity you are adding to get a better idea of what the trade-offs are.

Any new opcodes are extremely risky and we're going to need to think very carefully about what we're getting ourselves into. IMO it would be a good idea to implement a really simple one first, like OP_BLOCKHEIGHT, as a soft-fork using OP_NOP2 just to understand if our process for even doing that is sufficient. Heck, ignore OP_CHECKSIG for now, do that first as an exercise and write up a detailed plan - just like Gavin wrote for the P2SH BIP - to better understand what you are getting yourself into.

Remember what I said about upgrading flying airplanes...

Right, but the point is those problems aren't unique to Bitcoin-style consensus - what is unique to Bitcoin is the possibility that you'll get a transaction that isn't valid not because there is anything wrong with that transaction, but rather because the block it's in (or a previous block) has something that caused it to be rejected by the majority hashing power.

Those are all important risks, but they're all problems that happen in non-Bitcoin contexts too.

Indeed - this is why I think the problem Bitcoin faces is harder than writing avionics software: we have no choice but to start with a buggy and poorly designed piece of software, the original Bitcoin client written by Satoshi, and maintain and improve it. The avionics analogy would be to take a multi-engine WWII-era bomber, fly and maintain it continuously 24/7 via in-flight refueling, and gradually upgrade it to be as good as a brand new Boeing 777 without ever landing once. Needless to say, starting from scratch would be much easier, but we don't have that option.

You say to know a lot about writing quality software, why don't you write a from-scratch alt-coin implementation done correctly with solid software engineering? I'm sure if you succeed it'll get adopted; in fact it's quite possible to one day collectively decide to replace Satoshi's implementation with yours in a hard-fork, just copy the UTXO set over and get miners to switch on the appointed flag day. Even if it doesn't get a lot of use it'll still be a worthwhile and groundbreaking exercise: no-one has ever written a from-scratch implementation of a crypto-coin; every alt-coin has simply copied the Satoshi codebase. The closest thing is P2Pool, but it's not as hard because if consensus fails in P2Pool all that happens is variance goes up if one side of the fork doesn't have much hashing power; no money is actually lost.

Good idea.


Yeah, nothing wrong with re-implementations if you hide them behind a trusted bitcoind node (IE one you run yourself) and/or use them for small sums of money. Just remember you'll occasionally suffer downtime when your implementation mistakenly thinks a block or transaction is invalid, but your trusted bitcoind node will protect you from false confirmations.

In addition I should point out the existence of bitcoinj and python-bitcoinlib and other libraries written by competent developers who understand the consensus problem is a good thing - Matt has done a tonne of good work on testing that greatly helped by the process of creating bitcoinj. The problem is people who don't understand the consensus problem and think they'll succeed in replacing the reference client, and there are a lot of developers in that boat, often developers who are otherwise talented and productive people.


Good idea.

SPV mode is good for this application because SPV nodes don't verify anything but the transactions you are interested in.

If bitcoinj sees a transaction that it thinks is invalid, can you use it in a way that lets other transactions from the same block be processed normally? IE the customer receiving a payment with a weird transaction would simply have to wait until someone fixes the problem, but other customers would be unaffected.

Given that the disagreements we are talking about are things like "Does Alice have a valid check signed by Bob entitling her to take $1000 of Bob's money?" any type of majority voting protocol is unacceptable - you're proposing a system that would allow a majority of miners to decide that they are going to steal your money with no recourse.

Yup.

There's probably a dozen people in the world who understand Bitcoin as well as I do. If there exist Bitcoin experts at all, those dozen people are your experts. Yet even that group still keeps on finding stuff about Bitcoin that they just didn't know before.

Looking at how the code is written I'll bet you if you asked Satoshi himself what he thought would happen, he would say the transaction would be rejected. Sure, it's a bug really, yet because Bitcoin relies on 100% consensus, or all hell breaks loose, we can't "fix" that behavior now.

webbtc.com runs on bitcoin-ruby, a re-implementation of Bitcoin, and as we keep saying over and over again, re-implementations are dangerous because they invariably have bugs and don't duplicate bitcoin-qt behavior exactly.

I forked webbtc.com on testnet yesterday, still not quite sure how I pulled that off, but I was playing with non-standard transactions.

I also forked webbtc.com on mainnet today with transaction 315ac7d4c26d69668129cc352851d9389b4a6868f1509c6c8b66bead11e2619f It has two inputs, both using SIGHHASH_SINGLE, and only one output. SignatureHash() in script.cpp has the following code:


if (nOut >= txTmp.vout.size())
{
    printf("ERROR: SignatureHash() : nOut=%d out of range\n", nOut);
    return 1;
}


The thing is, that's not actually an error. SignatureHash() returns the hash of the signature, and CheckSig() doesn't check the return code, so it acts as though the hash the signature signed was 0000000000000000000000000000000000000000000000000000000000000001 You can create a signature for that, and it works just fine and the transaction is considered valid.

bitcoin-ruby and libbitcoin, and probably a few more, all treat this case as an actual error and fail, so they get forked from the main network and are vulnerable to an attacker creating fake confirmations.

I was auditing litecoin this weekend, and in the process found a half-dozen obscure edge cases in Bitcoin's scripting code that I didn't know about it, and I'm already an expert on how Bitcoin works. Frankly the science of software engineering just isn't at the point where we know how to re-implement bitcoin and get it right. This is a problem at least as hard as writing safety-critical flight avionics software; in my opinion it's probably harder.

DO NOT TRUST RE-IMPLEMENTATIONS OF BITCOIN WITH YOUR MONEY
DO NOT WRITE FULL-NODE RE-IMPLEMENTATIONS OF BITCOIN
YOU ARE NOT SMART ENOUGH TO MAKE THEM SECURE, NOBODY IS

pythion-bitcoinlib too if I ever get around to finishing up the unittests I wrote.

It doesn't: https://bitcointalk.org/index.php?topic=164072.msg2773286#msg2773286

Miners can already exploit  transaction malleability to screw up transaction chains so I don't see it as a big issue.  It's really not unlike a double-spend frankly.

Ultimately the 100 block rule is there because even without malice without the rule you're going to get a lot of chaos. Anyway the cost to have the rule isn't a big deal, that's less than a day.

edit: I'll also point out, this isn't specific to non-ECDSA transactions either: 1JwSSubhmg6iPtRjtyqhUYYH7bZg3Lfy1T

Actually I was thinking that even when Bitcoin worked that way OP_CODESEPARATOR would have served no purpose. However I was thinking about it more, and realized that if you didn't insert OP_CODESEPARATOR between the scriptSig and scriptPubKey before calling EvalScript() you could do some interesting things.

Basically I would be able to write a scriptPubKey like this:

<pubkey1> OP_CHECKSIG OP_VERIFY <pubkey2> OP_CHECKSIG

This is spendable by the following scriptSig:

<sig2> <sig1> OP_CODESEPARATOR

sig1 is a signature made by pubkey1 and what sig1 signs is the transaction; nothing exciting yet. But because the scriptSig and scriptPubkey are combined, and Bitcoin happily executes op's in scriptSigs, pubkey1 can also sign new opcodes that have to be present in the scriptSig for the signature to be valid. For instance pubkey1 could make a signature only spendable if the scriptSig includes this:

<pubkey3> OP_CHECKSIG OP_VERIFY

Essentially pubkey1 has delegated the ability to sign the transaction to pubkey3 after the fact. Now if pubkey2 wants to spend the transaction they create a signature of their own over the transaction including those new opcodes, and create the following scriptSig:

<sig2> <sig1> <sig3> OP_CODESEPARATOR <pubkey3> OP_CHECKSIG OP_VERIFY

sig3, sig1 and then sig2 are all checked, and they are all hash a transaction including the following combined scriptSig+scriptPubkey:

<pubkey3> OP_CHECKSIG OP_VERIFY <pubkey1> OP_CHECKSIG OP_VERIFY <pubkey2> OP_CHECKSIG

These days the scriptSig and scriptPubKey evaluated separately, and critically that means that when the scriptPubKey is evaluated pbegincodehash is set to the beginning of the scriptPubKey - there is no way to have a signature check any part of the scriptSig. Secondly there is no way for the scriptPubKey to determine what is in the scriptSig other than the contents of the data stack - there's no way to access the scripts themselves. Even though opcodes can be executed in the scriptSig there is no way that the scriptPubKey can make sure the right ones are executed.

Interestingly Luke's CHECKHASHVERIFY BIP is really proposing something very similar to what I'm talking about above, except in how it relies on a strict hash verification rather than signatures; Luke's implementation may have even enabled the above, although I can't seem to find the code to check. (it'd be bad if it did though because it's a hard-fork) The right design would probably be something like OP_GETCODE to put the serialized code on the stack, followed by OP_HASH/OP_EQUAL/OP_VERIFY for the P2SH case, and OP_HASH/OP_SIGCHECK/OP_VERIFY. Add in some more opcodes to be able to get the other parts of a transaction, and match that against templates, and you'd have a lot of interesting capabilities.

Really Satoshi's one mistake was allowing OP_RETURN to return true in the scriptSig; if he hadn't put in OP_RETURN, or had the current behavior where OP_RETURN can only make the script fail, we could have kept combined scriptSig and scriptPubKey processing. Worth thinking about for the future if a script v2.0 ever happens.

edit: Note how all of the above works even better if there is some kind of OP_CODESEPARATOR "stack", or if OP_CHECKSIG can accept a number for how many OP_CODESEPARATORS "back" it counts; consider what happens when you try to do the delegation a second time.

OP_CODESEPARATOR only comes into affect when it is actually executed: EvalScript() starts off with pbegincodehash = script.begin(); and only updates pbegincodehash when the OP_CODESEPARATOR is executed.

In a scriptPubKey of the following form:


<pubkey> OP_CHECKSIG OP_CODESEPARATOR


The OP_CODESEPARATOR has no effect. Interestingly the signature is the same for a scriptPubKey with no OP_CODESEPARATOR at all, because of the scriptCode.FindAndDelete(CScript(OP_CODESEPARATOR)); line in SignatureHash(), but that doesn't change anything with regard to your proposed attack.

I really should get around to writing some separator related unittests; we don't have any right now.