When I tell people I work on Bitcoin full-time, a somewhat common reaction is "Really?  I thought Bitcoin was finished, what do you work on?"

I spend half my development time working on new stuff (or testing new stuff that other people have submitted), but the other half I spend trying to anticipate problems or reacting to problems that are reported. That work tends to be unseen, partly because we want to keep problems quiet while we fix them and partly because quietly anticipating/fixing problems minimizes the 'lulz' that attackers might enjoy if every single-node-DoS attack caused us to run around like chickens with our heads cut off.

Anyway, good developers are hard to find, and one of the reasons I'm not thrilled by all of the AlternaCoins is because I'd rather a good developer help make Bitcoin better rather than spend their time with the busy-work of cross-porting the latest Bitcoin fixes to some other codebase. I would guess that some of the developers of the alternative chains underestimated the amount of work it takes to nurture them and keep them healthy. Maybe that will change when Bitcoin is truly mature and has dealt with another year or two or six of attacks and scaling issues...

I truly don't mean this to sound like a threat, but I think some of the blockchains that have been chugging along running on an ancient forked version of the Bitcoin codebase will be attacked; pretty soon we'll be fully disclosing the denial-of-service bugs that prompted the 0.6.2/0.6.3 releases, and it is highly likely somebody will decide to play with exploit code on a vulnerable chain.

I wish people would find more constructive things to do with their time, but I wish I could fly and never get old like Peter Pan, too.

See the raw transaction API I've been working on, and please help try to break it-- it needs more testing.

The main motivation is to move forward with multisignature transactions, but it also lets you have complete control over the source(s) for a transaction.

My work on the 'raw transaction' API has me thinking hard about the gettransaction/getblock RPC calls. I want it to be easy for the information returned by those calls to be used by the new createrawtx/etc API calls.

So:  in version 0.6.3 and earlier, gettransaction would only return information about transactions in your wallet, and would not return all of the information about the transaction.

A month or two ago Pieter and Luke wrote code to modify gettransaction/getblock to return information about any transaction in the blockchain and a new 'decompositions' Object argument was added to those RPC calls that would let you specify 5 different ways to get information about transaction inputs and outputs (no/hash/hex/asm/obj).  So you might do:

I think 5 different ways of getting (or not getting if you specify 'no') the information is 4 too many, so I'm modifying the code as follows:

• getblock doesn't change from version 0.6; it returns a list of transaction id's in a "tx" Array.
• gettransaction gets an extra boolean argument that defaults to 0 (false).  If non-zero is passed, then it is verbose and outputs information about the transaction inputs and outputs in all the different formats (same as what decomposition "obj" does).
• for compatibility with the 'raw transaction' API calls, gettransaction always return the full serialized-hex-encoded transaction data as a new "rawtx" field.

The thinking is either RPC users will want minimal information quickly, or will want full information and won't care much if they get extra information. If you want full information as quickly as possible, then you should write code to decode the "rawtx" array yourself.

This does mean getting full information for every transaction in a block means more than one RPC call, but Jeff has a pull request for JSON-2.0 "batch" calls, so getting full information about every transaction in a block will be just two RPC round-trips (one getblock, then one batch with a bunch of gettransaction calls in it).

And to be clear: these changes are meant to be 100% backwards-compatible with getblock/gettransaction in version 0.6.3 and earlier.  We're talking about extra arguments and extra information in the returned JSON objects.

---

Example output for a complicated multisignature testnet transaction:

Short, in-wallet transaction:Verbose:

---

The only feature that bothers me is reporting 'addresses/type' for transaction inputs. It bothers me for two reasons:

1) It is expensive to fetch that data, because it means finding the previous transaction in the blockchain and fetching it from disk. It violates the principle that "a feature shouldn't cost you anything if you're not using it."

2) If "we" implement spent-input pruning (Pieter has some fantastic early results) that data might not exist.

I think it would be better to remove the 'addresses/type' information from gettransaction output, so gettransaction never has to lookup previous transactions.

If you care about the previous transaction, then you would have to call gettransaction again, using vin.prevout.hash to look them up (and, again, using the JSON-2.0 'batch' feature to lookup all the prior transactions in one RPC roundtrip).

If/when transaction pruning happens then the semantics are clear: that gettransaction would return an error if the prior transaction was pruned.

I think "1 2 3 4" is bad.  4 is more than 1, so a level 4 vulnerability is worse, right?

I still like critical/serious/other.  More gradations than that and I think we'll just waste time arguing over "is this a level 3 vulnerability?  level 4?  ok, let's make it a pi vulnerability (3.1415...)"

Anybody know where I can buy an "I am a Space Lizard" T-shirt for bitcoin?

I usually answer with:

Why do screwdrivers have value? What backs the value of a screwdriver?
Because they're useful and it takes effort to create them.

Bitcoin has value for the same reasons.

In the 0.6.3 thread Graet asks a good question:  what's do we mean by "critical" versus "serious" vulnerability?

Here's what those terms mean to me.  All examples are hypothetical:

A critical vulnerability is one that will have disastrous consequences if it is exploited. Examples might be a remote code exploit, a bug that somebody could use to take down the entire bitcoin network, or a bug that could be exploited to steal your entire wallet.

A serious vulnerability is one that will have serious consequences if it is exploited.  Examples might be a bug that can be triggered by network traffic and will cause the software to stop working or a bug that could be exploited to misdirect your next bitcoin transaction so it goes to an attacker instead of the intended recipient.

The 0.6.3 denial-of-service problem I consider "serious" -- an attacker who figures out exactly what the vulnerability is (we haven't disclosed that yet) can make bitcoind or Bitcoin-Qt stop processing transactions.

Then there are run-of-the-mill vulnerabilities; things like Sybil attacks that require an attacker to round up hundreds or thousands of machines, or denial-of-service attacks that require that the attacker be able to send the victim gigabytes of network traffic. Dealing with these often doesn't even merit a mention in the release notes, because they affect so few people and require an attacker willing to spend a fair bit of money and/or effort just to be annoying.

I'll buy my grandma a terabyte drive for Christmas.

No, seriously, a better startup experience is part of "easy to use" -- waiting hours for the blockchain to sync sucks.

The things on my "good enough to be called 1.0" list are:

+ easy enough for my grandma to use
+ secure enough that it'd be hard for my grandma to lose her bitcoins, even if her computer is infected by 11 bitcoin-stealing trojans and then catches fire and explodes.
+ past the December block-reward-drops-to-25

Before you copy the blk* files, you must run with the -detachdb option and make sure you shutdown bitcoin and wait for it to exit completely.  With 0.6.2, we change the default from "take a long time to shut down, but leave standalone .dat files" to "shutdown quickly, but leave .dat files that are tied to the database/log.* files".

Too many ordinary users were quitting bitcoin and then immediately powering off their systems, leaving the database in a bad state.

We're releasing 0.6.3 to fix two important issues (a serious potential denial-of-service attack, and to improve new-block propagation times for blocks containing lots of transactions).

If you can, please help do some sanity testing-- shutdown bitcoin, then download, install, and run 0.6.3 and let us know "works nicely for me running on 64-bit Ubuntu 10.04" by posting a quick reply to this thread.

Release notes:

Bitcoin version 0.6.3rc1 is now available for download at:
  http://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.6.3/test/

This is a bug-fix release, with no new features.

CHANGE SUMMARY
==============

Fixed a serious denial-of-service attack that could cause the
bitcoin process to become unresponsive. Thanks to Sergio Lerner
for finding and responsibly reporting the problem. (CVE-2012-3789)

Optimized the process of checking transaction signatures, to
speed up processing of new block messages and make propagating
blocks across the network faster.

Fixed an obscure bug that could cause the bitcoin process to get
stuck on an invalid block-chain, if the invalid chain was
hundreds of blocks long.

Bitcoin-Qt no longer automatically selects the first address
in the address book (Issue #1384).

Fixed minimize-to-dock behavior of Bitcon-Qt on the Mac.

Added a block checkpoint at block 185,333 to speed up initial
blockchain download.


Thanks to everybody who contributed to this release:
====================================================

Chris Moore
Christian von Roques
Fordy
Gavin Andresen
Jeff Garzik
Luke Dashjr
Matt Corallo
Michael Hendricks
Peter Todd
Philip Kaufmann
Pieter Wuille
R E Broadley
Sergio Lerner
Wladimir J. van der Laan

Chain proof-of-work is calculated based on the hash target, so if you get another block at the same height there is no benefit to keeping the one with "the smaller hash".

Maybe if you receive a second block solution, keeping the block that removes the most transactions from the memory pool would be the right "good for the entire ecosystem" policy.  That way even if small blocks propagate slightly faster that might be offset if a larger, slower block was found.  (but making new-block-flooding independent of the size of the block is an even better solution, and that shouldn't be too hard to implement)

Creating a semi-trusted backbone of connections to other pools/miners so your new blocks propagate quickly is a good idea.

The RPC and GUI won't let you spend 0-confirmation transactions unless they are 'change' outputs from yourself.

If you REALLY want to shoot yourself in the foot, I've got a pull request for the 0.7 release that adds 'raw transaction' RPC commands that will let you do all sorts of dangerous things, including counting your chickens before they've hatched spending unconfirmed-from-somewhere-else inputs.

I gathered data on transaction first-confirmation times this weekend:

 https://docs.google.com/open?id=0B74FtkFP4j73TXZUY05kSFVzN00

Each line in the file (except the first one) represents one transaction. Each column is:

Number of seconds the transaction spent in the memory pool.

Size, in bytes, of the transaction.

Fees attached to the transaction (in satoshis).

Priority (floating-point number) of the transaction when it first entered the memory pool.  Priority 0.0 transactions have inputs that aren't yet confirmed.

Last column is "1" if the transaction exited the memory pool normally, by being included in the a block, or "0" if the transaction was still in the memory pool when I shut down bitcoind.

--------------

I haven't really analyzed it yet (feel free to help if you're a data geek); I plan on testing some algorithms for suggesting a reasonable fee if you want your transaction to get confirmed quickly, and suggesting an estimate of how long you'll have to wait if you don't attach any fee at all.

But it look like 35% of transactions this past weekend got into a block within 10 minutes, and 87% got into a block within one hour.

Luke is guessing it is SatoshiDice, it might just be a run of bad luck.

Or it might be a side effect of Eligius accepting non-standard transactions.

Does Eligius include transactions that have not been transmitted/relayed to the rest of the network?  If so, it might be a side effect of that (and if that isn't a side effect now, it might be in the future if Eligius blocks take longer to verify as other nodes need to fetch transaction inputs from disk, instead of already having them in cache memory like transactions that ARE transmitted/relayed).

The 0.7 release will include an optimization that should help a lot (transaction signature cache -- prior releases checked signatures when the transaction was first seen, and then checked again when a new block was received, it makes processing new blocks much faster).

This is why I think the so-called "stable" backports are a bad idea. I want people to spend time finding and fixing bugs in 0.6.2, and I want people to realize that the core developers are not supporting older releases.

Cross-posting from the bitcoin-development list:

I submitted a pull request yesterday that implements low-level "raw" transaction, and am looking for feedback on the API and help with trying to test/break it.

Design doc:  https://gist.github.com/2839617
Pull request: https://github.com/bitcoin/bitcoin/pull/1456
Test plan: https://secure.bettermeans.com/projects/4180/wiki/Raw_Transaction_RPC_Test_Plan

Playing around with this API on the command line I'm pretty happy with the level of abstraction and the way it interacts with existing RPC commands; for example, "createrawtx" is just like "sendmany" in the way outputs are specified.

The signrawtx method is the key new method; it takes a raw transaction, signs as many inputs as it can, and returns the same raw transaction with signatures. Typical usage would be:

Funds are sitting in a multisignature transaction output, and it is time to gather signatures and spend them.

Assumption: you know the multisignature transaction's [txid, outputNumber, amount].

Create a raw transaction to spend, using createrawtx.
Use signrawtx to add your signatures (after unlocking the wallet, if necessary).
Give the transaction to the other person(s) to sign.
You or they submit the transaction to the network using sendrawtx.
I don't imagine anybody but very-early-adopters or ultra-geeks will do this by calling these RPC methods at a command-line. They are really intended for people writing services on top of bitcoind. The service should be careful to include an appropriate transaction fee, or the sendrawtx method is likely to fail.

I've been asked a couple of times: why doesn't signrawtx handle the BIP 0010 (https://en.bitcoin.it/wiki/BIP_0010) transaction format?

I considered parsing/writing BIP 10 format for raw transactions, but decided that reading/writing BIP 10 format should happen at a higher level and not in the low-level RPC calls. So 'raw transactions' are simply hex-encoded into JSON strings, and encoding/decoding them is just a couple of lines of already-written-and-debugged code.

------

Here is the help output and example use for all the new RPC calls:

listunspent [minconf=1] [maxconf=999999]
Returns array of unspent transaction outputs
with between minconf and maxconf (inclusive) confirmations.
Returns an array of 4-element arrays, each of which is:
[transaction id, output, amount, confirmations]

E.g:  listunspent 1 2
Returns:
[
    [
        "2881b33a8c0bbdb45b0a65b36aa6611a05201e316ea3ad718762d48ef9588fb3",
        0,
        40.00000000,
        2
    ],
    [
        "894a0fc535c7b49f434ceb633d8555ea24c8f9775144efb42da85b853280bcd7",
        0,
        50.00000000,
        1
    ]
]

getrawtx <txid>
Returns hexadecimal-encoded, serialized transaction data
for <txid>. Returns an error if <txid> is unknown.

E.g.: getrawtx fce46ea2448820f7bb8091b5f5e3fd75b7b267e60b9a22af88a9eeabfb084233
Returns:
01000000016d40da062b6a0edcaf643b6e25b943baf103941589d287e39d6f425d84ae8b1c00000 0004847304402203fb648ff8381d8961e66ef61ab88afe52826a5179b8a7312742c8d93785ca563 02204240ea12de1211fffab49686f13ca0e78011d1985765be6e6aa8e747852f897d01ffffffff0 100f2052a0100000017a914f96e358e80e8b3660256b211a23ce3377d2f9cb18700000000


createrawtx [["txid",n],...] {address:amount,...}
Create a transaction spending given inputs
(array of (hex transaction id, output number) pairs),
sending to given address(es).
Returns the same information as gettransaction, plus an
extra "rawtx" key with the hex-encoded transaction.
Note that the transaction's inputs are not signed, and
it is not stored in the wallet or transmitted to the network.

E.g.: createrawtx '[ ["fce46ea2448820f7bb8091b5f5e3fd75b7b267e60b9a22af88a9eeabfb084233",0] ]' '{"mqYmZSQQuAWNQcdwBrDwmtTXg2TLNz748L":50}'
Returns:
{
    "version" : 1,
    "locktime" : 0,
    "size" : 85,
    "vin" : [
        {
            "prevout" : {
                "hash" : "fce46ea2448820f7bb8091b5f5e3fd75b7b267e60b9a22af88a9eeabfb084233",
                "n" : 0
            },
            "scriptSig" : "",
            "sequence" : 4294967295
        }
    ],
    "vout" : [
        {
            "value" : 50.00000000,
            "scriptPubKey" : "OP_DUP OP_HASH160 6e0920fc26383dc7e6101bc417cf87169d0cedbd OP_EQUALVERIFY OP_CHECKSIG"
        }
    ],
    "rawtx" : "0100000001334208fbabeea988af229a0be667b2b775fde3f5b59180bbf7208844a26ee4fc00000 00000ffffffff0100f2052a010000001976a9146e0920fc26383dc7e6101bc417cf87169d0cedbd 88ac00000000"
}

signrawtx <hex string> [<prevtx1>,<prevtx2>...]
Sign inputs for raw transaction (serialized, hex-encoded).
Second argument is an array of raw previous transactions that
this transaction depends on but are not yet in the blockchain.
Returns json object with keys:
  rawtx : raw transaction with signature(s) (hex-encoded string)
  complete : 1 if transaction has a complete set of signature (0 if not)

E.g.: signrawtx "0100000001334208fbabeea988af229a0be667b2b775fde3f5b59180bbf7208844a26ee4fc00000 00000ffffffff0100f2052a010000001976a9146e0920fc26383dc7e6101bc417cf87169d0cedbd 88ac00000000" '["01000000016d40da062b6a0edcaf643b6e25b943baf103941589d287e39d6f425d84ae8b1c00000 0004847304402203fb648ff8381d8961e66ef61ab88afe52826a5179b8a7312742c8d93785ca563 02204240ea12de1211fffab49686f13ca0e78011d1985765be6e6aa8e747852f897d01ffffffff0 100f2052a0100000017a914f96e358e80e8b3660256b211a23ce3377d2f9cb18700000000"]'
Returns:
{
    "rawtx" : "0100000001334208fbabeea988af229a0be667b2b775fde3f5b59180bbf7208844a26ee4fc00000 0009100473044022007f3ba1b8bdc156f2340ef1222eb287c3f5481a8078a8dad43aa09fd289ba1 9002201cc72e97406d546dc918159978dc78aee8215a6418375956665ee44e6eacc115014752210 2894ca6e7a6483d0f8fa6110c77c431035e8d462e3a932255d9dda65e8fada55c2103c556ef01e8 9a07ee9ba61581658fa007bf442232daed8b465c47c278550d3dab52aeffffffff0100f2052a010 000001976a9146e0920fc26383dc7e6101bc417cf87169d0cedbd88ac00000000",
    "complete" : false
}

sendrawtx <hex string>
Submits raw transaction (serialized, hex-encoded) to local node and network.
E.g.: sendrawtx 0100000001334208fbabeea988af229a0be667b2b775fde3f5b59180bbf7208844a26ee4fc00000 0009100473044022007f3ba1b8bdc156f2340ef1222eb287c3f5481a8078a8dad43aa09fd289ba1 9002201cc72e97406d546dc918159978dc78aee8215a6418375956665ee44e6eacc115014752210 2894ca6e7a6483d0f8fa6110c77c431035e8d462e3a932255d9dda65e8fada55c2103c556ef01e8 9a07ee9ba61581658fa007bf442232daed8b465c47c278550d3dab52aeffffffff0100f2052a010 000001976a9146e0920fc26383dc7e6101bc417cf87169d0cedbd88ac00000000
Returns:
error: {"code":-22,"message":"TX rejected"}

(Rejected because it doesn't have all required signatures, if it was accepted it would return the transaction id)

Relevant discussion here:
  http://crypto.stackexchange.com/questions/779/hashing-or-encrypting-twice-to-increase-security

You're a crazy-man.  Crazy in a good way... ever do any APL programming?