[PATCH] getblockbycount, getblockbyhash RPCs

[jgarzik]

Gavin's bitcointools are nice for dumping blocks, but they require you to shutdown bitcoind, because bitcoin sets DB_PRIVATE flag, preventing outside users from using the multi-user database

See "getblockbycount" branch of git repository https://github.com/jgarzik/bitcoin

Here is sample command usage and output for block at height 71995:

Code:

$ /spare/repo/bitcoin.hacks/bitcoind -datadir=/garz/bitcoin/data getblock 71995
{
    "hash" : "00000000002d1a4380793affbc610885aa2e0b224eeedd64ffe108044ec7d434",
    "ver" : 1,
    "prev_block" : "000000000103fcffbd8020ff7459f3635eb41102ee3b22fa466a7fdfc05bad58",
    "mrkl_root" : "9d436c694968454ea0d17f4aece3b829930027c3cb918e5107a1605aa2eeae33",
    "time" : 1280823515,
    "bits" : 469830746,
    "nonce" : 2918845955,
    "n_tx" : 4,
    "tx" : [
        {
            "hash" : "f85e77e4379694c8d2c1232d6fddfc7792073fb8484bdac37a9ba5ed1d245c57",
            "ver" : 1,
            "vin_sz" : 1,
            "vout_sz" : 1,
            "lock_time" : 0,
            "in" : [
                {
                    "prev_out" : {
                        "hash" : "0000000000000000000000000000000000000000000000000000000000000000",
                        "n" : 4294967295
                    },
                    "coinbase" : "045a0c011c0143"
                }
            ],
            "out" : [
                {
                    "value" : 50.00000000000000,
                    "scriptPubKey" : "0xAD2D2527C630A3CF951703C4F44BB70F8C7524823F7095253D9412A1E9CAD1782B6B83228083A02485C20BC1870FB1A06C09DB768A5D27326A3E2FD859E7799204 OP_CHECKSIG"
                }
            ]
        },
        {
            "hash" : "38431f2f029a37a74a5b5bf0327f41a67b83aef8ad60a2efe918a8f1f0e7df1b",
            "ver" : 1,
            "vin_sz" : 1,
            "vout_sz" : 1,
            "lock_time" : 0,
            "in" : [
                {
                    "prev_out" : {
                        "hash" : "d9b308ca3484b7be6599c5050fec2cd2d31e654a0d2560989ffd28590bef6e6a",
                        "n" : 0
                    },
                    "scriptSig" : "0x01D82CD24A2C12E108087B9D9F81C49EF550F24D12668381D1D333D383B8EEB717200209F92E3246912CE4965E728963DF65FA63D9CCA10513CEF35C9BDCDC8018695F20024430"
                }
            ],
            "out" : [
                {
                    "value" : 50.00000000000000,
                    "scriptPubKey" : "OP_DUP OP_HASH160 0x403FC36C7A1B5A9390F29343F4317F148A3ACB18 OP_EQUALVERIFY OP_CHECKSIG"
                }
            ]
        },
        {
            "hash" : "c61a96af68cce7329a450c18e3ac359d3052bda9187515a42fa7d262405213bd",
            "ver" : 1,
            "vin_sz" : 1,
            "vout_sz" : 2,
            "lock_time" : 0,
            "in" : [
                {
                    "prev_out" : {
                        "hash" : "1ff50ec1208497b333972da7dfdb2ba92c18da901df997d630b32981a013b783",
                        "n" : 0
                    },
                    "scriptSig" : "0x0132831D85F7395034FD537CADFB5BC3DF134347E73BD76352D31314686785B17320024351730466BEC02AD0A9D1F2BFBF4F246AFA8EE4B1BEE3FA8B83801B8301AF5820024430 0x51342C8A6D8C38C33413C63BE8CA93AD060BEA212D961EA63F4C76013E81978A06934604515C7941A729450A508CF556AD6B5061ADABF74C8F881C44B2405CDA04"
                }
            ],
            "out" : [
                {
                    "value" : 0.05000000000000000,
                    "scriptPubKey" : "OP_DUP OP_HASH160 0xE183BC5BB9CEF757C51BCD8B864A8F2210114373 OP_EQUALVERIFY OP_CHECKSIG"
                },
                {
                    "value" : 0.4300000000000000,
                    "scriptPubKey" : "OP_DUP OP_HASH160 0x209F7B58DC860A2B0D2547FE659D78DFFC68FA77 OP_EQUALVERIFY OP_CHECKSIG"
                }
            ]
        },
        {
            "hash" : "d561a3594fcf97dd1a1abe7a1eda15c8e335aaaecf97f959de0595298d87c6d5",
            "ver" : 1,
            "vin_sz" : 1,
            "vout_sz" : 2,
            "lock_time" : 0,
            "in" : [
                {
                    "prev_out" : {
                        "hash" : "c61a96af68cce7329a450c18e3ac359d3052bda9187515a42fa7d262405213bd",
                        "n" : 1
                    },
                    "scriptSig" : "0x01DCDA3BAD3620FE246F2D973D6128123FC5F054121624852D98CEA4A3333128D00021025FFA265DA89861FC609792AD193204FCF0B2C1823B2D23052BCBD468274367BA0021024630 0x53B6705A0E13FD31A87C4F5F43ABDA77995B7CDA87B521043488254B16A122AFE99EA0BAE780CA19C6DCB733ECA9F42404B1F77951703B4057362704C126BD2304"
                }
            ],
            "out" : [
                {
                    "value" : 0.05000000000000000,
                    "scriptPubKey" : "OP_DUP OP_HASH160 0x388E046D522A5EFDBFD272EAE11E9718F8C10FA0 OP_EQUALVERIFY OP_CHECKSIG"
                },
                {
                    "value" : 0.3800000000000000,
                    "scriptPubKey" : "OP_DUP OP_HASH160 0x1FC4B63556EB262B048F4A908875A62E338F7364 OP_EQUALVERIFY OP_CHECKSIG"
                }
            ]
        }
    ],
    "mrkl_tree" : [
        "f85e77e4379694c8d2c1232d6fddfc7792073fb8484bdac37a9ba5ed1d245c57",
        "38431f2f029a37a74a5b5bf0327f41a67b83aef8ad60a2efe918a8f1f0e7df1b",
        "c61a96af68cce7329a450c18e3ac359d3052bda9187515a42fa7d262405213bd",
        "d561a3594fcf97dd1a1abe7a1eda15c8e335aaaecf97f959de0595298d87c6d5",
        "b626f3cd7c1fd229bffbec34fab2700bc31659dbf2a74f7916701a18490125b4",
        "dd74eea07e9bf2744655a9bab3a8cdbc5fe9a0c86ad18f42a2c259d40a83decb",
        "9d436c694968454ea0d17f4aece3b829930027c3cb918e5107a1605aa2eeae33"
    ]
}

[knightmb]

Hey cool, you can even see the generation and other transactions in it, nice 

[jgarzik]

Updated patch to v2, with a simple cast fix for 32-bit machines.

Same URL: http://gtf.org/garzik/bitcoin/patch.bitcoin-getblock

[throughput]

Oh! You have already implemented the feature I need!

Great!

[jgarzik]

Patch updated for current SVN, r130.  Same patch URL, no change in implementation or operation.

[jgarzik]

Patch updated for current SVN, r146, aka the "break patches with a CR/LF policy change" revision.

Same patch URL (see top of this thread), no change in implementation or operation.

[jgarzik]

Patch updated to latest SVN.

[Gavin Andresen]

I've been working on a "monitoraddress" / "monitorblocks" patch, so bitcoind will POST transaction/block information to an arbitrary URL.

And it seems to me "getblock" should be split into "getblockbycount", "getblockbyhash" and "gettransaction".  I also have some suggestions for making the naming more consistent:

I'd suggest getblockby* return:

Code:

{
    "hash" : "00000000002d1a4380793affbc610885aa2e0b224eeedd64ffe108044ec7d434",
    "blockcount" : 71995,
    "version" : 1,
    "merkleroot" : "9d436c694968454ea0d17f4aece3b829930027c3cb918e5107a1605aa2eeae33",
    "time" : 1280823515,
    "nonce" : 2918845955,
    "tx" : [  "f85e77e4379694c8d2c1232d6fddfc7792073fb8484bdac37a9ba5ed1d245c57", etc ]
}

Reasoning:
  blockcount instead of 'height' or 'depth' because getinfo returns "blockcount"
  getblockbyhash might return a block not on the "best" chain; return blockcount=-1 in that case?
  version instead of 'ver' because abbrvs shd b avoided (thy cn b hrd to rd/rmbr)
  only return transaction hashes; if you need more transaction details, call gettransaction.
  No n_tx (or n_anything) because it is implicit in length of returned array

(stuff I'm not sure about:  what is 'bits' in the implemented getblock?  Is that difficulty?  Does anybody really need all the detailed merkle tree info?)

I'd suggest gettransaction return:

Code:

{
  "txid" : ...transaction id passed in,
  "version" : 1,
  "time" : ...transaction time...,
  "confirmations" : n,
  "block" : "block hash",   #  (if confirmations > 0)
  "from" : [ [ "txid", n ], ... ], # Empty if coin generation transaction
  "to" : [ [ "address", amount ], ... ],
}

Reasoning:
  Returning the CScript string seems like it is both too much detail AND not very useful (I want the bitcoin addresses, NOT OP_DUP 0xblahblahblah ...)
  "from" is just a list of pointers to previous transaction "to"s
  confirmations is duplicate info (you could getblockbyhash and then look at its blockcount), but is so useful I think it belongs here.

[jgarzik]

Is there an index that indexes all transaction ids for all blocks?

Or should 'gettransaction' only work for wallet transactions?

[Gavin Andresen]

blkindex.dat contains all the transactions, in records that look like:
  key:  "tx" + HASH_OF_TRANSACTION
  value: serialized CTxIndex

The CTxIndex points to a record in one of the blk000n.dat files.

That does bring up an interesting point, though:  if lightweight clients are implemented, they wouldn't be able to support an arbitrary 'gettransaction'.

[jgarzik]

Patch updated to latest SVN, with a few of the suggested gavinchanges:

• 'getblock' renamed to 'getblockbycount'
• s/ver/version/
• vin_sz and vout_sz removed, redundant

[mizerydearia]

Just a quick note that this patch is not compatible with http://bitcointalk.org/index.php?topic=1319.msg15065#msg15065

<gavinandresen> I reimplemented getblock in the monitor* patch
<gavinandresen> (so it returned the same JSON data structures as monitorblock )
<necrodearia> Ah, so your patch makes getblock patch deprecated, right?
<gavinandresen> Yup
<necrodearia> Yay! ^_^

[jgarzik]

That is incorrect.

The content in gavin's patch makes my getblock largely redundant -- but they are not incompatible.

[mizerydearia]

That is incorrect.

The content in gavin's patch makes my getblock largely redundant -- but they are not incompatible.

Hummm...  I was able to use one patch or the other.  When trying to use both patches, I couldn't apply the patches.  Perhaps I did something wrong.  I will try again.

When applying gavin's monitor patch first and your getblock patch second, the following error occurs: http://pastebin.com/CUr1cvgS
When applying your getblock patch first and gavin's monitor patch second, the following error occurs: http://pastebin.com/6UY4dscY

[theymos]

Hummm...  I was able to use one patch or the other.  When trying to use both patches, I couldn't apply the patches.  Perhaps I did something wrong.  I will try again.

patch is not meant to be used blindly. You're supposed to look at the reject file manually and figure it out.

The problem is caused when trying to patch the section of rpc.cpp around line 1330. Both patches try to apply their changes to the bottom of the list, with context looking up. They modify each other's context. To fix this, they should be applied to the top looking only up or the bottom looking only down. (Not sure if you can actually do this, however.)

[jgarzik]

Patch updated to latest SVN (v0.3.14).

[theymos]

I noticed a bug in getblock (and bitcointools). Block 67300 is missing several transactions when compared to the output of Bitcoin's printblock, and the hash is also wrong.

I ran into the bug while parsing the entire chain: transaction e7a995 is spent in 67301, but according to getblock data that transaction was never made.

[jgarzik]

I noticed a bug in getblock (and bitcointools). Block 67300 is missing several transactions when compared to the output of Bitcoin's printblock, and the hash is also wrong.

I ran into the bug while parsing the entire chain: transaction e7a995 is spent in 67301, but according to getblock data that transaction was never made.

hmmmm.  That's quite strange, considering the code simply loops over all 'vin' and 'vout', duplicating the code pattern used elsewhere to dump a block to the debug log.

[theymos]

ArtForz figured out that this is caused because getblock is showing an orphan block at that position instead of the real block. Deleting the block database files fixes it because you won't re-download orphan blocks.

[theymos]

I think that this can be fixed by changing this:

Code:

	if (pindex->nHeight == height) {

to this:

Code:

	if (pindex->nHeight == height && pindex->IsInMainChain()) {

jgarzik: I don't have any orphan blocks right now, so can you test this on your block 90539 and compare it to the real one?