Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: jgarzik on July 29, 2010, 12:34:53 AM



Title: [PATCH] implement 'xlisttransactions'
Post by: jgarzik on July 29, 2010, 12:34:53 AM
Here is a patch against SVN 117, implementing 'xlisttransactions' RPC: http://pastebin.ca/1910553 (http://pastebin.ca/1910553)   At present, the options are ignored, and it dumps all transactions it finds.

Raw patch: http://pastebin.ca/raw/1910553 (http://pastebin.ca/raw/1910553)

Edit: Patch's current home is http://yyz.us/bitcoin/patch.bitcoin-listtransactions (http://yyz.us/bitcoin/patch.bitcoin-listtransactions)

Edit2: RPC command has been renamed to 'xlisttransactions'



Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on July 29, 2010, 12:48:34 AM

Here's an updated version, that sorts by number of confirmations:  http://pastebin.ca/1910564 (http://pastebin.ca/1910564)

Raw patch:  http://pastebin.ca/raw/1910564 (http://pastebin.ca/raw/1910564)



Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on July 29, 2010, 03:00:24 AM

OK, here's version 4.  Output now matches what you see in the UI, and gives you information on debits/credit/generated/etc.  The 'count' and 'includegenerated' parameters are now honored.

Patch: http://pastebin.ca/1910623 (http://pastebin.ca/1910623)
Raw patch: http://pastebin.ca/raw/1910623 (http://pastebin.ca/raw/1910623)

Here's sample output from my dev box:
Code:
$ /usr/local/src/bitcoin/bitcoind -datadir=/usr/local/bitcoin/data listtransactions
[
    {
        "address" : "15uyjNbNzizz5r8UgWpF1ZayV82Pq3FHQ5",
        "label" : "",
        "class" : "credit",
        "amount" : 0.02000000000000000,
        "confirmations" : 160
    },
    {
        "address" : "15uyjNbNzizz5r8UgWpF1ZayV82Pq3FHQ5",
        "label" : "",
        "class" : "credit",
        "amount" : 0.01000000000000000,
        "confirmations" : 162
    },
    {
        "address" : "1GKgKYtV79jYHR2mr1SSDp9EjXQuLTmUTw",
        "label" : "",
        "class" : "debit",
        "amount" : 0.02000000000000000,
        "confirmations" : 1525
    },
    {
        "address" : "191ALqREPdXCGE6mhfS7HqRZCeQB2AHT6y",
        "label" : "",
        "class" : "credit",
        "amount" : 0.02000000000000000,
        "confirmations" : 1531
    },
    {
        "address" : "1HVYQQ5K489fMx5Aqt48M5oTJPsmUhrpkx",
        "label" : "",
        "class" : "debit",
        "amount" : 0.01000000000000000,
        "confirmations" : 1572
    },
    {
        "address" : "1KTpPjGWyhTBC5NNYFwNzkyjW6UDL9jKPG",
        "label" : "Your Address",
        "class" : "credit",
        "amount" : 0.01000000000000000,
        "confirmations" : 1587
    }
]


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on July 29, 2010, 09:22:06 PM

Added a couple small cleanups, and verified it still works under 0.3.6 / SVN r119.

Version 5 of listtransactions: http://pastebin.ca/1911295 (http://pastebin.ca/1911295)
Raw patch: http://pastebin.ca/raw/1911295 (http://pastebin.ca/raw/1911295)


FAQ:

Q1) How does 'listtransactions' behave if tail of the block chain changes, eg. 0 confirmations -> 1 confirmation -> 0 confirmations?

A1) 'listtransactions' behaves the same way 'listreceivedbyaddress' behaves...  the output changes accordingly.



Title: Re: [PATCH] implement 'listtransactions'
Post by: lachesis on July 29, 2010, 10:38:49 PM
Very nice! Send Satoshi an email and ask him to add it to trunk.

By the way, anyone having trouble applying this on unix can use:
Code:
curl http://pastebin.ca/raw/1911295 | awk 'sub("$", "\r")' | patch
That'll make the line endings on the removed parts match the Windows line endings in the code files.


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on July 29, 2010, 10:45:26 PM
Very nice! Send Satoshi an email and ask him to add it to trunk.

He is welcome to pick it up now.

But I didn't want to push the patch to him until the two FIXME's are resolved.  Those FIXMEs are for 0.01% cases, but still...


Title: Re: [PATCH] implement 'listtransactions'
Post by: lachesis on July 29, 2010, 10:53:00 PM
Could you make count=0 return all transactions?


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on July 29, 2010, 11:02:49 PM
Could you make count=0 return all transactions?

Returning all transactions is pretty easy, sure.

I wonder if a better interface might be to follow 'listreceivedbyaddress' and print everything by default, only applying limits if limits were specified.  ie.  make your "count=0" the default.


Title: Re: [PATCH] implement 'listtransactions'
Post by: lachesis on July 30, 2010, 12:51:03 AM
I wonder if a better interface might be to follow 'listreceivedbyaddress' and print everything by default, only applying limits if limits were specified.  ie.  make your "count=0" the default.
Perhaps. In that case, you should switch the order of includegenerated and count. Perhaps even make includegenerated true? Generations are included by default in the UI, after all.


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on July 30, 2010, 05:44:23 AM

Version 6 of listtransactions: http://pastebin.ca/1911570 (http://pastebin.ca/1911570)
Raw patch: http://pastebin.ca/raw/1911570 (http://pastebin.ca/raw/1911570)

listtransactions implementation should be complete at this point.  The following command syntax is used:
Code:
listtransactions [count=10] [minconf=1] [includegenerated=true]

As lachesis suggested, count=0 will dump all transactions.

FAQ:

Q1) How does 'listtransactions' behave if tail of the block chain changes, eg. 0 confirmations -> 1 confirmation -> 0 confirmations?

A1) 'listtransactions' behaves the same way 'listreceivedbyaddress' behaves...  the output changes accordingly.


Title: Re: [PATCH] implement 'listtransactions'
Post by: Gavin Andresen on July 30, 2010, 01:18:06 PM
Couple of quick suggestions:

Using the key name "class" will cause problems for, at least, JavaScript, and probably other languages where "class" is a reserved word.  "type" or "variety" or some other synonym will cause fewer problems later.

Or, maybe better, get rid of that field and just report credits as positive numbers and debits as negative.  And add a separate "generated" field (boolean true or false).

Since each entry refers to a transaction, I'd suggest adding a "tx_id" SHA256 hex-encoded transaction id.  Then listtransactions would play nicely with the refundtransaction JSON-RPC extension (and maybe a future gettransactiondetails that let you get transaction parents, which block the transaction was in, and so on).

Code to get that would look something like:
Code:
            uint256 tx_hash = transaction.GetHash();
            string tx_id = tx_hash.GetHex();
            mapJSONResponse.push_back(Pair("tx_id", tx_id));


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on July 30, 2010, 06:30:40 PM
Couple of quick suggestions:

Using the key name "class" will cause problems for, at least, JavaScript, and probably other languages where "class" is a reserved word.  "type" or "variety" or some other synonym will cause fewer problems later.

Can you be more specific?  All mainstream programming language seem sensibly insensitive to abitrary string contents, JS included.  String content can certainly include language-reserved keywords and parsing tokens.

Since each entry refers to a transaction, I'd suggest adding a "tx_id" SHA256 hex-encoded transaction id.  Then listtransactions would play nicely with the refundtransaction JSON-RPC extension (and maybe a future gettransactiondetails that let you get transaction parents, which block the transaction was in, and so on).

Code to get that would look something like:
Code:
            uint256 tx_hash = transaction.GetHash();
            string tx_id = tx_hash.GetHex();
            mapJSONResponse.push_back(Pair("tx_id", tx_id));

Added, thanks for the suggestion.


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on July 30, 2010, 06:43:37 PM

Here is 'listtransaction' version 7:
http://gtf.org/garzik/bitcoin/patch.bitcoin-listtransactions (http://gtf.org/garzik/bitcoin/patch.bitcoin-listtransactions)

This adds the suggested txn_id field -- very nice suggestion, gavin!  I wanted a unique transaction id, and now I have one :)


Title: Re: [PATCH] implement 'listtransactions'
Post by: satoshi on July 30, 2010, 07:40:54 PM
What are you needing to use listtransactions for?

The reason I didn't implement listtransactions is I want to make sure web programmers don't use it.  It would be very easy to latch onto that for watching for received payments.  There is no reliable way to do it that way and make sure nothing can slip through the cracks.  Until we have solid example code using getreceivedbyaddress and getreceivedbylabel to point to and say "use this! use this! don't use listtransactions!", I don't think we should implement listtransactions.

When we do implement listtransactions, maybe one way to fight that is to make it all text.  It should not break down the fields into e.g. comment, confirmations, credit, debit.  It could be one pretty formatted string like "0/unconfirmed   0:0:0 date   comment      debit 4  credit 0" or something so it's hard for programmers to do the wrong thing and process it.  It's only for viewing the status of your server.  I guess that would be kinda annoying for web interfaces that would rather format it into html columns though.


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on July 30, 2010, 07:45:21 PM
It would be very easy to latch onto that for watching for received payments.  There is no reliable way to do it that way and make sure nothing can slip through the cracks.  Until we have solid example code using getreceivedbyaddress and getreceivedbylabel to point to and say "use this! use this! don't use listtransactions!", I don't think we should implement listtransactions.

Can you be more specific about "no reliable way"?

The existing facility getreceivedby* is provably unreliable.  It combines transactions into summarized totals.  When you go to the bank ATM and perform two deposits, the second immediately following the first, you do not see "ATM deposit $80" on your bank statement, you see two transactions with different transaction IDs for "ATM deposit $40".

I am missing something here?  listtransaction seems more reliable than getreceivedby* summaries.



Title: Re: [PATCH] implement 'listtransactions'
Post by: Gavin Andresen on July 30, 2010, 07:48:33 PM
Couple of quick suggestions:

Using the key name "class" will cause problems for, at least, JavaScript, and probably other languages where "class" is a reserved word.  "type" or "variety" or some other synonym will cause fewer problems later.

Can you be more specific?  All mainstream programming language seem sensibly insensitive to abitrary string contents, JS included.  String content can certainly include language-reserved keywords and parsing tokens.
It's pretty common to turn maps into objects, so you can use syntax like:
  foo.tx_id
... instead of foo['tx_id'].  Especially if you're doing something like passing the data into a templating system (which may ONLY understand the object.field syntax).

And foo.class just doesn't work out nicely.


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on July 31, 2010, 03:24:10 AM

Updated to version 8 of listtransactions:
http://gtf.org/garzik/bitcoin/patch.bitcoin-listtransactions (http://gtf.org/garzik/bitcoin/patch.bitcoin-listtransactions)

Includes gavin's suggestion, renaming 'class' to 'category'.


Title: Re: [PATCH] implement 'listtransactions'
Post by: lachesis on July 31, 2010, 01:36:59 PM
The reason I didn't implement listtransactions is I want to make sure web programmers don't use it.  It would be very easy to latch onto that for watching for received payments.  There is no reliable way to do it that way and make sure nothing can slip through the cracks.  Until we have solid example code using getreceivedbyaddress and getreceivedbylabel to point to and say "use this! use this! don't use listtransactions!", I don't think we should implement listtransactions.
Why is it unreliable? It provides the same information that the GUI provides, and that works fine for watching for payments...

When we do implement listtransactions, maybe one way to fight that is to make it all text.  It should not break down the fields into e.g. comment, confirmations, credit, debit.  It could be one pretty formatted string like "0/unconfirmed   0:0:0 date   comment      debit 4  credit 0" or something so it's hard for programmers to do the wrong thing and process it.  It's only for viewing the status of your server.  I guess that would be kinda annoying for web interfaces that would rather format it into html columns though.
We really shouldn't adopt a policy of protecting users from themselves. If we do, there should at least be a "devmode" switch or config line to turn it off.


Title: Re: [PATCH] implement 'listtransactions'
Post by: Gavin Andresen on July 31, 2010, 01:58:09 PM
Updated to version 8 of listtransactions:
http://gtf.org/garzik/bitcoin/patch.bitcoin-listtransactions (http://gtf.org/garzik/bitcoin/patch.bitcoin-listtransactions)
Cool!  Hope you don't mind, I added it to my github network as a 'feature' branch.


Title: Re: [PATCH] implement 'listtransactions'
Post by: throughput on August 05, 2010, 09:12:06 AM
What are you needing to use listtransactions for?

The reason I didn't implement listtransactions is I want to make sure web programmers don't use it.  It would be very easy to latch onto that for watching for received payments.  There is no reliable way to do it that way and make sure nothing can slip through the cracks.  Until we have solid example code using getreceivedbyaddress and getreceivedbylabel to point to and say "use this! use this! don't use listtransactions!", I don't think we should implement listtransactions.

It seems, that you certainly prefer GUI in favor of CLI interfaces.
GUI is really awful type of interface, when you have, say 5 nodes with SSH access to them and
want to periodically collect some state in a loop like this:
Code:
#!/bin/bash
while read host;
do
   ssh "$host" "hostname; bitcoind listtransactions"
   echo =============
done > report.txt < hostlist
Then, report.txt may be emailed to a human.
I hope, that is a valid usecase for you.


Title: Re: [PATCH] implement 'listtransactions'
Post by: knightmb on August 05, 2010, 10:32:04 AM
What are you needing to use listtransactions for?

The reason I didn't implement listtransactions is I want to make sure web programmers don't use it.  It would be very easy to latch onto that for watching for received payments.  There is no reliable way to do it that way and make sure nothing can slip through the cracks.  Until we have solid example code using getreceivedbyaddress and getreceivedbylabel to point to and say "use this! use this! don't use listtransactions!", I don't think we should implement listtransactions.
I have to agree with Satoshi, other than the extra field of "credit/debit" how is this different than getreceivedbyaddress & getreceivedbylabel? Can someone explain it to me better?

My only beef with *all* those function calls is that direct IP to IP payments are never shown.  As it is right now, if I send a payment via IP to IP, I can't find it with those commands. I can see the balance is right, but I'm not sure where it came from because neither command will show me any info about the transaction.

If listtransactions (or getreceivedbyaddress & getreceivedbylabel) could list IP to IP transactions, that would be useful.

[edit] I see it will show generated coin, so that is useful actually. Also, the current getreceivedbyaddress & getreceivedbylabel seem to be buggy in which transactions they show to begin with.


Title: Re: [PATCH] implement 'listtransactions'
Post by: lachesis on August 05, 2010, 03:00:21 PM
I have to agree with Satoshi, other than the extra field of "credit/debit" how is this different than getreceivedbyaddress & getreceivedbylabel? Can someone explain it to me better?
It lists transactions while the getreceivedby* functions return the sum of all transactions to that label or address. There's no reliable way to see individual transactions using the getreceivedby* functions.


Title: Re: [PATCH] implement 'listtransactions'
Post by: knightmb on August 05, 2010, 06:12:42 PM
I have to agree with Satoshi, other than the extra field of "credit/debit" how is this different than getreceivedbyaddress & getreceivedbylabel? Can someone explain it to me better?
It lists transactions while the getreceivedby* functions return the sum of all transactions to that label or address. There's no reliable way to see individual transactions using the getreceivedby* functions.
Ah ok, does this new function also list individual transactions from IP to IP transfers? I took a glance at the source to get a rough view of what it's showing, didn't know it got into that much deeper detail.


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on August 05, 2010, 07:02:03 PM
Ah ok, does this new function also list individual transactions from IP to IP transfers? I took a glance at the source to get a rough view of what it's showing, didn't know it got into that much deeper detail.

It should present a list very similar to the GUI-presented display.


Title: Re: [PATCH] implement 'listtransactions'
Post by: knightmb on August 05, 2010, 07:33:11 PM
Ah ok, does this new function also list individual transactions from IP to IP transfers? I took a glance at the source to get a rough view of what it's showing, didn't know it got into that much deeper detail.

It should present a list very similar to the GUI-presented display.

Ah ok, good to know. When I read over this topic, I saw some "example" outputs that didn't look that much different than what I was use to seeing, but now I see how useful feature really is, so it certainly has my vote.


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on August 05, 2010, 08:19:41 PM
[edit] I see it will show generated coin, so that is useful actually. Also, the current getreceivedbyaddress & getreceivedbylabel seem to be buggy in which transactions they show to begin with.

Buggy, how?  Maybe they are just confusing, like they were to me, initially?

getreceivedby* are by-address or by-label totals, not individual transactions.

listtransactions was created while trying to implement web services for bitcoin.  The only alternative I can see for tracking transactions, outside of listtransactions, was watching individual bitcoin address totals.  getreceivedby* is not reliable, in that regard.


Title: Re: [PATCH] implement 'listtransactions'
Post by: knightmb on August 05, 2010, 09:41:24 PM
[edit] I see it will show generated coin, so that is useful actually. Also, the current getreceivedbyaddress & getreceivedbylabel seem to be buggy in which transactions they show to begin with.

Buggy, how?  Maybe they are just confusing, like they were to me, initially?

getreceivedby* are by-address or by-label totals, not individual transactions.

listtransactions was created while trying to implement web services for bitcoin.  The only alternative I can see for tracking transactions, outside of listtransactions, was watching individual bitcoin address totals.  getreceivedby* is not reliable, in that regard.
Confusing because I didn't know it was just "amounts" and not transactions. So in that regard it's still buggy because I have a system that I did a bunch of test transactions and yet none of the correct totals show up in the summary. If you run the GUI in server mode, you can throw server commands with it running at the same time to compare output and never match the GUI unless there is more detailed description about what output the function is suppose to give.


Title: Re: [PATCH] implement 'listtransactions'
Post by: throughput on August 09, 2010, 03:05:15 PM
I have a question about listtransactions output.

Hi, I've patched 0.3.7 with listtransactions patch and with getblock patch.

listtransactions show me this output (full):
Code:
[   
    {   
        "address" : "1111111111111111111114oLvT2",
        "label" : "",
        "txn_id" : "c28a3addc59907c51292467dde331d5ab38e3a0b3b1724ff88475d124f2ad021",
        "category" : "mixed_debit",
        "amount" : 0.000000000000000,
        "confirmations" : 3
    },
    {   
        "address" : "1111111111111111111114oLvT2",
        "label" : "",
        "txn_id" : "8e1209c4d78ad77b8429a27613b7d2092be49b0ee6373f2e786374f19237d543",
        "category" : "mixed_debit",
        "amount" : 0.000000000000000,
        "confirmations" : 29
    }
]
It looks like generation, but 0.0BTC is generated and category is "mixed_debit".
So I was curious to list all blocks and check what was that txn_ids and have found that they
was exactly the generation transactions, the value is 50.0 and it is the first transaction in the block.
Code:
...
{   
    "hash" : "0000000000486e8b9c7092ae2b7f340143872e907db0140d745367aeca4f1372",
    "ver" : 1,
    "prev_block" : "00000000003f10014e3a0691e1896b1772eea506f03cf33edf9c4827eff1d6a8",
    "mrkl_root" : "8b6d8d7b13b91fd3d0b520c21e76e9ba38e37b8515735be42bc9ab3a3f385783",
    "time" : 1281348385,
    "bits" : 469809688,
    "nonce" : 3417957380,
    "n_tx" : 3,
    "tx" : [
        {   
            "hash" : "8e1209c4d78ad77b8429a27613b7d2092be49b0ee6373f2e786374f19237d543",
            "ver" : 1,
            "vin_sz" : 1,
            "vout_sz" : 1,
            "lock_time" : 0,
            "in" : [
                {   
                    "prev_out" : {
                        "hash" : "0000000000000000000000000000000000000000000000000000000000000000",
                        "n" : 4294967295
                    },
                    "coinbase" : "0418ba001c026605"
                }
            ],
            "out" : [
                {   
                    "value" : 50.00000000000000,
                    "scriptPubKey" : "0x41E4A7E22ACA8CFF9024B87F3722DA704B16265CA25F47E729691A41CDBC4DF4009F18B61A317D6DF67F9
3660E658204EF50BA28693B187D26CE33233264A63104 OP_CHECKSIG"
                }
            ]
        },
...

Does listtransactions mistakenly list two transactions as being my? Total balance is 0. Or WHAT?..


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on August 09, 2010, 04:51:06 PM
What you are seeing is normal.

Generated blocks show up as mixed_debit until they "mature", which is around 120 confirmations.  After maturity, you are credited the 50 BTC.  This is standard policy for all generated blocks in bitcoin.

If you were looking at the GUI, you would see a "matures in N blocks" messages for that block.



Title: Re: [PATCH] implement 'listtransactions'
Post by: throughput on August 10, 2010, 05:37:29 AM
What you are seeing is normal.

Generated blocks show up as mixed_debit until they "mature", which is around 120 confirmations.  After maturity, you are credited the 50 BTC.  This is standard policy for all generated blocks in bitcoin.

If you were looking at the GUI, you would see a "matures in N blocks" messages for that block.



Great! And thanks for the patch!


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on August 13, 2010, 07:43:32 PM

Patch updated for current SVN, r130.  No change in implementation or operation.

Same patch URL:  http://gtf.org/garzik/bitcoin/patch.bitcoin-listtransactions (http://gtf.org/garzik/bitcoin/patch.bitcoin-listtransactions)



Title: Re: [PATCH] implement 'listtransactions'
Post by: lachesis on August 13, 2010, 08:45:01 PM
Can you add the date and time associated with each transaction?


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on August 13, 2010, 10:39:01 PM
Can you add the date and time associated with each transaction?

Added, as the "tx_time" field in listtransactions version 10.

Same patch URL, as always:  http://gtf.org/garzik/bitcoin/patch.bitcoin-listtransactions (http://gtf.org/garzik/bitcoin/patch.bitcoin-listtransactions)

Sample output, with transactions from a local miner & test box:
Code:
[
    {
        "address" : "15SXfhMc82xC5pPx9qZhPA6bDrkbgBWBJy",
        "label" : "",
        "txn_id" : "716d636922c0fe4fa438914bfbccbdd3f440f68718438a5f14a62a954cd3ef49",
        "tx_time" : 1281424940,
        "category" : "credit",
        "amount" : 0.05000000,
        "confirmations" : 795
    },
    {
        "address" : "1GudGhcqpDCwrJktYvktPHYCnDe6MHtBSr",
        "label" : "",
        "txn_id" : "2900f86ade72c8a312cac92f759c7d1efcd30253f32a76323b63d6278ce6da97",
        "tx_time" : 1281139947,
        "category" : "debit",
        "amount" : 50.00000000,
        "confirmations" : 1387
    },
    {
        "address" : "1111111111111111111114oLvT2",
        "label" : "",
        "txn_id" : "d9aae437e71347dcfafd52a514a4b67db83044ae95384611fb127e805a51b9e7",
        "tx_time" : 1281068407,
        "category" : "generated",
        "amount" : 50.00000000,
        "confirmations" : 1507
    }
]


Title: Re: [PATCH] implement 'listtransactions'
Post by: lachesis on August 14, 2010, 05:43:38 PM
Added, as the "tx_time" field in listtransactions version 10.
Thank you :)


Title: Re: [PATCH] implement 'listtransactions'
Post by: Gavin Andresen on August 15, 2010, 12:01:07 AM
The naming is getting messy: 'txn_id' and 'tx_time' mismatch for no good reason.

tx as short for 'transaction' is fine (bitcoin source uses that abbreviation all over the place), but the rest of the JSON-RPC API doesn't use underscores, so they aught to be txid and txtime.


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on August 15, 2010, 12:34:14 AM

Posted version 11 of listtransactoins at a new URL,

http://yyz.us/bitcoin/patch.bitcoin-listtransactions (http://yyz.us/bitcoin/patch.bitcoin-listtransactions)

This update renames output fields to 'txid' and 'txtime'.



Title: Re: [PATCH] implement 'listtransactions'
Post by: ByteCoin on August 15, 2010, 05:34:01 AM
Generated blocks show up as mixed_debit until they "mature", which is around 120 confirmations.  After maturity, you are credited the 50 BTC.  This is standard policy for all generated blocks in bitcoin.

I've never seen the point of this. Ostensibly it's to stop the coins being spent until they're definitely part of the block chain.
After a block is created, the maturation time of 120 blocks is to make absolutely sure the block is part of the main chain before it can be spent.  Your node isn't doing anything with the block during that time, just waiting for other blocks to be added after yours.  You don't have to be online during that time.
The thing is, if you allowed it to be spent immediately it could only be spent in a branch of the block chain that's includes it's own creation. If that branch is not the longest then both the transaction creating it and the transactions spending it will be rolled back anyway! There's no justification why they should be viewed any more skeptically than a normal transaction especially as they're so small. A 10000BTC transaction doesn't take any longer to mature than a 0.01BTC transaction.

ByteCoin


Title: Re: [PATCH] implement 'listtransactions'
Post by: theymos on August 15, 2010, 06:29:49 AM
Most transactions are not "rolled back" when two chains recombine. Transactions in the shorter chain get put back into the transaction pool, and will eventually appear in a block. Generations would normally be the only transactions removed, which is why they are treated specially.


Title: Re: [PATCH] implement 'listtransactions'
Post by: ByteCoin on August 15, 2010, 10:01:28 AM
Most transactions are not "rolled back" when two chains recombine. Transactions in the shorter chain get put back into the transaction pool, and will eventually appear in a block. Generations would normally be the only transactions removed, which is why they are treated specially.
Thank you for explaining that. It seems reasonable. The precaution makes more sense now

ByteCoin


Title: Re: [PATCH] implement 'listtransactions'
Post by: mizerydearia on August 19, 2010, 04:15:36 PM
Quote
<topace> listtransactions still hasnt made it into svn has it ?
<aceat64> satoshi is somewhat hostile towards it
<Kiba> we need to gitify satoshi's project!
<kisom_dev> truth
<topace> huh why's that? against the method in which it was implemented? or against the concept of it?
<aceat64> topace: I'll find the thread
<aceat64> http://bitcointalk.org/index.php?topic=611.msg6706#msg6706
<bitbot> [PATCH] implement 'listtransactions' : What are you needing to use listtransactions for?  The reason I didn't implement listtransactions is I want to make sure web programmers don't use it.  It would be very easy to latch onto that for watching for received payments.  There is no reliable way to do it that way and make sure nothing can slip through the crack...
<topace> yea reading the thread now...
<topace> i still dont understand why he thinks its unreliable or not good idea for web applications? its perfect for web applications as far as i can see
<nanotube> yea i'm not getting it either... and he things making a plain text structured string will make it "more difficult" for web applications, as if he hasn't heard of string.split() or regexps. :)
<nanotube> thinks
<topace> lol yea
<topace> when i read that i just laughed
<topace> if it outputs it, it can be parsed
<necrodearia> topace: The more discussion about it in the thread, perhaps the more realization Satoshi will have to incorporate it maybe?

Quote
<aceat64> he's somewhat correct, listtransactions is bad way to determine if someone in particular has sent you bitcoins
<topace> why ?
<aceat64> the standard route of getnewaddress then monitor it with getreceivedbyaddress is really the best way
<aceat64> if you just look at transactions you can't see _who_ sent you money
<topace> sure you can, if you've labelled the address
<topace> or even if you ahvent, tied the address to an external account
<aceat64> ...
<aceat64> it doesn't work like that, when I sent you btc it could come from any number of my addresses
<aceat64> send*
<topace> yea but the address that shows in "listtransactions" shows the address it was sent TO
<topace> aka, the address that i've assigned to your account in my web application
<aceat64> oh, I see what you are saying, basically you're talking about the same idea of generating a unique address for each person but using listtransaction instead of getreceivedbyaddress to monitor it
<aceat64> Satoshi is talking about people using only listtransactions I think
<topace> yea.. you'd NEED to assign a unique address to each account in your web application
<topace> otherwise listtransactions would be just as useless as getreceivedbyaddress
<topace> i just don tsee the difference, once you have an address assigned to an account, you can get the TOTAL of all deposits using getreceivedbyaddress, or you can get each individual deposit using listtransactions
<aceat64> I personally run a client with the listtransactions patch in it
<aceat64> I think it should be in the vanilla client
aceat64> if someone is dumb enough to use just listtransactions and try to match up users with a transaction based on time and/or amount only, I say let them
<topace> yea that'd be impossible to do reliably
<topace> but also very stupid :p
<aceat64> I'm sure someone will do it
<topace> you create an account, you generate an address for it, and tell the account holder what the address is... it seems like a no-brainer to me
<aceat64> by not having listtransactions it makes it a lot harder to make an alternate GUI
<topace> a lot harder / impossible :p
<aceat64> well, you can make an alternate GUI, it'll just suck
<topace> yea, cuz it cant see any transactions :p
<topace> maybe thats the ulterior motive - to force people to use the vanilla client
<topace> _his_ client
<aceat64> I doubt it, I think he's just being a bit too over-protective


Title: Re: [PATCH] implement 'listtransactions'
Post by: ByteCoin on August 19, 2010, 05:18:45 PM
I hope you have permission from the people in the converstation to post that. If not - bad form.
If you edit your post to included "quoted with permission" I will delete this post

ByteCoin


Title: Re: [PATCH] implement 'listtransactions'
Post by: aceat64 on August 19, 2010, 07:21:11 PM
Quoted with permission from me at least, so if need be mizerydearia can edit the post and make it a weird monologue.


Title: Re: [PATCH] implement 'listtransactions'
Post by: mizerydearia on August 26, 2010, 01:20:46 AM
I glanced at http://www.bitcoin.org/wiki/doku.php?id=list_of_patches and haven't noticed any updates for this.  Is listtransactions established or useful/functional yet?  Is there a binary release (official or unofficial) that integrates this patch that I could use?  I would like to use the data extracted from using the listtransactions method to implement into my website if it is reliable to do so.

Specifically in a current use scenario I would like to be able to determine the last transaction that occurred for a particular address label and to extract related data such as amount and date/time.


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on August 26, 2010, 01:43:53 AM
I glanced at http://www.bitcoin.org/wiki/doku.php?id=list_of_patches and haven't noticed any updates for this.  Is listtransactions established or useful/functional yet?

It has been functional since day one :)

Is there a binary release (official or unofficial) that integrates this patch that I could use?

Not AFAIK.



Title: Re: [PATCH] implement 'listtransactions'
Post by: aceat64 on September 01, 2010, 03:47:06 PM
Is there a binary release (official or unofficial) that integrates this patch that I could use? 

I'm now doing binary releases with my github repo: http://github.com/aceat64/bitcoin-patchwork

I've got the following patches included:

  • Jgarzik's listtransactions patch: http://bitcointalk.org/index.php?topic=611.20
  • Jgarzik's getblock patch: http://bitcointalk.org/index.php?topic=724.0
  • The listgenerated method, lifted from Lachesis's now deprecated RPC Calls patch: http://www.alloscomp.com/bitcoin/

I've also made the following changes:

  • Slight change so that JSON-RPC returns 200 instead of a 500 error with invalid methods/params.
  • Slight change so that JSON-RPC binds to all/any IPs instead of just the loopback.
  • Changed MAX_OUTBOUND_CONNECTIONS from 8 to 32.


Title: Re: [PATCH] implement 'listtransactions'
Post by: Gavin Andresen on September 01, 2010, 06:33:54 PM
  • Slight change so that JSON-RPC returns 200 instead of a 500 error with invalid methods/params.
  • Slight change so that JSON-RPC binds to all/any IPs instead of just the loopback.
RE: JSON-RPC error / HTTP status codes:  I'm about to submit a patch to Satoshi so bitcoin better follows the JSON standards.  I'll create a new thread describing the changes.

RE: binding to any/all IPs:  DANGER! Make sure you have good firewall rules for port 8332 or you are likely to find yourself with an empty wallet!  Bitcoin is using HTTP BASIC authentication, so anybody who can eavesdrop on the connection between your JSON-RPC client and the bitcoin server can see the password (it is NOT encrypted).


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on September 01, 2010, 06:35:40 PM
RE: binding to any/all IPs:  DANGER! Make sure you have good firewall rules for port 8332 or you are likely to find yourself with an empty wallet!  Bitcoin is using HTTP BASIC authentication, so anybody who can eavesdrop on the connection between your JSON-RPC client and the bitcoin server can see the password (it is NOT encrypted).

I was thinking about adding support for Digest auth...



Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on September 01, 2010, 06:40:17 PM

Patch updated for current SVN, r146, aka the "break patches with a CR/LF policy change" revision.

Same patch URL (see top of this thread), no change in implementation or operation.


Title: Re: [PATCH] implement 'listtransactions'
Post by: Gavin Andresen on September 01, 2010, 07:35:27 PM
RE: binding to any/all IPs:  DANGER! Make sure you have good firewall rules for port 8332 or you are likely to find yourself with an empty wallet!  Bitcoin is using HTTP BASIC authentication, so anybody who can eavesdrop on the connection between your JSON-RPC client and the bitcoin server can see the password (it is NOT encrypted).
I was thinking about adding support for Digest auth...
I wouldn't bother-- HTTP Digest auth is vulnerable to man-in-the-middle attacks, it'd be a lot of work for a probably false sense of security.


Title: Re: [PATCH] implement 'listtransactions'
Post by: mizerydearia on September 01, 2010, 08:20:16 PM
Is there a binary release (official or unofficial) that integrates this patch that I could use? 

I'm now doing binary releases with my github repo: http://github.com/aceat64/bitcoin-patchwork

I've got the following patches included:

  • Jgarzik's listtransactions patch: http://bitcointalk.org/index.php?topic=611.20
  • Jgarzik's getblock patch: http://bitcointalk.org/index.php?topic=724.0
  • The listgenerated method, lifted from Lachesis's now deprecated RPC Calls patch: http://www.alloscomp.com/bitcoin/

I've also made the following changes:

  • Slight change so that JSON-RPC returns 200 instead of a 500 error with invalid methods/params.
  • Slight change so that JSON-RPC binds to all/any IPs instead of just the loopback.
  • Changed MAX_OUTBOUND_CONNECTIONS from 8 to 32.

Can you provide link to patches for listgenerated method and the following three changes at the end?  I'd like to incorporate them into my Gentoo Linux ebuild. (http://bitcointalk.org/index.php?topic=930.0)


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on September 01, 2010, 08:36:28 PM
RE: binding to any/all IPs:  DANGER! Make sure you have good firewall rules for port 8332 or you are likely to find yourself with an empty wallet!  Bitcoin is using HTTP BASIC authentication, so anybody who can eavesdrop on the connection between your JSON-RPC client and the bitcoin server can see the password (it is NOT encrypted).
I was thinking about adding support for Digest auth...
I wouldn't bother-- HTTP Digest auth is vulnerable to man-in-the-middle attacks, it'd be a lot of work for a probably false sense of security.

For Some Value Of vulnerable.  :)  The password cannot be obtained, at least.  MiTM can (a) tell the client to use another auth, and (b) impersonate a server.  (a) can be mitigated by requiring RFC 2617, (b) by SSL.

But, use of HTTP-Basic is just a crime, because it is so trivial to obtain the shared secret.  If HTTP-Basic is to be kept, at least require SSL connections?



Title: Re: [PATCH] implement 'listtransactions'
Post by: aceat64 on September 01, 2010, 09:21:33 PM
Can you provide link to patches for listgenerated method and the following three changes at the end?  I'd like to incorporate them into my Gentoo Linux ebuild. (http://bitcointalk.org/index.php?topic=930.0)

In the repo I have a folder called "patches" where I keep a copy of all the patches I apply.


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on September 07, 2010, 07:21:30 AM

Patch updated for SVN r147, which introduces trivial breakage w/ listtransactions.  (see start of thread for URL)


Title: Re: [PATCH] implement 'listtransactions'
Post by: mizerydearia on September 15, 2010, 08:39:10 PM
I've also made the following changes:

  • Slight change so that JSON-RPC binds to all/any IPs instead of just the loopback.

This patch doesn't seem to be compatible with this patch (http://bitcointalk.org/index.php?topic=984.msg12291#msg12291).  Can these patches be combined or is one better than the other?


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on September 18, 2010, 07:44:33 PM
Patch updated to latest SVN.


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on September 29, 2010, 05:32:52 PM
Patch updated to latest SVN, and for a change suggested by gavin:

  • sort by time, after sorting by number of confirmations


Title: Re: [PATCH] implement 'listtransactions'
Post by: davout on October 18, 2010, 01:06:15 PM
Hello all !

Is this patch ever going to hit the main official vanilla client ?

I don't really get how it could be the 'wrong way' to track transfers for a web application as long as each user has a distinct address which really seems like an obvious pre-requisite for any kind of app that is going to support multiple users.

Without that patch it's not really possible for an user to have a single entry point for transfers to his account without doing some dangerous guesswork. And since, AFAIK, nothing prevents re-using by mistake a payment address that has already been used there is the possibility that some transferred funds will end up in some no man's land :/



Title: Re: [PATCH] implement 'listtransactions'
Post by: teppy on October 20, 2010, 02:35:39 PM
jgarzik,

Could you have listtransactions also include the comment field associated with a transaction? Here's the problem I'm running into:

Dragon's Tale MMO does periodic (every 3 minutes) checkpoints. If someone has withdrawn BTC since the last checkpoint and the server crashes, then when it comes back up (restoring the most recent checkpoint), they could withdraw the same BTC again!

I deal with the opposite case cleanly: When someone requests an address to use to deposit BTC, I label that address and use getreceivedbylabel when computing their balance: My server and bitcoind are always in agreement WRT how many BTC the person has deposited. But unless I can query bitcoind about how many BTC were sent out on behalf of a player, I just have to trust the value stored in the last checkpoint file - very dangerous!

My solution is to record a comment consisting of the player's name, in each outbound transaction, and then when doing a restore, read in the last N transactions to correct for unrecorded outbound transactions in the game's database.

FWIW, this is almost exactly the same technique we use to sync the game database and our credit card merchant's database in A Tale in the Desert.


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on October 21, 2010, 07:36:24 PM

Patch updated to latest SVN (v0.3.14).


Could you have listtransactions also include the comment field associated with a transaction? Here's the problem I'm running into:

Yeah, I think this is doable.  Will look into it.


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on November 13, 2010, 07:20:58 PM
Patch updated to latest SVN.  See top of thread for URL.


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on November 23, 2010, 12:04:58 AM
Updated to SVN r188 (accounts).


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on November 27, 2010, 07:04:28 PM
Patch updated to latest SVN.


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on December 02, 2010, 05:50:35 AM
Patch updated for latest SVN.

RPC has been renamed to 'xlisttransactions', because mainline is now including an incompatible version.


Title: Re: [PATCH] implement 'listtransactions'
Post by: jgarzik on December 17, 2010, 05:23:15 PM
This patch is officially considered deprecated, as of mainline SVN r204.


Title: Re: [PATCH] implement 'xlisttransactions'
Post by: jgarzik on February 24, 2011, 12:46:32 AM
This patch is now un-deprecated.  People continue to find value in a 'listtransactions' display that uses [almost] the same logic as the GUI's wallet display.

The patch has been updated to latest git.  See top of thread for URL.