Bitcoin Forum
September 25, 2016, 03:40:47 PM *
News: Latest stable version of Bitcoin Core: 0.13.0 (New!) [Torrent]. Make sure you verify it.
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: JSON-RPC Multiple Invocations  (Read 4198 times)
lachesis
Full Member
***
Offline Offline

Activity: 210


View Profile
July 22, 2010, 02:09:49 AM
 #1

I noticed from a hint that Satoshi dropped in the JSON-RPC password thread about so-called "Multiple Invocation" support in Bitcoin's JSON-RPC.

As I run a site that polls bitcoind for payments to a large number of addresses twice a minute, I was intrigued. First of all, this isn't JSON-RPC 2.0's "Batch" support, where requests are submitted in an array and responses are received the same way:
Code:
request = [
        {"jsonrpc": "2.0", "method": "sum", "params": [1,2,4], "id": "1"},
        {"jsonrpc": "2.0", "method": "subtract", "params": [42,23], "id": "2"},
        {"jsonrpc": "2.0", "method": "get_data", "id": "9"}
    ]
response = [
        {"jsonrpc": "2.0", "result": 7, "id": "1"},
        {"jsonrpc": "2.0", "result": 19, "id": "2"},
        {"jsonrpc": "2.0", "result": ["hello", 5], "id": "9"}
    ]

Instead, it's something different, and I can't figure out how to parse the responses in Python. Here's a screen capture of a telnet session to the Bitcoin RPC server:
Code:
$ telnet localhost 8332
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
POST / HTTP/1.1
Content-Type: text/plain
Content-Length: 97

{"params":[],"id":1,"method":"getconnectioncount"}
{"params":[],"id":2,"method":"getdifficulty"}
HTTP/1.1 200 OK
Connection: close
Content-Length: 33
Content-Type: application/json
Date: Sat, 08 Jul 2006 12:04:08 GMT
Server: json-rpc/1.0

{"result":8,"error":null,"id":1}
HTTP/1.1 200 OK
Connection: close
Content-Length: 49
Content-Type: application/json
Date: Sat, 08 Jul 2006 12:04:08 GMT
Server: json-rpc/1.0

{"result":181.5432893640505,"error":null,"id":2}
Connection closed by foreign host.
As you can see, the server replies with two complete HTTP 200 responses instead of (as I would have expected) one response with the two lines concatenated as I did in the request.

I can't figure out how to parse that with anything at all semi-automated in Python. urllib2 and httplib both return after the first response and drop the second one on the floor.

Has anyone encountered this problem before? Does anyone know of a Python library that can handle this strange multi-request behaviour?

Bitcoin Calculator | Scallion | GPG Key | WoT Rating | 1QGacAtYA7E8V3BAiM7sgvLg7PZHk5WnYc
1474818047
Hero Member
*
Offline Offline

Posts: 1474818047

View Profile Personal Message (Offline)

Ignore
1474818047
Reply with quote  #2

1474818047
Report to moderator
1474818047
Hero Member
*
Offline Offline

Posts: 1474818047

View Profile Personal Message (Offline)

Ignore
1474818047
Reply with quote  #2

1474818047
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1474818047
Hero Member
*
Offline Offline

Posts: 1474818047

View Profile Personal Message (Offline)

Ignore
1474818047
Reply with quote  #2

1474818047
Report to moderator
lachesis
Full Member
***
Offline Offline

Activity: 210


View Profile
July 22, 2010, 03:18:36 AM
 #2

Alright, I know it hasn't been more than an hour since I last posted about this, but I decided to solve this problem myself.

If there's a better solution out there, I'd be glad to hear it, but here's my version 0.1 attempt to make this work:
http://www.alloscomp.com/bitcoin/btcjsonrpc.pys

Let me know if you like it, hate it, or see any obvious flaws with it. There's some getting started docs in the source, but here's the code of the test suite:
Code:
   from btcjsonrpc import Service
    s = Service()
    print 'preparing getbalance; id:',s.getbalance() # Each call returns its ID so you can find it later in the results
    print 'preparing getdifficulty; id:',s.getdifficulty()
    print 'preparing listreceivedbyaddress; id:',s.listreceivedbyaddress(10000) # Call with a parameter
    print 'preparing getbalance; id:',s.getbalance(id='getbalance 2') # You can also specify your own ID
    print '\nexecuting call\n\nresults:'
    results = s() # Get the results by calling the Service object
    for id,value in results.iteritems():
        print id,value
    # If you'd prefer to work directly with the JSON responses instead of a dict of IDs, then access the list Service.responses.
    print '\njson responses'
    print s.responses

And here's the output, including the docstr:
Code:
$ ./btcjsonrpc.py
 Socket-based, Bitcoin-compatible JSON-RPC v1.0 client.
 By: Eric Swanson (http://www.alloscomp.com/bitcoin)
 Version 0.1, July 21, 2010
Don't use this for one-off request->response pairs. Use something like the reference python-jsonrpc library, or urllib2 + json. This client is hackish, but it works for me (and has sped up my JSON-RPC accesses tremendously).

For details of WHY exactly I felt the need to redo python-jsonrpc using a raw socket, check out the follow forum post: http://bitcointalk.org/index.php?topic=528.0

Usage is fairly straightforward, and a code sample can be found below the library code (in the if __name__=='__main__' clause).

preparing getbalance; id: jss-1
preparing getdifficulty; id: jss-2
preparing listreceivedbyaddress; id: jss-3
preparing getbalance; id: getbalance 2

executing call

results:
jss-2 181.543289364
jss-3 []
getbalance 2 2345.94
jss-1 2345.94

json responses
[{u'id': u'jss-2', u'result': 181.54328936405051, u'error': None}, {u'id': u'jss-3', u'result': [], u'error': None}, {u'id': u'getbalance 2', u'result': 2345.9400000000001, u'error': None}, {u'id': u'jss-1', u'result': 2345.9400000000001, u'error': None}]

Bitcoin Calculator | Scallion | GPG Key | WoT Rating | 1QGacAtYA7E8V3BAiM7sgvLg7PZHk5WnYc
satoshi
Founder
Sr. Member
*
Offline Offline

Activity: 364


View Profile
July 24, 2010, 12:59:08 AM
 #3

Obviously it's a bug that it repeats the header.

I was trying to follow the 1.0 spec: http://json-rpc.org/wiki/specification   It called for multiple invocation.

I think they mean it's like this, but I'm not sure:

Post:
{"method": "postMessage", "params": ["Hello all!"], "id": 99}
{"method": "postMessage", "params": ["I have a question:"], "id": 101}

Reply:
{"result": 1, "error": null, "id": 99}
{"result": 1, "error": null, "id": 101}

I can't remember where I think I saw that it's supposed to send back HTTP status 500 for an error reply.  If it contains multiple responses and one is an error, I wonder if that makes the status 500 for the whole thing, I guess so.  Maybe it should always return 200.  I think someone sounded like the 500 might be causing a problem.

This probably gets fixed after 0.3.3.  Until then, just use single invocation.  I wonder if any JSON-RPC package even supports multiple invocation, probably not.

It would be nice if we could pin down better how multiple-invocation is supposed to work, if at all, before trying to fix it, and whether returning HTTP status 500 for error response is right.

lachesis
Full Member
***
Offline Offline

Activity: 210


View Profile
July 24, 2010, 02:36:35 AM
 #4

Yeah the 1.0 spec is a bit strange. JSON-RPC 2.0 pins it down quite nicely: a list of requests begets a list of responses. Afaik, none of the JSON-RPC client libraries handle the error code. They all just look at the "error" field of the JSON-RPC response. At the very least, I would return 200 if ANY of the requests succeed.

Bitcoin Calculator | Scallion | GPG Key | WoT Rating | 1QGacAtYA7E8V3BAiM7sgvLg7PZHk5WnYc
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!