JSON-RPC password

[satoshi]

I uploaded to SVN my changes to add a password to JSON-RPC.  If you're set up to build, please test it.

The -server switch is replaced with -rpcpw=<password>, which is also used with bitcoind.
bitcoin -rpcpw=<password>    -- runs with JSON-RPC port open
bitcoind -rpcpw=<password>   -- daemon with password

If you have a better idea for the switch name, let me know, but keep in mind there will eventually be a password for encrypting the database too.  I'm not sure but I think they may want to use different passwords for the two.

It gives a warning if you don't set a password.

All commands now require the password as the first parameter.  It'll tell you that if you run "bitcoind help".

The central code:

  // Check password
  if (params.size() < 1 || params[0].type() != str_type)
      throw runtime_error("First parameter must be the password.");
  if (params[0].get_str() != strRPCPassword)
  {
      if (strRPCPassword.size() < 15)
          Sleep(50);
      begin = strRequest.end();
      printf("ThreadRPCServer incorrect password attempt\n");
      throw runtime_error("Incorrect password.");
  }

Any comments on these decisions?

1) if (strRPCPassword.size() < 15) Sleep(50);  -- this means if it's a short password, it'll wait 50ms after each attempt.  This might be used as a DoS attack, but I figured if it's a short password, it's more important to protect against brute force password scan.  This may tell outsiders whether the password is less than 15 characters, but less than 15 isn't all that noteworthy, most passwords are less than 15.  If you want to close the DoS possibility, just use a password 15 characters or longer.

2) begin = strRequest.end();  -- if it's a single request with multiple invocations, I throw away the rest if one has a bad password.  This is so you can't stuff it with millions of password attempts in one packet.  What do you think, is this the right thing to do?  (multiple invocation is probably almost never used anyway)

I also fixed the two duplicated commands listed in the help:

getaddressesbylabel <pw> <label>
getbalance <pw>
getblockcount <pw>
getblocknumber <pw>
getconnectioncount <pw>
getdifficulty <pw>
getgenerate <pw>
getinfo <pw>
getlabel <pw> <bitcoinaddress>
getnewaddress <pw> [label]
getreceivedbyaddress <pw> <bitcoinaddress> [minconf=1]
getreceivedbylabel <pw> <label> [minconf=1]
help <pw>
listreceivedbyaddress <pw> [minconf=1] [includeempty=false]
listreceivedbylabel <pw> [minconf=1] [includeempty=false]
sendtoaddress <pw> <bitcoinaddress> <amount> [comment] [comment-to]
setgenerate <pw> <generate> [genproclimit]
setlabel <pw> <bitcoinaddress> <label>
stop <pw>

[1714175590]

1714175590

[1714175590]

1714175590

[1714175590]

1714175590

[1714175590]

1714175590

[laszlo]

I guess it's ok for remotely doing it but if your concern is that someone else on the same unix machine can steal your bitcoins this still doesn't help because they can see your command line in /proc, top, ps etc.  It could read the password on stdin or use readline or something, to guard against that particular thing at least.  Allowing it to be passed on the command line is not good, in my opinion.

Even better might be to use a key file, then you can use unix permissions to make it readable to only that user, kind of like ssh does.. then the bitcoind could have an 'authorized_keys' file with the public keys.  Anyway I don't mean to be an ass but the command line thing is just a false sense of security.

[martin]

I'm afraid I have to agree with laszlo here, using a certificate/keyfile would be far more secure. Saying that, thanks for adding some security to the JSON api

[satoshi]

Right, that is quite a bit better. 

Can you give me any examples of other stuff that does it that way?  (and what the command line looks like)

The main change you're talking about here is instead of -rpcpw= when you start bitcoind, you'd use a switch that specifies a text file to go and read it from, right?  (any ideas what I should name the switch?)

[Gavin Andresen]

The Bitcoin Faucets (production and TEST) are now running with this change.

I was confused for a bit because the password is given LAST on the command line, but FIRST in the JSON-RPC params list.  I agree that reading the command-line password from a file would be more convenient and more secure.

I'll try to do some research on how other projects tackle JSON-RPC authentication.

[Gavin Andresen]

The Transmission BitTorrent client does authenticated JSON-RPC; see "Remote Control" section of:
 https://trac.transmissionbt.com/wiki/ConfigurationParameters

E.g. setting.json file might look like:

Code:

{
    "rpc-enabled":1
    "rpc-authentication-required": 1,
    "rpc-password": "xxxxxxxxxx",
    "rpc-port": 9091,
    "rpc-username": "xxxxxxxxxx",
    "rpc-whitelist-enabled":1
    "rpc-whitelist":"127.0.0.1,192.168.*.*"
}

It uses HTTP 'basic' authentication (Authorization: basic base64(username:password) in the HTTP headers).

[Gavin Andresen]

After further research...  I think the Transmission approach, combined with the existing "only allow connections from 127.0.0.1" is a good short/medium-term solution.

Putting the username:password in a settings.json file in the Bitcoin directory aught to work nicely (since Bitcoin can already parse JSON).  And keeping the authentication stuff off the command line and in the HTTP headers instead of the JSON request params is nice and clean.

Long term, the "right" way to do authenticated, secure JSON-RPC is with client-side certificates and https.    But that looks like it would be a lot of work to implement and a big learning curve for users to figure out how to generate client-side certificates and how to get both sides of the JSON-RPC connection using them.   And I'm not even certain a full-blown client certificate solution would solve the problem of malicious Javascript making JSON-RPC requests via XMLHttpRequests to localhost; if the user installed the client certificate in the browser (because maybe there was a nifty JSON-RPC-powered web front-end to controlling Bitcoin), would the browser automatically send the client certificate if a malicious website made requests?

[satoshi]

So you drop a settings file in the ~/.bitcoin directory, that sounds better.  In the "no password is set" warning, it could tell you where the file is and what to do.

What is the most popular and common settings file format?

HTTP basic authentication should be considered.  In actual practice though, it's more work for web developers to figure out how to specify the password through some extra parameter in the HTTP or JSON-RPC wrapper than to just stick an extra parameter at the beginning of the parameter list.  What do you think?  Does HTTP basic authentication get us any additional benefits?  Moving it off the parameter list but then you still have to specific it in a more esoteric place I'm not sure is a net win.

I was confused for a bit because the password is given LAST on the command line, but FIRST in the JSON-RPC params list.  I agree that reading the command-line password from a file would be more convenient and more secure.

You're also confusing me, what do you mean?  Did I do something unintended?

[Gavin Andresen]

So you drop a settings file in the ~/.bitcoin directory, that sounds better.  In the "no password is set" warning, it could tell you where the file is and what to do.
What is the most popular and common settings file format?

You ask hard questions!  Most common: probably Windows INI files, because Windows is most common OS.

I'd lobby for using JSON; it's (mostly) a subset of YAML (which is a common choice for config files), so any JSON or YAML parser will read it.

HTTP basic authentication should be considered.  In actual practice though, it's more work for web developers to figure out how to specify the password through some extra parameter in the HTTP or JSON-RPC wrapper than to just stick an extra parameter at the beginning of the parameter list.  What do you think?  Does HTTP basic authentication get us any additional benefits?

I think the only big advantage is that it keeps authentication where it belongs in the transport layer, so if, in the future, you do want to go with full-fledged HTTPS with certificates the API doesn't have to change.

I was confused for a bit because the password is given LAST on the command line, but FIRST in the JSON-RPC params list.  I agree that reading the command-line password from a file would be more convenient and more secure.

You're also confusing me, what do you mean?  Did I do something unintended?

No, I just confused "command" with "parameter", and did this:

Code:

> bitcoind help
error: First parameter must be the password.
> bitcoind <my password> help
error: unknown command: <my password>
>bitcoind help <my password>
 ... that works.

[Vasili Sviridov]

Should bitcoind be more of a system-wide daemon? With PAM authentication and support for all system users, not just the one running it?
Like scanning /home/*/.bitcoin folders for a wallet file and provide access by both username and system password...

[laszlo]

If you're using another JSON-RPC client that you wrote you can take care to protect the password, but using the bitcoin binary as the client and passing the password on the command line has the same issue as starting the daemon with it.  It's still visible to every user that way.

So both the server and the client mode invocation need to use the file and not accept the password on the command line.  Generally programs like this refuse to start if the mode on the file isn't 600 or something like that, because that means other users can read it.

[satoshi]

Still need to know what's the most typical settings file format on Linux.  Is there a standard file extension?  I've never seen a settings file using JSON, and it doesn't look very human friendly with everything required to be in quotes.  I think what I usually see is like:
# comment
setting=value

Is there a settings file thing in Boost?

When you're using bitcoind to issue commands from the command line as a client, can we have it get the password from the settings file then too?

Gavin pointed out I forgot to increment the column of numbers in CommandLineRPC, so the current -rpcpw= implementation doesn't work right from the command line with non-string parameters.  (JSON-RPC is fine)  Still under construction.

[martin]

You could do worse than using yaml for the settings

[satoshi]

I was researching config file formats, here's a comparison.

YAML is massive.  I'm not sure there's a lightweight easy to build library we can integrate into our project.  Seems overkill.

JSON is tempting and I'm inclined to like it, but two main sticking points:
1) No comments!  How can you have a config file where you can't comment out a line to disable it?
2) Not very user friendly to have to "quote" all the strings, including the keys, and also have to remember the comma at the end of lines.
{
    "key" : "value",
}

I suppose we could easily preprocess JSON reading the config file one line at a time, truncate the lines at any # character (and/or "//"?), concatenate them into a string and pass it to JSON, so you could go:
# comment
"key" : "value",   # still have to remember the comma
"key2" : "value",   // comment like this or both

Boost has boost::program_options.

We could read lines ourselves and feed them into a map<string, string> mapConfig.

while (!eof)
  read line
  if '#' found, truncate line
  split line at first ':' -> key, value
  mapConfig.insert(key, value)

If we use the syntax:
# comment
key : value

...and don't allow whitespace indenting before the keys, I guess we would be a subset of YAML and could switch to YAML someday if we need more complexity.

If we go with self parsed, that doesn't mean we can't use JSON on particular parameter values as needed.  If an option needs a list or more structured data, it could always parse its value as json:
key : ["item1", "item2", "item3"]

Although it has to be all on one line then.

I guess I'm leaning towards self parsed mapConfig:
# comment
key : value

[lachesis]

The simplest Linux config files typically have formats that look something like this:

Code:

# how often (in minutes) data is saved when all interface are offline
OfflineSaveInterval 30

# force data save when interface status changes (1 = enabled, 0 = disabled)
SaveOnStatusChange 1

# file used for logging if UseLogging is set to 1
LogFile "/var/log/vnstat.log"

# file used as daemon pid / lock file
PidFile "/var/run/vnstat.pid"

The standard extension, if there is one, is .conf. Hash (#) is the most common comment character by far. Semicolon ( and C-style comments crop up too.

I think a good, human-manipulatable config file that ISN'T part of the wallet would be a big step forward. There are a lot of options that are currently specified on the command line (noirc, for example, or -minimizetotray) which might be better specified in a config file.

[martin]

Out of those options Satoshi presented, I'd go with the self parsed file with the key/value separator character being a space like lachesis suggested, especially if that's the typical format for at least one OS.

[Gavin Andresen]

I think there's no such thing a a "typical" settings file on Linux!

I just did a quick survey of 20 .conf files in /etc on my debian system, and found:
 1 file used "key value"
 5 used "key=value"  (actually, a couple were  "key = value", allowing whitespace around the "=")
 14 did their own thing.

The 14 that did their own thing were all over the map; from one-value-per-line to "key:value" to full-blown XML.  # is
the universal comment character in the Linux world.

My vote would be for:

# comment
key1=value1

[satoshi]

I just did a quick survey of 20 .conf files in /etc on my debian system, and found:
 1 file used "key value"
 5 used "key=value" 

Thanks for that survey!

I find "key value" a little unnatural.  There ought to be a more definite separator between key and value that suggests assignment.  The space people may just be getting lazy using their language's split function.
key=some full sentence with spaces in it.  # seems more clear
key some full sentence with spaces in it.  # than this

Allright then, lets go with self-parsed mapConfig, syntax:
# comment
key=value

file extension .conf.  What's the filename, is it ~/.bitcoin/settings.conf or ~/.bitcoin/bitcoin.conf or what?   

I think we better strip whitespace at the beginning and end of the key and the value.
# user who likes column formatted
k            = value
key         = value
longerkey =   this sentence would be this    # "this sentence would be this"
        key = value   # guess this is ok too
  nextkey = value
      right = justified

The normal syntax should be "key=value", but you can't blame people for the occasional "key = value".

[satoshi]

boost::program_options has the same "key=value" format.  Gavin pointed out we can use it in a simple way as a parser without getting into all the esoteric c++ syntax like typed value extraction.  We can use more features if we want later.

Lets go ahead with HTTP basic authentication instead of password as a parameter.

[Gavin Andresen]

I volunteered to implement this, and made good progress today.  Satoshi: I should have patches for you tomorrow.

Done: teach Bitcoin to read settings from {BITCOIN_DIR}/bitcoin.conf file, and added -conf=path_to_config_file.conf command-line option. 
Done: teach Bitcoin RPC to require HTTP Basic authentication, and reject requests with the wrong username/password.

TODO: teach Bitcoin command-line RPC to add the Authorization: header.  You won't have to give the username/password when controlling bitcoin from the command line, it'll read them from the bitcoin.conf file and Do the Right Thing.
TODO: dialog box or debug.log warning if no rpc.user/rpc.password is set, explaining how to set.
TODO: limit password guessing attempts if the rpc.password is < 15 characters long.
TODO: update the JSON-RPC wiki page

After all that is done and I've sent patches to Satoshi, I'm going to add a couple more things to bitcoin.conf :

port=   # to set the listen port (override default 8333)
rpc.port= # to set the JSON-RPC port (override default 8332)

With the existing -datadir option, that'll make it easier for me to run multiple bitcoins on one box.