Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: bytemaster on August 01, 2010, 08:14:29 PM



Title: Multiple Wallets, one computer (multiple accounts)
Post by: bytemaster on August 01, 2010, 08:14:29 PM
I want to have multiple "accounts" with unique balances and to send and receive coins on a per-account basis.  The equiv. of having multiple wallets running at the same time.  

It would help to simply list the balance for each 'receiving address' and to specify a 'source' address when sending coins.  


Title: Re: Multiple Wallets, one computer
Post by: Insti on August 01, 2010, 11:14:23 PM
It would help to simply list the balance for each 'receiving address' and to specify a 'source' address when sending coins. 

+1


Title: Re: Multiple Wallets, one computer
Post by: lfm on August 01, 2010, 11:24:57 PM
I want to have multiple "accounts" with unique balances and to send and receive coins on a per-account basis.  The equiv. of having multiple wallets running at the same time.   

It would help to simply list the balance for each 'receiving address' and to specify a 'source' address when sending coins. 

Would it help to use the attached messages you have when sending to an IP address? It should be easy to record a "sending ip address" in that case also.


Title: Re: Multiple Wallets, one computer
Post by: bytemaster on August 02, 2010, 01:17:15 AM
Nope, I really need it on a per-receiving address so that it is easy to directly track the balances from customers paying me. 


Title: Re: Multiple Wallets, one computer
Post by: kwukduck on September 22, 2010, 06:59:52 PM
It's something im missing too, it would be nice to have seperate 'accounts' with each it's own balance and transactions. One for personal use, one for business, etc.


Title: Re: Multiple Wallets, one computer
Post by: lfm on September 26, 2010, 08:24:11 AM
I want to have multiple "accounts" with unique balances and to send and receive coins on a per-account basis.  The equiv. of having multiple wallets running at the same time.   

It would help to simply list the balance for each 'receiving address' and to specify a 'source' address when sending coins. 

I guess one way to do this would be use the -datadir=<dir>  command line arg to specify a data directory (other than the default "$(HOME)/.bitcoin/"). You would only run the client on one directory at a time. (Note the directory is relative to the current directory for the SERVER bitcoind. Its safest to specify a fully qualified directory starting at the root.) Also of course the client does not need to be running  on any specific directory nor running at all to receive payments to addresses in any/all of the wallets.




Title: Re: Multiple Wallets, one computer
Post by: nelisky on September 26, 2010, 10:14:31 AM
I want to have multiple "accounts" with unique balances and to send and receive coins on a per-account basis.  The equiv. of having multiple wallets running at the same time.   

It would help to simply list the balance for each 'receiving address' and to specify a 'source' address when sending coins. 

I guess one way to do this would be use the -datadir=<dir>  command line arg to specify a data directory (other than the default "$(HOME)/.bitcoin/"). You would only run the client on one directory at a time. (Note the directory is relative to the current directory for the SERVER bitcoind. Its safest to specify a fully qualified directory starting at the root.) Also of course the client does not need to be running  on any specific directory nor running at all to receive payments to addresses in any/all of the wallets.




Yeah, I've been considering how to best go about this for the web services. Right now multiple services on the same machine run multiple clients bound to separate rpc ports. It works... to a point.

What I want is a separate wallet per user, and that scales pretty badly, so what I thought could be done is have multiple wallets in memory at the same time, based on some key (which would be the subdir inside the datadir for the wallet) and the rpc calls would take a -wallet= switch. I just don't know how many wallets I would be able to keep in memory at the same time, and I would loose the ability to move money instantly amongst them, but that's ok, I guess.

A slightly more scalable alternative would be to lazily load the wallets on demand. I know wallets keep a bunch of information, but maybe we could limit that to own addresses and transactions?

Satoshi: are you planning on doing any of this? As I'm not familiar with this part of the code, it would suck to spend a week trying to figure out the best way to do this and then you pushing the perfect implementation to svn :)


Title: Re: Multiple Wallets, one computer
Post by: jimbobway on September 27, 2010, 03:53:38 AM
Can you just create a new windows user and log in with that user?  Bitcoin creates a directory per windows user, I think.


Title: Re: Multiple Wallets, one computer
Post by: theymos on September 27, 2010, 04:34:04 AM
You could create an additional database file with public keys associated with usernames/passwords. Creating an address logs the resulting public key to the new database (in addition to wallet.dat); CreateTransaction is modified to only select coins from those addresses associated with the account; the various "list transactions" commands are modified to only tally/show transactions to addresses associated with the account; new commands are added to transfer keys from one account to another. You could make it possible to transfer funds (without the keys) from one account to another without a real transaction, but this would involve much more complexity.

Implementation wouldn't be too difficult, I think (if you don't have to deal with the UI). The worst part would be dealing with the new database file and figuring out how to replace the existing authentication with your own stuff.


Title: Re: Multiple Wallets, one computer
Post by: throughput on September 27, 2010, 01:48:18 PM
Satoshi: are you planning on doing any of this? As I'm not familiar with this part of the code, it would suck to spend a week trying to figure out the best way to do this and then you pushing the perfect implementation to svn :)

I'm sure, Satoshi should better plan documenting the protocol as a standard, so we will have interoperable implementations,
free from his dictatorship and tyranny. That is solely my own opinion.

For the techical issue discussed: It is clear, that the "node" instance, that holds the blocks database, connects to the network,
exchanges data with peers and with users, is architecturally independent from the "wallet" instance(s).
Wallet(s) may be viewed as peer(s), that request the block data and publish transactions/blocks, like network peers do now.

So it is feasible to have "block chain" daemon, that will cache block database on disk and may also act as an intermediary
between the local users and "the network", so it will have no secret keys at all, no sensitive information.
And the local "wallets" will have no networking part, no network interoperability problems, except for connecting to the daemon. So there will be no "bind to port 8XXX" issue.
They will be free from the P2P stuff, will only work in a client-server paradigm, will trust the server for executing their requests, but will not trust the server with any secret information.
And so, they may be as simple, as system-on-a-chip cards, capable of iterating through the blocks/transactions list, but not required to store it. The block chain database may also return narrow results, parameterised by the list of public keys, like "getreceivedbyaddr" does now.

Generation should happen at admin's local wallet, not at the networking node. That means "wallets" will be permitted to publish not only transactions, but solved blocks too.

For current codebase this is a major refactoring, I'm sure Satoshi will veto on it, and he may have his own merits.
So the only hope is the alternative implementations, that aren't possible without the standard.


Title: Re: Multiple Wallets, one computer
Post by: nelisky on September 27, 2010, 01:56:05 PM
Satoshi: are you planning on doing any of this? As I'm not familiar with this part of the code, it would suck to spend a week trying to figure out the best way to do this and then you pushing the perfect implementation to svn :)

I'm sure, Satoshi should better plan documenting the protocol as a standard, so we will have interoperable implementations,
free from his dictatorship and tyranny. That is solely my own opinion.

For the techical issue discussed: It is clear, that the "node" instance, that holds the blocks database, connects to the network,
exchanges data with peers and with users, is architecturally independent from the "wallet" instance(s).
Wallet(s) may be viewed as peer(s), that request the block data and publish transactions/blocks, like network peers do now.

So it is feasible to have "block chain" daemon, that will cache block database on disk and may also act as an intermediary
between the local users and "the network", so it will have no secret keys at all, no sensitive information.
And the local "wallets" will have no networking part, no network interoperability problems, except for connecting to the daemon. So there will be no "bind to port 8XXX" issue.
They will be free from the P2P stuff, will only work in a client-server paradigm, will trust the server for executing their requests, but will not trust the server with any secret information.
And so, they may be as simple, as system-on-a-chip cards, capable of iterating through the blocks/transactions list, but not required to store it. The block chain database may also return narrow results, parameterised by the list of public keys, like "getreceivedbyaddr" does now.

Generation should happen at admin's local wallet, not at the networking node. That means "wallets" will be permitted to publish not only transactions, but solved blocks too.

For current codebase this is a major refactoring, I'm sure Satoshi will veto on it, and he may have his own merits.
So the only hope is the alternative implementations, that aren't possible without the standard.


Well put, the standard is certainly needed to move this into mainstream, and source code as documentation sucks. But that is slightly longer, more important goal, not one I will hold off on projects for. I'm sure you agree the standard definition alone will take a good amount of time, especially when people look at it and start saying what if, or wouldn't it be better if... Then we still have to develop standards compliant clients.

I'm not saying that effort isn't worth it, much the opposite, it's mandatory for the immediate future. This hack jobs we do are actually a great way to get to know how things are being done, thus gathering any knowledge to help in describing the protocol :)


Title: Re: Multiple Wallets, one computer
Post by: Gavin Andresen on September 27, 2010, 03:14:55 PM
The "label" mechanism (setlabel / getreceivedbylabel) is supposed to meet this need, but only solves part of the problem.

If the API was extended as I describe below, would it solve the same problems as having multiple wallets?

Proposal:

+ new send method: send TO a given bitcoin address specifically FROM the bitcoins sent to <label>
  (change generated would be automatically tagged with <label>)
+ add optional [label] param to getbalance.
+ new method: listsentbylabel
  (returns array of [ "address" : "bcaddresssentto", "amount" : x.yz, "confirmations": n ])

Each customer "account" would be a bitcoin <label>.  Account handling would look like:

Create account / create new address for account:
  getnewaddress [account_id_label]
   ... tell user "fund your account by sending coins to {the address returned}"

Customer withdraws/spends:
  sendfrom [account_id_label] [address] [amount]
   (FAILS if balance for that account too low)

Show customer their balance:
  getbalance [account_id_label]

Show customer their transactions in/out
  listreceivedbylabel [account_id_label]
  listsentbylabel [account_id_label]

---------

Seems to me this would be a much better direction to go in, rather than having separate wallet.dat files for each customer.  At the very least, backing up thousands of customer's wallet files would be inefficient and error-prone, and constantly switching between them would also be incredibly inefficient.



Title: Re: Multiple Wallets, one computer
Post by: nelisky on September 27, 2010, 03:33:59 PM
The "label" mechanism (setlabel / getreceivedbylabel) is supposed to meet this need, but only solves part of the problem.

If the API was extended as I describe below, would it solve the same problems as having multiple wallets?

Proposal:

+ new send method: send TO a given bitcoin address specifically FROM the bitcoins sent to <label>
  (change generated would be automatically tagged with <label>)
+ add optional [label] param to getbalance.
+ new method: listsentbylabel
  (returns array of [ "address" : "bcaddresssentto", "amount" : x.yz, "confirmations": n ])

Each customer "account" would be a bitcoin <label>.  Account handling would look like:

Create account / create new address for account:
  getnewaddress [account_id_label]
   ... tell user "fund your account by sending coins to {the address returned}"

Customer withdraws/spends:
  sendfrom [account_id_label] [address] [amount]
   (FAILS if balance for that account too low)

Show customer their balance:
  getbalance [account_id_label]

Show customer their transactions in/out
  listreceivedbylabel [account_id_label]
  listsentbylabel [account_id_label]

---------

Seems to me this would be a much better direction to go in, rather than having separate wallet.dat files for each customer.  At the very least, backing up thousands of customer's wallet files would be inefficient and error-prone, and constantly switching between them would also be incredibly inefficient.



That is a much better approach, agreed. Assuming one label spans multiple addresses, so a client can accept transactions from an imported priv key (this being discussed on another thread), it seems to fit my needs perfectly, with only one small caveat which is internal transfers; say I use this approach for the lottery users (though that is not my use case, I actually would be more about site separation than user separation, but still):
- Each account has on inbound address, annotated with the login label
- Every time a user withdraws, I use the sendfrom mechanism you describe
- The user hits the jackpot, but the prize is actually held in multiple users' accounts so
  a) I need to, when users buy tickets, move the amount to a separate address (forces external transaction, sucks)
  b) I leave coins where they are, and then on payout I make a normal sendtoaddress (both external transaction and breaks the label accounting logic)
  c) I leave coins where they are, payouts do nothing to the wallet, withdrawals do as many transactions as needed from each user account (what a nightmare!)

I don't see any good option here, and the best one, if I'm doing external transactions anyway, is a). I can trust these transactions with 0 blocks confirmed, because I'm both the sender and the recipient, and this would keep the accounts/labels in check, which serves as a great verification to the overall balances. Can you think of any alternative?


Title: Re: Multiple Wallets, one computer
Post by: Gavin Andresen on September 27, 2010, 05:10:33 PM
Separate "accounts" (addresses with labels) to accumulate Jackpots is the right idea.  Users buy tickets, bitcoins are moved to the appropriate Jackpot account.  When a Jackpot is won, transactions flow out of its account back to whoever won.


Title: Re: Multiple Wallets, one computer
Post by: nelisky on September 27, 2010, 05:16:18 PM
Separate "accounts" (addresses with labels) to accumulate Jackpots is the right idea.  Users buy tickets, bitcoins are moved to the appropriate Jackpot account.  When a Jackpot is won, transactions flow out of its account back to whoever won.


Yeah, it is quite obvious. I suffer for over optimization obsession syndrome :) I feel that as data flows only inside the application there's no need to register transactions and put that 'extra load' on the system, but quite frankly the extra load is perfectly ignorable, and the transactions help keep everything tidy. Are you planning to do anything on this regard? I'm sure I can find my way around to do it myself, it will just take time, but I don't want to duplicate any work though.


Title: Re: Multiple Wallets, one computer
Post by: satoshi on October 24, 2010, 07:17:51 PM
I have the beginning of something like this.  It's mostly like what Gavin described.

Some more rpc interface:

move <fromaccount> <toaccount> <amount>
   Move from one internal account to another.  I think blank account name ("") will be your default account.  If you sell something to a user, you could do move "theiraccount" "" 123.45.
   Is "move" the best name for this?  I shied away from "transfer" because that sounds too close to sending a transaction.

I'm thinking a new function getaccountaddress instead of overloading getnewaddress:

getaccountaddress <account>
   Gives you an address allocated from getnewaddress <account>.  It'll keep giving the same address until something is received on the address, then it allocates a new address.  (It automatically does what the sample code I posted some time ago did)

Would these commands make it possible in simple cases to implement your website without needing a database of your own?


Title: Re: Multiple Wallets, one computer
Post by: grondilu on October 24, 2010, 07:29:24 PM
Wouldn't it be much simpler to be able to run several instances of bitcoin, each of them running on a specific "wallet.dat" file ?

I mean, we would just add a --wallet long option to the client/server app.  The name of your account would be the name of your wallet file.


Title: Re: Multiple Wallets, one computer
Post by: SmokeTooMuch on October 24, 2010, 08:58:32 PM
I want to have multiple "accounts" with unique balances and to send and receive coins on a per-account basis.  The equiv. of having multiple wallets running at the same time.   

It would help to simply list the balance for each 'receiving address' and to specify a 'source' address when sending coins. 
use -datadir and change the port on the source code and then use a different dir and port for every bitcoin instance


Title: Re: Multiple Wallets, one computer
Post by: satoshi on October 25, 2010, 04:53:53 PM
Here's some pseudocode of how you would use the account based commands.  It sure makes website integration a lot easier.

print "send to " + getaccountaddress(username) + " to fund your account"
print "balance: " + getbalance(username, 0)
print "available balance: " + getbalance(username, 6)

// if you make a sale, move the money out of their account
move(username, "", amount, 6)

// withdrawal
sendfrom(username, bitcoinaddress, amount, 6)


Title: Re: Multiple Wallets, one computer (multiple accounts)
Post by: jgarzik on October 25, 2010, 05:18:58 PM

Why try so hard to prevent running multiple instances of bitcoin?

It is a valid use case to separate bitcoin processes for separate users using SElinux MAC protection, a protection far stronger than anything that can be coded into a shared user process.


Title: Re: Multiple Wallets, one computer (multiple accounts)
Post by: Gavin Andresen on October 25, 2010, 09:08:56 PM
If you're running something like mybitcoin or mtgox or any other site that lets customers keep bitcoins in accounts, then one-customer-per-bitcoin-process isn't at all practical.  To start with, every bitcoin process has a complete copy of the block chain...


Title: Re: Multiple Wallets, one computer (multiple accounts)
Post by: grondilu on October 25, 2010, 09:14:50 PM
If you're running something like mybitcoin or mtgox or any other site that lets customers keep bitcoins in accounts, then one-customer-per-bitcoin-process isn't at all practical.  To start with, every bitcoin process has a complete copy of the block chain...

Indeed.  I hadn't thought about this.  I guess there is also the problem of port mapping.


Title: Re: Multiple Wallets, one computer (multiple accounts)
Post by: ByteCoin on October 26, 2010, 01:40:50 AM
If you're running something like mybitcoin or mtgox or any other site that lets customers keep bitcoins in accounts, then one-customer-per-bitcoin-process isn't at all practical.  To start with, every bitcoin process has a complete copy of the block chain...

This would be one reason for splitting the bitcoin client functionality into two orthogonal parts.
One part would maintain the block chain, verify incoming transactions from the network and provide the interface for obtaining useful information from the block chain. It would neither contain nor have access to any private data. It would have large data files which need never be backed up but re-downloaded if lost. This would be the "server".

The other part would run the UI and allow the user to generate transactions, maintain the user's wallet(s), show the user's balance, incoming and outgoing transactions. It would contain the minimum of information that is also encoded in the block chain and the files would be small and easily backed up. This would be the "client"

Possibly hash generation could be a third independent process which is optimised for the individual CPU and other hardware. It requires no storage and just communicates with the server portion. Possibly the server portion could come with the default hash generator built in but most users with serious hardware like CUDA would run specialized hash generators using the interface on the server.

ByteCoin


Title: Re: Multiple Wallets, one computer (multiple accounts)
Post by: jgarzik on October 26, 2010, 01:53:52 AM
If you're running something like mybitcoin or mtgox or any other site that lets customers keep bitcoins in accounts, then one-customer-per-bitcoin-process isn't at all practical.  To start with, every bitcoin process has a complete copy of the block chain...

100% agreed, but that is not the only "multiple wallet" (multiple instance) use case.  This account system sounds useful, but it should not be used as a justification for avoiding the addition of "-bindport" feature, to change the default 8333 port.


Title: Re: Multiple Wallets, one computer (multiple accounts)
Post by: nelisky on October 26, 2010, 02:52:59 AM
Yeah, I'm already running multiple processes, but not one per client, rather one per service. It's not practical at all, keeping the ports under control, had to create bitcoind wrappers for each one of these or it was total hell.

I just don't see this scaling to one process per client, even if you don't have a huge number of clients.


Title: Re: Multiple Wallets, one computer (multiple accounts)
Post by: Gavin Andresen on October 27, 2010, 06:41:36 PM
An impromptu brainstorm this morning in IRC chat (thanks everybody!) helped me think through a few issues for an "accounts" API.

The big idea:

Replace the JSON-RPC "label" API with the notion of "accounts".

What is broken about the current API?
 + you can't get a list of all transactions that make up an account's balance
 + if your bitcoin service has the notion of maintaining a balance for multiple
   customers then you end up mirroring information stored in the bitcoin wallet database.
   Mirroring is a problem because if the connection between your service and bitcoin
   fails at the wrong time (e.g. between sending a "sendtoaddress" and getting
   back "sent") your database can be out of sync with the bitcoin reality.

Problems this proposal does NOT tackle:
 + multiple "wallets" for GUI bitcoin users
 + improving anonymity by keeping "coins" from different people in different "wallets"
 + "push" notifications from bitcoin when coins are received (or blocks generated)

NEW METHODS:
------------
getaccountaddress <account>
move <fromaccount> <toaccount> <amount>
sendfrom <account> <tobitcoinaddress> <amount> [minconf=1] [comment] [comment-to]
listtransactions <account> [minconf=1] [count=10] [start=0]

CHANGES TO EXISTING METHODS:
----------------------------
getbalance [account] [minconf=1]

listreceivedbyaddress:
  return "account" instead of "label" in JSON-RPC result

sendtoaddress:
  same API, but debits accounts as described below

METHODS RENAMED:
----------------
setlabel --> setaccount
getlabel --> getaccountfromaddress
getaddressesbylabel -> getaddressesbyaccount
getreceivedbylabel -> getreceivedbyaccount
listreceivedbylabel -> listreceivedbyaccount
  ... returns "account" instead of "label" in result


METHODS REMOVED (deprecated):
-----------------------------
setlabel



NOTES:
------

All existing routines would continue to be supported for a while (e.g. listreceivedbylabel would be kept as a synonym for listreceivedbyaccount, and would return both "account" and "label" in the JSON result).

Coins going into or out of the wallet that don't have an associated account will be associated with a default account (named the empty string: ""). So sum(account balances) will always equal server account balance.

Generated coins will be assigned to the default account when they mature.

sendtoaddress, and the GUI "send" button, will debit accounts starting with the default account (if it has a non-zero balance) and continuing in alphabetical (ascii) order.the default address, which will be allowed to go negative.

None of these changes will be visible in the graphical user interface. These changes are for people running bitcoind to support bitcoin-related services, not for end-users.

It would be nice to support transactions by batching up several commmands and ensuring that they either all succeed or all fail.  But: this should be useful even without that feature.  The cleanest way of doing that is JSON-RPC-2.0 "batch send", and that all can come in a later version.

Why remove setlabel?  Because it is not clear what "setaccount <bitcoinaddrress>" means for old transactions that were received on that address-- do they remain credited to the old account (confusing!) or does history get rewritten so they are credited to the new account (your accountant/auditor will surely protest!).

UPDATE: svn rev 188 implements most of this (all but gettransactions).  And due to the disaster recovery scenario described below, 'setlabel' stays as 'setaccount'.


Title: Re: Multiple Wallets, one computer (multiple accounts)
Post by: jgarzik on October 27, 2010, 07:20:43 PM
Comments:

  • Very nice, though this makes using at least one account mandatory?  If yes, I suppose we can create a default 'master' account, just like git creates a 'master' branch by convention.
  • I found it useful to label individual transactions, and will continue to find that useful even when using multiple accounts.  Labels remain useful as an ad hoc tx grouping mechanism (use case A) or an easy way to associate a user-generated (website-generated) unique transaction id with a bitcoin address (use case B)


Title: Re: Multiple Wallets, one computer (multiple accounts)
Post by: Gavin Andresen on October 28, 2010, 02:54:38 PM
RE: one mandatory account:  yes, the empty-string-named-account will be the "master" account.

RE: existing use cases:  you should be able to do everything you can do now... EXCEPT for change the label of bitcoin address after it is created.  Although associating a unique transaction ID to a bitcoin address seems like the wrong thing to do (since the same bitcoin address might be re-used for multiple transactions; your application might not allow that, but the addresses are publicly available in the block chain and you might be opening yourself up to unpleasant hacks if you don't consider what should happen if an old bitcoin address that you THOUGHT would be used only once receives a few bit-pennies....)

This morning I thought through what might happen in case of a disaster and a service using accounts had to restore from a wallet backup and some other, alternative source of transaction history (maybe application-specific transaction logs sent to an offsite logging server).

Assuming your backup is recent enough for your keypoolsize, no bitcoins will be lost, but account balances will be wrong because any "move", "sendfrom", and associate-bitcoin-address-with-account actions will be lost.

I've tweaked the proposed API to make recovery easier; I'm still thinking about "setlabel" -- in the recovery case, you definitely DO want to be able to re-associate bitcoin addresses with accounts...


Title: Re: Multiple Wallets, one computer (multiple accounts)
Post by: Alex Beckenham on May 16, 2011, 03:23:42 PM
Really old thread, but I couldn't find this discussed elsewhere...

What I am finding is that simply having 'accounts' isn't actually detailed enough and I'd love to be able to have subaccounts and subsubaccounts, etc.

Example:

[Australia]
   [Australian Web Site #1]
       [Bruce]
       [Kylie]
   [Australian Web Site #2]
       [Jack]
       [Nicole]
[New Zealand]
    [Kiwi Site #1]        
    [Kiwi Site #2]


Reason being that I'd like to know Bruce's balance for example, but the best I can do is find out the total balance of 'Australia'.

Currently to get Bruce's balance, I have to check his address with getreceivedbyaddress, and then keep track of any outgoing funds from this customer using other means such as SQL.