Securely Generating Bitcoin Addresses for Users

[vicknick]

I have a couple of questions regarding the handling of public/private keys on bitcoin services websites:

1. For those bitcoin services website, like bitcoin exchanges, how are they able to generate unique bitcoin address for each users? Do they use a BIP wallet to generate those bitcoin address, or some other methods?

2. Do they store those bitcoin address on their public web server? I am thinking that when you visit their website, their back-end will surely get your assigned bitcoin address from somewhere in their system, so they can check for the balance, so I am guessing bitcoin address are stored on their webserver. Will storing bitcoin address on webserver pose a security threat?

3. Likewise, do they store private keys of those addresses on webserver? If no due to security concerns, then how are users able to withdraw their bitcoin from the website, especially since the private keys are in cold-wallet and thus they are not connected to the public webserver?

4. Most website will have their own bitcoin addresses to pool in all the funds deposited from the users' bitcoin address. This addresses are then used for user's withdrawals. How are they able to check which user's public address have funds in it, so they can transfer the funds to their own address, especially if they have so many users? Also, how they able to handle user's withdrawal in this case, assuming the private keys of their own pool addresses are stored offline? What if users require instant withdrawal?

Hope someone can shed some lights on the system design of the bitcoin websites.

[BitcoinGirl.Club]

Hope someone can shed some lights on the system design of the bitcoin websites.

What I believe that, all of them has hot wallet and cold wallet. Whatever address they give to their users to deposit the funds, once the users made deposit they then move the funds in the cold wallet. I am not much technical guy in terms of bitcoin technology but I am sure someone else will be able to answer you the number by number.

[vicknick]

Yeah that's the only system I can think of which balances convenience for users, and security of their funds. Storing all in hot wallets is not secure, but storing all in cold wallet would require a human to periodically move all the users withdrawal's bitcoin address to the cold wallet's system, and transfer from there .

[2double0]

1. For those bitcoin services website, like bitcoin exchanges, how are they able to generate unique bitcoin address for each users? Do they use a BIP wallet to generate those bitcoin address, or some other methods?

Running their own Bitcoind for JSON rpc calls via php can let them generate a new address for each of their customers using the following json rpc call in php:

Code:

echo $wallet->getnewaddress();

[pooya87]

i can't really say what exchanges do because they aren't releasing their source code or any information about how they do it but one way could be this using BIP32:
- the exchange has 2 wallets, a hot wallet and a cold wallet. the cold wallet could be a single address (eg. multi-sig) and the key to it is kept offline. it holds the backup bitcoins and the bulk of their balance. the hot wallet is online and uses BIP32 so instead of one key it stores a master private key.
- when a user signs up it creates a new address for him at a path like this: m/42'/1'/0'/0 then for the next user it would be m/42'/1'/0'/1
- now their database has to only store the "index" of each user and the private key could either be derived on request or be stored alongside the username. for example each time the user makes a deposit it triggers something on the server that first gets the user index, derives his private key from the master key then adds the output to the queue so that next withdrawal request (by another user) is paid with that output.
- it also triggers another event that adds the value of that output to the total hot wallet balance and if it exceeded the set threshold it makes a transaction sending the excess amount to the cold storage.

[HCP]

1. For those bitcoin services website, like bitcoin exchanges, how are they able to generate unique bitcoin address for each users? Do they use a BIP wallet to generate those bitcoin address, or some other methods?

Yes

It's impossible to know as most don't openly advertise how they handle these types of things. I'm sure there are websites that are using "HD" wallets to generate keys from seeds, and others that simply use the old "RNG" method to just randomly generate keys as required. Others probably use hardware wallets etc.

How long is a piece of string?

2. Do they store those bitcoin address on their public web server? I am thinking that when you visit their website, their back-end will surely get your assigned bitcoin address from somewhere in their system, so they can check for the balance, so I am guessing bitcoin address are stored on their webserver. Will storing bitcoin address on webserver pose a security threat?

Addresses aren't really a "security threat" as such... they are after all, ALL published within the blockchain data for everyone to see when they receive coins.

I would assume that they are storing their address list somewhere... whether that is on their public webserver or not is probably more a function of performance and latency than anything else.

3. Likewise, do they store private keys of those addresses on webserver? If no due to security concerns, then how are users able to withdraw their bitcoin from the website, especially since the private keys are in cold-wallet and thus they are not connected to the public webserver?

If they are storing private keys on a webserver, they are pretty stupid. Private keys are called "private" for a reason. Putting them anywhere near a public facing server is a recipe for disaster.

Users don't generally create a transaction, sign it and then broadcast it... they will create a "withdrawal request", which is then processed by the system. As long as the system has access to the appropriate keys, that's all that really matters.

4. Most website will have their own bitcoin addresses to pool in all the funds deposited from the users' bitcoin address. This addresses are then used for user's withdrawals. How are they able to check which user's public address have funds in it, so they can transfer the funds to their own address, especially if they have so many users? Also, how they able to handle user's withdrawal in this case, assuming the private keys of their own pool addresses are stored offline? What if users require instant withdrawal?

I would think that most of the sites don't do it like this. Instead, each user will have an account (SomeUser) and some associated Database values (like AccountBalance, DepositAddress etc). Each DepositAddress is assigned to the user either semi-permanently (and re-used for each deposit, I've seen a couple of gambling sites in particular that do this) or they automatically get assigned a new DepositAddress whenever the current one is "used" and has received funds for the first time (most exchanges seem to do this).

SomeUser then logs in, and clicks on the "deposit BTC" link... they are presented with the DepositAddress that is currently linked in the backend database with their SomeUser account. SomeUser then sends some BTC from their external wallet to DepositAddress. Custom software (or maybe a modified version of Bitcoin Core etc), monitoring the sites wallet addresses, detects a transaction to DepositAddress. Once that transaction is confirmed, the "AccountBalance" for SomeUser is then incremented by value of the transaction.

When a user wants to withdraw, they issue a request to withdraw a certain amount of BTC (AmountRequested) to a certain address (WithdrawAddress)... The system simply checks that SomeUser.AccountBalance >= AmountRequested and that WithdrawAddress is a "valid" BTC address. If they are, then the backend can either immediately create a transaction that sends AmountRequest to WithdrawAddress using whatever hotwallet funds they have (basically using funds from ANY address(es) in the sites hot wallet)... or, if they're a site that likes to batch transactions, they can add the request to the queue. At this point, SomeUser.AccountBalance will be decremented by AmountRequested (+fee )... and most likely a "pending" transaction entry will be added to their account history etc.

Scheduled tasks could be used to shift funds from hot wallet deposit addresses to cold storage... or you could simply do it whenever hot wallet balance exceeds a certain threshold. Likewise the refilling of hotwallet is likely done when the balance falls below a certain threshold to ensure enough funds are available to satisfy "instant withdrawal".

Obviously there is a lot of room for debate as to which is the "best" method for these types of systems... or what is an "acceptable" amount to hold in a hot wallet before shifting to cold... or what is minimum amount required in hot wallet. It will be very much site specific and vary according to the level of risk the site is willing to carry, what their incoming/outgoing transaction numbers/values are etc.


Note that the above is a mixture of guesswork/assumption and casual conversation with site owners on how they handle things in "general terms" without low level specifics etc.

[vicknick]

If they are storing private keys on a webserver, they are pretty stupid. Private keys are called "private" for a reason. Putting them anywhere near a public facing server is a recipe for disaster.

Users don't generally create a transaction, sign it and then broadcast it... they will create a "withdrawal request", which is then processed by the system. As long as the system has access to the appropriate keys, that's all that really matters.

Then I assume this processing system will have to get the private keys somewhere to create a transaction. Wouldn't this mean the private keys are linked to the system and thus, the public webserver? Because when the server is hacked, the hacker will be able to trace to where the private keys are stored?

So far, the only system I can think of which is secure is to have the processing system cut off from the webserver, where staffs will manually check for withdrawal request on the webserver, write down the user's id and withdrawal address, and then move to the offline processing system to create a transaction based on the user's id. Correct me if I am wrong.

Custom software (or maybe a modified version of Bitcoin Core etc), monitoring the sites wallet addresses, detects a transaction to DepositAddress. Once that transaction is confirmed, the "AccountBalance" for SomeUser is then incremented by value of the transaction.

Will the transaction id gets recorded in the database as well? Because I assume that when the server need to check/audit the user's balance, they will have to know which transaction have already been added before?

Also, are there any software/website that have this feature to monitor an address, and notify once someone made a deposit to it?

If they are, then the backend can either immediately create a transaction that sends AmountRequest to WithdrawAddress using whatever hotwallet funds they have (basically using funds from ANY address(es) in the sites hot wallet)...

What if the hot wallets have lower funds than the requested amount? I thought the blockchain only allows one-to-one wallet transaction? Pooling funds together into one large wallet will avoid this issue?

Lastly, even if the hacker doesn't have the private keys, wouldn't changing the balance/data in the webserver database will allow hacker to withdraw the amount they want?

[o_e_l_e_o]

Wouldn't this mean the private keys are linked to the system and thus, the public webserver?

Potentially. Most exchanges use a combination of a hot wallet which will automatically perform withdrawals as requested, with the majority of funds in a cold wallet which would require staff intervention to access.

So far, the only system I can think of which is secure is to have the processing system cut off from the webserver, where staffs will manually check for withdrawal request on the webserver, write down the user's id and withdrawal address, and then move to the offline processing system to create a transaction based on the user's id.

Yes, a system where private keys are stored either on a hardware wallet or an airgapped device will require some manual intervention by staff to process a transaction. The vast majority can be automated though - once the live server has had enough withdrawal requests, it can generate a transaction and save it to a file, and all the staff member would have to do is take the file to air gapped device, sign it, and then return it to the live server for broadcast. Most exchanges don't do this though - withdrawals are processed automatically from hot wallets, and these hot wallets are topped up manually from cold wallets as required.

Will the transaction id gets recorded in the database as well?

Not necessarily. As long as the system has a note of which addresses were assigned to which users, then it can look up any previous transactions on the blockchain.

Also, are there any software/website that have this feature to monitor an address, and notify once someone made a deposit to it?

I couldn't tell you what software major exchanges are using to do this, but if you are interested in monitoring addresses at a personal level, sites like https://www.blockonomics.co/views/wallet-watcher.html should do the trick.

What if the hot wallets have lower funds than the requested amount?

Then the withdrawal will be delayed until a staff member can top up the hot wallet from a cold wallet.

I thought the blockchain only allows one-to-one wallet transaction? Pooling funds together into one large wallet will avoid this issue?

Transactions can have inputs from multiple addresses and outputs to multiple addresses. Its common for exchanges to pool all their funds in to one address, wait until they have a reasonable number of withdrawal requests (such as 50 or 100), and then process them all in a single transaction to save on fees.

Lastly, even if the hacker doesn't have the private keys, wouldn't changing the balance/data in the webserver database will allow hacker to withdraw the amount they want?

If they could update the balance on their account without being detected, then theoretically yes.

[Theb]

Do they store those bitcoin address on their public web server? I am thinking that when you visit their website, their back-end will surely get your assigned bitcoin address from somewhere in their system, so they can check for the balance, so I am guessing bitcoin address are stored on their webserver. Will storing bitcoin address on webserver pose a security threat?

I'm not that very familiar on how crypto exchanges handle it but I know custodial wallets like Coinbase stores your private keys offline using cold storages that they claimed are stored in "security boxes and vaults" which they also claimed are distributed around the world. Also both private keys and wallets are also encrypted via AES-256 encryption before they store it in the vaults. So what is all that is left is your public address found not only in their servers but also in Bitcoin's blockchain and hackers can't really do anything on your BTC not unless they managed to get into your account.