fee_per_full_coin should be adjusted if the median fee, averaged over a prolonged period, becomes significantly higher (or lower). And even if you lag a bit with this adjustment, your utxo set will not degrade instantly. You can monitor the recent median fee and tell if your fee_per_full_coin setting is too low or too high - you basically want it so that with median fee, you will be mostly choosing averagely small txs (assuming your utxo set is already in good shape). 2 or 3 inputs seem like a good choice, but as the size of inputs varies depending on the properties of addresses that the wallet contains, the actual setting will be specific to the concrete wallet setup.


70 out of 100, more than 2x of usable amount, is quite a big extra, IMO.

An interesting concept. One of the imperatives that was set when we created our system was that the risks should be eliminated or mitigated to the maximum extent possible, so this was not considered. But this is quite easy to add, as an option for those who can tolerate the risk, so we might think about implementing it.

Another important requirement was that if there is a spendable balance on wallet, and a payment order comes to spend this balance, it should be sent - the developer using an API should not bother about if there enough big utxo available, or if the transaction limitation might be hit, or whatever - if there's enough funds for the order amount (and network fee, if the fee is taken from the wallet, and not from recipient)  - send it, use every utxo available, even if it means sending several txs.

CPFP means you just paying more fee in the child tx - if you wanted to use the result of the sweep soon, you could just have added more fee at the tx creation


If you do not conflate utxo from different wallets (or sub-wallets), there's no problem - user should have used different wallets/sub-wallets for different purposes, if this is a concern

Same. (see: https://simplexum.com), but it is not focused solely on coin selection, so probably you've done more work in this direction.


What would prevent this from scaling ?


Yes, but if you have too much small utxo, and you need to send large payment at inconvenient time (when fee is high) - you will pay a premium, as delaying payment is often not an option. Also you might hit transaction size limitations, and will need to send several transactions to satisfy one payment.

Holding both fair amount of big utxo and a lot of small utxo in hot wallet might mean your hot wallet balance is higher than necessary, and your risks are also higher.


But this means you are freezing this funds for a week, and need to get funds for operation from somewhere else, like your cold wallet.

Also, big sweeping transaction is more visible.

You can do special sweeping tx without this formula. The idea here is that if you do payments regularly, the fee in the network varies, and you set your fee_per_full_coin reasonably high - you will get gradual UTXO consolidation, without the need for separate sweeping transaction.

It may not be very useful for a single user wallet that spends occasionally, but for busy service that receives and sends a lot of payments - might mean that when they suddenly need to send a big payment, and the network fee is high - they won't need to send a huge-sized tx and pay a lot in fees, because their UTXO set will be in good shape.

For UTXO-based cryptocurrencies, it is important to have the set of managed Unspent Transaction Outputs in optimal shape, because if you have too much small UTXO, you will have to create bigger transactions to send the same amount of funds, and hence pay bigger network fee, or even need to send several transactions to transfer the needed amount.

Typical approach to handle this is to do periodical sweep of UTXO, bundling them all together, at the time when network fee is low.

We implemented another approach, that does not require manual UTXO consolidation, and at the same time, conserves network fee - when fee in the network is high, the algorithm tries to use less UTXO per transaction, and when network fee is low, it will create bigger transaction, effectively performing UTXO consolidation along with regular payments. This organic UTXO consolidation allows to maintain healthy UTXO set automatically, without manual consolidation runs.

We implemented it this way:

Define a rate that you are OK to pay in terms of monetary percentage: we call it "fee per full coin". Basically, you say, "to send 1 btc, I am willing to pay 0.001 btc"

When building a transaction, we calculate optimal tx size like this:

    optimal_tx_vsize = utxo_sum * fee_per_full_coin / fee_per_kb

The closer candidate transaction to the optimal size, the better score it will get.

Of course we cannot always find the optimal transaction, but we will try to choose the one with best score, depending on available UTXO set.

The resulting effect is that when fee_per_kb is high, optimal_tx_vsize will be low (may be even zero), and we will try to find the smallest tx possible with current UTXO set. When fee_per_kb is low, optimal_tx_vsize will be high, and we will choose transactions with more UTXO, performing organic UTXO consolidation.

---

If more wallets implement this type of optimizations, it will result in more healthy global UTXO set.

how we do it:

separate tx signer into independent node, that periodically connects to the main system to download transaction proposals, sign them and upload signatures. you protect this node to the maximum degree possible. can put it behind TOR so the attacker will have serious difficulties locating it.

make the signer track the spends and enforce money flow limits (in satoshi per second, and in total amount). it refuses to sign transactions that exceed the limits. limits are per-key, and if the key is HD, the limit is enforced for any descendent keys, collectively

for limiting the spending speed, we use (a variation of) an algorithm similar to that is used for network traffic shaping (https://en.wikipedia.org/wiki/Leaky_bucket), but with satoshis instead of bytes

The QR codes with encrypted privkeys can even be transferred via open channels, provided that the value on the address corresponding to the privkey is not too large.

Like, top manager have some qr codes in his wallet, and if the need arise, just snap a picture of one, and send to an employee via message app. Employee shows qr code on his phone to the camera, and it entered into the system.

Only signer node can decrypt the privkey, so there's little risk in sending it via open channel.

If someone steals encryption key used by the signer node, he can steal the amount of the top-up, but will reveal himself that way.

Or someone might steal a wallet from top manager to get his hands on more qr codes, but it is much more complex to perform - you need to both to get encryption key used by signer node, and target the top manager. You can also add another level of encryption, so that after the qr code is received from top manager, it needs to be decrypted with the key stored in the office, before it can reach the signer node. But this may be an overkill.

The signer node enforces money flow limits, and can be protected much better than the main system, because it does not require a lot of resources, and can be in a different location, but if someone takes over this node, and also the main system (if attacker does not control main system, an alerts will be sent out if it detects that funds were spent outside the system), and starts to siphon funds out, than yes - lower-level employess might be duped into thinking that it is OK to add another 1btc into hot wallet for now.

This can be mitigated with multisig - when you have more than one independent signer nodes in different locations (maybe even on different OS and hardware), that all enforce the same limits, and attacker needs to take over them all, to circumvent the limits. To use the paper wallet top-up with multisig you'll need several employees to scan several keys of one address to increase spendable balance.

Or this can be solved with administrative methods - like, employees need to report to the management that they just added X btc to hot wallet, and this way the management can monitor the usage and intervene if it feels not right.

Or both - multisig and administrative.

Signer node is a part of the system of hot wallet (or you can say 'warm' wallet - because it can do money flow limiting, like network traffic shaping, but for money).

Signer node is not a cold wallet - it cannot be, because it have to actually connect to the internet, at least periodically, to be able to download transaction proposals and upload signatures. For busy hot wallets you probably want it to connect to the main system once a minute.

For cold wallet, you download transaction proposals yourself, put them onto SD card, sign on offline system, and upload signatures to the main system manually (with a script)

If signer node goes down, you cannot use hot wallet, until you bring it back up. You can have several signer nodes for one key on stand-by, or in multisig setup - but that means you have more hosts to secure and protect.


That's what money flow limits are for - you set how fast the funds can outflow, and the maximum amount of money that can be sent out before manual intervention is needed . Even if the funds are available, the signer won't sign the transactions until money cap is increased (scan QR code with the command 'allow withdrawal of another 10 btc' - and this QR code is in big boss's safe - she can give it out only if convinced that the limit exhaustion is not suspicious)

maybe someone just wanted to encode something and put it into blockchain, but for some reason decided to not use op_return approach, and encoded it in this strange manner?

I try to create a scheme that will be as low-tech as possible, from the user's perspective.

Working with confidential paper documents have much longer history than working with digital documents, and the 'safe' and 'confidential paper' is the concepts that is understood by much larger pool of people, than the concept of signing requests with PGP.

For employees, in my view, the scheme should be very simple - "I take these envelope and show it to the camera. I do not give this envelope to anyone, because it bears a lot of value, and this is from my individual safe (we can have individual encryption keys per employee), so they will know that it is me who added this value to hot wallet"

The idea is to not access the cold storage at all - in a sense that when you need to top-up hot wallet, you do not create a transaction, do not sign it and do not send it.

The transaction is already in blockchain and is already confirmed, and as this can be done long before the top-up, the fees for this transaction can be minimal.

Everything is already in the system - the addresses and UTXOs are already known.

The wallet shows full balance (let's say, 123 btc), and spendable balance (for example, 0.3 btc).

We add one key, so hot wallet will be able to spend from one additional UTXO (value 1 btc)

The wallet shows full balance (123 btc) and spendable balance (1.3 btc)

We increased spendable balance without sending any transaction (it was sent long ago)

And we do not have to wait for funding transaction confirmation, or send 0-conf coins out.

Let's say an exchange got a hot wallet, and if there's more withdrawals than deposits, after some time it will need to move some funds from their cold wallet to hot wallet, to satisfy withdrawal requests.

Moving funds from cold wallet might require actions from high management, and will be visible in blockchain.

The idea to speed this up and allow less senior personnel to perform the top-up:

Generate 100 new addresses, encrypt their privkeys with symmetric key, print encrypted privkey (along with other metadata about the address) on paper as QR codes, put them into envelopes, and put envelopes into safe. Send 1 btc to each address. (edit: and of course you do not reuse these addresses! 1 address = 1 UTXO)

When there's a need to top-up hot wallet, three employees go to the safe, open it (while the procedure is recorded on video), take one envelope, put a log entry into paper journal, sign that journal entry with their manual signatures, and then open the envelope and scan qr code into payment system.

As the privkey is encrypted, and the decryption key is only present on separate signer node (that may be anywhere in the world and even connect to the main system via TOR), employees or anyone who sees that QR code cannot steal the funds.

There's no blockchain transaction at the time of top-up, and higher management is not needed to perform it, because there is much less trust involved, and with paper journal and video, the level of collusion between employees needs to be much higher for them to steal.

If someone takes over the signer node, he cannot steal the keys that are not yet scanned into the system.

(currently we can do similar setup, but with money flow limiter settings - for example, an employee can increase money flow limit from 0.3 btc/minute to 1 btc/minute by scanning a QR code with the command for the signer, but if anyone finds and takes over that signer node, and this is not a multisig setup - its game over)

We can try to use asymmetric encryption to encrypt private keys for addresses, and then the key to decrypt them need not be known at the time when QR codes are created, but using asymmetric encryption will mean that the data size will be much larger (256 bytes for RSA 2048) and the QR code needs to be larger. Private keys for addresses are much more valuable data, and we are working with them directly when we are encrypting them anyway, so I think symmetric encryption will suffice here.

I think we are going to implement support for this scheme in the near future, and I wanted to bounce my idea against a collective mind before diving into coding.

Is there anything that can be done to make this scheme more secure or more convenient ?

Do the exchanges already use similar schemes ? What challenges they face with these schemes ?

Is it too involved and complex ?