Need Some logical help on UTXOS for my CEX.

[nonlogs]

In my CEX, I have built a fair withdrawal system for my users.

Users get to choose the withdrawal options such as fast, medium, or slow each with different network fees:

* Fast ($8.89 ~ arrival in 2 blocks)
* Medium ($1.89 ~ arrival in 6 blocks)
* Slow ($0.90 ~ arrival in 25 blocks)

I set this fee based on mempool congestion, updated every 15 minutes.

Now, the problem comes into play when there are hundreds of withdrawals happening every minutes. As you know, users can choose any option to withdraw, so the problem occurs when I don't have enough UTXOs to create the transactions. As you know, when a hot wallet spends UTXOs, it sends back the remaining UTXOs, and it takes 2 block confirmations to avoid block reorg. Other people who are trying to withdraw would then need to wait 20 minutes for the UTXOs to become available again.

I consolidate the utxo's in hot wallet so that txn size can be reduced (done periodically).

Suppose I could batch transactions into fast, medium, and slow then if I spend UTXOs in the fast batch, I cannot push the medium batch until I get the UTXOs back.

So this is the problem I'm facing.

[Hatchy]

It's quite complex to add this features to a centralized exchange because of the large number of traders you'd be handling. Most centralized exchanges have likely considered this which is why you don't typically see these features. When you place withdrawal you usually only get an estimated fee not an idea of how quickly your funds will arrive. I believe other CEXs handle this by batching multiple transactions and broadcasting them all at once.

I have two suggestions for you. First you could offer two withdrawal speed options, fast and medium. There's no need for a slow option. This would allow you to focus on prioritizing your unspent transaction outputs more effectively. You would also need to increase your UTXO reserves so there are enough funds for medium speed withdrawals without waiting for change from fast withdrawals.

Second, you could use two separate hot wallets, one for medium and one for fast withdrawals. You'd need to manage these carefully, perhaps by splitting your total available funds between them. Then, you can configure your system to draw from the appropriate wallet for either medium or fast transactions. It might not be the best idea though..

[Mia Chloe]

I think  your issue comes from UTXO contention due to the 2 block confirmation delay for spent outputs which kinda bottlenecks your tiered withdrawals especially for the fast option. I'm not a pro though  but I guess to fix this you should first adjust your UTXO selection algorithms to prioritize confirmed UTXOs for fast withdrawals and intelligently select from a larger pool for other tiers which should maintain a kinda buffer of readily available UTXOs.

I think you'll need to also improve your queuing system, instead of blocking, pre-select and reserve UTXOs for upcoming batches from other available UTXOs. Also you should add a constant and adaptive UTXO consolidation during low fee periods to combine fragmented UTXOs. Then also increase your number for wallets so UTXO don't become a problem.

[Stalker22]

I agree with Hatchy, two speed options would be enough. Fast and slow.  A "fast" option would mean paying a premium fee, ensuring inclusion in the next few blocks. A "slow" option would mean being part of the regular batched transactions.

The second suggestion, using multiple hot wallets, may not be an ideal solution because it increases the attack surface and adds layers of complexity. I think better UTXO management would be a smarter solution.  By creating multiple, roughly equal-sized UTXOs, you can select the most appropriate number of UTXOs for the exact amount being withdrawn, minimizing the size of the transaction (and thus the fee) for a given speed. This will also allow you to service a "fast" withdrawal without necessarily needing to wait for a change output from a larger, slower batch transaction.

[nonlogs]

It's quite complex to add this features to a centralized exchange because of the large number of traders you'd be handling. Most centralized exchanges have likely considered this which is why you don't typically see these features. When you place withdrawal you usually only get an estimated fee not an idea of how quickly your funds will arrive. I believe other CEXs handle this by batching multiple transactions and broadcasting them all at once.

I have two suggestions for you. First you could offer two withdrawal speed options, fast and medium. There's no need for a slow option. This would allow you to focus on prioritizing your unspent transaction outputs more effectively. You would also need to increase your UTXO reserves so there are enough funds for medium speed withdrawals without waiting for change from fast withdrawals.

Second, you could use two separate hot wallets, one for medium and one for fast withdrawals. You'd need to manage these carefully, perhaps by splitting your total available funds between them. Then, you can configure your system to draw from the appropriate wallet for either medium or fast transactions. It might not be the best idea though..

It is indeed complex that is why no CEX has implemented it yet, and that’s why I'm asking for suggestions. If it were easy, everyone would have done it. I got the same response from AI like what you said, and from my viewpoint, managing multiple hot wallets is very difficult because I have no idea who is going to request fast or medium. And even if I maintain equal amounts on both by splitting, some user's could choose medium and the balance would flip at any given moment.

[nonlogs]

I agree with Hatchy, two speed options would be enough. Fast and slow.  A "fast" option would mean paying a premium fee, ensuring inclusion in the next few blocks. A "slow" option would mean being part of the regular batched transactions.

The second suggestion, using multiple hot wallets, may not be an ideal solution because it increases the attack surface and adds layers of complexity. I think better UTXO management would be a smarter solution.  By creating multiple, roughly equal-sized UTXOs, you can select the most appropriate number of UTXOs for the exact amount being withdrawn, minimizing the size of the transaction (and thus the fee) for a given speed. This will also allow you to service a "fast" withdrawal without necessarily needing to wait for a change output from a larger, slower batch transaction.

I think most of the exchanges use consolidation of UTXOs to reduce the number of inputs so that it reduces the transaction size. But even if I run something like deconsolidation, it would still put a heavy load on my pocket by doing that periodically. I have to spend more network fees to keep a certain size of UTXOs. Even if I do that, it never guarantees that the user would request the same amount of withdrawal, and I would have to use UTXOs reserved for medium so that I can allow withdrawal for fast users or put the fast users on wait based on withdrawal time and process the medium user's withdrawal, because technically medium user requested before the fast user and we’ve got no UTXOs. If I don't follow this rule, those who are on medium would stay pending until all fast withdrawals are processed, and if there are thousands of withdrawals, it could even take days for a medium withdrawal to be processed.

[Sticky Bomb]

Second, you could use two separate hot wallets, one for medium and one for fast withdrawals. You'd need to manage these carefully, perhaps by splitting your total available funds between them. Then, you can configure your system to draw from the appropriate wallet for either medium or fast transactions. It might not be the best idea though..

You have spoken well Hatchy and I support using two separate hot wallets for this, permit me to add little more logics to your response, This is the deal, create a logic to filter withdrawals with respect to the highest fee (fast category), those of lower fees (Medium) and finally for the last (slow). Use one of the hot wallets for only fast category and the other second one to mix other transactions in the event there are more in the fast category than you can batch at once (for example, of you batch 100 transactions at once, you include all remaining from the fast category which maybe 60, then select 30 from the medium and finally 10 from the slow category or you can even leave out the slow category, only to add some of them after a given time frame), this would ensure that the transactions that are in the fast category are given adequate priority, while those in other categories receives minimal attention with respect to their position on the respective withdrawal queue).

because technically medium user requested before the fast user and we’ve got no UTXOs. If I don't follow this rule, those who are on medium would stay pending until all fast withdrawals are processed, and if there are thousands of withdrawals, it could even take days for a medium withdrawal to be processed.

Fast users are always top priority, but you can create a logic to address medium users transaction being considered stuck. Make a count of these transactions in each category filtered as I mentioned earlier and add them to the second hot wallet in percentages. Example 60% for the fast transactions, 35% of the medium and 5% of the slow transactions and keep it up like that. Additionally, you can create a withdrawal tracker to inform users who requested for withdrawals of their position in the queue. This would help soothe their nerves and give them hope that their transactions are in the process of being sent to the mempool and they can monitor the progress by themselves instead of bugging your support network.

[Stalker22]

I agree with Hatchy, two speed options would be enough. Fast and slow.  A "fast" option would mean paying a premium fee, ensuring inclusion in the next few blocks. A "slow" option would mean being part of the regular batched transactions.

The second suggestion, using multiple hot wallets, may not be an ideal solution because it increases the attack surface and adds layers of complexity. I think better UTXO management would be a smarter solution.  By creating multiple, roughly equal-sized UTXOs, you can select the most appropriate number of UTXOs for the exact amount being withdrawn, minimizing the size of the transaction (and thus the fee) for a given speed. This will also allow you to service a "fast" withdrawal without necessarily needing to wait for a change output from a larger, slower batch transaction.

I think most of the exchanges use consolidation of UTXOs to reduce the number of inputs so that it reduces the transaction size. But even if I run something like deconsolidation, it would still put a heavy load on my pocket by doing that periodically. I have to spend more network fees to keep a certain size of UTXOs. Even if I do that, it never guarantees that the user would request the same amount of withdrawal, and I would have to use UTXOs reserved for medium so that I can allow withdrawal for fast users or put the fast users on wait based on withdrawal time and process the medium user's withdrawal, because technically medium user requested before the fast user and we’ve got no UTXOs. If I don't follow this rule, those who are on medium would stay pending until all fast withdrawals are processed, and if there are thousands of withdrawals, it could even take days for a medium withdrawal to be processed.

Maybe you didnt understand what I meant. Im not suggesting a constant, periodic deconsolidation that would rack up fees. My idea is more about intelligently structuring your UTXOs during the initial consolidation process, rather than just focusing strictly on making everything one big input. 

When a withdrawal request comes in for a specific amount, if you only have large UTXOs, you are forced to spend one, get a large change output, and then potentially wait for that change to confirm before you can send another fast withdrawal right after. This is where the bottleneck for "fast" withdrawals comes in.

But if you break up your wallet balance into a bunch of smaller, equal-sized UTXOs at first (say ten 0.1 BTC inputs instead of a single 1 BTC input), you get way more flexibility.  If someone wants 0.3 BTC withdrawal, you just grab three of those 0.1 inputs, keep the transaction small and dont have as much change output to deal with.  This means you dont have to wait as long for changes to confirm before youve got more accessible money to handle the next withdrawal. The core idea here is to minimize the need for change outputs.