Best Approach to Processing Bitcoin Deposits Using External APIs

[maxloop]

Hello!

I would like to process deposits, but I am not quite sure how to best go about it. Unfortunately, I cannot and do not want to operate my own node at the moment (because then I have to download the entire blockchain and it is complicated to work directly with the node), to process deposits, so only an external API comes into question.

Here's the scenario: I have only one address to which all deposits should go. My service works without accounts and the sender's address is the UserID. The user should be directly shown on the website that the transaction has been received. In addition, the confirmations of the transactions should be updated and displayed promptly. But only up to 6 confirmations. After that, the deposit is successful, and we do not need to query the additional confirmations. It is important to me that if my web server goes offline, the system also processes the deposits during the time the server was offline.

My solution with which I am not quite satisfied and I hope there is a better and more efficient solution: I query all transactions of my deposit address every 2 minutes. But this also gives me the old deposits that have already been processed. Therefore, the server must first look at which deposits are new to include them in the database. Then I still run a websocket, which immediately informs me when there is a new block. And if that is the case, I query the transaction details for all deposits that still do not have 6 confirmations, to get the current number of confirmations. And I only do that whenever a new block was calculated, since no new confirmation can happen before that. And then I query all transactions of my deposit address again every 2 minutes and look for the new deposits that are not yet in the system. And again and again.

My problem with my solution: The API I currently use only outputs the last 10,000 transactions. This means that as soon as 10,001 transactions come in new within 2 minutes, then one transaction is lost. Although that is already a very unrealistic scenario to receive that many deposits in such a short time. However, it also bothers me that it is not really efficient. Because I always query all transactions and then have to sort out again and again, although I only need the new transactions. I hope that there might be a better way. I thought I could ask Blockchair for transactions within a certain period of time. So if the last query was for the period 12:04:00 to 12:06:00, then I query the transactions in the time between 12:06:00 to 12:08:00 in the next API call. And if there is ever downtime, then I just take the time of the last retrieval and the current time. But Blockchair somehow does not allow filtering by date. And other providers do not offer that either...

My question: How would you proceed? What is the best and most efficient way to process deposits? Deposits should always go to just one address and be booked as successful at 6 confirmations. In addition, users should always be promptly informed about new confirmations. Maybe you also know directly a provider with a good and affordable API... I would be very grateful. I'm still a bit new to the topic.

Many, many thanks and have a good day!

[BitMaxz]

That's complicated I suggest if this is all about accepting Bitcoin payments you can use BTCpayserver and skip downloading the whole blockchain.
Deploying BTCpayserver in your own hardware is much better if you have Raspi you can run BTCpay without syncing by enabling pruned mode.
Here's how to skip syncing https://docs.btcpayserver.org/FAQ/Synchronization/#can-i-skip-the-synchronization

Instead of using the same address every 2 minutes, BTCpayserver gives a unique BTC address derived from your wallet.


Or if you don't want to run it on your hardware there is an alternative way you can use https://www.blockonomics.co/merchants
The first 20 transactions are free above transaction it ask for a 1% fee for every transaction. This is easier to manage it only requires your xPub or your wallet master public key meaning they don't have full control of your wallet all unique addresses are generated from your wallet.

I've tested both of them on WordPress and Shopify.

[maxloop]

I don't understand why it's so complicated. Cryptocurrencies are meant to be used for payments. So why is it so hard to process them? Even the API providers on the market don't seem to offer really optimal solutions. That's strange. I'm going to take a look at BTCpayserver. However, I also need a solution for Ethereum and Litecoin. And I definitely want to use only one deposit address. I don't want new addresses to be generated all the time.

It's very disappointing and incomprehensible that it's so complicated. But if I run my own node, then it's much easier to process deposits, right? Or does it also require a lot of makeshift solutions?

Thank you for your help!

[alextechlead]

You can use payment gateways to handle this. Sites like CryptoApi and nowpayments offer this service. Hosting the complete blockchain on your server is very resource intensive and harder to code if i'm honest

[alexeyneu]

quicknode (they have option to add bcoin which i've used) . Only somewhat hardcore part is to set up ssl with them. Balance and this stuff is easy then.

Code:

       hostent *host_btc = gethostbyname("xxx-xxx-xxx.bcoin.quiknode.pro");
       sockaddr_in server_btc = {AF_INET, htons( 443 ), *((unsigned long*)host_btc->h_addr)};


        q = socket(AF_INET , SOCK_STREAM , 0);
        connect(q, (sockaddr *)&server_btc, sizeof(server_btc));
        sslctx = SSL_CTX_new(bxf);
        cSSL = SSL_new(sslctx);
        SSL_set_fd(cSSL, q);
        char wtf[] = "xxx-xxx-xxx.bcoin.quiknode.pro";
        SSL_ctrl(cSSL, SSL_CTRL_SET_TLSEXT_HOSTNAME, TLSEXT_NAMETYPE_host_name, (void *)wtf);
        SSL_connect(cSSL);
        fbtcquest << fvea.str().length() << "\r\n\r\n" << fvea.str();
        SSL_write(cSSL, fbtcquest.str().c_str(), fbtcquest.str().length());
        memset(server_reply, 0, 5000);
        long long h = SSL_read(cSSL, server_reply, 4400);

i mean that that you need to use SSL_ctrl()

[BitMaxz]

I don't understand why it's so complicated. Cryptocurrencies are meant to be used for payments. So why is it so hard to process them? Even the API providers on the market don't seem to offer really optimal solutions. That's strange. I'm going to take a look at BTCpayserver. However, I also need a solution for Ethereum and Litecoin. And I definitely want to use only one deposit address. I don't want new addresses to be generated all the time.

It's very disappointing and incomprehensible that it's so complicated. But if I run my own node, then it's much easier to process deposits, right? Or does it also require a lot of makeshift solutions?

Well for me it's complicated it's technical I let handle my payments with BTCpay it's open-source and you can add any coins or tokens you want to accept you just need to learn how to add them.

Read this

With using the additional token configuration you will be able to have separate payment methods for each configured Currency, Asset, Altcoin or Token. This means you can have separate payment methods for BTC, Lightning Network, LTC, ETH (and ERC20 tokens), Liquid assets, ... you name it. This allows you to issue and use Liquid Assets as coupons or vouchers, see more details below

Source: https://docs.btcpayserver.org/FAQ/Integrations/

Meaning LTC and Ethereum are both supported.
Using only one deposit address will make the payment gateway confused about who pays you. They fix this to give a unique address to every customer it should be easier to manage than having only a single deposit address. Look what if you have many customers buying something from your website at the same time so how do you know who pays you? That's why this payment gateway came to fix this kind of issue for any store owners.

[maxloop]

quicknode (they have option to add bcoin which i've used) . Only somewhat hardcore part is to set up ssl with them. Balance and this stuff is easy then.

What do you mean with "they have the option to add bitcoin"? So I can tell them I want to use their Bitcoin API and then they enable it for me? But why don't they offer it regularly?

Unfortunately, they do not offer Litecoin. Many other unknown coins, but no Litecoin, which is used extremely often for payments. Very strange.

Read this

With using the additional token configuration you will be able to have separate payment methods for each configured Currency, Asset, Altcoin or Token. This means you can have separate payment methods for BTC, Lightning Network, LTC, ETH (and ERC20 tokens), Liquid assets, ... you name it. This allows you to issue and use Liquid Assets as coupons or vouchers, see more details below

Source: https://docs.btcpayserver.org/FAQ/Integrations/

Meaning LTC and Ethereum are both supported.
Using only one deposit address will make the payment gateway confused about who pays you. They fix this to give a unique address to every customer it should be easier to manage than having only a single deposit address. Look what if you have many customers buying something from your website at the same time so how do you know who pays you? That's why this payment gateway came to fix this kind of issue for any store owners.

Thanks. So with BTCpayserver I can process deposits through Litecoin and Ethereum as well? Without fees? I will look into it. But its strange that the name is btcpayserver, when its not just for Bitcoin.

And I know, but I have to run everything via one payment address. There's no other way. Thanks!

[BitMaxz]

Thanks. So with BTCpayserver I can process deposits through Litecoin and Ethereum as well? Without fees? I will look into it. But its strange that the name is btcpayserver, when its not just for Bitcoin.

And I know, but I have to run everything via one payment address. There's no other way. Thanks!

BTCpayserver doesn't ask for any fees all transactions go directly to your own wallet if you have a hardware wallet like Trezor then all transactions will be deposited to that wallet.

I agree the name is strange it was created before for Bitcoin only as time passed by they also developed adding other coins.

What wallet do you use why still insist on using one payment address? if you are using a custodial wallet BTCpay will not work because it requires xPub or a master public key.

[alexeyneu]

quicknode (they have option to add bcoin which i've used) . Only somewhat hardcore part is to set up ssl with them. Balance and this stuff is easy then.

What do you mean with "they have the option to add bitcoin"? So I can tell them I want to use their Bitcoin API and then they enable it for me? But why don't they offer it regularly?

Unfortunately, they do not offer Litecoin. Many other unknown coins, but no Litecoin, which is used extremely often for payments. Very strange.

bcoin is a library that provides tracking of the balance on addresses. So you can use it via api then. Yeah no litecoin there

[maxloop]

What wallet do you use why still insist on using one payment address? if you are using a custodial wallet BTCpay will not work because it requires xPub or a master public key.

It's not about the wallet, it's about my project concept. There can only be one deposit address for it. So this won't work with BTCpay, because BTCpay always generates new addresses?

bcoin is a library that provides tracking of the balance on addresses. So you can use it via api then. Yeah no litecoin there

Oh, I see, then I am dependent on a specific client. I thought I could just make a regular API call to request transaction data. That's not so nice. But it could be noted as a temporary solution.


I wonder how Bustabit or Luckybit processes or processed deposits. I'm probably imagining it too simply to process deposits. All I want is to save new deposits into my database and keep checking them until they have 6 confirmations.

[DannyHamilton]

I would like to process deposits, but I am not quite sure how to best go about it. Unfortunately, I cannot and do not want to operate my own node at the moment

Here's the scenario: I have only one address to which all deposits should go.

No. Just no.  A bitcoin address is like an invoice number. You give out a new one for every deposit. That way you know exactly who you gave the address to and when.  Then when a payment shows up on that address, you know who's deposit it is, since htey are the only one you gave the address to.

My service works without accounts and the sender's address is the UserID.

If I send you a single deposit in a single transaction, the bitcoins you receive from me will have been received by me at 5 different addresses. How are you going to decide which "sender's address" is my userID?  Do I now have 5 userID's, just because I sent you one transaction? this is not the right way to do this. You're making it complicated and you clearly are trying to write software without any understanding of how any of this actually works.  This is a bomb with the fuse already lit.  Please stop until you've learned a bit more and you've re-architected your service to make the best use of what's available to you.

The user should be directly shown on the website that the transaction has been received. In addition, the confirmations of the transactions should be updated and displayed promptly. But only up to 6 confirmations. After that, the deposit is successful, and we do not need to query the additional confirmations.

Quite easy to do if you give out a new address for each depoist.

It is important to me that if my web server goes offline, the system also processes the deposits during the time the server was offline.

Another good reason to use new addresses for every deposit, and to build recovery processes into your system for handling downtime.

My solution with which I am not quite satisfied

I wouldn't be either from what I've read so far.

How would you proceed?

I'd start over. I'd make sure I understood how bitcoin works. I'd take advantage of the tools available. I'd run a node. I'd process transactions and blocks as they are received by the node. I'd create a recovery process that can scan the most recent X blocks. I'd create backups/snapshots of point in time to recover from if needed. I'd receive all deposits into cold wallets whose private keys have never been online. I'd maintain a hot wallet with a small but adequate balance if I need to also send out payments. I'd have very strict and careful process in place to fund the hot wallet.

Deposits should always go to just one address

Again, no.

I don't understand why it's so complicated. Cryptocurrencies are meant to be used for payments. So why is it so hard to process them?

It's not.  You are choosing to MAKE it hard by artificially putting constraints on yourself and refusing to architect your product/service properly.

I definitely want to use only one deposit address. I don't want new addresses to be generated all the time.

Then you don't understand how bitcoin works. This would be like saying "I want to accept U.S. physical cash, but I don't want to anything made of metal or or cotton-linen blends. Why is this so difficult to do?"

If I run my own node, then it's much easier to process deposits, right?

Easier?  That depends on your capabilities, knowledge, and skills. But it's certainly more reliable and less prone to the whims of a service provider.

Or does it also require a lot of makeshift solutions?

That depends on how complicated the design and architecture of your product/service is.

[alexeyneu]

you can use it. bitcoin itself has no api for checking balance on external addresses that're outside of node

[maxloop]

Thanks @DannyHamilton for the answer. It also feels good to be hit with such clear words. So thank you for that! However, I must remain stubborn and unfortunately cannot be persuaded to use anything but a single deposit address. And the matter with the userID is not a problem either, I just don't want to reveal exactly how I do it. It's not so relevant anyway. The assignment to users would then be a different topic.

It can be thought of as a donation address. And this address remains the same. I would like to now record the donations that come in into my database. But it's also good to hear that it's indeed not so simple. I do know quite a bit about cryptocurrencies, as long as it doesn't get too specialized (like with Bitcoin Core development). It's therefore good to read that I haven't overlooked a simple solution for my problem and that it actually presents a challenge, which I'm happy to take on.

[BitMaxz]

It can be thought of as a donation address. And this address remains the same. I would like to now record the donations that come in into my database. But it's also good to hear that it's indeed not so simple. I do know quite a bit about cryptocurrencies, as long as it doesn't get too specialized (like with Bitcoin Core development). It's therefore good to read that I haven't overlooked a simple solution for my problem and that it actually presents a challenge, which I'm happy to take on.

If it's a donation address then no need to use BTCpayserver because a single BTC address is enough to receive donations. You don't also need to use 3rd party APIs just to receive donations.

However, if you want to keep track of who donates you then use BTCpayserver. To understand how donation works with BTCpay check this video below.

- https://youtu.be/MIWGvl6_WzI?si=KUD07gAnwdSDG9Dg

[maxloop]

And the best solution is to run my own node with bitcoin core where I directly interact with Bitcoin core to get all the data I need, right? Is it really recommended to interact with Bitcoin core directly or do you guys can suggest any other tool for it?