As a developer, what's the best way to accept BTC without using third-parties

[contactlight]

Hi,

I am a developer with a lot of experience in C++, Java and Python. I am familiar with BitcoinJ and similar frameworks that help you interact with the Bitcoin network.

I am trying to accept Bitcoin payments for a project and I would like to avoid using third-party services such as Blockchain's Payment API, Coinbase and BitPay.

I'm thinking of working with "btcd" or "Bitcoinj". I have a prototype that uses the PostgresFullPrunedBlockStore. It works perfectly except when the service is down for a while. It misses transactions and syncing the blockchain doesn't seem to help. I end up not getting notified of "relevant" transactions that occurred when my service was down, despite syncing the blockchain.

I do realize that I can easily interact with "bitcoind" using RPC and add new addresses but I don't want to rely on bitcoind for that. I'm thinking of potential scalability issues.

What do you think would be the right approach here? I'm hoping to come up with a generic solution so that I can open-source it and other people can integrate it into their own systems.

I have one potential solution in mind. Let me know if you see any problems with this.

Code:

1- Join the network, wait for new blocks from peers
2- Loop indefinitely and wait for new blocks from peers. Receive block N:
  2a- Look at the block, check the transactions included in the block, are they relevant?
  2b- If they are relevant, note the unspent output and store it (making the assumption that its inputs are valid, if they weren't, it wouldn't be included in the block and propagated? Is this a safe assumption to make?)
  2c- Sweep the address and send it to cold wallet
  2d- Increment the balance of the user

[jonald_fyookball]

I dont think you need to worry about peers, blocks, unspent outputs, etc.

Just generate a key/address pair , give the public address to your customer.
What's wrong with that?

[contactlight]

I dont think you need to worry about peers, blocks, unspent outputs, etc.

Just generate a key/address pair , give the public address to your customer.
What's wrong with that?

I need to be able to programmatically check for the incoming transactions to that address and sweep all the unspent outputs. I am already planning on generating a key/address pair and give the public address to the customer.

I just need to have a reliable way to be notified of incoming transactions to that address. My current Bitcoinj setup notifies me and sweeps the transactions when notified. However, this only happens if my service is up and running when the user makes the transaction. If the service is down, the transactions go unnoticed and I don't get notified during the blockchain sync process.

I was using BlockChain and PostgresFullPrunedBlockStore, and I noticed that it wasn't populating the unspent outputs table. This was probably due to me using BlockChain instead of FullPrunedBlockChain. I refactored that into my code and now I am syncing again. Let's see what happens with that.

[jonald_fyookball]

best practice is not to re use addresses.  You get one transaction per address, and then you can sweep it.

[contactlight]

best practice is not to re use addresses.  You get one transaction per address, and then you can sweep it.

Thanks but that's what I will be doing anyway. After sweeping, the address will be taken out of the "active_addresses" list and won't be checked ever again. This detail however doesn't make downtime any better. When the Bitcoin payment processing daemon is down, I won't be notified of the new transactions. BitcoinJ, when catching up with the blockchain, refuses to notify me of my transactions.

[jonald_fyookball]

so you do need to worry about outputs then.  Will you need to write a "balance check" function from scratch?
I assume you are running a full node.

[contactlight]

so you do need to worry about outputs then.  Will you need to write a "balance check" function from scratch?
I assume you are running a full node.

I'm debating if I should run a full node or if I should code a "pretend" full node. Either way, you can assume that I can store the entire blockchain. I just don't want to store all the unspent outputs, but only the ones for my addresses. Because indexing unspent outputs for all addresses can get tedious and require up to 320GB of disk space.

I will likely need to write a balance check function from scratch. I was planning on using "PostgresFullPrunedBlockStore" from BitcoinJ and call calculateBalanceForAddress(Address) but they don't seem to have implemented this class well.

[gweedo]

Generate keys offline, put them in database table, check using an api, then you don't have worry about sweeping them.

Can't be any easier.

[jonald_fyookball]

Generate keys offline, put them in database table, check using an api, then you don't have worry about sweeping them.

Can't be any easier.

I think he wants to check without relying on an api to some third party service.
He want to validate the balances on his own by parsing the blockchain.

[justusranvier]

Generate keys offline, put them in database table, check using an api, then you don't have worry about sweeping them.

Can't be any easier.

I think he wants to check without relying on an api to some third party service.
He want to validate the balances on his own by parsing the blockchain.

All the decent full node implementations have a good API that you can use locally.

API does not imply a third party service.

[contactlight]

Generate keys offline, put them in database table, check using an api, then you don't have worry about sweeping them.

Can't be any easier.

I think he wants to check without relying on an api to some third party service.
He want to validate the balances on his own by parsing the blockchain.

All the decent full node implementations have a good API that you can use locally.

API does not imply a third party service.

I do not want to rely on a separate process. I want my daemon to be self reliant. That's why I have been focusing on BitcoinJ.

My implementation is now a SPV. Instead of keeping track of all unspent outputs, it checks for transactions relevant to my addresses on each block and sweeps them. It seems to be working now but I need to test it further.

[justusranvier]

I do not want to rely on a separate process. I want my daemon to be self reliant.

You should have mentioned up front that intentionally getting hax0red was part of the design requirements.

[contactlight]

I do not want to rely on a separate process. I want my daemon to be self reliant.

You should have mentioned up front that intentionally getting hax0red was part of the design requirements.

Why do you think that this design would lead to a breach? I'm not going to be running everything else on it. It will just be a payment processor connected to the network and the database, keeping track of balances and transactions.

One issue I can see with it would be that we wouldn't run validation on the inputs. Receiving the new block from multiple nodes should fix that, though. They wouldn't have relayed the blocks if the transactions in it weren't valid. So I need 1 confirmation.

[gweedo]

I do not want to rely on a separate process. I want my daemon to be self reliant.

You should have mentioned up front that intentionally getting hax0red was part of the design requirements.

Some people don't have any idea how bitcoin works but they use their programming certificates as "I know everything yet I am still asking". Now I know why I left this forum for while.

[contactlight]

I do not want to rely on a separate process. I want my daemon to be self reliant.

You should have mentioned up front that intentionally getting hax0red was part of the design requirements.

Some people don't have any idea how bitcoin works but they use their programming certificates as "I know everything yet I am still asking". Now I know why I left this forum for while.

Not sure if that is directed towards me but if it is, that's very uncalled for and childish. You can avoid this drama by not looking at threads with questions and moving on your way on your high horse.

[gmaxwell]

Have you looked into Baron? https://github.com/slickage/baron

[nuno12345]

If you can run a daemon you dont need to rely on any third partys.
You can create an account for each address in your daemon and use getbalance account to check that specific address balance.
Addresses need to be created on that daemon or imported and an account needs to be assign to that address.

[dabura667]

If you can run a daemon you dont need to rely on any third partys.
You can create an account for each address in your daemon and use getbalance account to check that specific address balance.
Addresses need to be created on that daemon or imported and an account needs to be assign to that address.

Then once you have 100000000000000 keys on your wallet, you'll need 30 GB for the blockchain, and 500TB for your wallet.dat file... lol

[gweedo]

If you can run a daemon you dont need to rely on any third partys.
You can create an account for each address in your daemon and use getbalance account to check that specific address balance.
Addresses need to be created on that daemon or imported and an account needs to be assign to that address.

Then once you have 100000000000000 keys on your wallet, you'll need 30 GB for the blockchain, and 500TB for your wallet.dat file... lol

Someone here that actually has a wallet with over 2 million addresses generated. It actually doesn't take up that much space. Plus you can use tools to remove private keys from your wallet.

[BitcoinExchangeIndia.com]

Hi,

I am a developer with a lot of experience in C++, Java and Python. I am familiar with BitcoinJ and similar frameworks that help you interact with the Bitcoin network.

I am trying to accept Bitcoin payments for a project and I would like to avoid using third-party services such as Blockchain's Payment API, Coinbase and BitPay.

I'm thinking of working with "btcd" or "Bitcoinj". I have a prototype that uses the PostgresFullPrunedBlockStore. It works perfectly except when the service is down for a while. It misses transactions and syncing the blockchain doesn't seem to help. I end up not getting notified of "relevant" transactions that occurred when my service was down, despite syncing the blockchain.

I do realize that I can easily interact with "bitcoind" using RPC and add new addresses but I don't want to rely on bitcoind for that. I'm thinking of potential scalability issues.

What do you think would be the right approach here? I'm hoping to come up with a generic solution so that I can open-source it and other people can integrate it into their own systems.

I have one potential solution in mind. Let me know if you see any problems with this.

Code:

1- Join the network, wait for new blocks from peers
2- Loop indefinitely and wait for new blocks from peers. Receive block N:
  2a- Look at the block, check the transactions included in the block, are they relevant?
  2b- If they are relevant, note the unspent output and store it (making the assumption that its inputs are valid, if they weren't, it wouldn't be included in the block and propagated? Is this a safe assumption to make?)
  2c- Sweep the address and send it to cold wallet
  2d- Increment the balance of the user

What's wrong in using a 3rd party API ?