Bitcoin Forum
May 22, 2024, 09:53:12 AM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: How To Correctly Poll New Blocks for Payments?  (Read 1023 times)
BkkCoins (OP)
Hero Member
*****
Offline Offline

Activity: 784
Merit: 1009


firstbits:1MinerQ


View Profile WWW
August 23, 2012, 03:04:19 AM
Last edit: August 23, 2012, 05:13:44 AM by BkkCoins
 #1

I'm going to have a new web service that will require I give out addresses and then monitor if payments are made to me to update an account balance.

I plan to poll blockchain.info every few minutes for each new block. This seems easier than running the bitcoin client and having to keep 3 GB of data on my web server.

Right now I have code and it checks the json data for each block and scans for the block that is marked "main_chain": true and then scans those transactions looking for "out" records in my address database. When one is found I add that transaction to my pmts table (so I can also check for duplicates). After N blocks pass, for confirmation, I can add the amount to a user balance.

This method seems like the most efficient in terms of polling or checking addresses.

I'm just wondering if there is some indicators/flags or gotchas I have to watch out for?

HeavyMetal
Newbie
*
Offline Offline

Activity: 42
Merit: 0



View Profile
August 23, 2012, 05:00:50 AM
 #2

I would also like to know this.
BkkCoins (OP)
Hero Member
*****
Offline Offline

Activity: 784
Merit: 1009


firstbits:1MinerQ


View Profile WWW
August 23, 2012, 06:10:58 AM
 #3

Here's one concern I have. When I record a pmt transaction from block N and then wait til block N+6 to apply the payment - how do I verify it is still in the main_chain? I should somehow make sure the transaction did not become invalid in the meantime because if I just wait 6 blocks but not check back then I cannot know if it was confirmed after all. Not sure how to best handle that.

Mike Hearn
Legendary
*
expert
Offline Offline

Activity: 1526
Merit: 1129


View Profile
August 23, 2012, 11:30:08 AM
 #4

I plan to poll blockchain.info every few minutes for each new block. This seems easier than running the bitcoin client and having to keep 3 GB of data on my web server.

I really wouldn't recommend that for a few reasons.

One is that it's impolite to everyone else who uses the site, you're draining resources away that people who want interactive exploration of the block chain need, for a very simple functionality. 3GB is really not much storage at all.

Another reason is that if blockchain.info goes down or decides to block you for causing excessive load, your site will break.

There are several ways to do what you want. One is to just use the Satoshi client and send it RPCs.

Another, if you're comfortable with Java, is to write a bitcoinj app. You'll get callbacks when new transactions that are relevant to the wallet arrive. You don't need to have the private keys in the wallet so if the server gets hacked the money can't be stolen.

D.H.
Sr. Member
****
Offline Offline

Activity: 311
Merit: 250


Bitcoin.se site owner


View Profile WWW
August 23, 2012, 03:21:40 PM
 #5

One is that it's impolite to everyone else who uses the site, you're draining resources away that people who want interactive exploration of the block chain need, for a very simple functionality.

Well, they do provide an API that lets you do exactly what OP wants. I don't see how it can be considered impolite to use that API, especially if you only poll it once every few minutes. Your other reason is very valid though.

www.bitcoin.se - Forum, nyheter och information på svenska! (Forum, news and information in Swedish)
HeavyMetal
Newbie
*
Offline Offline

Activity: 42
Merit: 0



View Profile
August 23, 2012, 04:18:46 PM
 #6

You could always run the bitcoind server on another machine if 3gb of space is a concern on your server.

I don't know if the bitcoind server allows fine grained access control, ie you can check balance but not send etc. But one could write their own api that goes between the bt server and your web server.
BkkCoins (OP)
Hero Member
*****
Offline Offline

Activity: 784
Merit: 1009


firstbits:1MinerQ


View Profile WWW
August 23, 2012, 04:47:06 PM
 #7

One is that it's impolite to everyone else who uses the site, you're draining resources away that people who want interactive exploration of the block chain need, for a very simple functionality.

Well, they do provide an API that lets you do exactly what OP wants. I don't see how it can be considered impolite to use that API, especially if you only poll it once every few minutes. Your other reason is very valid though.
The API info states that by default the limit is 300 calls/5 min. or 5760 / 8 hours, and that you can contact them to have the limit increased. I don't see how a call every 3-5 minutes is going to be impolite or a problem given the stated limits. It's very clear from the API docs that using it this way is one of it's intended purposes.

It's not just that I don't want to have bitcoind running on the server and the space it uses but also that AFAIK it doesn't provide a useful interface for monitoring a large collection of randomly generated addresses. There are some calls to check various wallet values but I don't think there is a way to setup callbacks upon payment arrival. So I would end up repeatedly polling by RPC for potentially thousands of addresses that only a few may have a payment arrive on. And then it doesn't seem to provide a way to add addreses without keys. So unless there is a better suited daemon I can run then making use of the JSON API looks like the best way.

After trying some stuff and coding for a while I decided that likely the best way was to poll the simplest "getblockcount" call to see when a new block arrives. This call only returns a number and is very lightweight. When a new block arrives I don't grab it. Instead I grab the one that is N block before it and check those transaction addresses. They already have the needed confirmation level and so can be applied to balances in one step. Much simpler.

One thing that would make such use much lighter on the API server would be to offer a much reduced block-height call. For payment processing it seems you only really need the (trx,address,value) output tuples and that is a small fraction of the overall block data. Blocks that currently are 500K in size would likely be 5-10K if useless data was pared away.



Revalin
Hero Member
*****
Offline Offline

Activity: 728
Merit: 500


165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g


View Profile
August 24, 2012, 02:15:27 AM
 #8

Just announced: https://bitcointalk.org/index.php?topic=102740

Quote
The purpose of BTC Sniffer is the add an easy way to listen to Bitcoin Network events and facilitate extensions...

      War is God's way of teaching Americans geography.  --Ambrose Bierce
Bitcoin is the Devil's way of teaching geeks economics.  --Revalin 165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g
BkkCoins (OP)
Hero Member
*****
Offline Offline

Activity: 784
Merit: 1009


firstbits:1MinerQ


View Profile WWW
August 24, 2012, 03:05:48 AM
 #9

Just announced: https://bitcointalk.org/index.php?topic=102740

Quote
The purpose of BTC Sniffer is the add an easy way to listen to Bitcoin Network events and facilitate extensions...
Thank you. I'm going to look at that.

Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!