Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: torba on August 08, 2013, 01:07:51 AM



Title: Is there a better way to get notified of receives with bitcoind?
Post by: torba on August 08, 2013, 01:07:51 AM
My current method is to just poll bitcoind API for a new transaction. I feel like this isn't the right way.


Title: Re: Is there a better way to get notified of receives with bitcoind?
Post by: Jouke on August 08, 2013, 07:11:02 AM
My current method is to just poll bitcoind API for a new transaction. I feel like this isn't the right way.
-walletnotify=


Title: Re: Is there a better way to get notified of receives with bitcoind?
Post by: freeloader247 on August 09, 2013, 03:44:37 AM
Is there any example implementation of  -walletnotify in PHP? Right now my code look something like this

Code:
$bitcoin = new bitcoin(jsoncredentials);

$bitcoin->getbalance(  account  );

if ($balance >= get_Price( id )) {
}

and it feels very sloppy/hackish.


Title: Re: Is there a better way to get notified of receives with bitcoind?
Post by: kjj on August 09, 2013, 03:59:04 AM
Yeah, don't do that unless you are sure that your transaction volume will be extremely low.

You want the process run by -walletnotify to be very fast.  Ideally, it should spit out a line into a pipe (or file or socket) and end promptly.

You should then have a different process, either a daemon or a cron job, read that stream and gather whatever information you need.  You should be checking the confirmation count of the transactions, not the balance of the account.  For that matter, using accounts can get you into trouble.  Make sure you know what you are doing if you are going to use them at all.

-walletnotify will trigger when a transaction is included in a block, but it might also trigger if the transaction shows up over the network as a loose transaction.  So, you should check the confirmation count each time -walletnotify hits, until you see at least one confirmation.  After that, you should check it every time you see a block to make sure the count has reached your threshold.  You could optimize that a bit by skipping the slow RPC calls until <threshold> number of blocks have been seen after the transaction was first included.


Title: Re: Is there a better way to get notified of receives with bitcoind?
Post by: freeloader247 on August 09, 2013, 06:53:47 AM
Yeah, don't do that unless you are sure that your transaction volume will be extremely low.

You want the process run by -walletnotify to be very fast.  Ideally, it should spit out a line into a pipe (or file or socket) and end promptly.

You should then have a different process, either a daemon or a cron job, read that stream and gather whatever information you need.  You should be checking the confirmation count of the transactions, not the balance of the account.  For that matter, using accounts can get you into trouble.  Make sure you know what you are doing if you are going to use them at all.

-walletnotify will trigger when a transaction is included in a block, but it might also trigger if the transaction shows up over the network as a loose transaction.  So, you should check the confirmation count each time -walletnotify hits, until you see at least one confirmation.  After that, you should check it every time you see a block to make sure the count has reached your threshold.  You could optimize that a bit by skipping the slow RPC calls until <threshold> number of blocks have been seen after the transaction was first included.

To be honest when I read your comment a few hours ago I had no clue what a pipeline, socket and cron job was, but after much googling, coffee and Wu Tang later I have come up with something like this.
Code:
walletnotify=/home/btcdev/walletnotifyclient 127.0.0.1 1337 %s

walletnotifyclient.c
Code:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netdb.h>

int main(int argc, char *argv[])
{
    if (argc < 4) {
printf("\n Usage: %s <hostname> <port> <TxID> \n",argv[0]);
exit(0);
    }

    int sockfd, portno, n;
    struct sockaddr_in serv_addr;
    struct hostent *server;

    char buffer[256];

    portno = atoi(argv[2]);
    sockfd = socket(AF_INET, SOCK_STREAM, 0);
    if (sockfd < 0) {
fprintf(stderr,"ERROR, opening socket\n");
exit(0);
    }
    server = gethostbyname(argv[1]);
    if (server == NULL) {
        fprintf(stderr,"ERROR, no such host\n");
        exit(0);
    }
    bzero((char *) &serv_addr, sizeof(serv_addr));
    serv_addr.sin_family = AF_INET;
    bcopy((char *)server->h_addr,
         (char *)&serv_addr.sin_addr.s_addr,
         server->h_length);
    serv_addr.sin_port = htons(portno);
    if (connect(sockfd,(struct sockaddr *) &serv_addr,sizeof(serv_addr)) < 0) {
fprintf(stderr,"ERROR, connecting\n");
exit(0);
    }
    snprintf(buffer, sizeof(buffer), "%s",argv[3]);
    n = write(sockfd, buffer, strlen(buffer));
    if (n < 0) {
fprintf(stderr,"ERROR, writing to socket\n");
exit(0);
    }
    close(sockfd);
    return 0;
    exit(0);
}

Server,php
Code:
<?php
include 'bitcoin_lib.php';
$bitcoin = new bitcoin('http://user:pass@127.0.0.1:8332/');

$txid = array(
    
'unconfirmed' => array(),
    
'confirmed' => array()
);

$currentBlock 0//the current block
$minConfirmation 3//minimum tansaction confirmations
$blockThreshold 3//blocks to wait checking transaction for confirmations

$server stream_socket_server("tcp://127.0.0.1:1337"$errno$errorMessage);
if (
$server === false) {
throw new UnexpectedValueException("Could not bind to socket: $errorMessage");
}

for (;;) {
$client = @stream_socket_accept($server);
if ($client) {
$message stream_get_contents($client);
if (strlen($message) == 64) { //recived a TXID
echo 'New tx:'.$message."\n";
$txid['unconfirmed'][] = array(
    'onBlock' => $currentBlock $blockThreshold,
    'txid' => $message
);
}
else { //recived the current block
$currentBlock $bitcoin->getblockcount();
echo 'New Block:'.$currentBlock."\n";
//loop through unconfirmed transactions on new block
foreach ($txid['unconfirmed'] as $key => $unconfirmedTX) {
if($currentBlock >= $unconfirmedTX['onBlock']) {
echo $unconfirmedTX['txid'].' : reached blockThreshold'."\n";
$gettransaction $bitcoin->gettransaction($unconfirmedTX['txid']);
if($gettransaction['confirmations'] >= $minConfirmation) { //transaction has been confirmed
echo $unconfirmedTX['txid'].' : confirmed'."\n";
unset($txid['unconfirmed'][$key]);
$txid['confirmed'][] = $gettransaction;
}
else { //Transaction does not have min confirmations
echo $unconfirmedTX['txid'].' : unconfirmed wait '.$unconfirmedTX['confirmations'].' blocks'."\n";
$blockstowait $minConfirmation $unconfirmedTX['confirmations']; //set blocks to wait before rechecking transaction
$gettransaction['onBlock'] = $currentBlock $blockstowait
$txid['unconfirmed'][$key] = $gettransaction;
}
}
else {
$blockstowait $unconfirmedTX['onBlock'] - $currentBlock;
echo $unconfirmedTX['txid'].' : has not reached blockThreshold, wait '.$blockstowait.' blocks'."\n";
}
}
}
fclose($client);
}

}
?>


Am I on the right track or have I gone crazy?


Title: Re: Is there a better way to get notified of receives with bitcoind?
Post by: freeloader247 on August 10, 2013, 06:32:52 AM
Also what is a proper transaction flow? Currently I have things set up like this, but as it is advised to avoid both accounts and getbalance this is clearly wrong.

  • Registration: getnewaddress for user setaccount to user ID.
  • Transaction: getbalance of users account and confirm that is larger than purchase price.



Title: Re: Is there a better way to get notified of receives with bitcoind?
Post by: Remember remember the 5th of November on August 10, 2013, 10:25:42 AM
Yeah, don't do that unless you are sure that your transaction volume will be extremely low.

You want the process run by -walletnotify to be very fast.  Ideally, it should spit out a line into a pipe (or file or socket) and end promptly.

You should then have a different process, either a daemon or a cron job, read that stream and gather whatever information you need.  You should be checking the confirmation count of the transactions, not the balance of the account.  For that matter, using accounts can get you into trouble.  Make sure you know what you are doing if you are going to use them at all.

-walletnotify will trigger when a transaction is included in a block, but it might also trigger if the transaction shows up over the network as a loose transaction.  So, you should check the confirmation count each time -walletnotify hits, until you see at least one confirmation.  After that, you should check it every time you see a block to make sure the count has reached your threshold.  You could optimize that a bit by skipping the slow RPC calls until <threshold> number of blocks have been seen after the transaction was first included.

To be honest when I read your comment a few hours ago I had no clue what a pipeline, socket and cron job was, but after much googling, coffee and Wu Tang later I have come up with something like this.
Code:
walletnotify=/home/btcdev/walletnotifyclient 127.0.0.1 1337 %s

walletnotifyclient.c
Code:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netdb.h>

int main(int argc, char *argv[])
{
    if (argc < 4) {
printf("\n Usage: %s <hostname> <port> <TxID> \n",argv[0]);
exit(0);
    }

    int sockfd, portno, n;
    struct sockaddr_in serv_addr;
    struct hostent *server;

    char buffer[256];

    portno = atoi(argv[2]);
    sockfd = socket(AF_INET, SOCK_STREAM, 0);
    if (sockfd < 0) {
fprintf(stderr,"ERROR, opening socket\n");
exit(0);
    }
    server = gethostbyname(argv[1]);
    if (server == NULL) {
        fprintf(stderr,"ERROR, no such host\n");
        exit(0);
    }
    bzero((char *) &serv_addr, sizeof(serv_addr));
    serv_addr.sin_family = AF_INET;
    bcopy((char *)server->h_addr,
         (char *)&serv_addr.sin_addr.s_addr,
         server->h_length);
    serv_addr.sin_port = htons(portno);
    if (connect(sockfd,(struct sockaddr *) &serv_addr,sizeof(serv_addr)) < 0) {
fprintf(stderr,"ERROR, connecting\n");
exit(0);
    }
    snprintf(buffer, sizeof(buffer), "%s",argv[3]);
    n = write(sockfd, buffer, strlen(buffer));
    if (n < 0) {
fprintf(stderr,"ERROR, writing to socket\n");
exit(0);
    }
    close(sockfd);
    return 0;
    exit(0);
}

Server,php
Code:
<?php
$txid 
= array();
$server stream_socket_server("tcp://127.0.0.1:1337"$errno$errorMessage);

if (
$server === false) {
    throw new 
UnexpectedValueException("Could not bind to socket: $errorMessage");
}

for (;;) {
    
$client = @stream_socket_accept($server);

    if (
$client) {
$txid[] = stream_get_contents($client);
var_dump($txid);
        
fclose($client);
    }
}
?>


Am I on the right track or have I gone crazy?
Quoting for archival purposes. But wow, you catch on quick.


Title: Re: Is there a better way to get notified of receives with bitcoind?
Post by: kjj on August 10, 2013, 12:15:13 PM
Quoting for archival purposes. But wow, you catch on quick.

No kidding.  When I saw this, freeloader247 became my favorite person.

I still haven't had a chance to read his programs in detail, but he's certainly on the right track.

Also what is a proper transaction flow? Currently I have things set up like this, but as it is advised to avoid both accounts and getbalance this is clearly wrong.

  • Registration: getnewaddress for user setaccount to user ID.
  • Transaction: getbalance of users account and confirm that is larger than purchase price.

When you create a new address to accept a payment, stash a copy in your local database, along with whatever identification you need.  This could be as simple as {address,order number} or {address,userid}.  Since the notification system is transaction based, you use something like gettransaction or getrawtransaction with the verbose flag for processing.

If you pair this up with -blocknotify, you can really minimize the number of slow RPC calls you have to make.

Basically, you gettransaction each time -walletnotify triggers until it returns 1 confirmation.  If you want 6 confirmations, you make a note to come back and check again after -blocknotify hits 5 more times.  At that point, gettransaction should read 6 confirmations.  If it doesn't, then congratulations, your system was clever enough to not get fooled by a chain reorg, put it back in the queue to be checked again after X more blocks.


Title: Re: Is there a better way to get notified of receives with bitcoind?
Post by: freeloader247 on August 10, 2013, 05:34:27 PM
[deleted]


Title: Re: Is there a better way to get notified of receives with bitcoind?
Post by: Abdussamad on August 12, 2013, 02:49:31 PM
php can be run over the command line too. Look up php-cli. What I mean is that the command run by walletnotify can be the php script itself.


Title: Re: Is there a better way to get notified of receives with bitcoind?
Post by: kjj on August 12, 2013, 03:02:30 PM
Edit: I'm an idiot -blocknotify returns the new block hash not the current block height.
Edit: if anyone had the code to identify block hashes from transaction ids hook me up, otherwise I'm going to sleep and will do it tomorrow.

Blocknotify just gives the hash.  The hash isn't even important, just the event.  Most of the time, getting a block means that you've extended the main chain, but not always.

If you need 5 more confirmations, then there is no point checking the status of the transaction until you see 5 new blocks.  But, seeing 5 new blocks doesn't automatically mean that the transaction you are looking at has seen 5 additional confirmations.  If there is a fork, or a reorg, or whatever else, your confirmation count could have actually gone down.  So, once you have enough blocks that you might have enough confirmations, you then verify by checking gettransaction to be sure.


Title: Re: Is there a better way to get notified of receives with bitcoind?
Post by: freeloader247 on August 13, 2013, 05:15:38 AM
[deleted]