walletnotify doesn't work

[arixpcx]

Hi i'm creating an exchange website and the first step is to connect my website to a bitcoind client.
I've installed bitcoin-qt, downloaded the blockchain and configured the bitcoin.conf file.

bitcoin.conf:

Code:

 # bitcoin.conf configuration file. Lines beginning with # are comments.
 
 
 # Network-related settings:
 
 # Run on the test network instead of the real bitcoin network.
 #testnet=0
 
 # Connect via a socks4 proxy
 #proxy=127.0.0.1:9050
 
 ##############################################################
 ##            Quick Primer on addnode vs connect            ##
 ##  Let's say for instance you use addnode=4.2.2.4          ##
 ##  addnode will connect you to and tell you about the      ##
 ##    nodes connected to 4.2.2.4.  In addition it will tell ##
 ##    the other nodes connected to it that you exist so     ##
 ##    they can connect to you.                              ##
 ##  connect will not do the above when you 'connect' to it. ##
 ##    It will *only* connect you to 4.2.2.4 and no one else.##
 ##                                                          ##
 ##  So if you're behind a firewall, or have other problems  ##
 ##  finding nodes, add some using 'addnode'.                ##
 ##                                                          ##
 ##  If you want to stay private, use 'connect' to only      ##
 ##  connect to "trusted" nodes.                             ##
 ##                                                          ##
 ##  If you run multiple nodes on a LAN, there's no need for ##
 ##  all of them to open lots of connections.  Instead       ##
 ##  'connect' them all to one node that is port forwarded   ##
 ##  and has lots of connections.                            ##
 ##       Thanks goes to [Noodle] on Freenode.               ##
 ##############################################################
 
 # Use as many addnode= settings as you like to connect to specific peers
 #addnode=69.164.218.197
 #addnode=10.0.0.2:8333
 
 # ... or use as many connect= settings as you like to connect ONLY
 # to specific peers:
 #connect=69.164.218.197
 #connect=10.0.0.1:8333
 
 
 # Maximum number of inbound+outbound connections.
 #maxconnections=
 
 
 # JSON-RPC options (for controlling a running Bitcoin/bitcoind process)
 
 # server=1 tells Bitcoin-QT to accept JSON-RPC commands.
 server=1
 
 # You must set rpcuser and rpcpassword to secure the JSON-RPC api
 rpcuser=user
 rpcpassword=password
 
 # How many seconds bitcoin will wait for a complete RPC HTTP request.
 # after the HTTP connection is established. 
 #rpctimeout=30
 
 # By default, only RPC connections from localhost are allowed.  Specify
 # as many rpcallowip= settings as you like to allow connections from
 # other hosts (and you may use * as a wildcard character).
 # NOTE: opening up the RPC port to hosts outside your local
 # trusted network is NOT RECOMMENDED, because the rpcpassword
 # is transmitted over the network unencrypted.
 rpcallowip=10.0.0.3
 rpcallowip=10.0.0.19
 rpcallowip=127.0.0.1
 
 # Listen for RPC connections on this TCP port:
 rpcport=8332

 walletnotify=/home/arik/scripts/transaction.sh %s
 
 # You can use Bitcoin or bitcoind to send commands to Bitcoin/bitcoind
 # running on another host using this option:
 #rpcconnect=127.0.0.1
 
 # Use Secure Sockets Layer (also known as TLS or HTTPS) to communicate
 # with Bitcoin -server or bitcoind
 #rpcssl=1
 
 # OpenSSL settings used when rpcssl=1
 #rpcsslciphers=TLSv1+HIGH:!SSLv2:!aNULL:!eNULL:!AH:!3DES:@STRENGTH
 #rpcsslcertificatechainfile=server.cert
 #rpcsslprivatekeyfile=server.pem
 
 
 # Miscellaneous options
 
 # Set gen=1 to attempt to generate bitcoins
 #gen=0
  
 # Pre-generate this many public/private key pairs, so wallet backups will be valid for
 # both prior transactions and several dozen future transactions.
 #keypool=100
 
 # Pay an optional transaction fee every time you send bitcoins.  Transactions with fees
 # are more likely than free transactions to be included in generated blocks, so may
 # be validated sooner.
 #paytxfee=0.00
 
 # Allow direct connections for the 'pay via IP address' feature.
 #allowreceivebyip=1
   
 # User interface options
 
 # Start Bitcoin minimized
 #min=1
 
 # Minimize to the system tray
 #minimizetotray=1

My bitcoin-qt client runs on ubuntu 14.04

I use asp.net to develop my website, the rpc commands seem to work fine (used getnewaddress and sendtoaddress so far).
The problem is with the walletnotify.

As you can see in my bitcoin.conf file, i have the line:

Code:

walletnotify=/home/arik/scripts/transaction.sh %s

But whenever my wallet receives an incoming transaction the walletnotify doesn't get invoked.
In /home/arik/scripts i have the transaction.sh file:

Code:

#!/bin/sh
curl -d "txid=${1}" http://10.0.0.3:51827/Callbacks/BTC.aspx?secret=666

I have curl installed.
I tried to invoked it manually from the terminal: bash /home/arik/scripts/transaction.sh blablabla
and it works (the asp.net page receives the request with the post parameter blablabla).

So the problem seems to be in the bitcoin.conf file.
Can you help me find the problem please?

Also, i've heard that walletnotify will get called only for the first confirmation and after that i'll have to use blocknotify but the problem is that walletnotify sends the transaction id and i can get all the information about the transaction with the gettransaction method but blocknotify sends the blockid (so i've heard), would i still be able to use gettransaction?

Thanks for the help

[vit1988]

- did you try to execute it without "bash"?
- does bitcoind's user has proper rights to execute the script? (what does "ls -l /home/arik/scripts/transaction.sh" say?)

[arixpcx]

- I just tried executing it without "bash", it says "Permission denied".
- I'm not sure, i'm pretty new to linux as well..
ls -l .../transaction.sh says "-rw-rw-r-- 1 arik arik 82 אוג 11 17:45 /home/arik/scripts/transaction.sh"

So you reckon that the problem is with the permissions?
I'm an idiot...   

[rarkenin]

Yes, the issue is with permissions. You don't have the execute bit set on transaction.sh. Execute the following command:

Code:

chmod a+x /home/arik/scripts/transaction.sh

to set the execute bit.

[arixpcx]

Thanks for the help guys
Could you also address my second questions(about block notify) please?

[DannyHamilton]

Thanks for the help guys
Could you also address my second questions(about block notify) please?

blocknotify provides the block hash

Your script can then use the API call "getblock" to get the list of transaction hashes that are included in the block.

Your script can then loop through each transaction hash using either the API call "getrawtransaction" or "gettransaction" to get and analyze each of the transactions in the block to see if they matter to you.

[arixpcx]

Ohh.. so just tell me if this pseudocode is right:

Code:

// Blocknotify callback page

blockhash = fromPost("blockhash");

transactions = getblock(blockhash);

foreach(tx in transactions) {
	if(tx.hash exists in deposits table) {
		txdb = pull transaction from deposits table where transactionHash = tx.hash;
		txdb.confirmations = tx.confirmations;
		txdb.save();
	}
}

I loop through all the transaction in the block, for each transaction i check if it exists in the deposits table and if it does exist there, i update the number of confirmations of this transaction to the current number.

Is it right?
Thank you

[DannyHamilton]

Ohh.. so just tell me if this pseudocode is right:

Code:

// Blocknotify callback page

blockhash = fromPost("blockhash");

transactions = getblock(blockhash);

foreach(tx in transactions) {
	if(tx.hash exists in deposits table) {
		txdb = pull transaction from deposits table where transactionHash = tx.hash;
		txdb.confirmations = tx.confirmations;
		txdb.save();
	}
}

I loop through all the transaction in the block, for each transaction i check if it exists in the deposits table and if it does exist there, i update the number of confirmations of this transaction to the current number.

Is it right?
Thank you

Unconfirmed transaction hashes are malleable.

You should not assume that a transaction will have the same hash when it is confirmed as it had when it was sent.  MtGox lost a lot of money that way.

Instead you should use "gettransaction" or "getrawtransaction" to find out exactly what address is being paid and how much is being sent.

If you want to be able to match the transaction up with a previously seen unconfirmed transaction, then you need to look at the inputs of the transaction and see if they are the same.

[arixpcx]

So perhaps the best way is to match the address, date and amount?

Code:

txdb = pull transaction from deposits table where address = tx.address and date = tx.date and amount = tx.amount

Is that right?

[vit1988]

Another approach is to process all transactions from rpc->listtransactions instead of processing all txs from a new block. You'll get only transactions which affect your wallet including txid and number of confirmations.

https://bitcointalk.org/index.php?topic=716114.msg8186720#msg8186720

[arixpcx]

What is the difference between txid and txhash? the comment above me says that i shouldn't rely on txhash as an identifier for a transaction since it can change...

About this code:

Code:

$transactions = $bitcoinrpc->listtransactions('accountname', 500);
foreach ($transactions as $tx) {

    if ($tx['category'] == 'receive' && $tx['confirmations'] < 10) {
        $id_user = $db->query('SELECT id_user FROM addresses WHERE address={$tx['address']}');
        $db->query('INSERT INTO balance (txid, id_user, amount, confirmations) VALUES ({$tx['txid']}, {$id_user}, {$tx['amount']}, {$tx['confirmations']}) 
                           ON DUPLICATE KEY UPDATE confirmations={$tx['confirmations']}');
        // (balance.txid must be defined as unique key of course)

        if ($db->affectedRows() > 0) {
            // number of confirmations for $tx has changed, do any prefered action 
        }
    }
}

Where should i write it? in the blocknotify callback page? or do you say that i shouldn't call this code right after the wallet/block notify event and just save the txid in a file and use cronjobs to collect the data every couple minutes?

[DannyHamilton]

What is the difference between txid and txhash? the comment above me says that i shouldn't rely on txhash as an identifier for a transaction since it can change...

The terms txid and txhash are generally used interchangeably.  Once a transaction is confirmed, it is very unlikely (although not impossible) that the txhash will change.  When a transaction is not yet confirmed, you should never rely on the txhash (txid) remaining the same.  Doing so will just cause you long term problems.

[arixpcx]

You said that i should keep the inputs of the transaction as an identifier.
Could you please be more specific about what part of the inputs i should keep?
The transaction hashes of the input transactions? the signatures of the input transaction?

[fbueller]

I used to add pending transactions (< 6 confirmations) to one table, and then looked up each txid to see if confirmations changed. You need to have it tolerate conflicted transactions, which are the results of malleated txids. Sometimes, even after confirming in the blockchain (usually only on or two confirmations) your client might realize there is a longer proof of work chain which may or may not have your transaction in it. Because of this, your client will track the longer chain, so you need to have a way for your app to realize that a confirmed transaction is no longer valid to avoid duplicates. These are some of the reasons we use 6 confirmations instead of one.

The problem is two-fold:

1. When I broadcast a transaction to you, the p2p network will relay it to everyone. If there's any malleability bots running (that was the issue before) they'd try to change the sig so it's still valid, but would the effect of altering the TxID. TXID's are just a hash of the entire transaction. If any detail at all changes you get a new txid.

The txid you hear about before confirmation might be a malleated copy, and maybe a miner has the original or another copy. So while your client thinks it has txid A when unconfirmed, a block could be published with a completely different TxID.

2. Before it's confirmed in the blockchain, there could be multiple competing copies floating around the network. Your app needs to tolerate this because as mentioned above, even if one transaction makes it into the blockchain with TxID A, if the longest proof of work chain changes the effects could be:
 - Your txid isn't confirmed in this new longer set of blocks whatsoever. Maybe.
 - Your txid is included in the chain, but it could be in a different block height.
 - Your txid is included in the chain, but it could have a completely different TxID!

Back when the bot was running, there were visible effects of all of this. I'd receive two payments instead of one.. If I tried spending them, obviously only one output is spendable not two.

So, you really need to pay attention to what's in the chain. You need to accept that chain reorgs (your client noticing there is a better chain available) happen, and so does malleability. While malleability won't always happen, chain reorgs are frequent. Not too deep, but frequent. Scanning backwards with each new block you hear about until you find a common ancestor would probably be a good idea, so you know if it's a new block, or a reorg.

This just means - your application needs to be able to delete, and replace transactions it learned about, but realizes later weren't confirmed properly. Once confirmed you should store the txid, the vout, and the scriptPubKey. This is all you need to create new transactions in future (even raw transactions purely in code).

I think the safest way to track inputs is this:

Each blocknotify, your app should look at the block hash, and keep asking the node for the 'previousblockhash' until you find a block hash you've already seen before. Normally the previousblockhash will already be known to your app, but if not, then your client has realized you were on the wrong fork and switched.
Once it finds this common ancestor block, you know how many blocks are between the leading block and the ancestor (subtract two heights) - n. If it's > 1, you need to remove all transactions that were confirmed in the last n blocks, and scan from the common ancestor all the way up to the most recent block in the chain.


I haven't said how to deal with transactions:

Decode each tx in a block, and analyze its outputs. If any pay to you, you need to store that txid, the vout, and the scriptPubKey. Usually other data like the blockhash, blockheight are useful to have around to assist with the pruning of no-longer-valid transactions. This will be used to maintain a list of all incoming transactions.

While you're looking at the transaction, you need to see if any of the inputs being spent are already in your table of stored incoming transactions. That way you know to remove it, or mark it as spent and keep it for historical purposes.

The important thing here is to stress that the top of the blockchain can be in turmoil, but know that older blocks are harder to replace. So this is why 6 confirmations is important - it buys you enough time so that you know it should be nearly impossible for blocks that deep to be replaced. Since blocks can be replaced, and malleability can change txids, you can't expect txs confirmed in one side of the fork to be in the other.

Bearing these in mind, you're able to continue writing your application. Have fun

[arixpcx]

This is a lot to bear in mind lol
but i'll handle.
Thank for the help

[vit1988]

About this code:

Code:

$transactions = $bitcoinrpc->listtransactions('accountname', 500);
foreach ($transactions as $tx) {

    if ($tx['category'] == 'receive' && $tx['confirmations'] < 10) {
        $id_user = $db->query('SELECT id_user FROM addresses WHERE address={$tx['address']}');
        $db->query('INSERT INTO balance (txid, id_user, amount, confirmations) VALUES ({$tx['txid']}, {$id_user}, {$tx['amount']}, {$tx['confirmations']}) 
                           ON DUPLICATE KEY UPDATE confirmations={$tx['confirmations']}');
        // (balance.txid must be defined as unique key of course)

        if ($db->affectedRows() > 0) {
            // number of confirmations for $tx has changed, do any prefered action 
        }
    }
}

Where should i write it? in the blocknotify callback page? or do you say that i shouldn't call this code right after the wallet/block notify event and just save the txid in a file and use cronjobs to collect the data every couple minutes?

I actually run this in a endless loop (+sleep(5)) cause I was not aware of bitcoinds notification feature when I implemented this.

If I had to refactor it, I would run this once a minute as cronjob and make a simple INSERT INTO balance (..., confirmations) VALUES (..., 0) in walletnotify.

[herry57]

The Wallet card will let you pay for items at any retailer, even those that don't have the little NFC kiosks Wallet has thus far relied upon. The charge should be relayed to your Wallet account just like you had used an NFC phone. It's unclear if that's just the Wallet balance, or if cards are included too. This fits nicely with Google's strategy of uncoupling the NFC payments in KitKat from the hardware secure element. Google is really making Wallet easier to use on multiple fronts.