Linux Electrum Daemon Generating Bitcoin Addresses For Payment

[JollyGood]

I am testing an ecommerce store that allows bitcoins as a form of payment. I successfully coded python and added it to a server running bitcoind. Whenever a customer gets to the checkout and chooses to pay using bitcoin a new address along with grand total on a qrcode is generated. The bitcoin daemon is pre-populating the database with 10 addresses. When the number falls below 3 the daemon generates and pre-populates the database to top up to 10 addresses again. Everything works but before exploring it further with a view to taking it live I would like to know about electrum. I heard of it but never used it.

If I run electrum on a server effectively can it do the same thing the bitcoind is doing? If I modify the python scripts can electrum generate bitcoin addresses and populate the database whenever the top is needed? If it can I prefer using electrum over bitcoind because the bitcoin blockchain is massive and takes several days to sync on an AWS instance. Electrum I understand is instant.

Please can someone with electrum experience give some tips. Thank you.

[JollyGood]

Please any advice?

[Abdussamad]

If you only need to receive and not spend on the server create a watch only wallet on your server and then use the addrequest command with the --force switch to get new addresses. You can give it any dummy amount. See `electrum help addrequest` on the command line for more info. This command can also be used over a JSON RPC interface.

Alternatively you can use pybitcoin tools or some other implementation of bip32 in python to generate addresses from your wallet's master public key aka xpub.

[JollyGood]

If you only need to receive and not spend on the server create a watch only wallet on your server and then use the addrequest command with the --force switch to get new addresses. You can give it any dummy amount. See `electrum help addrequest` on the command line for more info. This command can also be used over a JSON RPC interface.

Alternatively you can use pybitcoin tools or some other implementation of bip32 in python to generate addresses from your wallet's master public key aka xpub.

I will look in to this, thank you for the reply. In the end if it can be done then whichever is easier will be used but I will find it a struggle as I have never used electrum before.

As long as a new unique address is generated at the checkout whenever a customer wants to pay then it works. The balance will be sent to a desktop wallet after reaching a pre-configured amount. That is what I am doing using my python script for my bitcoind. I hope to replicate it with electrum.

[Abdussamad]

A watch-only wallet does not contain any private keys so it is not possible to spend from that. So the way it works is you install electrum on your PC and create a wallet with it. Then you go to wallet menu > information and there you'll find your master public key (mpk) which begins with xpub.. You then install electrum on the server and restore your wallet using this mpk. The electrum command line option `electrum restore <mpk>` will do that. Run `electrum help` or `electrum help restore` for more info. Restoring from mpk on the server creates a watch only wallet there. You can then use the addrequest command mentioned before to get addresses for use in your app on the server. The addresses that you generate on the server will be for the wallet on your PC and using that you can spend the coins as you see fit. This setup is actually taking advantage of one of the best features of deterministic wallet i.e. the ability to generate addresses without needing the private key.

If you use pybitcointools then you would do the same thing. Generate addresses from your wallet's xpub.

[JollyGood]

Great information, thank you so much.

It seems the function of what I want to achieve are all available. The advantage here with electrum seems to be that instead of running a aws instance server with a massive HD for the bitcoin blockchain I can in theory run this on a nano instance for less than $5 a month. I will try and check it out this weekend.

As I never used electrum much was in the dark. Thank you for the help Abdussamad

[JollyGood]

Abdussamad thank you so much for your help. I installed the Electrum daemon on an Ubuntu EC2 instance and restored the wallet using the MPK. The 'electrum' commands are so eay to use. Everything went smooth and everything is going well except one problem.

I generated 40 wallets on my desktop electrum wallet using: wallet.change_gap_limit(50)

On my instance when I use 'electrum listaddresses' in the command line it lists the first 20 addresses correctly in order and then ignores addresses 21-40. Instead it then shows a further 5 addresses, none of which are on my desktop wallet. Any idea why this happens? I stopped/restarted the daemon and alsorestored the wallet again but it shows the same thing.

On my desktop even when I reduce the wallet addresses back to the default 20, the instance daemon command line shows 20 identical to desktop wallet addresses and then those extra unknown 5 addresses.

Please advise if I am doing something wrong or if this is the way it is supposed to work.

[Abdussamad]

Abdussamad thank you so much for your help. I installed the Electrum daemon on an Ubuntu EC2 instance and restored the wallet using the MPK. The 'electrum' commands are so eay to use. Everything went smooth and everything is going well except one problem.

I generated 40 wallets on my desktop electrum wallet using: wallet.change_gap_limit(50)

Do you mean 40 wallets or 40 addresses? I think you mean addresses. Don't confuse wallets and addresses. A wallet has many addresses.

On my instance when I use 'electrum listaddresses' in the command line it lists the first 20 addresses correctly in order and then ignores addresses 21-40. Instead it then shows a further 5 addresses, none of which are on my desktop wallet. Any idea why this happens? I stopped/restarted the daemon and alsorestored the wallet again but it shows the same thing.

On my desktop even when I reduce the wallet addresses back to the default 20, the instance daemon command line shows 20 identical to desktop wallet addresses and then those extra unknown 5 addresses.

Please advise if I am doing something wrong or if this is the way it is supposed to work.

On the desktop wallet go to view menu > addresses and switch to addresses tab. Up top you will see filter selections. Change the filter to "change" and "all". You will find the extra addresses there. These are change addresses that electrum uses internally to send back money to itself:

https://en.bitcoin.it/Change

Use addrequest --force to get addresses on the server. That way you'll always get an unused addresses from the receiving chain.

[JollyGood]

I am seeing 20 addresses in my desktop wallet. I am seeing the same 20 addresses plus 6 extra addresses in my putty console for my ubuntu daemon after using the MPK.

If I generate extra wallets on my desktop, say for example make it 30 instead of 20 by using wallet.change_gap_limit(30). The putty command console command still shows the same 20 + 6 addresses no matter what I do (reboot, stop/start daemon, delete wallet and use MPK again etc).

Is this supposed to happen?

[Abdussamad]

If you read my last post you'll see that I've explained what those 6 addresses you see are. They are change addresses.

As for the gap limit changes you need to increase the gap limit on the ubuntu system too if you want to see more addresses there. Unfortunately I don't know of any good way to do that via CLI. However it doesn't really matter because if you use addrequest --force it'll generate addresses past the gap limit too so it's not going to be a problem for you. Please understand that with deterministic wallets, which is what electrum wallets are, you have virtually unlimited addresses.

Also once again please don't call an address a wallet or vice versa. Wallets have many addresses, the keypairs behind them and related meta data.

[JollyGood]

Apologies. Yes writing "wallets" instead of "addresses" was an oversight on my part.

[JollyGood]

I got the export of addresses to the database tables figured out. Just need to make a transaction to see if the confirmations get populated too.

Does anyone know if Electrum Litecoin uses the same file and folder structure as Electrum (bitcoin). I want to use the same files for addresses and confirmations export to database if possible. Can I do that without tinkering the code?

Ideally I will run 2 servers, one with electrum and another with electrum litecoin both exporting addresses to the same remote database (wallet_litecoin and wallet_bitcoin) but can electrum litecoin and electrum daemons run on the same server?

[Abdussamad]

There is a notify command line option that'll tell you when funds are received to an address. There are also various get* options like getaddressbalance and gettransaction. Please run `electrum help` on the command line to see all these options. It is better to use the CLI or JSON RPC interface than reading the wallet files directly.

You can do `electrum setconfig rpcport` to set an rpc port. Different ports for the different daemons.

Why are you exporting addresses to a database? You can get electrum to generate them on demand for you.

[JollyGood]

Thanks Abdussamad

Well about running multiple daemons on one server, I will skip that. Since an instance is so cheap I will run separate bitcoin and litecoin daemons and learn along the way. I will not run any ecommerce stores on servers running daemons and will keep them separate. 1x btc daemon, 1xltc daemon and 1x ecommerce store/database

I am exporting addresses to an ecommerce store database so when an address is dislayed in checkout it is stored and associated specifically for that customers order (a new address for each order). Having an address in the database allows the amounts and confirmations to be stored too so if there any disputes with customers they can be sorted out quickly. Also in some cases it will be important to give an address to a customer upon sign up without the need to create a payment request (or order).

Also having prepopulated addresses helps if there are problems connecting with electrum for any reason at least addresses are available while the issue can be looked at.

If I have it on demand (http://docs.electrum.org/en/latest/merchant.html#) I think I have to have the ecommerce store and electrum on the same server?

[Abdussamad]

There is no harm in having an electrum watch-only wallet on the same server as the ecommerce store. A watch-only wallet can't be spent from so there is no risk to having it on the same server as the store.

Alternatively you can skip using electrum entirely and use a bip32 library to derive addresses from the MPK on the fly.

Yes assigning different addresses to different customers is the right thing to do.

[JollyGood]

There is no harm in having an electrum watch-only wallet on the same server as the ecommerce store. A watch-only wallet can't be spent from so there is no risk to having it on the same server as the store.

Yes I am going to do this.

Alternatively you can skip using electrum entirely and use a bip32 library to derive addresses from the MPK on the fly.

I needed to find another solution because the orders placed in the store need to be updated as and when confirmations were received. I think I have that part covered using a python scriptbut needs further testing. I was going to do it by testing using a small bitcoin payment but the tx fee is quite high. I am looking at trying to learn about transactions and exporting to database using testnet.

Yes assigning different addresses to different customers is the right thing to do.

Thanks for confirming I will need to have new addresses for each order and in some cases I will also need to assign a fixed address for specific customers.

[JollyGood]

Absolutely loving electrum. The more I play with the code the more I love it. Experimenting with python and learning along the way

Thank heavens I will not be paying for a massive blockchain or waiting for it to sync !

[JollyGood]

Am about to put electrum on the ecommerce server but need to know about the electrum payment function from here (http://docs.electrum.org/en/latest/merchant.html#)

electrum setconfig requests_dir /var/www/r/

and

electrum setconfig url_rewrite "['file:///var/www/','https://mydomain.com/']"

then

electrum addrequest 3.14 -m "this is a test"
{
   "URI": "bitcoin:1MP49h5fbfLXiFpomsXeqJHGHUfNf3mCo4?amount=3.14&r=https://electrum.org/r/7c2888541a",
   "address": "1MP49h5fbfLXiFpomsXeqJHGHUfNf3mCo4",
   "amount": 314000000,
   "amount (BTC)": "3.14",
   "exp": 3600,
   "id": "7c2888541a",
   "index_url": "https://electrum.org/r/index.html?id=7c2888541a",
   "memo": "this is a test",
   "request_url": "https://electrum.org/r/7c2888541a",
   "status": "Pending",
   "time": 1450175741
}

Are the above guidelines mandatory for electrums payment structure to work?

For it to work and best fit my needs I think instead of having a pop up at index.html showing the qrcode and timer slider I would rather have the qrcode and timer slider on my checking_out.php. The checking_out.php file also has other information related to the ecommerce store on it so the above code would be incorporated but would the timer slider still work if the default guidelines were modified? Avoiding a pop up qrcode is preferred. I suppose pop ups are not exactly great when using mobile phone right?

[JollyGood]

I have got the required information sent to the database but the following python code has no effect at all:

def update_confirmations_and_date(connection, confirmations, tx_date, txid):
with connection.cursor() as cursor:
cursor.execute('''
update tx_history
set confirmations = %s,
date = %s
where txid = %s ''',
(confirmations, tx_date, txid))

update_confirmations_and_date(connection, confirmations, tx_date, txid)

Please advise on what needs to be changed. When I execute the python file all new addresses and associated payments are sent to the database but there is a problem. When the information is sent first time in the tx_history table it does not get updated. Specifcally the confirmations to not get updated. When I trunacate the table, execute the python file and then reload the database all the address information in updated and current with all confirmations showing. If I do not truncate the table then exisiting addresses confirmations do not get updated.

Executing the python file does everything else correctly, including sending correct information to the 'addresses' table to update the "balance_confirmed" and "balance_unconformed" status but it does not collect the confirmations information from the daemon and send it to the database.

What am I doing wrong with the above code?

[HCP]

Are you "committing" the transaction? Most DB libraries will create transactions such that any updates etc need to be explicitly committed or rolled back etc... check for a commit() method on whatever "connection" class you are using (looks like Psycopg based on the connection.cursor() call, if so then read here: http://initd.org/psycopg/docs/connection.html#connection