Bitcoin Forum

Bitcoin => Project Development => Topic started by: donGeilo on August 17, 2016, 06:52:09 PM

Title: Python scripting
Post by: donGeilo on August 17, 2016, 06:52:09 PM

Hi, i'm working on a small project and got stuck.
Maybe somebody coud lead me?

I'm running a full node and want a py-script to check the balance of an address,
That's the code i am using, but really no idea, how the url needs to look, to get get the balance from my bitcore full node, already tried "http://localhost:3001/insight/"+address+"" but, didn't work!

That's the original code i was using:

Code:

def scan(address,pkey): 
	req = urllib2.Request("http://btc.blockr.io/api/v1/address/info/"+address+"")
	res = urllib2.urlopen(req)
json_data = json.load(res)

Title: Re: Python scripting
Post by: psycodad on August 17, 2016, 07:13:36 PM

If you want to check against your local node, the below python module would probably be better suited for the job (though it probably can be done in some more complicated way with urllib2):

https://en.bitcoin.it/wiki/API_reference_%28JSON-RPC%29#Python

Also AFAIK to query the balance of any address and not just those of your wallet you'd need to run the node with -txindex, see below URL:

https://en.bitcoin.it/wiki/Running_Bitcoin

HTH

Title: Re: Python scripting
Post by: donGeilo on August 17, 2016, 07:28:53 PM

First thx for your reply, but
how should the code lines look then?

It's a bitcored full node with insight api, so think -txindex is not suitable?

Title: Re: Python scripting
Post by: Adriandmen on August 17, 2016, 07:36:16 PM

The syntax seems wrong. I'm assuming this is Python 2.

I haven't tested it yet, but does the following work?:

Code:

req = urllib2.Request("http://btc.blockr.io/api/v1/address/info/"+address)
res = urllib2.urlopen(req)
json_data = json.loads(res.read())

Title: Re: Python scripting
Post by: psycodad on August 17, 2016, 08:09:13 PM

I don't know about insight api, however what I said regarding txindex is wrong: that doesn't give you the ability query balances, just all transactions (from which you can extract all balances if you for example store them in a db like a block explorer does).

Title: Re: Python scripting
Post by: donGeilo on August 17, 2016, 08:37:14 PM

Thx again, but nothing worked!

I think i'm doing somthing wrong with the Url and the json_data quering.

Title: Re: Python scripting
Post by: kaicrypzen on August 17, 2016, 09:16:07 PM

The original code seems okay to me, though there are somethings that bother me:
- The scan function has a parameter named pkey which it doesn't use ...
- I don't know why +"" is added.
- urlopen can directly take the (string) url as parameter, I don't see why call Request first ...

Normally, both json.load(res) and json.loads(res.read()) should work ...

What version of python are you using?

Do you get an exception when running your code?

Title: Re: Python scripting
Post by: donGeilo on August 17, 2016, 09:31:28 PM

ATM i'm using python 2.7.12

i copied these line from github from a script. Tried to modify it, but till now nothing worked.

If i'm typing an addres in the browser, the node works fine.

Edit:
The code like it is, works, just when i try to point it to my localhost, nothing happens

Title: Re: Python scripting
Post by: kaicrypzen on August 17, 2016, 09:45:48 PM

Quote from: donGeilo on August 17, 2016, 09:31:28 PM

The code like it is, works, just when i try to point it to my localhost, nothing happens

What does print res.read() return?

Title: Re: Python scripting
Post by: donGeilo on August 17, 2016, 09:51:32 PM

Just an error: unexpected indent

Title: Re: Python scripting
Post by: donGeilo on August 17, 2016, 09:58:31 PM

Maybe you got an easy clean solution, to check the balance of some addresses in a list, on a local node?

Cause the script from github has got much crap in it!
https://github.com/TheZ3ro/bitcoin-privkey-bruteforce/blob/master/btcscan.py (https://github.com/TheZ3ro/bitcoin-privkey-bruteforce/blob/master/btcscan.py)

Title: Re: Python scripting
Post by: kaicrypzen on August 17, 2016, 10:04:34 PM

Quote from: donGeilo on August 17, 2016, 09:51:32 PM

Just an error: unexpected indent

So there seem to be an issue with indentation, maybe a mix of tabs and spaces ...

Did you try running the lines directly in a python interpreter, like this:

Code:

import urllib2
import json
address = "your_address"
req = urllib2.Request("http://btc.blockr.io/api/v1/address/info/"+address)
res = urllib2.urlopen(req)
print res.read()
json_data = json.load(res)
print json_data

Title: Re: Python scripting
Post by: Adriandmen on August 17, 2016, 10:09:12 PM

Quote from: donGeilo on August 17, 2016, 09:58:31 PM

This one also works:

Code:

import urllib2
import json

address = raw_input("Enter Address >> ")
urlRequest = urllib2.Request("https://blockchain.info/rawaddr/" + address)
data = urllib2.urlopen(urlRequest).read()

json_data = json.loads(data)
print "Final Balance: " + str(json_data['final_balance'])

Title: Re: Python scripting
Post by: donGeilo on August 17, 2016, 10:14:57 PM

Yes, these works great!

But need it to read the addresses from a list.

Title: Re: Python scripting
Post by: kaicrypzen on August 17, 2016, 10:16:39 PM

Quote from: donGeilo on August 17, 2016, 09:58:31 PM

Maybe you got an easy clean solution, to check the balance of some addresses in a list, on a local node?

If it is parsing the output of a url to get the balance then I think it can be done.

I just need some info:
- Where are the addresses? in a file? in the code in a list object?
- What is the call you perform for the local node i.e. the url which you use to access information about the BTC address? And do you expect the same kind of output that http://btc.blockr.io/api/v1/address/info/<address> gives?

Title: Re: Python scripting
Post by: donGeilo on August 17, 2016, 10:22:06 PM

I want to put the addresses in a textfile.
The output is ok.
Url supposed to be: http://localhost:3001/insight/ (not working right now)

Title: Re: Python scripting
Post by: kaicrypzen on August 17, 2016, 10:24:29 PM

Quote from: donGeilo on August 17, 2016, 10:14:57 PM

Yes, these works great!

But need it to read the addresses from a list.

Then you might loop through your list, something like that:

Code:

balances = []

for address in addresses:
    urlRequest = urllib2.Request("https://blockchain.info/rawaddr/" + address)
    data = urllib2.urlopen(urlRequest).read()
    json_data = json.loads(data)
    balances.append("Balance of " + address + " is " + str(json_data['final_balance']))

"\n".join(balances)

Title: Re: Python scripting
Post by: kaicrypzen on August 17, 2016, 10:29:58 PM

Quote from: donGeilo on August 17, 2016, 10:22:06 PM

I want to put the addresses in a textfile.

Let's assume there is one address per line, then you might replace the previous loop with:

Code:

f = open("addresses.txt", "r")

for address in f:
....

And if you want to write the results somewhere then use:

Code:

f = open("balances.txt", "w")

f.write(....)

Finally you might have something like:

Code:

addresses = open("addresses.txt", "r")
balances = open("balances.txt", "w")

for address in addresses:
    urlRequest = urllib2.Request("http://localhost:3001/insight/" + address)
    data = urllib2.urlopen(urlRequest).read()
    json_data = json.loads(data)
    balances.write("Balance of " + address + " is " + str(json_data["data"]["balance"]) + "\n") #use the right keys

addresses.close()
balances.close()

Title: Re: Python scripting
Post by: donGeilo on August 17, 2016, 10:37:51 PM

then i need the prefix address in each line right?

Title: Re: Python scripting
Post by: kaicrypzen on August 17, 2016, 10:43:28 PM

Quote from: donGeilo on August 17, 2016, 10:37:51 PM

then i need the prefix address in each line right?

If your txt file is like:

Code:

first_addr
second_addr
.....
last_addr

the code will work.

The github code seems to loop through lines which are prefixed with Address: and Privkey: but if your file contains only addresses, one per line, then the code will work just fine (it loops through the lines and affects the line's value to the variable address). Is this case? Or will you also have private keys in your txt file?

Title: Re: Python scripting
Post by: donGeilo on August 17, 2016, 10:46:46 PM

No, like it is is great i'll give it a try now, just need to figure out how i reach the local host.

Then i'll give you a reply.

But THANK YOU VERY MUCH!!!

Title: Re: Python scripting
Post by: kaicrypzen on August 17, 2016, 10:51:47 PM

Quote from: donGeilo on August 17, 2016, 10:46:46 PM

No, like it is is great i'll give it a try now, just need to figure out how i reach the local host.

Then i'll give you a reply.

But THANK YOU VERY MUCH!!!

Ok then, I can go to sleep :P.

You are very welcome. Good luck ;).

Title: Re: Python scripting
Post by: xcbtrader on August 18, 2016, 07:16:57 AM

Hello

If you use a external API like https://blockchain.info/rawaddr/, the problem is if your make more connections to the same IP in a small time, server blockeds your IP (2 o 3 connections per second is max).
The solution is add sleep instruction.

Code:

import time
...
...
...
for address in addresses:
    urlRequest = urllib2.Request("https://blockchain.info/rawaddr/" + address)
    data = urllib2.urlopen(urlRequest).read()
    json_data = json.loads(data)
    balances.append("Balance of " + address + " is " + str(json_data['final_balance']))
    time.sleep(0.5) # 2 connectios per second

...
...
...

Another solution is connect differents APIs.

Title: Re: Python scripting
Post by: donGeilo on August 18, 2016, 07:49:42 AM

Yes, thx

that s the reason I want to use my own node.

But unfortunately i can't reach it from the script.

Title: Re: Python scripting
Post by: kaicrypzen on August 18, 2016, 08:44:16 AM

Quote from: donGeilo on August 18, 2016, 07:49:42 AM

Yes, thx

that s the reason I want to use my own node.

But unfortunately i can't reach it from the script.

Still not working?

Can you post your code and/or any error/exception you're getting?

Title: Re: Python scripting
Post by: donGeilo on August 18, 2016, 09:00:09 AM

Good morning!

lol No still doesn't work!

My code:

Code:

import urllib2

import json



addresses = open("addresses.txt", "r")
balances = open("balances.txt", "w")

for address in addresses:
    urlRequest = urllib2.Request("http://localhost:3001/insight-api/" + address)
    data = urllib2.urlopen(urlRequest).read()
    json_data = json.loads(data)
    balances.write("Balance of " + address + " is " + str(json_data["data"]["balance"]) + "\n") #use the right keys

addresses.close()
balances.close()

My errors:

Code:




Traceback (most recent call last):
 
 File "self4.py", line 10, in <module>

    data = urllib2.urlopen(urlRequest).read()
 
 File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
  
  return opener.open(url, data, timeout)
 
 File "/usr/lib/python2.7/urllib2.py", line 435, in open
  
  response = meth(req, response)
 
 File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
  
  'http', request, response, code, msg, hdrs)
 
 File "/usr/lib/python2.7/urllib2.py", line 473, in error
  
  return self._call_chain(*args)
 
 File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
  
  result = func(*args)
 
 File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
  
  raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

urllib2.HTTPError: HTTP Error 404: Not Found

Edit:
I think i'm missing something inside the bitcoin.conf file

Edtit2:

Just recognized, seems to be really close! It's the first time the bitcored-Terminal shows the GET requets from python (but with the 404 error in it).
Never appeard before!!!

Title: Re: Python scripting
Post by: kaicrypzen on August 18, 2016, 09:51:54 AM

Quote from: donGeilo on August 18, 2016, 09:00:09 AM

Good morning!

Good morning to you too :) (or maybe good evening, or good day :))

Quote from: donGeilo on August 18, 2016, 09:00:09 AM

My errors:

Code:

urllib2.HTTPError: HTTP Error 404: Not Found

If you have this error, then I don't understand why it would work when you enter the url in your browser ... Are you a 100% sure that when you enter http://localhost:3001/insight-api/<address> it works in your browser?

You might try two things:
- Clean your browser's cache and retry (entering the url directly in the browser).
- Add a print statement to see what is actually passed to Request:

Code:

url = "http://localhost:3001/insight-api/" + address
print url

Title: Re: Python scripting
Post by: donGeilo on August 18, 2016, 10:06:14 AM

Alright,

if i put: localhost:3001/insight/ +address in the address-bar i get the error 404!

If i just open localhost:3001/insight/ -the page opens an i can put the address in the search field, and it shows me the balance!!! ???

EDIT:

I think it supposed to be http://localhost:3001/insight-api/address/ +address

'cause that's the url it shows me if i enter the address in the search field (in fact it shows me"/insight/address/")

Title: Re: Python scripting
Post by: kaicrypzen on August 18, 2016, 10:35:10 AM

Quote from: donGeilo on August 18, 2016, 10:06:14 AM

Ah! There might be your answer then. If indeed http://localhost:3001/insight-api/address/<address> is the right url then just update the code and see if it works better ;)

Title: Re: Python scripting
Post by: donGeilo on August 18, 2016, 10:37:35 AM

Nope,
same 404 error

Title: Re: Python scripting
Post by: kaicrypzen on August 18, 2016, 10:56:58 AM

Quote from: donGeilo on August 18, 2016, 10:37:35 AM

Nope,
same 404 error

Well you'll just have to figure out what the right url is.

When you put the address in the search field, do you have to click on a search button (or hit enter) ? If so, then what is the url that appears in the search bar once you've done that? This would be the url you should be using.

Title: Re: Python scripting
Post by: donGeilo on August 18, 2016, 11:05:00 AM

I copy-paste the addres in the search field and hit enter,
and the exact url is:

localhost:3001/insight/address/1NH5FzSuo........

but from the bitcore forum i got the info:
/insight/* is routed via client-side javascript application, the api is accessible at /insight-api/

Title: Re: Python scripting
Post by: kaicrypzen on August 18, 2016, 11:27:32 AM

Quote from: donGeilo on August 18, 2016, 11:05:00 AM

I found this https://github.com/bitpay/insight-api (https://github.com/bitpay/insight-api)

It says an address is accessed with:

Code:

/insight-api/addr/<address>

and a balance with:

Code:

/insight-api/addr/<address>/balance

Title: Re: Python scripting
Post by: donGeilo on August 18, 2016, 11:39:53 AM

If i use:

Code:

 http://localhost:3001/insight-api/addr/+address")

Code:

 http://localhost:3001/insight-api/addr/+address/balance")

i get the error 400 bad request

EDIT:

misstyped in the code!!

Now it is just KeyError: 'data'

So it passsed the url!!!

Title: Re: Python scripting
Post by: donGeilo on August 18, 2016, 11:44:48 AM

YAY!!!
removed 'data'

and ......

it works!!!!!!!!!!!

But gives no output!!!

EDIT:

SOLVED!!!

Title: Re: Python scripting
Post by: kaicrypzen on August 18, 2016, 11:53:12 AM

Quote from: donGeilo on August 18, 2016, 11:44:48 AM

YAY!!!
removed 'data'

That's why I commented

Code:

#use the right keys

;)

Quote from: donGeilo on August 18, 2016, 11:44:48 AM

and ......

it works!!!!!!!!!!!

But gives no output!!!

The code isn't supposed to give any output ... it just stores balances in balances.txt ;)

Quote from: donGeilo on August 18, 2016, 11:44:48 AM

EDIT:

SOLVED!!!

Cool :), so what was the matter?

Title: Re: Python scripting
Post by: kaicrypzen on August 18, 2016, 12:12:46 PM

Quote from: donGeilo on August 18, 2016, 12:01:45 PM

during the tests i edited a few balances -output text files

Just looked into the wrong one!

Quote from: donGeilo on August 18, 2016, 12:01:45 PM

Balance of 1NH5FzSuo.......
is 0.1

Do you mean there is a carriage return after the address? If that's the case, then use:

Code:

balances.write("Balance of " + address.strip("\n") + " is " + str(json_data["balance"]) + "\n")

Title: Re: Python scripting
Post by: donGeilo on August 18, 2016, 12:19:01 PM

No everything works fine, like it should!!

Just one more question:

If my list would contain more lines, like:

John Smith
Address: 1234567890...

Benjamin Franklin
Address: 1098765432.....

How would that look, to just output all balances ?

Title: Re: Python scripting
Post by: kaicrypzen on August 18, 2016, 12:47:32 PM

Quote from: donGeilo on August 18, 2016, 12:19:01 PM

If you have more info in your text file, maybe you should consider using a csv format or make the lines very distinguishable:

Code:

Name: John Smith
Address: 1234567890...

Then you'll have to test, let's assume that you don't care about the name:

Code:

....

for line in addresses:
    if line.startswith("Address: "): # Test that the line is an address line
        address = line.strip("Address: ") # Remove the prefix Address: 
        urlRequest = urllib2.Request("http://localhost:3001/insight-api/addr/" + address)
        data = urllib2.urlopen(urlRequest).read()
        json_data = json.loads(data)
        balances.write("Balance of " + address + " is " + str(json_data["balance"]) + "\n") 

....

Title: Re: Python scripting
Post by: donGeilo on August 18, 2016, 01:18:39 PM

Perfect!!!

Thanks alot!!!

Title: Re: Python scripting
Post by: kaicrypzen on August 18, 2016, 01:49:29 PM

Quote from: donGeilo on August 18, 2016, 01:18:39 PM

Perfect!!!

Thanks alot!!!

You're welcome.

You might wanna check dive into python (http://www.diveintopython.net) and the python standard library for python 2 (https://docs.python.org/2/library) and python 3 (https://docs.python.org/3/library).

Also, maybe lock this thread if all is okay ;) and don't hesitate to shoot me a pm if you need anything.

Good luck ;).