Bitcoin Forum

Hello everyone!

I am relatively new to bitcoin, but I have general idea of how it works, so I know about wallets, confirmations, bitcoin addresses and all this general informations.

What i am looking for right now is some easy and reliable way of checking/crawling the blockchain automatically. I have a certain skills as a programmer and I am thinking about doing something around bitcoin and related stuff (idk, maybe some new wallet or service or who knows). What I would like to ask IT guys here is what is the most effective (not necessarily the easiest) way of keeping track of what is going on in blockchain.

Let's say I will have a wallet and inside that wallet few addresses and I will want to check every minute what's going on with those addresses. Did someone send BTC to one of those addresses? How many BTC they've send? How many confirmations are there already? At what time it was done? etc.

I realize there are some API services for example on blockchain.info where you can request an API key and then you even get notified by them when something happenes. But I tried to apply there, they declined my request and then I realized "Hey, I must be stupid, why I just don't check the blockchain directly somehow like they do?" - don't get this wrong I don't need anything complex, all I am looking for right now is just a simple way of checking few things like:

- latest incoming transactions into an address
- how many bitcoins are stored on a certain address
- how many confirmations certain transaction gets

That's basically all. Would be great if I could just crawl some public site for all these informations or something. Or if there is some reliable API someone provides (where you won't get declined like on blockchain.info) that would be great. I am probably asking very basic and stupid question and I expect the solution to be quite easy, but I am really new to this.

you can just use JSON-RPC
https://en.bitcoin.it/wiki/API_reference_(JSON-RPC)#PHP
https://en.bitcoin.it/wiki/Original_Bitcoin_client/API_calls_list

pretty easy to work with: just install bitcoin core, let it sync and call the API directly

Awesome, I knew there has to be something like this. Looks like exactly what I am looking for, thanks!

Still: Is there some public API service or something that just allows you to call few basic commands? What you've posted is great, but may be a little overkill for what I actually need, but I might to learn to use it anyway. Any other suggestions anyone?

If i'm not mistaking, blockchain.info also has an api for which you don't need an api key, if i remember correctly, it's limited since it only allows x calls every 24 hours.

https://blockchain.info/api/json_rpc_api

The big plus for this api is that it is fully compatible with the original Bitcoind RPC protocol, so if you develop code and use this API, you should be able to switch to the bitcoind api without to much troubles.

If you just want a block explorer, you can also have a look at the insight api, i hear it's easy to use and install, but in essence, you're running a full node.

Check this thread for how-to videos on getting started with the basics.
https://bitcointalk.org/index.php?topic=990348.0

While you can using blockchain.info to get started, you might not want to rely on them for a commercial solution. They've had a lot of issue in the past with security and reliability and generally looked down upon for wallet services, especially holding bitcoin. They're often targeted on ddos attacks which could interrupt your API calls. Good to experiment with, but not for production.

Look at listtransactions this will put the transactions into a json format that you cycle through.

Look at getreceivedbyaddress to see what's been received to that address and then listunspent for an array of unspent values.

Look at gettransaction and then the object 'confirmations' in that array.

I highly recommend running your own node.  ;D

Thanks for link and basic examples. What benefits has running my own node? Does it mean I am just closer to BTC network so everything works faster (faster propagation of transactions and all)? Why not just use RPC to access bitcoin network?

First, in short, you help the network by accepting and validating the transactions and blocks and relaying them to other nodes. The number of full nodes has been going down over time as bitcoin popularity continues to grow. Secondly, you're in control over your wallet's private keys and don't have to worry about anyone getting them through other means. I know you can export them (encrypted) from other wallet services, but by managing your own security, you have full control. Your API calls might be slightly faster, but you also wont have any limits on the number of calls you can have. You can make unlimited API calls to your own node.

As mentioned above ,Blockchain.info does give you access to the free API which doesn't need a Key.Mocachino gave you the most simplest solution using core,to avoid installing it if you're looking for a quick set up with few commands you can try the below 3rd party API's.

https://developers.xapo.com/

http://blockr.io/documentation/api

http://dev.blockcypher.com/

What else came to mind when thinking about this:

When accessing blockchain via JSON-RPC or node, is there any way for the outsiders to see some info about who and from where is he doing it? Le'ts say I will set up my own VPS and will run send commands for example - can anybody see the IP address of my VPS server for example? Or is this entirely untrackable? Meaning if I send BTC from one address to another using JSON-RPC which runs on IP address 155.55.11.1 (just an example), can someone find it that the 'order' for sending those BTC somewhere came from this IP address?

I mean if BTC is all about privacy and anonymity, if I pay for my own VPS and run some BTC services there then I guess I wouldn't want to know the world that it's done on that server. Because you know you can have frontend interface/domain for users on one server and manage BTC transactions from entirely different place/server (or at least that's what feels right to do), but I am still curious if it's possible to track who (from what IP) is sending "orders" to the addresses.

IP's are not recorded in the blockchain. As far as i know, the worst thing that can happen is that each node can potentially log which node sent them the transaction first. It would be very hard for somebody to track down the original node that broadcasted the transaction to the network. For example, blockchain.info shows the ip address that broadcasted the transaction to their node first. This might or might not be the true source of the transaction.

I think there are other things you want to worry about when you're very privacy-concerned: your id info when buying BTC, you address when buying physical stuff with BTC, any service that can potentially link your id, ip,facebook addy, email,... to a BTC address...
Also, if you're trying to stay anonymous, i'd be carefull with social contacts... Don't mention to much on social media, or even at the pub after a couple of beers...

Edited to add some more explanation

Yeah, these things go without saying, I was just wondering whether there are some less obvious methods of tracking something down in blockchain. Thanks.

If you disclose your public key in some fashion like here in the forums or your name in OP_RETURN, etc.  This paper from a couple of years ago discusses a particular
"attack" to reveal information on a percentage of transactions: http://arxiv.org/abs/1405.7418 (http://arxiv.org/abs/1405.7418)  Because bitcoin still relies on the network stack, you can still use anything that inspects the stack.

Your VPS provider or anyone up/downstream could monitor the network traffic and see that your VPS is connecting to bitcoin. You could can see what's available to wireshark here: https://www.wireshark.org/docs/dfref/b/bitcoin.html. There's mitigation for these types of things, Tor for example. Version 12 of core, has some new tor functionality: https://bitcoin.org/en/release/v0.12.0

Okay, I'm starting to run into specific problems when trying to install things on my debian VPS - currently I try to install bitcoin-gt, which requires libboost but I keep getting this

 dependency problems prevent configuration of bitcoin-qt:
 bitcoin-qt depends on libboost-chrono1.54.0; however:
  Package libboost-chrono1.54.0 is not installed.
 bitcoin-qt depends on libboost-filesystem1.54.0; however:
  Package libboost-filesystem1.54.0 is not installed.
 bitcoin-qt depends on libboost-program-options1.54.0; however:
  Package libboost-program-options1.54.0 is not installed.
 bitcoin-qt depends on libboost-system1.54.0; however:
  Package libboost-system1.54.0 is not installed.
 bitcoin-qt depends on libboost-thread1.54.0; however:
  Package libboost-thread1.54.0 is not installed.
 bitcoin-qt depends on libdb4.8++; however:
  Package libdb4.8++ is not installed.
 bitcoin-qt depends on libprotobuf8; however:
  Package libprotobuf8 is not installed.

and have no idea how to install those libboost.

(idk if this is a proper section for dealing with these things anymore)

Is this a freshly installed system (or VPS), in other words, can you still switch distro's to ubuntu?
I ask because installing bitcoind on ubuntu is as simple as running:
sudo apt-add-repository ppa:bitcoin/bitcoin
sudo apt-get update
sudo apt-get install bitcoin-qt bitcoind

I did manage to install bitcoind on other distro's, but the process was a bit more complex... In case you cannot switch distro's, i'll have a look at the output this afternoon.

Try the following:


Debian is a bit of a... rough ride. Unbuntu or CentOS can be an easier journey if linux isn't your main squeeze (pun intended).  8)

I am running Debian Jessie on that VPS - yes it's brand new, I can switch to whatever I want (I don't have any data there), altough there was a preinstalled LAMP which I also have no idea how to install so I will have do it manually after I switch to Ubuntu...

Okay then, I've started the re-installation, says it's gonna take up to 1 hour before it's done...

I honestly don't have much of an experience with setting up server, so your help is much appreciated guys. I was actually thinking of putting up a "job offer" to help me set things up, if you would like I can even pay you (in BTC) for some more extensive help with this (PM me if you are interested).

Alright it's done, I have now

Description:    Ubuntu 16.04 LTS
Release:        16.04
Codename:       xenial

installed on my VPS.

Even if you aren't using digitalocean as a vps provider, they have some good step by step tutorials for both ubuntu and centos.  Here's one for 16.04:

https://www.digitalocean.com/community/tutorials/how-to-install-linux-apache-mysql-php-lamp-stack-on-ubuntu-16-04

Alright, have the bitcoind finally up and running! So...

Is it now possible to manipulate (sending btc to other addresses etc.) wallets/addresses anywhere in blockchain as long as I have access the to it? (xpub) For example I have a wallet on blockchain.info and I know addresses and their xpubs - now I should be able to access them right? I mean it doesn't matter they are "at blockchain.info" cuz it's just some node in network and I don't have to use RPC JSON protocol just to access/store things on my server/node, I can now access any address in the network no matter where it is, true?

Because so far I don't feel like storing btc via my own VPS, I hardly know what I am doing when using linux commands via shell, I am definitely not going to store any btc there, but it would be awesome if I could access addresses/wallets which I've for example created using my browser and regular blockchain.info user interface like everyone else. Or do the blockchain.info where I currently have my addresses with my BTC somehow doesn't allow this? Like idk they hold some other keys or something... If I just go to my wallet on blockchain.info, display my address and then display xpub for that address thats all I need to send commands for that address via my newly installed bitcoind? (hope I am not missing something important here)

Awesome, congrats. So, you can send to any address you want or receive any address you create (own the private key to). You can access nodes that you have access to, so if you're running bitcoind on your server or at blockchain.info.  If you want to code against the wallet services, you're going to be using json for the most part. If you just want to use wallet services like sending bitcoin, you're going to use their wallet user web interface.


Okay, so there's bitcoind, which is a headless daemon and then blockchain.info API which is a way for you to talk to your wallet(s) through code. The front end for blockchain.info is a web UI on their wallet site, for bitcoind, you have to issue commands via the command line or use RPCJSON. Bitcoind doens't have a UI per say unless you install one for it.

Bitcoind - https://en.bitcoin.it/wiki/Running_Bitcoin
blockchain.info - https://blockchain.info/api

Check out coinables tutorials, he has some for blockchain.info and bitcoind:
https://bitcointalk.org/index.php?topic=990348.0

I don't think blockchain.info has a testnet, so be careful.  You can run testnet on bitcoind, and if you need testnet coins, ask me and I'll send you some (the faucets kind of suck).

Wait, let me get this straight.

I go here https://blockchain.info/wallet/#/login I log into my wallet, I click on Settings->addresses-> there I found my address which is 'blablabla123345blabla', I click on more options and I display public key to that address which is 'publicblablabla56789blabla'.

Now: This is NOT enough then to manage sending/receiving for that address? How do I get private key to that address? Do they hold it? I have another address listed as 'imported address' on blockchain.info and it has private key not public one. But all other addresses I create have just public key. Does this mean I still have to use their god damn API which they declined for me?

Or is the private key just for accessing the wallet as a whole and public keys are for every single addresses so I have one private key for all the addresses but many public keys for each and every one address...?

Because what I basically want to achieve is to be able to send/receive BTC no matter the API. I mean they can't limit number of requests if I use just a blockchain network right? I won't use any blockchain.info API, I will just access my wallet with a key I own throught the network and nothing else.  How could they limit that? What am I missing here?

I think you've crossed a couple of wires someplace.

A wallet service, like blockchain.info and coinbase allows you to send and receive bitcoins. Your public key allows you to receive bitcoins, this is the address that you provide to people that you want to get bitcoins from.  Your private key is something that you want to keep to yourself and not share with anyone.  The bitcoin private key is universal, you can typically import it into any bitcoin wallet.  This is important because let's say, blockchain.info shuts down tomorrow, you can just import your private key into another wallet and reclaim any bitcoins you may have. Be very careful with your private key.

Both blockchain.info and coinbase offer API services, this allows you to use code to access your wallet to send/receive/sign bitcoin transactions through code. Both of them have limits in how you interact with the bitcoin network, this is to prevent bad actors and others from abusing their network.

With your own wallet installed on your own system, you can all of the above with no limitations.


You might want to learn more about bitcoin on a technical level, there's a video series here that's free (only the mining video is outdated): http://bitcoin.cbtnuggets.com/

I'd be happy to skype with you at some point if you're still confused and I can walk you through some more information if you're still confused, just PM me.

What is the current size of blockchain in MB/GB? Looks like my VPS with 30GB disk size isn't enough to run a full node? If I understand correctly I have to download whole blockchain for it to work.

This graph https://blockchain.info/charts/blocks-size shows 74k MB? Does it mean I need at least 74GB hard drive to run bitcoind on VPS these days?