[ANN] chainsnort (live transaction monitoring and fingerprinting tool)

[flatfly]

This command-line tool displays transactions as they are happening on the blockchain.
It's similar to a network trace, but for blockchain activity. It has the ability to automatically flag large transfers.  It can be useful in identifying interesting patterns in transactions.

Warning: watching the live blockchain action can be hypnotic!

Features:
- Print new transactions to the console, in the following format:
   UTC Time, Total Amount (sum of all outputs), Main recipient, Transaction Fee
- Big (50+ BTC) transactions and extreme fees are flagged.
- Also shows "new block" events, with the corresponding block reward
- No private keys, admin privileges, or credentials of any sort are needed.
- 100% Python -> runs fine on OS X, Linux, Windows and Android!
- NEW August 2014: transaction type guessing engine  (in private beta build only)
- Low-latency encrypted websocket connections
- Extremely light on resources and bandwidth
 
I will be adding new features and parameters on a regular basis.


The current data source is the blockchain.info websocket API. Direct local bitcoind support, while a little more complex, could also be implemented in case there is enough demand for that.

Many thanks for your support - donations will help me continue development!

As a little bonus, you will be treated to a colorful ASCII easter egg (in the Windows build only, for now), if chainsnort sees a tip being sent to my tip jar [1111127SpvabYpoeDoiz5L7QPkfiSh2Q] - The easter egg changes depending on the size of the transaction

I also have a separate edition with some features that are useful for forensic and technical investigation purposes. I'm not releasing it at the moment but PM me if interested.


Screenshots:






Latest public version: 0.43.3

OS X and Linux

Just copy and paste into a terminal:

1/ curl -o websocket.py https://raw.github.com/liris/websocket-client/master/websocket.py

2/ curl -o chainsnort.py https://gist.github.com/flatfly/8253870/raw/9733e34e469e1bcd8d0ae13565ade4eaa0490618/gistfile1.py

3/ python chainsnort.py


Windows build

Download

Please do leave some feedback or send a tip if you like this tool. Highly appreciated!


Legal stuff:
I share the source code of chainsnort for educational and reviewing purposes only. If you plan to reuse portions of the code in commercial, academic or government projects, please contact me.
                    
Also, I am developing this on my free time and with a VERY limited budget, so i welcome any donations. Also, all donators are entitled to a special color edition of chainsnort.  (Proper website coming soon)


SHA1: ac7c67d48d4f11aff93c4c8a5e3c304f9097d578

GPG signature for Windows executable:

Code:

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (MingW32)

iEYEABECAAYFAlLSrX4ACgkQ0dQqIfkZdf7H2wCgqhxvqnr9bAEuuemn0DruGjxF
c54An0NNSZOj4l72Z8RWpydTdkNlUVHc
=/Bo1
-----END PGP SIGNATURE-----

Source code for all platforms (Windows, Linux, OS X, Android): available at this gist.

[1715543938]

1715543938

[1715543938]

1715543938

[1715543938]

1715543938

[betyourbits]

This is very cool! What kind of uses do you think it will serve? For traders?

[carajillu]

Cool!

[flatfly]

Optimized buffer size to drastically reduce bandwidth usage.

[bitpop]

Wow

[bitpop]

server='https://blockchain.info' ssl?

[bitpop]

Error in cygwin, but works without

$ python chainwatch.py
ChainWatch v0.35:  52789 confirmed transactions in last 24 hours - 0.61 tx/sec


ValueError: zero length field name in format

[bitpop]

How are you making the beep? I can't find it in the code :/

[flatfly]

server='https://blockchain.info' ssl?

on my to-do list!

[bitpop]

server='https://blockchain.info' ssl?

on my to-do list!

It redirects anyway

[flatfly]

Error in cygwin, but works without

$ python chainwatch.py
ChainWatch v0.35:  52789 confirmed transactions in last 24 hours - 0.61 tx/sec


ValueError: zero length field name in format

Perhaps you have a different version of Python in cygwin?
ChainWatch uses Python 2.7.

[flatfly]

How are you making the beep? I can't find it in the code :/

Code:

print '\a'

[bitpop]

How are you making the beep? I can't find it in the code :/

Code:

print '\a'

crazy

[bitpop]

yup

$ python --version
Python 2.6.7

[flatfly]

Updated to version 0.384 (see code in OP).

Changes:
- some transactions were occasionally missing - fixed
- cleaner timestamp ordering
- lower network usage
- terminal color output for OS X, Linux and Windows  (this feature is for donators only)



Note: Do not tweak the various timer and buffer values unless you like playing with fire.
You might get your IP address banned from blockchain.info.

[bitpop]

How'd you know I changed the sleeps

[flatfly]

How'd you know I changed the sleeps

Just a hunch...

[bitpop]

Wow

[flatfly]

Next thing to be added: the fee paid for each transaction is worked out based on the total inputs and outputs and displayed in the right-most column. It gives you a quick insight into what people are paying to get their coins transferred. The color version will flag outliers in red.

Preview:

[bitpop]

The best

[flatfly]

The best

Thanks, glad you like it.
I guess I need to take some rest now...

[bitpop]

Ok good night

[flatfly]

Added fee column.
Cleaned up output style.

[bitpop]

Cool

Still no ssl tho

[flatfly]

Cool

Still no ssl tho

After some more testing, I have now enabled SSL in version 0.387 (I just added the extra 's' actually.)
It works just fine as long as you have a version of Python compiled against OpenSSL - which is usually the case.

[flatfly]

Version 0.4 is now out.

It is almost a complete rewrite, due to the discovery of a serious bug in the blockchain.info JSON API.
The best alternative was to switch to the websocket API, which offers better performance and reliability for this type of usage.

The code is simpler and more efficient, but the websocket library is now a requirement. (sudo pip install websocket)

To make this less of a pain to Windows users, I plan to provide signed precompiled binaries soon.

[bitpop]

Cant wait for windows but then again, security risk

[flatfly]

Here it is...
And for a limited amount of time, it is the color version that is freely available.

Of course standard security precautions apply, as always with Windows software.
(More detailed warning to be added)

Link: Download (2MB)

SHA1:1bae78311549deea03e8f1176d3315d4b22239ff

GPG signature:

Code:

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (MingW32)

iEYEABECAAYFAlK9SvMACgkQ0dQqIfkZdf6ZegCgsmR5e76DxWMvYvTnNtR6TcLi
fcgAnAs0h4R6JjZbxMYi2tTQyXjSLbOl
=M0v5
-----END PGP SIGNATURE-----

[opentoe]

Just sent you a few bucks as a donation. All I can afford these days.

Cool little utility. And when I see amounts like 300+ being traded makes me cringe! Big time money going through.

[flatfly]

Thanks for the donation!
OK, I have decided to rename the tool to "chainsnort". My geeky friends like the new name better!

[flatfly]

OK, version 0.410 is out, which also detects "new block" events, and has a bugfix in the windows build.

Also, you will be treated to a colorful ASCII easter egg (in the windows build only, for now), if chainsnort sees a tip being sent to this special tip jar: 1111127SpvabYpoeDoiz5L7QPkfiSh2Q  
The easter egg changes depending on the size of the transaction

I also have a separate edition with some features that are useful for forensic and technical investigation purposes. I'm not releasing it at the moment but PM me if interested.

Download Link

SHA1: 082689acee8baef4eb45c67609e096f48d2350b8

GPG signature:

Code:

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (MingW32)

iEYEABECAAYFAlLBRvkACgkQ0dQqIfkZdf45FACfRQJjxxYo55FkPE4g6y8su9z8
GrcAn0Qyddc8V4h881wK06lLNIoYWF0v
=31vC
-----END PGP SIGNATURE-----

[Sukrim]

Please move away from trusting blockchain.info and run this against bitcoind RPC... the added benefit would be that it could be easier to do this for altcoins too.

[Gonzo418]

Very cool stuff. Also thanks for the code !

[flatfly]

Please move away from trusting blockchain.info and run this against bitcoind RPC... the added benefit would be that it could be easier to do this for altcoins too.

Support for bitcoind RPC is already on my to-do list.

[flatfly]

Very cool stuff. Also thanks for the code !

Thanks, glad you like it.

[JDBound]

I'm getting:

File "chainsnort.py", line 36
    print '{:12.8f}'.format(feepaid) +
                                                ^
SyntaxError: invalid syntax

Help?

[bitpop]

I'm getting:

File "chainsnort.py", line 36
    print '{:12.8f}'.format(feepaid) +
                                                ^
SyntaxError: invalid syntax

Help?

Might be python 2.6/7

[flatfly]

I'm getting:

File "chainsnort.py", line 36
    print '{:12.8f}'.format(feepaid) +
                                                ^
SyntaxError: invalid syntax

Help?

My bad, copy paste issues! (It was a non-final version that was mistakenly posted)
I've fixed the code in the OP, can you try again and let me know if that helps?

If it doesn't, please post your OS and Python version.

[arnuschky]

Which version of websocket do you use? Somehow my install says

Code:

AttributeError: 'module' object has no attribute 'create_connection'

Installed using pip

[flatfly]

Great work, great functionality from a small amount of code. Thank you!

(I'd also like to see Bitcoin RPC usage, it's a little more complex on the user's end but it's more malleable! https://en.bitcoin.it/wiki/API_reference_(JSON-RPC))

Thanks, always love this kind of feedback!
This started out as a quick hack to teach myself the joys of JSON, so I'm really glad to see that people like the result.

As for bitcoin RPC support, I'd really want to add that as soon as possible, but I will have less
free time as of tomorrow, so it might take a while, so if someone wants to have a go at it...
Which reminds me, I should probably set up a proper github repo.

[flatfly]

Which version of websocket do you use? Somehow my install says

Code:

AttributeError: 'module' object has no attribute 'create_connection'

Installed using pip

This should be the correct library: https://github.com/liris/websocket-client
"Pip install websocket" should grab it, IIRC...
Which version of Python do you have?
2.7 would work best.

[Sukrim]

"Pip install websocket" should grab it, IIRC...

https://pypi.python.org/pypi/websocket-client/

"pip install websocket-client" <-- the client part might be actually important! (otherwise you'd get https://pypi.python.org/pypi/websocket/ )

[flatfly]

"Pip install websocket" should grab it, IIRC...

https://pypi.python.org/pypi/websocket-client/

"pip install websocket-client" <-- the client part might be actually important! (otherwise you'd get https://pypi.python.org/pypi/websocket/ )

Thanks for pointing this out, https://pypi.python.org/pypi/websocket-client/ is indeed the correct one.

[flatfly]

Version 0.415 is now available.

Changes:
- millisecond display for timestamps
- switched to UTC format for timestamps (will make that an config option soon)
- fixed transaction density calculation on Linux / OS X

[zeryl]

What would it take, to get you to either do a gist, or put this on github proper, to make it easy to stay up to date?

[flatfly]

What would it take, to get you to either do a gist, or put this on github proper, to make it easy to stay up to date?

I might put up a gist soon, but it could take a little while, as I have to deal with a serious issue right now, which is taking up a lot of mental energy - I prefer not to go into detail. In the meantime, please watch or subscribe to this thread for updates.  

Thanks!

[Praeconium]

Hi I am trying to use Your tool, and I also have issue with

Code:

ubuntu@ip-172-31-47-182:~/bot$ sudo python first.py
chainsnort 0.415:   51406 new transactions in the last 24 hours  -  0.59 tx/sec

AttributeError: 'module' object has no attribute 'create_connection'
ubuntu@ip-172-31-47-182:~/bot$ 

I installed websocket-client

Code:

ubuntu@ip-172-31-47-182:~/bot$ sudo easy_install websocket-client
Searching for websocket-client
Reading http://pypi.python.org/simple/websocket-client/
Best match: websocket-client 0.12.0
Downloading https://pypi.python.org/packages/source/w/websocket-client/websocket-client-0.12.0.tar.gz#md5=15bf7acb9caec87144d39ffcff765966
Processing websocket-client-0.12.0.tar.gz
Running websocket-client-0.12.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-D9v0Tt/websocket-client-0.12.0/egg-dist-tmp-QruTjI
zip_safe flag not set; analyzing archive contents...
Adding websocket-client 0.12.0 to easy-install.pth file
Installing wsdump.py script to /usr/local/bin

Installed /usr/local/lib/python2.7/dist-packages/websocket_client-0.12.0-py2.7.egg
Processing dependencies for websocket-client
Finished processing dependencies for websocket-client

it doesnt seems to work properly

Code:

ubuntu@ip-172-31-47-182:~/bot$ python -c 'import websocket; print websocket.__file__'
/usr/local/lib/python2.7/dist-packages/websocket-0.2.1-py2.7.egg/websocket/__init__.pyc

ubuntu@ip-172-31-47-182:~/bot$ python -c 'import websocket; print websocket.create_connection'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: 'module' object has no attribute 'create_connection'

Anyone has some kind of solution?

[flatfly]

Hi I am trying to use Your tool, and I also have issue with

Code:

ubuntu@ip-172-31-47-182:~/bot$ sudo python first.py
chainsnort 0.415:   51406 new transactions in the last 24 hours  -  0.59 tx/sec

AttributeError: 'module' object has no attribute 'create_connection'
ubuntu@ip-172-31-47-182:~/bot$ 

I installed websocket-client

Code:

ubuntu@ip-172-31-47-182:~/bot$ sudo easy_install websocket-client
Searching for websocket-client
Reading http://pypi.python.org/simple/websocket-client/
Best match: websocket-client 0.12.0
Downloading https://pypi.python.org/packages/source/w/websocket-client/websocket-client-0.12.0.tar.gz#md5=15bf7acb9caec87144d39ffcff765966
Processing websocket-client-0.12.0.tar.gz
Running websocket-client-0.12.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-D9v0Tt/websocket-client-0.12.0/egg-dist-tmp-QruTjI
zip_safe flag not set; analyzing archive contents...
Adding websocket-client 0.12.0 to easy-install.pth file
Installing wsdump.py script to /usr/local/bin

Installed /usr/local/lib/python2.7/dist-packages/websocket_client-0.12.0-py2.7.egg
Processing dependencies for websocket-client
Finished processing dependencies for websocket-client

it doesnt seems to work properly

Code:

ubuntu@ip-172-31-47-182:~/bot$ python -c 'import websocket; print websocket.__file__'
/usr/local/lib/python2.7/dist-packages/websocket-0.2.1-py2.7.egg/websocket/__init__.pyc

ubuntu@ip-172-31-47-182:~/bot$ python -c 'import websocket; print websocket.create_connection'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: 'module' object has no attribute 'create_connection'

Anyone has some kind of solution?

Try using wget instead of pip for a quick and dirty way to fix the issue:

Code:

wget https://raw.github.com/liris/websocket-client/master/websocket.py

[Praeconium]

It worked, thank You so much!

[arnuschky]

Which version of websocket do you use? Somehow my install says

Code:

AttributeError: 'module' object has no attribute 'create_connection'

Installed using pip

This should be the correct library: https://github.com/liris/websocket-client
"Pip install websocket" should grab it, IIRC...
Which version of Python do you have?
2.7 would work best.

2.7. I had to grab the websocket-client package, didn't work without it. Maybe you should update your docs...

May I suggest sticking the script on github? Using copy and paste is really annoying, especially with python due to the indentation problems.

Thanks for the help!

[flatfly]

Which version of websocket do you use? Somehow my install says

Code:

AttributeError: 'module' object has no attribute 'create_connection'

Installed using pip

This should be the correct library: https://github.com/liris/websocket-client
"Pip install websocket" should grab it, IIRC...
Which version of Python do you have?
2.7 would work best.

2.7. I had to grab the websocket-client package, didn't work without it. Maybe you should update your docs...

May I suggest sticking the script on github? Using copy and paste is really annoying, especially with python due to the indentation problems.

Thanks for the help!

I've just made a quick gist with the latest version.
When I have more time, I'll put up a proper little web page for it.

[flatfly]

Version 0.42.2 is ready... (I just changed the version numbers so they make more sense, at least to me...)

Changes:
- Switched to fully encrypted websockets (WSS protocol)
- Minor code cleanup

Source code only for now - Windows build will follow soon.

[flatfly]

Windows build for 0.42.2 is ready, and as a way to thank early users, it is the color version that is currently available for free. (See link and signatures in OP)

[bitpop]

Colors!

Edit: Not getting colors. Also, I don't see any code for the easter egg.

[RoxxR]

Colors!

Edit: Not getting colors. Also, I don't see any code for the easter egg.

I think only the executable windows version has colors and the easter egg

[RoxxR]

I'm happily chainsnorting on OS X:  

[flatfly]

I'm happily chainsnorting on OS X:  

Nice!

Actually installing chainsnort on OS X is really easy. Just copy and paste into a terminal:

1/ curl -o websocket.py https://raw.github.com/liris/websocket-client/master/websocket.py

2/ curl -o chainsnort.py https://gist.github.com/flatfly/8253870/raw/9733e34e469e1bcd8d0ae13565ade4eaa0490618/gistfile1.py

3/ python chainsnort.py

[flatfly]

I have just updated the OP with some legal stuff, to cover some use cases - However it's still a draft while I work out a proper license. Feel free to ask if there are any questions.  (I'll be offline until tomorrow, though)

[flatfly]

Now performing some (very early) testing of the bitcoind RPC connector...

[robanswe]

Now performing some (very early) testing of the bitcoind RPC connector...

Good luck! I'm really looking forward to the RPC version:)

[RoxxR]

Hey, how about adding support for twitter API - ie, sending automatic tweets when certain conditions are met? That would be dope...

[flatfly]

Version 0.43.3 is out!

Fixes a minor indentation issue with large amounts, and adds output multiplexing (tee-style) support, which allows you to capture traffic both to the console and a log file (just start the tool with the -o argument for that). The log file is called "txtrace.log" and is located in the same path as the script, or in the following path if you are running the Windows executable: %AppData%\Chainsnort\0.43.3\Log.

No documentation or manual at this stage. It will come sooner or later.

Also, I didn't have much time to make progress on direct bitcoind support.
Of course, if anybody wants to help out, they are welcome to do so

[bitpop]

Man this is the third coincidence today. Creepy. I ran into tee for the first time earlier and now you mention it.

[flatfly]

Man this is the third coincidence today. Creepy. I ran into tee for the first time earlier and now you mention it.

The lack of any coincidences would be the greatest coincidence.

[flatfly]

Hey, how about adding support for twitter API - ie, sending automatic tweets when certain conditions are met? That would be dope...

Yes, this is actually on my to-do list! I am looking at the following notification types:
email, syslog, twitter and pastebin.

[flatfly]

Next feature (currently under testing): color highlighting of "interesting" addresses, based on a very simple external text file containing a list of addresses to watch for. For instance, in the below screenshot, the 1dice and 1A6TJ addresses are defined as 'interesting'.



When such an address is detected, an alert containing the relevant transaction details could also be emailed.

[RoxxR]

I tinkered a little bit with this and now I've got a dynamic chainsnort desktop wallpaper 



I'm thinking of making a little blog post on how to do this, if someone is interested.

[robanswe]

I tinkered a little bit with this and now I've got a dynamic chainsnort desktop wallpaper 

https://i.imgur.com/RieKGhl.png

I'm thinking of making a little blog post on how to do this, if someone is interested.

I'm interested it looks great!

[flatfly]

OK, I will remove the Windows binary download of chainsnort (complete with color output) very shortly, so get it while you can. Any early users who already own a copy of the chainsnort binary are allowed to freely use it forever. 

Now I don't want to turn it into a typical paid app, but I would like to try a new idea: as i do need thé donations and almost none were received so far, any time a tip comes in through the official donation address <1111127SpvabYpoeDoiz5L7QPkfiSh2Q>, the download is made freely available to anyone (not just the donator) for the next 24h.

[Sukrim]

... seriously?! This is hardly more than a few dozen lines of code.

Anyways, please open a proper development environment on github, so we can also improve this. On the other hand it is just a websocket listener that reformats stuff a little bit so far...

[cshpoet]

IS RPC version out yet?  Really looking forward to this! 

[BTCisthefuture]

OK, I will remove the Windows binary download of chainsnort (complete with color output) very shortly, so get it while you can. Any early users who already own a copy of the chainsnort binary are allowed to freely use it forever. 

Now I don't want to turn it into a typical paid app, but I would like to try a new idea: as i do need thé donations and almost none were received so far, any time a tip comes in through the official donation address <1111127SpvabYpoeDoiz5L7QPkfiSh2Q>, the download is made freely available to anyone (not just the donator) for the next 24h.

How much does the donation size have to be, I was wanting this app but realized the dl link is gone now.

[flatfly]

OK, I will remove the Windows binary download of chainsnort (complete with color output) very shortly, so get it while you can. Any early users who already own a copy of the chainsnort binary are allowed to freely use it forever. 

Now I don't want to turn it into a typical paid app, but I would like to try a new idea: as i do need thé donations and almost none were received so far, any time a tip comes in through the official donation address <1111127SpvabYpoeDoiz5L7QPkfiSh2Q>, the download is made freely available to anyone (not just the donator) for the next 24h.

How much does the donation size have to be, I was wanting this app but realized the dl link is gone now.

It can be anything you like! Donations help me further develop and maintain the tool. Once the donation is made please post here again in this thread to confirm. Thanks!

[flatfly]

IS RPC version out yet?  Really looking forward to this!  

No, I really could use some help with this. It's a little more complex (and time-consuming) than expected to implement it properly, and I don't even have the resources to run a full node to experiment with, unfortunately.  

[robanswe]

No, I really could use some help with this. It's a little more complex (and time-consuming) than expected to implement it properly, and I don't even have the resources to run a full node to experiment with, unfortunately.  

Run the full node in testnet?

[bitpop]

Blockchain has a public rpc

[flatfly]

Blockchain has a public rpc

Unfortunately, some key RPC methods required by chainsnort are disabled on their API.

[flatfly]

No, I really could use some help with this. It's a little more complex (and time-consuming) than expected to implement it properly, and I don't even have the resources to run a full node to experiment with, unfortunately.  

Run the full node in testnet?

I'm investigating this...

[flatfly]

I'd like to thank robanswe, who kindly gave me access to a full node for developing the RPC connector.

[flatfly]

The next version will also flag microtransactions (sub $1) in red. Due to the fee structure, only 1% of transactions are microtransactions currently.

I've also adopted the transparent background idea, with RoxxR's help

[robanswe]

I'd like to thank robanswe, who kindly gave me access to a full node for developing the RPC connector.

No problems at all just make sure you always test connect before you run your test because the RPC server will not be online 24/7. Also don't hesitate to pm me if you get any problems. Chainsnort with RPC connector will be awesome for sure and I can barley wait:) Anyway keep up the good work and take the time you need.

[vm_mpn]

I just can't stop admiring developers on this forum - what you guys are doing is amazing!

Ok, moving on to harass other coders... They deserved it 

Vadim

[flatfly]

Here's a very early preview of the 100% RPC version... Only transaction hashes are displayed in this proof of concept, but the live capture method is already working out nicely:

[robanswe]

Here's a very early preview of the 100% RPC version... Only transaction hashes are displayed in this proof of concept, but the live capture method is already working out nicely:

It's nice to see that you are making progress:)

[flatfly]

Here's a very early preview of the 100% RPC version... Only transaction hashes are displayed in this proof of concept, but the live capture method is already working out nicely:

It's nice to see that you are making progress:)

Indeed it's been progressing nicely:



A few kinks remain, though, and will take longer to iron out.

[robanswe]

Indeed it's been progressing nicely:

https://i.imgur.com/3sImjAC.png

A few kinks remain, though, and will take longer to iron out.

Wow looks a lot better then I had hoped for! Anyway I had an idea now when the program can connect to the bitcoin-qt client wouldn't it be cool to have it write out network alert messages with bright red text or something? https://en.bitcoin.it/wiki/Alerts Maybe even add some alert piping sound.


(getinfo will show the alert info in the error field when a alert is active)

[flatfly]

Indeed it's been progressing nicely:



A few kinks remain, though, and will take longer to iron out.

Wow looks a lot better then I had hoped for! Anyway I had an idea now when the program can connect to the bitcoin-qt client wouldn't it be cool to have it write out network alert messages with bright red text or something? https://en.bitcoin.it/wiki/Alerts Maybe even add some alert piping sound.


(getinfo will show the alert info in the error field when a alert is active)

Sounds like a good idea - I'll look into implementing this and put it on my to-do list!

[flatfly]

OK, here's a quick development status update on the RPC version:

Open issues:

- some transactions fail to be captured. this is being investigated.
- real-time fee calculation appears to be unfeasible/too slow
  given the current bitcoind API and exposed data structures.
- new block notification is not implemented yet.

[robanswe]

OK, here's a quick development status update on the RPC version:

Open issues:

- some transactions fail to be captured. this is being investigated.
- real-time fee calculation appears to be unfeasible/too slow
  given the current bitcoind API and exposed data structures.
- new block notification is not implemented yet.

It's great to see some progress being made:)

-Can this just be to my not so well connected node? I mean blockchain.info got around 1000 connections with nodes all around the world I got 8 connections with one node..
-Can't this be due to latency I bet you got around 80+ ms to my rpc server instead of 1< ms on localhost.

[flatfly]

OK, here's a quick development status update on the RPC version:

Open issues:

- some transactions fail to be captured. this is being investigated.
- real-time fee calculation appears to be unfeasible/too slow
  given the current bitcoind API and exposed data structures.
- new block notification is not implemented yet.

It's great to see some progress being made:)

-Can this just be to my not so well connected node? I mean blockchain.info got around 1000 connections with nodes all around the world I got 8 connections with one node..
-Can't this be due to latency I bet you got around 80+ ms to my rpc server instead of 1< ms on localhost.

- Yes this is the most likely explanation in my opinion too. Based on my estimates, I believe that with 15 connections, the RPC version should be able to see 99-100% of all transactions in real time. This still needs to be verified in practice (and also depends on how well connected those 15 nodes are.)

- If someone wants to take a stab at implementing fee calculation, all the better! I don't have enough time for that currently.

[robanswe]

I changed to 16 connections. Anyway if you could upload the latest source code to github or something someone might be willing to help. I think my coding skills is a little to bad but I will take a look for sure.

[flatfly]

I changed to 16 connections. Anyway if you could upload the latest source code to github or something someone might be willing to help. I think my coding skills is a little to bad but I will take a look for sure.

Thanks. Seems to be back to 8 connections now though. Did it consume too much bandwidth?
The blockchain.info websocket stream was down for much of the day so I couldn't compare results yet.

Later this week, I will clean up the code and post it so other devs can help.

[robanswe]

Thanks. Seems to be back to 8 connections now though. Did it consume too much bandwidth?
The blockchain.info websocket stream was down for much of the day so I couldn't compare results yet.

Later this week, I will clean up the code and post it so other devs can help.

Yes I was forced to cut back to 8. I'm currently on a adsl line with less then 1mbit up stream. So when I play online games with voip I really need all the bandwidth I got. I might be able to start a server on 2,5mbit upload rate at a different location. But until then I will keep changing from 8 to 15 and this week I will need the Internet nearly all the time so it will mostly be limited to 8
Also I know about the blockchain.info thing the api gets blocked by cloude fair. Thats one of the reasons I like rpc support. I also know there are other live transactions monitors like the one Bit Pay just released. But I like the simplicity of this small python script:) It also looks nice in a console window.

[flatfly]

I've posted the current RPC skeleton code there:
 https://gist.github.com/flatfly/9151814

I didn't have time to further work on this in the last few days, unfortunately.
If people are interested in Windows builds, do let me know.
(And donations do help get things done! )

[robanswe]

I've posted the current RPC skeleton code there:
 https://gist.github.com/flatfly/9151814

I didn't have time to further work on this in the last few days, unfortunately.
If people are interested in Windows builds, do let me know.
(And donations do help get things done! )

Got it working after some trial and error. Anyway it seems to work fine and I will see if I can improve it. A Windows build would be nice but a colored version for Linux would be even better.

[flatfly]

Demonstrating the highlight-arbitrary-address feature:



At this time, this is only available to donators and private beta testers.

Note: in case you're wondering, the "1Dorian" address is part of a fundraising campaign started today by Andreas M. Antonopoulos for Dorian Nakamoto.  Read more about it here:

 http://www.reddit.com/r/Bitcoin/comments/1ztjmg/andreas_im_fundraising_for_dorian_nakamoto/

[RoxxR]

I'm getting the below error message when starting up chainsnort. Any help?

Code:

ValueError: invalid literal for int() with base 10: '<!DOCTYPE html> <!--[if lt
IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]--> <!--[if IE 7]>
  <html class="no-js ie7 oldie" lang="en-US"> <![endif]--> <!--[if IE 8]>    <ht
ml class="no-'

[bitpop]

I'm getting the below error message when starting up chainsnort. Any help?

Code:

ValueError: invalid literal for int() with base 10: '<!DOCTYPE html> <!--[if lt
IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]--> <!--[if IE 7]>
  <html class="no-js ie7 oldie" lang="en-US"> <![endif]--> <!--[if IE 8]>    <ht
ml class="no-'

Looks like blockchain was down

[bitpop]

moment I still need time to fully understand this because every read and understand that I am here several times makes me confused whether it was easy to make a profit?

 

This is not for profiting, just fun

[T-rage_11]

please reup the windows version !!!!!!!!!!!!!  

Edit:
>>> I made something similar .. check this: http://btcaudio.tk/live  

[RoxxR]

please reup the windows version !!!!!!!!!!!!! 

Zero donations or contributions in 5 months, people take and don't give anything back... No wonder this project is dying...

[RoxxR]

Yeeha!

[Dabs]

Interesting. I don't know how I missed this. But I guess all the data is coming out from bitcoin-qt / bitcoind, there is no need to use the blocknotify or walletnotify if you have this tool.

[instacalm]

Love this little app, soothing to observe all the transactions flowing

[Mitchell]

I just found out about this program and it works great! Send you a (very) small tip. Keep up the good work

[flatfly]

I just found out about this program and it works great! Send you a (very) small tip. Keep up the good work

Thanks for the tip! As a sign of gratitude, I will PM you a special version of chainsnort with a couple of additional features (within a few days).

[Mitchell]

Thanks for the tip! As a sign of gratitude, I will PM you a special version of chainsnort with a couple of additional features (within a few days).

No problem man. You deserved it. I will patiently await your PM.

[flatfly]

In the meantime, here's a teaser of the extended version, which will include a verbose mode showing all inputs and outputs for each transaction, and simple pattern-matching capabilities.

[flatfly]

I have also implemented a new advanced feature called transaction fingerprinting, which attempts to classify transactions in one (or more) categories based on certain behavioral patterns. Of course this can never be 100% perfect, but I'm aiming for a decent level of accuracy. It is still experimental at this time.


Example:

[Mitchell]

Those are some awesome features you are working on! I'm glad that I send you a donation

[flatfly]

OK, the "transaction-type guessing engine" now has 14 different tags recognized.
Feel free to PM me if interested in trying this new private beta (Windows only for now).

[flatfly]

The download link for the latest published windows build (0.43.3c - January 12, 2014) is available again.

I've also updated the link in the OP.

[Meuh6879]

please reup the windows version !!!!!!!!!!!!! 

Zero donations or contributions in 5 months, people take and don't give anything back... No wonder this project is dying...

I don't see a bitcoin adress for donation ... 

[Propulsion]

The websocket.py download is a 404.

Workaround was to grab it from here: https://github.com/liris/websocket-client

[UberNifty]

It never ceases to amaze me the awesome innovations such as this that Bitcoin has inspired... will definitely be keeping an eye on this thread.

[Dabs]

I'd like to try to private beta.

[flatfly]

Hi,

- sorry for the delayed responses, I've been literally overwhelmed by the amount of stuff and work lately. Trying to redefine my priorities but it's easier said than done.

- will send the latest private beta build to Dabs as soon as it's ready, later this week most likely

- about donations: they can be sent to either 1SEPsD35L3C6nKUghwNvRHeyGSyTUSRYS or 1111127SpvabYpoeDoiz5L7QPkfiSh2Q and are always warmly accepted

- a new transaction type (XCP_OP) is now detected by the fingerprinting module, bringing the number of different types recognized to 15.  Note: Don't expect to see that new type very easily, there are only a couple of XCP transactions done per hour.

And thanks for all the feedback!

[RoxxR]

Is it possible to mark multisig transactions?

[flatfly]

Chainsnort now recognizes P2SH (multisig) transactions and tags them in real-time!

Screenshot upcoming...

[bitpop]

Can this be added to Bitcoin-ncurses?

[flatfly]

Can this be added to Bitcoin-ncurses?

Haven't heard of Bitcoin-ncurses - will look it up!

[bitpop]

Can this be added to Bitcoin-ncurses?

Haven't heard of Bitcoin-ncurses - will look it up!

It's awesome to check on my node, but it's missing live transactions

[flatfly]

A little screenshot of the latest windows build, highlighting a large multisig transaction:





Also, starting today I will be experimenting a new funding model and see if it's sustainable:
  

• on odd-numbered weeks, the latest stable Windows build for chainsnort can be obtained at no cost - just PM me for the download link
   
• on even-numbered weeks, the download will cost 0.255 BTC - send the transaction and PM me for the download link
  
  

Hopefully the revenue will help cover part of my development time.

[bitpop]

I have a business idea for you. Traders would pay to see large transactions entering and leaving exchanges.

[Sukrim]

Exchanges don't publish lists of addresses they hold, this would make them easily auditable.

[Balloon]

Very useful code!

One question though. I wrote my own version which basically just takes in any unconfirmed transaction and write it to a file. And I also tried the json API provided by blockchain.info.
***
https://blockchain.info/unconfirmed-transactions?format=json
***
What surprises me is that websocket missed several transactions. Is there any way to make sure that the websocket API will return ALL unconfirmed transactions?

Balloon

[bitpop]

Very useful code!

One question though. I wrote my own version which basically just takes in any unconfirmed transaction and write it to a file. And I also tried the json API provided by blockchain.info.
***
https://blockchain.info/unconfirmed-transactions?format=json
***
What surprises me is that websocket missed several transactions. Is there any way to make sure that the websocket API will return ALL unconfirmed transactions?

Balloon

I doubt it, it's not meant to be a reliable api, you'll have to roll your own node possibly with that bitpay software

[BootstrapCoinDev]

Exchanges don't publish lists of addresses they hold, this would make them easily auditable.

You can verify whether the address belong to exchange or no by dint of big transactions with huge number of inputs. No software required.

[lissacoffey]

Blockchain.info uses a special system that doesn't involve Python (as far as we know), they're most likely using a modified node that notifies them of transactions.

The easy way

Just use Blockchain.info's transaction notification service. Their web sockets api lets anyone subscribe to any bitcoin address
The hard way

If you want to listen to all latest transactions, and not to addresses you own, you're going to need more in depth control. Otherwise use Bitcoind and its wallet notify parameter. Check out PyNode, a Bitcoin node implementation in Python.

Thanks
Lissa
http://driverrestore.com/

[bitpop]

Can someone help me with a little tut for linux, im new with linux

Just copy and paste into a terminal:

1/ curl -o websocket.py https://raw.github.com/liris/websocket-client/master/websocket.py

2/ curl -o chainsnort.py https://gist.github.com/flatfly/8253870/raw/9733e34e469e1bcd8d0ae13565ade4eaa0490618/gistfile1.py

3/ python chainsnort.py

[Newar]

it said: bash: 1/: No such file or directory

Code:

curl -o websocket.py https://raw.github.com/liris/websocket-client/master/websocket.py

curl -o chainsnort.py https://gist.github.com/flatfly/8253870/raw/9733e34e469e1bcd8d0ae13565ade4eaa0490618/gistfile1.py

python chainsnort.py

[Newar]

Sorry, that websocket link is broken. curl -o didn't work either anymore for me. Try:

Code:

git clone https://github.com/liris/websocket-client.git

cd websocket-client

python setup.py install

wget https://gist.github.com/flatfly/8253870/raw/9733e34e469e1bcd8d0ae13565ade4eaa0490618/gistfile1.py

python gistfile1.py

[lutinus]

Sorry, that websocket link is broken. curl -o didn't work either anymore for me. Try:

Code:

git clone https://github.com/liris/websocket-client.git

cd websocket-client

python setup.py install

wget https://gist.github.com/flatfly/8253870/raw/9733e34e469e1bcd8d0ae13565ade4eaa0490618/gistfile1.py

python gistfile1.py

I get this :

 # python setup.py install
Traceback (most recent call last):
  File "setup.py", line 1, in <module>
    from setuptools import setup
ImportError: No module named setuptools

OS : Debian GNU/Linux 7

Any idea please ?

[bitpop]

Adding to ncurses would be awesome

https://github.com/azeteki/bitcoind-ncurses/issues/15

[bitpop]

So it's gone?

[gregwtmtno]

Does anyone have a copy of the code? I'm trying to build something similar.

[Newar]

Does anyone have a copy of the code? I'm trying to build something similar.

I thought I did, but turns out I didn't   . All I can find for gistfile1.py (posted Jul 6, 2015): https://www.snip2code.com/Snippet/576729/chainsnort-0-484--latest_stable-

I gave it a quick go and after creating an empty file called prefixes.txt, it seems to work ok.

flatfly was last active on August 01, 2015, so until he comes back I put it on github as a backup. You can get it using:

Code:

wget https://raw.githubusercontent.com/Newar417/flatfly_archived_chainsnort/master/gistfile1.py

Code from snip2code:

Code:

import json, os, sys, datetime, time, urllib, websocket, operator, random, re
from colorama import Fore, Style, init   ; init()
  
t0 = time.time() ; Z = 0 ; nboutputs = 0 ; nbinputs = 0
VERBOSE = 1

print 'chainsnort v0.484: ' , 
os.system('title chainsnort_inspect [mainnet]')
matches = open('prefixes.txt').read().split()
matches2 = ['111111','XXXXXX']  

mini_easter_egg = ['** THANKS! **    (' +chr(3)+ ' _ ' +chr(3)+ ')        ','** PEACE! **     (^ _ ^)        ']


class Plex(object):
    def __init__(self, name):
        self.file = open(name, 'w')
        self.stdout = sys.stdout
        sys.stdout = self
    def __del__(self):
        sys.stdout = self.stdout
        self.file.close()
    def write(self, data):
     
        self.stdout.write(data)
        data=data.replace(chr(8),'')
        data=data.replace(chr(0x1B),'')
        data=re.sub('\[.*m','',data)
        # data=re.sub('\t ','X', data)

        self.file.write(data)


def main():
  global Z, btcseen
  btcseen = 0
  
  while (1):
    bigtag = '  '
    result = ws.recv()
    time.sleep(0.05)
    result = json.loads(result)
    if result['op'] == 'block' :
      print
      reward = result['x']['reward']/1e8	  	
      print datetime.datetime.utcnow().strftime("%H:%M:%S.%f")[:-3] , bigtag + ' {:11.4f}'.format(reward) ,'B  [NEW BLOCK SOLVED] ' ,
      print Fore.CYAN+Style.BRIGHT + str(result['x']['height']) + Fore.RESET+Style.NORMAL
      print 
      continue 
	  	  
    inputs = sum(p['prev_out']['value'] for p in result['x']['inputs']  )
    outputs = sum(p['value'] for p in result['x']['out']  )
    feepaid = (inputs-outputs)/1e8 
    val = sum(p['value'] for p in result['x']['out'])/1e8
    btcseen = btcseen + val
	
    if val >= 50:
     if val >= 5000:
      print '\a' ,
     bigtag = Fore.GREEN+Style.BRIGHT + '>>' #  + chr(16)*2 
     time.sleep(0.3)
	 
    feetag = ''
    if feepaid < 0.0001 or feepaid >= 0.01:  
     feetag = Fore.YELLOW+Style.BRIGHT	
	
    if VERBOSE == 0:
        max_addr = sorted(result['x']['out'], key=operator.itemgetter('value'), reverse=True)[0]['addr']
        for ku in result['x']['out']:
         if any(s in ku['addr'] for s in matches):
           max_addr =  Fore.MAGENTA+Style.BRIGHT + max_addr + Fore.RESET+Style.NORMAL
             
        print '\b'+ datetime.datetime.utcnow().strftime("%H:%M:%S.%f")[:-3], bigtag + ' {:11.4f}'.format(val) ,'B '  + Fore.RESET+Style.NORMAL , max_addr , '\t' , feetag ,  '\b'*6 + '{:12.8f}'.format(feepaid) + Fore.RESET+Style.NORMAL
   
    if VERBOSE == 1:
       nboutputs = 0 ; nbinputs = 0 
	   
       for ki in result['x']['inputs']:
           print ' '.ljust(30) , 'i', ki['prev_out']['addr']
           nbinputs += 1
       for ku in result['x']['out']:
           print ' '.ljust(30) , 'o', ku['addr']
           nboutputs += 1

       txcat =  ''  
       if any(s['prev_out']['addr'] in ku['addr'] for s in result['x']['inputs']):    
           txcat = '[WEB_SPEND]'
       if nboutputs > 2: 
 		   txcat = '[AUTOMATED_PAYMENT]'
		   if (nboutputs > 20) & (nbinputs == 1):
		       txcat = '[FAUCET_PAYMENT]'
		       if val >= 0.2: 
		           txcat = '[MIXER]'
		       if val >= 20:
		           txcat = '[MININGPOOL_PAYMENT]'
       if (val >= 50) & (nbinputs == 1): 
 		   txcat = '[COLDSTORAGE_MOVE]'
       if (val <= 0.0011) & (nbinputs == 1): 
 		   txcat = '[MICROTRANSACTION]'		   
       if (nboutputs == 1) & (nbinputs == 1): 
 		   txcat = '[ADDRESS_SWEEP]'
       if (nboutputs == 1) & (nbinputs > 1): 
 		   txcat = '[ADDRESS_CONSOLIDATION]'      
       if txcat == '':  
           txcat = '[NATIVE_SPEND]'
       for ku in result['x']['out']:
           if any(s in ku['addr'] for s in matches) | ('1Lucky' in result['x']['inputs'][0]) :    # to fix
               txcat +=    ' [ONCHAIN_GAMING]'           # or in inputs as well
               break 
       for ku in result['x']['out']:
           if any(s in ku['addr'] for s in matches2):
               txcat = '[COIN_DESTRUCTION]'      # also add DONATION
               break
       for ku in result['x']['out']:
           if ku['value'] < 546:   
               txcat = '[ILLEGAL]'
               break               
       for ku in result['x']['out']:
           if ku['value'] == 7800:
               if (nboutputs == 3):		   
                  txcat = '[XCP_OP]'
               # print '\a'
               break   

       for ku in result['x']['out']:
           if ku['addr'][0] == '3':   
               txcat = Fore.CYAN+Style.BRIGHT + '[MULTISIG]'
               break 
               			  
       print 'Transaction:   ' + result['x']['hash'] 
			 
       print '\b'+ datetime.datetime.utcnow().strftime("%H:%M:%S.%f")[:-3], bigtag + ' {:11.4f}'.format(val) ,'B '  + Fore.RESET+Style.NORMAL , Fore.RESET+Style.BRIGHT + txcat.ljust(32) + Fore.RESET+Style.NORMAL , ' ', feetag , '{:12.8f}'.format(feepaid) + Fore.RESET+Style.NORMAL
 
    Z += 1 ; print
 
try:
  sys.tracebacklimit = 0
  domain='blockchain.info'
  stat = int(urllib.urlopen('https://'+domain+'/q/24hrtransactioncount').read().decode("utf8"))
  print Fore.CYAN+Style.BRIGHT + '\b' + str(stat)  + Fore.RESET+Style.NORMAL +  ' new transactions in the last 24 hours  -  ' + Fore.CYAN+Style.BRIGHT  + '%02.02f' % (stat/86400.0) + Fore.RESET+Style.NORMAL + ' tx/sec' 
  # print '\nEstablishing TLS tunnel...',  
  print '\nCapturing transactions on mainnet...  (Fingerprinting enabled)\r',  
  ws = websocket.create_connection("wss://ws."+domain+"/inv")
  ws.send('{"op":"unconfirmed_sub"}')  ;
  ws.send('{"op":"blocks_sub"}')
  # print 'Done. Waiting for new transactions...\r',
   
  if '-o' in sys.argv:
       Plex('../../../Log/txtrace.log')       
  
  time.sleep(0.3)
  main()
      
except KeyboardInterrupt:
  t1 = time.time()
  print
  print 'Closing threads... Done.'
  print 'Captured',Z, 'events - Transaction density: %02.02f' % (Z/(t1 - t0)) , 'tx/sec - Coins moved:' + ' {:5.4f}'.format(btcseen) 
  sys.exit()

[gregwtmtno]

Thanks a ton, Newar. God, this code is so much simpler than the Rust I'm doing my version in.

[azeteki]

Hello,

I've been playing around recently with adding this functionality to my bitcoind front-end.

Those of you who aren't running full nodes will not find this useful. However, if you do have a full node, you can do all of this locally without external API requests. It may even be faster (I have not tested chainsnort, so I do not know.)

https://github.com/esotericnonsense/bitcoind-ticker

At the moment this is just a proof of concept and does the absolute bare minimum required. No highlighting or anything fancy, just spews out transactions to the terminal. The idea is there though.

[frogCorporation]

Very cool now we have a look and evaluate mainly under Linux

[xelwenq]

This tool is fantastic for my node. A great idea.
Thank you for release

[Meuh6879]

Can someone build an update for this ? 

[RoxxR]

Can someone build an update for this ? 

Define update - do you mean adding new features, or updated binaries?