Tools to Navigate and Analyze the Blockchain

[charleshoskinson]

I've been using blockchain explorer for a bit now and I'm very impressed. However, I'd like to download and analyze a local copy of the blockchain and I'm wondering if anyone wrote some software to do this?

Edit:

And Yes I'm going to shamelessly promote my course:
https://www.udemy.com/bitcoin-or-how-i-learned-to-stop-worrying-and-love-crypto/

[1715032577]

1715032577

[1715032577]

1715032577

[1715032577]

1715032577

[1715032577]

1715032577

[1715032577]

1715032577

[1715032577]

1715032577

[etotheipi]

I've been using blockchain explorer for a bit now and I'm very impressed. However, I'd like to download and analyze a local copy of the blockchain and I'm wondering if anyone wrote some software to do this?

You could make one pretty easily out of armoryengine (obviously what Armory is based on).  It scans and indexes the blockchain and allows for arbitrary header and tx-lookup.  Within a python script, you can traverse the entire blockchain in any way you want to (though you'd want to use the lower-level C++ stuff if you want to do full blockchain scans in less than an 30 min).  There's a bunch of examples of it in the extras/sample_armory_code.py in the github project.  I know znort had done something similar.

At one point, I had created a block explorer with an ancient version of armoryengine, but there's no way that works anymore.  It would require more work upgrading it probably, than just making your own wrapper around armoryengine that serves your purposes. 

No idea if that's what you're looking for.  But it's out there.

[charleshoskinson]

I'm trying to get a sense of how much software exits to explore the blockchain. I'd like to include it in my course in the Blockchain section and also write some software to analyze the economy.

[etotheipi]

I might put up a bounty to create a block explorer out of armoryengine.  I know it's possible, because I've done it.  But I don't have time to do it, myself.

Again, in the sample_armory_code, there is actually a full script for parsing the blockchain since the inception of SatoshiDice, and collecting every bet that's ever been made, calculate bet statistics, profits ,etc.  Dooglus maintains that thread now (and script), but it demonstrates what is possible with armoryengine. 

Granted, I mentioned that full blockchain scans from the python side will be really slow.  So dooglus updated his script to maintain memory between calls.  It's just a caveat worth mentioning if you want to do full scanning, not just point-and-click exploring.  The C++ code can scan the entire 7 GB of blockchain in 2-4 minutes.

[charleshoskinson]

If I committed to spending the time to produce said software, would you help my market my course?

[etotheipi]

If I committed to spending the time to produce said software, would you help my market my course?

I would want to see the course first, before promoting it.  But there is definitely a shortage of informational material out there, so most likely I'd be down.  I could at least put a link to you on the website, and maybe in the help section. 

[charleshoskinson]

https://www.udemy.com/bitcoin-or-how-i-learned-to-stop-worrying-and-love-crypto/

I'd be happen to include a lecture on Armory if you'd live me to

[etotheipi]

https://www.udemy.com/bitcoin-or-how-i-learned-to-stop-worrying-and-love-crypto/

I'd be happen to include a lecture on Armory if you'd live me to

That sounds like a fair trade!    I promote you, you put up a lecture about "Cold Storage" and Armory. 

The blockexplorer would be cool, too, but it's probably no short amount of work if you are not familiar with Armory code and/or python and/or PyQt.  I'd like people to have the functionality but it's not high priority for me, and in the end I'm not sure that many people would use it...

[etotheipi]

I do have to admit:  talking about "the Bitcoin" kind of makes me cringe.  That's not the normal grammatical context we use.  Either "we are talking about Bitcoin" or "we are talking about the Bitcoin system/network/protocol".  Otherwise, I perused the content and it looks like good information and context.

I'm still amazed that people spend so much time talking about Cold Storage and never even mention Armory.  Armory has been doing this for over a year (an eon in BTC time), and is the most trusted and intuitive solution for Cold Storage.  I'm up to about 10,000 downloads per month!  Yet somehow, so few people actually know about!

I guess I need better marketing.

[charleshoskinson]

I do have to admit:  talking about "the Bitcoin" kind of makes me cringe.  That's not the normal grammatical context we use.  Either "we are talking about Bitcoin" or "we are talking about the Bitcoin system/network/protocol".  Otherwise, I perused the content and it looks like good information and context.

I'm still amazed that people spend so much time talking about Cold Storage and never even mention Armory.  Armory has been doing this for over a year (an eon in BTC time), and is the most trusted and intuitive solution for Cold Storage.  I'm up to about 10,000 downloads per month!  Yet somehow, so few people actually know about! Sad

I guess I need better marketing.

It's like saying the Dollar. It is a contextual phrase and avoids me having to say things like the Bitcoin Open Source Project or the Bitcoin Ecosystem. Also many journalists are starting to use this convention. I agree that we should have more control, but I figured for a first generation course subject to lots of community review and modification that it's sufficient for now.

That said, what would you like me to introduce about Armory to my users? It's a great product with a host of features thus I figured I'd ask what you consider to be the best and most appealing?

[etotheipi]

I do have to admit:  talking about "the Bitcoin" kind of makes me cringe.  That's not the normal grammatical context we use.  Either "we are talking about Bitcoin" or "we are talking about the Bitcoin system/network/protocol".  Otherwise, I perused the content and it looks like good information and context.

I'm still amazed that people spend so much time talking about Cold Storage and never even mention Armory.  Armory has been doing this for over a year (an eon in BTC time), and is the most trusted and intuitive solution for Cold Storage.  I'm up to about 10,000 downloads per month!  Yet somehow, so few people actually know about! Sad

I guess I need better marketing.

It's like saying the Dollar. It is a contextual phrase and avoids me having to say things like the Bitcoin Open Source Project or the Bitcoin Ecosystem. Also many journalists are starting to use this convention. I agree that we should have more control, but I figured for a first generation course subject to lots of community review and modification that it's sufficient for now.

That said, what would you like me to introduce about Armory to my users? It's a great product with a host of features thus I figured I'd ask what you consider to be the best and most appealing?

• You only need to backup your wallet once, ever
• You can backup your wallet to a single sheet of paper with only 4 lines, which is superior in may ways to digital backups (inexpensive, durable, hidable, visual integrity)
• Multiple wallets interface to segregate your funds into different categories or security tiers
• The absolute easiest offline wallets imagineable.  You can keep your coins offline, but it only takes 1-2 minutes extra with a USB key to move coins from it like a regular wallet.  Your online wallet produces the exact same addresses as the offline wallet, but the online wallet cannot be compromised (well, your privacy can be compromised, but not the security).  You can easily transition from an online to offline wallet, if you want.
• Create clickable payment requests to copy into emails or webpages
• A lot of really advanced features if you switch to "Expert" usermode

I'm sure there's more stuff, but those are the things that are probably the most appealing.  You can pare it down to what you find most impressive.  The cold storage thing is critical though:  I'm constantly running across "wishlists" on the forums, of people wishing that .... well Armory existed (listing all these features without realizing it already exists in Armory).

By the way, if you do a lecture about it.  Spend some time getting to know it.  Use the offline wallet feature.  And be sure to add the warning about resource usage.  It's still very heavyweight, and not suitable for all computers (the offline version will run on anything, though).  This will be resolved in the next month, but until then, it may not be usable by everyone.

Some links:

https://bitcoinarmory.com/using-offline-wallets-in-armory/
https://bitcoinarmory.com/armory-quick-start-guide/
https://bitcoinarmory.com/get-armory/

[charleshoskinson]

Ok, I tell you what. I'm doing another lecture on Bitcoin-Qt to address some questions I've been getting. After I finish that lecture, I'll make an Armory lecture and then send it to you. Could you watch it and give me some advice on what I should change or add? After you like the content, I'll upload it to my class

[bezzeb]

Hey guys, i know this evolved into a private thread sort of, and hate to intrude.  But it is a forum (lol) and I wanted to pitch in as I got here by searching and was delighted to learn i wasn't the only one who would like to have a local blockchain explorer application to hunt and analyze the QT blockchain database.

Reason:  a) i find it fascinating to explore the chain and peek into interesting addresses.  b)  Websites like blockchain.info - while extremely cool and useful - seem to go against the grain in terms of lost privacy.  I'm not a huge tin-foil hat wearer but i do think it's creepy that we go online and share the addresses we are curious about, thus making a strong IP link for those who watch IP numbers.  This when we have all the data in our QT client already but only lack a local interface to do what blockchain.info does.  God bless the blockchain.info guys - they don't seem to be using google analytics in their page source to help google link your queries to your gmail account, but at some point i can see strange happenings on this front somewhere out there with the end result being that pseudonymous is rendered moot or something cooky like that.  Imagine big data profiling your bitcoin activity.  yuck.

So bottom line - I'd pitch in some BTC for the time it would take to add this feature as I feel it would be good for the project in the long run to make clients as self sufficient and powerful as possible without the need to use online services for that which could be done locally.  I can't afford much, but it seems I'm not the only one right? 

I have a lot of thoughts on this as I've been thinking about it ever since the first day I saw blockchain.info long ago, but I'll stop here for now.

And to etotheipi:  I'm an armory user and I love it   We need more *simple* mainstream methods for the masses to get into because Armory would probably (for example) make my mother's head explode.  But as a "power wallet" it's the best IMHO.  God bless and I'll definitely toss a few coin slices into your collection hat.