Bitcoin Forum
December 11, 2016, 08:23:37 AM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: [1] 2 »  All
  Print  
Author Topic: [ANN] pynode: Simple bitcoin P2P node  (Read 5826 times)
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1470


View Profile
July 22, 2012, 05:23:00 PM
 #1

Github URL: https://github.com/jgarzik/pynode

pynode is a simple bitcoin P2P client, originally based on ArtForz' half-a-node, which maintains a blockchain database and TX memory pool.  All bitcoin-related code is separated into a "bitcoin" module, permitting you to use core features such as CBlock or P2P message serialization, without actually using the node itself.

Therefore, pynode has two goals:

1. As a generic "python-bitcoin" library, for any project.  Update: Now a separate project, https://github.com/jgarzik/python-bitcoinlib
2. As a P2P node that may be used as a base for other projects, such as a network monitoring node or P2P proxy.

At present, it is mainly for developers and not end users.  Here is what pynode does not do:
  • key management (wallet)

Here are the limited things that pynode can do at present:
  • Outgoing TCP connection to a single remote P2P node
  • Initial blockchain download
  • Maintain a transaction memory pool, with TX's removed as they are confirmed in blocks
  • Maintain an indexed blockchain database
  • Respond to JSON-RPC requests via an internal HTTP server

Over time it will become more capable, but this is an alpha quality release for developers at present.


Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
1481444617
Hero Member
*
Offline Offline

Posts: 1481444617

View Profile Personal Message (Offline)

Ignore
1481444617
Reply with quote  #2

1481444617
Report to moderator
1481444617
Hero Member
*
Offline Offline

Posts: 1481444617

View Profile Personal Message (Offline)

Ignore
1481444617
Reply with quote  #2

1481444617
Report to moderator
1481444617
Hero Member
*
Offline Offline

Posts: 1481444617

View Profile Personal Message (Offline)

Ignore
1481444617
Reply with quote  #2

1481444617
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481444617
Hero Member
*
Offline Offline

Posts: 1481444617

View Profile Personal Message (Offline)

Ignore
1481444617
Reply with quote  #2

1481444617
Report to moderator
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1470


View Profile
July 22, 2012, 05:25:14 PM
 #2

Reserved.

Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1470


View Profile
August 03, 2012, 09:07:14 PM
 #3

pynode now successfully verifies all transaction scripts on mainnet.  testnet3 verification still a WIP.

Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
maaku
Legendary
*
expert
Offline Offline

Activity: 905


View Profile
August 05, 2012, 06:48:59 AM
 #4

Great work @jgarzik, this is critically important stuff. Even if nobody seems to notice Wink

I'm an independent developer working on bitcoin-core, making my living off community donations.
If you like my work, please consider donating yourself: 13snZ4ZyCzaL7358SmgvHGC9AxskqumNxP
gweedo
Legendary
*
Offline Offline

Activity: 1246


Java, PHP, HTML/CSS Programmer for Hire!


View Profile WWW
August 05, 2012, 06:55:48 AM
 #5

Great work @jgarzik, this is critically important stuff. Even if nobody seems to notice Wink

I noticed it! I am waiting for it to get a little more stable, cause this would make a perfect proxy, have a bitcoind, turn off the RPC and use this so even if the rpc name, and password are figured out, they can't steal any coins.

Want to earn 2500 SATOSHIS per hour? Come Chat and Chill in https://goseemybits.com/lobby
weex
Legendary
*
Offline Offline

Activity: 1062


Give me your tired, your poor, your huddled...


View Profile
August 05, 2012, 07:41:55 AM
 #6

Definitely interested in this and running it now. Is it normal to see a bunch of blocks labeled as orphan at the first startup?

 Also the example lists /tmp/chaindb for the database location. Perhaps that should be .pynode/ if it will be helpful later to keep the database?

jgarzik
Legendary
*
qt
Offline Offline

Activity: 1470


View Profile
August 05, 2012, 03:58:23 PM
 #7

Updated status:

pynode now verifies all known scripts on mainnet and testnet3, except for the OP_IF opcodes.  OP_IF is not present on mainnet right now, so this should not be a problem for testing.

pynode now includes a JSON-RPC server like bitcoind.  Only a few RPC API methods are implemented right now.  Check back in a few days, to see more RPC API calls implemented, such as parts of the raw transaction API.

pynode is not officially stable yet, as script verification is disabled (run testscript.py instead) and chain reorg is not implemented yet.

Definitely interested in this and running it now. Is it normal to see a bunch of blocks labeled as orphan at the first startup?

Possibly yes.  Chain reorganization is not yet implemented.  Small 1-2 block forks happen on mainnet every few days.  If the chain forks, pynode will not understand this, and get stuck.  Known problem, will be fixed.

The workaround is to delete your chain database, and re-download it.  If you know python, you could save download time by writing a simple script that deleted the last 100 blocks.

You may ignore orphans, as long as you see new network blocks being stored in the database, with 'height' increasing.

Quote
Also the example lists /tmp/chaindb for the database location. Perhaps that should be .pynode/ if it will be helpful later to keep the database?

I trust people can figure this one out on their own Smiley  They will need avoid using RPC username "XXXX" and password "YYYY" also Smiley

Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1470


View Profile
August 05, 2012, 04:27:53 PM
 #8

Updated OP to indicate the "python-bitcoin" nature of the project.  Programmers may use some or all of pynode's library, without using the node itself.

Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1470


View Profile
August 06, 2012, 11:02:45 PM
 #9

pynode's script engine now successfully verifies all scripts in the mainnet and testnet3 chains.

Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
Ferroh
Full Member
***
Offline Offline

Activity: 141



View Profile WWW
August 08, 2012, 06:56:01 AM
 #10

This is awesome!

Looking forward to seeing chain reorg support.
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1470


View Profile
August 18, 2012, 04:20:50 AM
 #11

Because pynode startup may take a while, here is a set of pre-indexed database files you may download:

     http://gtf.org/garzik/bitcoin/chaindb.tar.bz2  (2.3 gigabytes)
     SHA1 a8e093ce7eedb3434a3a97570be8c3b855fa7529

This was generated by pynode git HEAD (commit b93b91b1bb659b0477eeff8b0a6771bdb595f0f4).

If you wish to treat this as untrusted data, you may verify hashes and TX connectivity with the dbck.py script.  Runtime should be under 1 hour in most cases.

If you wish to verify scripts, this will take much longer, possibly 10-12 hours.  testscript.py will verify scripts in the database.




Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1470


View Profile
August 20, 2012, 01:23:44 AM
 #12


Just pushed chain reorg code to github.  It seems to survive a static test... let's see how it fares on the live network.


Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1470


View Profile
August 23, 2012, 02:20:09 PM
 #13

Script verification is now enabled by default.

Unfortunately it is quite slow, for an unexpected reason: Python is very slow copying data structures.  Therefore, I strongly urge users to download the chaindb.tar.bz2 archive mentioned earlier in the thread.  Otherwise the block download could take a day or two.

Plans to mitigate this include:
  • Do not verify scripts earlier than a checkpoint (reference client does this)
  • Store database of only unspent CTxOut (sipa's reference client ultraprune branch does this)
  • Swap out slow GDBM database for new pagedb database


Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1470


View Profile
August 23, 2012, 04:04:51 PM
 #14

In addition to the fully validating node and python-bitcoin module, pynode also maintains a branch that contains a non-validating node calling a "mini-node", naturally found in the "mini-node" branch of pynode.git:

      https://github.com/jgarzik/pynode/tree/mini-node

This is essentially ArtForz' half-a-node, with the minimum needed to work on mainnet.

Forks have already started appearing.  blkmond inside pushpoold existed for a long time.  And bitcoin network sniffer fork also just appeared.


Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
sebicas
Member
**
Offline Offline

Activity: 68


View Profile WWW
August 23, 2012, 04:16:05 PM
 #15

In addition to the fully validating node and python-bitcoin module, pynode also maintains a branch that contains a non-validating node calling a "mini-node", naturally found in the "mini-node" branch of pynode.git:

      https://github.com/jgarzik/pynode/tree/mini-node

This is essentially ArtForz' half-a-node, with the minimum needed to work on mainnet.

Forks have already started appearing.  blkmond inside pushpoold existed for a long time.  And bitcoin network sniffer fork also just appeared.



Thanks jgarzik for pointing this out, I haven't seen this before, so I went and create my own... Sad
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1470


View Profile
September 23, 2012, 05:14:35 AM
 #16


Thanks to socrates1024, pynode got a much needed upgrade from python's asyncore to the more modern gevent. 

This opens the door to easy multi-peer operation, so expect full node status very soon.


Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
marcus_of_augustus
Legendary
*
Offline Offline

Activity: 2100



View Profile
September 23, 2012, 07:24:23 AM
 #17

Not sure how this one slipped under my radar ...

Wondering if an Electrum client might be able to modified to talk to a pynode ... or vice versa?

FactoredPrimes
Newbie
*
Offline Offline

Activity: 14


View Profile
September 23, 2012, 02:38:20 PM
 #18

Very nice tool. Am I correct that this could be the basis of a tool to monitor large numbers of addresses for activity?
Severian
Sr. Member
****
Offline Offline

Activity: 476



View Profile
September 23, 2012, 02:43:10 PM
 #19

preserved
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1470


View Profile
September 23, 2012, 04:27:42 PM
 #20

Wondering if an Electrum client might be able to modified to talk to a pynode ... or vice versa?

What does Electrum use to talk to bitcoind?   pynode provides a JSON-RPC HTTP server interface, just like bitcoind...


Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
Pages: [1] 2 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!