[ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client

[jgarzik]

Added two new utilities, "txmod" and "blkstats"

Added some P2SH scripting ability, also.

[jgarzik]

A poor first attempt at writing some TX_MULTISIG code here:

https://github.com/aido/picocoin/commit/d6635879c1f6ab4812a0b123db3800d555c2993d

It's probably not worthy of a pull request but there may be something of use that can be copied and pasted into the main code.

It's a good start.  The main bug there is that 'pubkeys' is plural.  You have to parse multiple entities out of the script to match a single "OP_PUBKEYS"  That requires a bit more reworking of the script parser, which fundamentally assumes there is a 1:1 ratio between opcodes and script data items.  With "OP_PUBKEYS", that assumption fails.

(I was looking at this last night )

[fbueller]

Hey folks, wondering if anyone here could help me with this: https://bitcointalk.org/index.php?topic=526089

I was looking at libbitcoin, it seems pretty great, although I haven't looked at C++ in years.. Would this be easy to do: a parser for the blockchain to load every transaction into a mongodb database? I'd like to decode the transactions as they're read and log some information about each.

Once I have all the old transactions done I'd just write something simple in PHP to continuously add information as it happens, but there's no quick way to decode every single transaction if I'm asking bitcoind to decode the transaction every time.

[Aido]

Added two new utilities, "txmod" and "blkstats"

Added some P2SH scripting ability, also.

Always good to see picocoin getting some more TLC 

[jgarzik]

A poor first attempt at writing some TX_MULTISIG code here:

https://github.com/aido/picocoin/commit/d6635879c1f6ab4812a0b123db3800d555c2993d

It's probably not worthy of a pull request but there may be something of use that can be copied and pasted into the main code.

Just pushed out a commit that implements this.

[Aido]

Also Jeff,

You have seen my attempts at swapping out the OpenSSL dependency in favour of PolarSSL to facilitate having picocoin run on low powered and embedded devices.

PolarSSL developers have indicated that they plan on releasing a new update early next week: https://github.com/polarssl/polarssl/issues/71

This new release should mean that PolarSSL now implements all bitcoin requirements. The PolarSSL guys have been most helpful on this.

Once this new PolarSSL release drops I will be able to finish up my PolarSSL pull request fairly quickly after that. A period of rigorous testing should follow as removing the OpenSSL dependency is a fundamental change.

[jgarzik]

Also Jeff,

You have seen my attempts at swapping out the OpenSSL dependency in favour of PolarSSL to facilitate having picocoin run on low powered and embedded devices.

PolarSSL developers have indicated that they plan on releasing a new update early next week: https://github.com/polarssl/polarssl/issues/71

This new release should mean that PolarSSL now implements all bitcoin requirements. The PolarSSL guys have been most helpful on this.

Yep, looking forward to it.

[Aido]

I noticed that PolarSSL also provides some networking functionality like:

Code:

net_connect()
net_bind()
net_accept()
net_set_block()
net_set_nonblock()
net_usleep()
net_recv()
net_send()

I know picocoin's networking functionality works perfectly well and it's probably a case of "if it ain't broke, don't fix it". But I might investigate getting PolarSSL to do some of the networking heavy lifting if possible. It may turn out to be a quick and easy way of making picocoin IPv6 compatible (I haven't looked too closely at the networking code yet but I don't think it is currently).

Then again, I probably shouldn't be getting ahead of myself and should just wait until the PolarSSL crypto related stuff is working first.  


EDIT:

OK, a quick scan of the picocoin network code shows that picocoin may already be IPv6 compatible. Ignore above statements.

[hu_ciao]

No recent progress, but any and all pull requests will be reviewed, tested and merged with lightning speed

I look forward to it. Thanks!


Been trying to compile picocoin on tinycore (http://tinycorelinux.net).

Complied jansson package ok. Then started with picocoin.

I did 'autoreconf -vi' and './configure'. All good.
Did not get to 'make DESTDIR=/mnt/sdb1/tmp/picocoin install-strip'.

Shame, as this looks like a great little package. I searched for and found it!
Now to test drive... I hope the SPV client is working (or soon)!


Fatal make errors are reported at 'make test' (and so 'make install' will also fail).

My c-coding is weak, so as to not make a mess of it I ask that someone take a look
Perhaps my method is obviously wrong?

Rather than register at github I posted here. OK?


Here is the end of 'make test' output:

<output>
Making check in lib
make[1]: Entering directory `/mnt/sdb1/picocoin-master/lib'
gcc -DHAVE_CONFIG_H -I. -I.. -I/usr/local/include/glib-2.0 -I/usr/local/lib/gli
b-2.0/include   -I../include    -march=i486 -mtune=i686 -Os -pipe -MT addr_matc
h.o -MD -MP -MF .deps/addr_match.Tpo -c -o addr_match.o addr_match.c
In file included from addr_match.c:11:
../include/ccoin/compat.h:26: error: static declaration of 'g_list_free_full' f
ollows non-static declaration
/usr/local/include/glib-2.0/glib/glist.h:53: note: previous declaration of 'g_l
ist_free_full' was here
make[2]: *** [addr_match.o] Error 1
make[2]: Leaving directory `/mnt/sdb1/picocoin-master/lib'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/mnt/sdb1/picocoin-master'
make: *** [all] Error 2
</output>

Seems these pages refer to something along the same lines.
--error: static declaration of 'foo' follows non-static declaration--


http://stackoverflow.com/questions/3148244/static-declaration-follows-non-static-declaration

https://www.linuxquestions.org/questions/programming-9/error-previous-declaration-of-xxxxx-was-here-627195/

[jgarzik]

../include/ccoin/compat.h:26: error: static declaration of 'g_list_free_full' follows non-static declaration
/usr/local/include/glib-2.0/glib/glist.h:53: note: previous declaration of 'g_list_free_full' was here

Diagnosis:  Typically a system either (a) needs compat.h help due to older GLib, or (b) does not need such help.  All systems should be either "a" or "b".  Your system configured as both "a" and "b".

The most likely fix is to make sure you have GLib > 2.30, properly installed with pkg-config support.

Alternately, you may try manually editing compat.h to comment out the offending lines, and see if that works for you.

Code:

#if ((GLIB_MAJOR_VERSION == 2) && (GLIB_MINOR_VERSION < 30))
...
#endif /* GLIB_VERSION < 2.30 */

[hu_ciao]

@ jgarzik

Thanks for doing the diagnosis. It was my fault, of course.

Made no headway for quite a while tried changing... and many other fails. Gave up those routes.
Compiled new glib 2.40 from source.

Eventually it compiled very quickly

Now I have it running. Test drive time.
I really like it. Keep up the good work!

Being able to move the existing bitcoid (etc.) wallet to picocon would encourage users to try it.
Maybe one should be able import the wallet-addr to picocon.wallet?

Is this possible?

It doesn't matter to me, as I have no stacks of coins to move or spend yet.
I'm still trying out the software and reading up. Learning.

Netsync still times out -- must get the right timeout value, I guess.
$ picocoin net.connect.timeout=240 netsync

You wrote on Github:
   

This is unfortunately normal -- you should try to addnode. Otherwise it takes a while to cycle through addresses, until it hits a live node.

The network peer manager code needs improvement.

Noob question: Which node/how do i find it?

Does picocoin connect through a proxy like polipo/privoxy or tor/SOCKS yet? This will be a popular feature.

Can we use nickname with 'picocoin new-address'? How about delete after address used/emptied of coins?

A secure way to "run it" on a MSWin system is from a quemu image, possibly on a removable drive.
Vbox (or portable vbox) is faster, but quemu doesn't require administrative privileges.
MSWin users can still get fork.

Of course TC and other small linux OSs can be booted from usb... without emulation or virtualization.

Maybe these are silly ideas. I'm sure you've heard them all.

I made a small script to get the passphrase and put it into the environmental var for the shell instance.
Saves me typing export... and reminds me not to use the window for anything else.

I wish I could help, but can not write -- read only 

[d_oilen]

Hi,
i pulled down and tried to compile picocoin on my system.
Linux XXXXXXXXXXX 2.6.32-431.5.1.el6.i686 #1 SMP Tue Feb 11 21:56:33 UTC 2014 i686 i686 i386 GNU/Linux ( centos 6.5 final)
now with the latest and greatest i found the following small mistakes:

1. in brd.c you need to include socket.h and netinet/in.h in order to successfuly compile.
2. in blkscan.c and most probably in many other places you create a dependency on glib 2.28 to 2.30. if i might suggest:
 in blkscan.c "print_txout(bool show_from, unsigned int i, struct bp_txout *txout)" you can do the following:
 

Code:

                #if GLIB_CHECK_VERSION (2, 28, 0)
		   g_list_free_full(addrs.pub, g_buffer_free);
		   g_list_free_full(addrs.pubhash, g_buffer_free);
		#else
		  //addrs.pub cleanup
		  g_list_foreach (addrs.pub, (GFunc) g_buffer_free, NULL);
		  g_list_free (addrs.pub);
		  //addrs.pubhash cleanup
		  g_list_foreach (addrs.pubhash, (GFunc) g_buffer_free, NULL);
		  g_list_free (addrs.pubhash);
		#endif

3. similar as above in txmod.c "mutate_outputs(void)" you can:

Code:

if (!tx.vout){
			#if GLIB_CHECK_VERSION (2, 30, 0)
				tx.vout = g_ptr_array_new_full(8, g_bp_txout_free);
			#else
				tx.vout = g_ptr_array_sized_new (8);
				g_ptr_array_set_free_func (tx.vout, (GDestroyNotify)g_bp_txout_free);
			#endif	
	}	

as a side note, be aware that on centos systems the stock libevent is not working ( it is at version is 1.4)  and you have to install libevent 2.x from source

regards,
JD

[hu_ciao]

Still getting timeout errors and no connection. My mistake?

Tried 'picocoin addnode=012.345.678.9 8333 netsync', but does not (yet) succeed. I used addresses from dns-seeds.

The best I can get is a long list of peers (wc -l picocoin.peers gives almost 2000); dns-seeds lists 97-131 each time.
Also got a few 'llnet: ... read EOF' results in among the 'net: ... connection timeout' results.

Anyone else succeeding/failing to get a picocoin.blkdb with the current version? Mine is still 0 KB.


Made a few icons and 'nix Bourne shell scripts to help run picocoin.
Icons made with ImageMagick on Debian.

Scripts should go in /usr/sbin/, or wherever your script bin is. Maybe they're useful for somebody?  
Extraxt with 'tar -xvjf buttn.tar.bz2'. Sorry, no scripts for msWin users (-_-)
...maybe Darwin (mac) can use the scripts? OK for Android, too.

I hope this encourages testing, exploration, and development of the project.

One script, pcoin.sh, loads a new CLI window and runs the second script, picocoin.sh. Aterm is called in pcoin.sh and a code example for rxvt is commented. One might rename it to the shorter pcoin, as there's no conflict with the program name (also p is the SI prefix for pico).
Edit the scripts as you like.

The picocoin.sh makes '.picocoin' directory, if not yet created, in users' home and changes to it.
User is asked for a passphrase, confirmed by retyping. That's exported to environmental variable in a new subshell.
The command prompt waits for picocoin commands.

When done there is no need to 'cd' back to original directory.
With command 'exit' the window closes if run from pcoin.sh or clears if picocoin.sh is run alone. Either way the passphrase is no longer in memory. Try 'echo $PICOCOIN_PASSPHRASE' to confirm.

I prefer to have the config and generated files in one place, uncluttered, hidden from view. We don't need regular access exept through the Bitcoin client. This also prevents the dropping of files all over the filesystem when picocoin is started from different curdirs.


Included with scripts are icons and a toy html page to show/play with the icons. The first 'button' has a radio option where down image is/not very small.
Click and hold down a second, release, and wait...

Eagerly awaiting developments.


Have fun


File: buttns.tar.bz2   51787 B (52 KB)
(P2P Torrent)

magnet:?xt=urn:btih:ee75467fd552464eeb266bf5bf46fd2581e46fa5&dn=buttns.tar.bz2

[edit removed trackers from torrent]

[jgarzik]

Hi,
i pulled down and tried to compile picocoin on my system.
Linux XXXXXXXXXXX 2.6.32-431.5.1.el6.i686 #1 SMP Tue Feb 11 21:56:33 UTC 2014 i686 i686 i386 GNU/Linux ( centos 6.5 final)
now with the latest and greatest i found the following small mistakes:

1. in brd.c you need to include socket.h and netinet/in.h in order to successfuly compile.
2. in blkscan.c and most probably in many other places you create a dependency on glib 2.28 to 2.30. if i might suggest:
 in blkscan.c "print_txout(bool show_from, unsigned int i, struct bp_txout *txout)" you can do the following:
 

Code:

                #if GLIB_CHECK_VERSION (2, 28, 0)
		   g_list_free_full(addrs.pub, g_buffer_free);
		   g_list_free_full(addrs.pubhash, g_buffer_free);
		#else
		  //addrs.pub cleanup
		  g_list_foreach (addrs.pub, (GFunc) g_buffer_free, NULL);
		  g_list_free (addrs.pub);
		  //addrs.pubhash cleanup
		  g_list_foreach (addrs.pubhash, (GFunc) g_buffer_free, NULL);
		  g_list_free (addrs.pubhash);
		#endif

3. similar as above in txmod.c "mutate_outputs(void)" you can:

Code:

if (!tx.vout){
			#if GLIB_CHECK_VERSION (2, 30, 0)
				tx.vout = g_ptr_array_new_full(8, g_bp_txout_free);
			#else
				tx.vout = g_ptr_array_sized_new (8);
				g_ptr_array_set_free_func (tx.vout, (GDestroyNotify)g_bp_txout_free);
			#endif	
	}	

Thanks for pointing out several locations incompatible with older GLib versions.

In general, #ifdefs directly placed in the library code are to be avoided.  The pattern to follow is updating compat.h with

Code:

#if VERSION < X
  provide wrapper or full implementation of missing function
#endif

If there is a missing function, add the function to compat.h, inside a version check.

[hu_ciao]

A bit more detail on test of picocoin.

picocoin debug netsync &> debug.txt

net: parent exec NC_START
net: have 46188/46188 peers
net: blkdb opened
net: gc'd 0 connections
net: open connections (have 0, want 8 more)
...     [ tries/connects to 8 ips ]
net: parent after NC_START
net: engine started. sleeping 10 minutes (cxn tmout 11 sec)
net: connected to 68.224.[scrubbed]
net: 68.224.[scrubbed] message version
net: 68.224.[scrubbed] version(70002, 0x1, 1397475165, To:[scrubbed], From:68.224.[scrubbed], /Satoshi:0.9.0/, 295782)
net: 68.224.[scrubbed] message verack
net: 68.224.[scrubbed] message alert
net: 68.224.[scrubbed] message inv
...     [ first timeout error reported - it's the first ip tried/connected ]
net: gc'd 1 connections
net: open connections (have 7, want 1 more)
...     [ keeps trying connections as previous connections timeout ]
net: 68.224.[scrubbed] message inv
net: 68.224.[scrubbed] message addr
net: 68.224.[scrubbed] addr(2 addresses, 0 old)
...     [ as before: new made, old dropped ]
...     [ a few ipv6 fails: '' ]
socket connect [scrubbed]:  Network is unreachable
net: failed to start connection to [scrubbed]
...
net: gc'd 1 connections
net: open connections (have 7, want 1 more)
net: connecting to [scrubbed]
net: 68.224.[scrubbed] message inv
net: stopping engine
peerman: 46707 peers to write

(Got 53982 peers last time tried.)


Looked at net connections: netstat -tp | grep picocoin  (ran in loop).

Shows only one ESTABLISHED others are SYN_SENT.

Not likely a firewall block, as picocoin talks to the one.
Connections being ignored?
Get SYN_SENT status only before timeouts; no further SYN-ACK or ACK steps in the connection handshake ...except for the one.

Can get other IPs to connect with block relay daemon (brd). Timeout stderr messages persist, though.

Adding an address which connected with brd doesn't increase number of connections made by picocoin; still speaks only to the one address.

Thanks for today!


(Not complaining, just saying.)

[jgarzik]

Because of stupid node selection, it takes a while to cycle through dead addresses passed to your node from other nodes.  You can use addnode etc.

[Aido]

Hi Jeff,

I'm trying to get my head around the brd.c code with a view to introducing LMDB to the picocoin project.

There's a few things I don't yet get. It seems that brd no longer stores blocks in brd.blkdb (it's commented out in code) but uses brd.blocks instead. But, there are still attempts to write to brd.blkdb, like calling blkdb_add(), which I think will always fail.

So, can the blkdb.c code be removed or does it just need a tidy-up?

[jgarzik]

Hi Jeff,

I'm trying to get my head around the brd.c code with a view to introducing LMDB to the picocoin project.

There's a few things I don't yet get. It seems that brd no longer stores blocks in brd.blkdb (it's commented out in code) but uses brd.blocks instead. But, there are still attempts to write to brd.blkdb, like calling blkdb_add(), which I think will always fail.

So, can the blkdb.c code be removed or does it just need a tidy-up?

blkdb is required and is always used.  It is the block header database.  It is kept in RAM, as bitcoind does.  It is optionally stored in a file, in addition to RAM.  It is optionally regenerated from the 17+GB blockchain file, if blkdb is missing.

[Cryddit]

What I don't see in these libraries, that I would very much like to see, is comments that say which things you cannot touch or what kind of changes you can make in the block structure, without breaking compatibility with existing ASIC miners. 

Anybody know? 

Anybody know who knows? 

[Aido]

blkdb is required and is always used.  It is the block header database.  It is kept in RAM, as bitcoind does.  It is optionally stored in a file, in addition to RAM.  It is optionally regenerated from the 17+GB blockchain file, if blkdb is missing.

OK, thanks for the explanation.

Along with some other handy features LMDB can have multiple sub-databases. I haven't thought things out fully yet but blkdb could possibly be stored as a sub-database with the blockchain as another sub-database or collection of sub-databases, performance permitting. Or maybe plagiarise the libbitcoin database layout a bit.