Bitcoin Forum

Bitcoin => Wallet software => Topic started by: jgarzik on November 27, 2012, 08:25:01 PM



Title: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on November 27, 2012, 08:25:01 PM
Source code URL:  https://github.com/jgarzik/picocoin/ (https://github.com/jgarzik/picocoin/)

I'd like to announce another bitcoin implementation, which is really two useful pieces in one:

     libccoin - a bitcoin library, written in C
     picocoin - A lightweight, C-based SPV (https://en.bitcoin.it/wiki/Thin_Client_Security#Simplified_Payment_Verification_.28SPV.29) bitcoin wallet client

libccoin supports all key network datastructures (block, transaction, etc.), script parsing and validation, transaction and block validation, a "headers-only" or full block database, and many other features essential to any bitcoin client.  libccoin passes all key encoding, script and transaction tests available in the Satoshi reference bitcoin client.

picocoin is much more under construction.  When complete, it will be a very low resource, command line / JSON-driven bitcoin wallet.  Advanced security features already implemented include required wallet encryption, fork (http://en.wikipedia.org/wiki/Fork_%28operating_system%29)-based process separation of P2P networking and wallet (and chroot/SELinux jailing coming soon), something that the reference Satoshi client does not even support.

Status:  Alpha quality, developer release.  Passes reference client base58/script/transaction tests, but is still a developer-only preview.

Feature list:
  • Liberal Mit/X11 free software license
  • A full-feature bitcoin support library.  The library will not be limited to "what picocoin needs."
  • Supports all core data structures and network messages
  • Full script implementation
  • Passes hundreds of available reference client tests
  • Supports multiple block chains: main or testnet3
  • Very low resource usage (cpu, disk, and memory)
  • Small codebase (both source code and compiled object)
  • Fast.  Loads all block headers in under 2 seconds.
  • Supports advanced thin-client features such as bloom filtering (https://en.bitcoin.it/wiki/BIP_0037), an upcoming proposal that will reduce client bandwidth usage.
  • Works on big endian machines, as well as little endian machines
  • Multi-platform: Linux, Mac OSX, BSD are working.  Windows support in progress.
  • Library works on Windows.  picocoin will work on Windows with some modifications, but be a bit less secure than other platforms due to lack of fork.
  • Improved security:  fork-based process separation firewall between networking and wallet code -- your wallet is never directly exposed to the network.
  • Follows the philosophy of "do, not hype."  This library is already far more secure and capable than other libraries hyped as the "future of bitcoin" by their authors.

Code contributions are welcome (see github URL above).
Comments are welcome.
Donations are welcome too (see signature).



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on November 27, 2012, 08:27:00 PM
Reserved for FAQ etc.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: marcus_of_augustus on November 27, 2012, 09:11:51 PM
Nice work, thanks. This should be useful.

Edit: did you use anything from cbitcoin https://github.com/MatthewLM/cbitcoin ?


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on November 27, 2012, 09:16:24 PM
Edit: did you use anything from cbitcoin https://github.com/MatthewLM/cbitcoin ?

Definitely not.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: 2112 on November 27, 2012, 09:22:49 PM
  • Very low resource usage (cpu, disk, and memory)
  • Small codebase (both source code and compiled object)
  • Multi-platform: Linux supported currently.  Should work on OSX/BSD/Windows with minimal modifications.
Comments are welcome.
Jeff, I have one comment that has value only very early in the design of your architecture. Since you've made a choice of C as a implementation language I suggest that you add one more target to the list of supported platforms: standalone a.k.a. bare metal.

It will be nearly impossible to convert your code later unless you very early make a commitment to supporting the event-driven programming like using lwIP-style network interface in addition to socket-style networking.

The same architectural choice needs to be made early to accomodate various lightweight flash-storage access methods instead of Unix-style open/lseek/read/write/close. Same for fork/wait/pthreads versus any RTOS-style explicit synchronization primitives.

One possible advantage of such choices would be that instead of relying purely on donations you may be able to obtain funds via cooperation with future manufacturers of the future Bitcoin devices that will greatly benefit from the "very low resource usage".


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on November 27, 2012, 09:26:57 PM
Jeff, I have one comment that has value only very early in the design of your architecture. Since you've made a choice of C as a implementation language I suggest that you add one more target to the list of supported platforms: standalone a.k.a. bare metal.

Yes, embedded usage is another target.

Each module in libccoin is carefully designed to minimize internal dependencies.  The core data structures, address generation, script execution and transaction validation are wholly independent of any filesystem or network design.

These modules may be used on a non-POSIX flash filesystem, with zero network support, today.

picocoin (the client) requires certain network, filesystem and process features, but libccoin (the library) does not.  This is by design.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jimbobway on November 27, 2012, 09:36:49 PM
Do you think you could give a distinct (and cool) name for this library?  There are several now and it gets confusing.  For example:

libcoin - Gronager
libccoin - jgarzik
libbitcoin - genjix
cbitcoin - MathhewLM

How about GarzikBitcoinLib?


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: dserrano5 on November 27, 2012, 09:58:45 PM
Quote
AES encryption is applied to the wallet.  Passphrase is specified via environment variable PICOCOIN_PASSPHRASE.

Is this secure? My understanding is that the environment isn't a safe place to store/transmit information.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on November 27, 2012, 10:05:41 PM
Quote
AES encryption is applied to the wallet.  Passphrase is specified via environment variable PICOCOIN_PASSPHRASE.

Is this secure? My understanding is that the environment isn't a safe place to store/transmit information.

That's temporary.  It will input via fd like GPG soon.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: 2112 on November 27, 2012, 10:45:58 PM
Yes, embedded usage is another target.

Each module in libccoin is carefully designed to minimize internal dependencies.  The core data structures, address generation, script execution and transaction validation are wholly independent of any filesystem or network design.

These modules may be used on a non-POSIX flash filesystem, with zero network support, today.

picocoin (the client) requires certain network, filesystem and process features, but libccoin (the library) does not.  This is by design.
But even the definitions in your "ccoin" include files are deeply incompatible with any standalone environment, even if it uses GCC toolchain. Major rework would be required even now, when there's just couple of hundred lines of code.

The big-endian compatibility is already super-confusing. Istead of using uint32_t in bu256_t use an "union le_int32" to at least flag the mixing of little endian uint32_t with guint32_t.

From my extensive commercial experience of porting from Linux/Posix/Intel-only code to anything standalone or lightweight OS will unfortunatly go through the "first deny then rewrite from almost scratch" stages.

If I may suggest two things:

1) from the start build also on Mac Leopard or Snow Leopard to verify the endian-neutrality with "-arch ppc" with Rosetta on Intel CPU. Edit: Never mind, I forgot that you have access to the Linux on POWER and z/Arch through your association with RedHat.
2) obtain any embedded development/simulation environment. If you have any Xilinx Spartan6 boards (with memory) laying around you could use them to test using MicroBlaze CPU with Xilinx EDK/SDK.

Thank you for yor time.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on November 27, 2012, 11:34:42 PM
But even the definitions in your "ccoin" include files are deeply incompatible with any standalone environment, even if it uses GCC toolchain. Major rework would be required even now, when there's just couple of hundred lines of code.

This seems quite an exaggeration, with no supporting evidence of what needs a major rework.  I do embedded programming for the day job, so I don't see this.  The Linux kernel is as bare metal as it gets, and must work in all environments, all platforms.  :)

The main chore of embedded use is simply the dependencies, most notably OpenSSL, and to a lesser extent GLib.  Once you have that, or its replacement, embedded use is straightforward.

Quote
The big-endian compatibility is already super-confusing. Istead of using uint32_t in bu256_t use an "union le_int32" to at least flag the mixing of little endian uint32_t with guint32_t.

Using "union" decreases the performance and quality of code output for zero gain.  It is trivial enough to use sparse (https://sparse.wiki.kernel.org/index.php/Main_Page) with a type specifier to guarantee endian purity, like the Linux kernel does, if that is desired ("__le32" etc.)

Quote
1) from the start build also on Mac Leopard or Snow Leopard to verify the endian-neutrality with "-arch ppc" with Rosetta on Intel CPU. Edit: Never mind, I forgot that you have access to the Linux on POWER and z/Arch through your association with RedHat.

The code has long since been verified to work on big endian.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Bitcoin Oz on November 27, 2012, 11:40:04 PM
Do you think you could give a distinct (and cool) name for this library?  There are several now and it gets confusing.  For example:

libcoin - Gronager
libccoin - jgarzik
libbitcoin - genjix
cbitcoin - MathhewLM

How about GarzikBitcoinLib?

Just call it gbitcoin  :)


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: BkkCoins on November 28, 2012, 01:22:27 AM
So am I understanding right? I could run picocoin and it would be lightweight like Electrum but also support the Bitcoin network by interacting on the network like a full satoshi client. I've been using Electrum mostly to get away from having to use around 5GB disk space and for quick install/startup but I'd really prefer to be supporting Bitcoin more by being another validating node in it's network.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: 2112 on November 28, 2012, 01:49:32 AM
This seems quite an exaggeration, with no supporting evidence of what needs a major rework.  I do embedded programming for the day job, so I don't see this.  The Linux kernel is as bare metal as it gets, and must work in all environments, all platforms.  :)
For starters, you'll have heck of a time to remove your dependency on glib.h from GTK. But there's more. You could actually try to compile your code from within the Linux kernel tree to see for yourself.

For a die-hard Linux developer like you I think the minimum definition of bare-metal would be uClinux. Full Linux doesn't count anymore for a true embedded development.

The main chore of embedded use is simply the dependencies, most notably OpenSSL, and to a lesser extent GLib.  Once you have that, or its replacement, embedded use is straightforward.
Both GLib and OpenSSL are heavy duty harvester combines. Removing the dependencies on them will at least halve the resource usage.

For a RedHat-ter at least make the code work with newlib instead of glib. The required knowledge should be no more than the internal phone call away for you. This includes a whole sermon on why you don't want to build a dependency on them.

Using "union" decreases the performance and quality of code output for zero gain.  It is trivial enough to use sparse (https://sparse.wiki.kernel.org/index.php/Main_Page) with a type specifier to guarantee endian purity, like the Linux kernel does, if that is desired ("__le32" etc.)

The code has long since been verified to work on big endian.
As I've noted: I've forgot that you are one of the very few Bitcoin developers with regular access to big-endian hardware. I'm thinking that you will continue to be the only one developer who can test picocoin on the big-endian architecture. The mojibake-endian problem will continue to badly affect the future contributors/users of your library if you aren't going to use your own medicine (that is "sparse" and "__le32")

The performance loss for passing unions around may have been true for GCC 2.96. I'm not normally working with Linux kernel and/or GCC.

But I have a great, positive experience with code of the following style(C++ used for brevity, ... expands to long #ifdef trees using platform- and compiler-specific intrinsics)
Code:
union be_int {
  unsigned char b[4];
  int i;
  inline operator =(int a) { ... }
  inline operator int() { ... }
};
The explicit strong-typing of endianness blocks the vast majority of possible endian errors and allows to produce meaningfull, endian-bug-free contributions from people who are exclusively using little-endian hardware. In my (rather extensive) experience it actually increases the performance because programmers can visualize the data structures better and can spend more time designing the algorithms instead of debugging trivial omissions or transpositions.

All in all, I apologise for jumping in into the battle between you and MatthewLM. Regretfully I can't easily build neither of yours code, because I'm almost exclusively working on the closed-source platforms like Windows, MacOS, etc. and standalone environments not relying on GCC/GLIB/autogen. But it is a shame that another C implementation of Bitcoin is useless for the developers of Bitcoin hardware wallets.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on November 28, 2012, 02:54:09 AM
So am I understanding right? I could run picocoin and it would be lightweight like Electrum but also support the Bitcoin network by interacting on the network like a full satoshi client. I've been using Electrum mostly to get away from having to use around 5GB disk space and for quick install/startup but I'd really prefer to be supporting Bitcoin more by being another validating node in it's network.

picocoin is only a lightweight client.  It is not a "full node" and probably never will be.

The included library, libccoin, does provide all the tools and gadgetry needed to build your own "full node" bitcoin client that takes 5GB disk space, if you have the programming chops to do that.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on November 28, 2012, 03:11:49 AM
For starters, you'll have heck of a time to remove your dependency on glib.h from GTK.

You mean "remove your dependency on glib.h from libccoin", I presume?  GTK has no relevance to this, besides being the administrative umbrella project for GLib.

Removing the GLib dependency is straightforward for any developer.  GLib is used for a few ADTs like variable length strings, hash tables and arrays.

Quote
Both GLib and OpenSSL are heavy duty harvester combines. Removing the dependencies on them will at least halve the resource usage.

For a RedHat-ter at least make the code work with newlib instead of glib. The required knowledge should be no more than the internal phone call away for you. This includes a whole sermon on why you don't want to build a dependency on them.

Not sure you understand the problem space...  First, newlib is wholly different from GLib, and does not provide the ADTs that GLib provides.  Second, replacing GLib and OpenSSL would not change resource usage much at all.  ADTs, Big Integer support, SHA hashing and ECDSA are required regardless of lib chosen.  They would simply be replaced with...  code that did the same thing under name other than "OpenSSL" or "GLib."

Quote
The performance loss for passing unions around may have been true for GCC 2.96. I'm not normally working with Linux kernel and/or GCC.

It continues to be true for all known compilers I've seen or heard of, when compared with a simple, native machine integer as used by the current implementation.

Quote
All in all, I apologise for jumping in into the battle between you and MatthewLM. Regretfully I can't easily build neither of yours code, because I'm almost exclusively working on the closed-source platforms like Windows, MacOS, etc. and standalone environments not relying on GCC/GLIB/autogen.

It will soon be building on MacOSX and Windows.  Will post when this happens (volunteers willing to post building outputs are helpful).  As stated, it is a developer-only release right now, so there are plenty of rough edges.

Quote
But it is a shame that another C implementation of Bitcoin is useless for the developers of Bitcoin hardware wallets.

I respectfully disagree with that assessment.  Anyone looking at the code may see precisely what dependencies need replacing, for an embedded environment.  If anyone actually developing hardware or embedded bitcoin solutions wishes to contact me, I would be happy to illustrate how it would work.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: ldrgn on November 28, 2012, 03:49:25 AM
Everything builds and tests pass on MacOS X 10.7.  Good job on this code, you've written the client Satoshi should have.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on November 28, 2012, 03:56:53 AM
Everything builds and tests pass on MacOS X 10.7.  Good job on this code, you've written the client Satoshi should have.

Appreciated.  Thanks in particular for testing OSX.  That was a big item on the checklist.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Steve on November 28, 2012, 05:34:29 AM
Very nice!  looking forward to playing around with it.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: 2112 on November 28, 2012, 06:22:56 AM
It continues to be true for all known compilers I've seen or heard of, when compared with a simple, native machine integer as used by the current implementation.
OK, show of hands. You must have heard of the existience of Microsoft Visual Studio C++/C compiler. I wrote a short test program that tests strong-typed little-endian integer on the Intel architecture, which will be the most important case.
Code:
#include <cstdio>

union le_int {
unsigned char b[4];
int i;
void operator =(int a) { i = a; }
operator int() { return i; }
};

int lehmer1(int n)
{
int r = 1;
while (n--)
r = (16807 * r) % 2147483647;
return r;
}

int lehmer2(int n)
{
union le_int r;

r = 1;
while (n--)
r = (16807 * r) % 2147483647;
return r;
}

void main()
{
printf("%d\n",lehmer1(2112));
printf("%d\n",lehmer2(2112));
}

and the generated "Release Win32" code, with everything at default:

Code:
00311000  push        edi  
printf("%d\n",lehmer1(2112));
00311001  mov         ecx,840h 
00311006  mov         edx,1 
0031100B  mov         edi,7FFFFFFFh 
00311010  imul        edx,edx,41A7h 
00311016  mov         eax,edx 
00311018  cdq 
00311019  idiv        eax,edi 
0031101B  dec         ecx 
0031101C  jne         main+10h (0311010h) 
0031101E  push        edx 
0031101F  push        312100h 
00311024  call        dword ptr ds:[312090h] 
0031102A  add         esp,8 
printf("%d\n",lehmer2(2112));
0031102D  mov         ecx,840h 
00311032  mov         edx,1 
00311037  imul        edx,edx,41A7h 
0031103D  mov         eax,edx 
0031103F  cdq 
00311040  idiv        eax,edi 
00311042  dec         ecx 
00311043  jne         main+37h (0311037h) 
00311045  push        edx 
00311046  push        312100h 
0031104B  call        dword ptr ds:[312090h] 
00311051  add         esp,8

Could anyone here can give an example of a popular compiler that produces different code for those two little functions? Maybe the test program needs to be somehow changed to trip up the optimizer? I'm seriously asking. I have code like this (and pure C equivalent with #define macros instead of operators()) in wide deployment for many years now on a variety of platforms.

I really only have my Windows laptop with me. I would have to ask people to unpack and boot my usual development environment for remote access.

Meanwhile I'm going to see if I can trip-up the Visual Studio somehow by rewriting this code in various ways.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on November 28, 2012, 06:42:13 AM
Thanks to denisx on IRC, who helped get picocoin/libccoin going on FreeBSD and GLib < 2.30 as well.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: ShadowOfHarbringer on November 28, 2012, 09:28:22 AM
First thing, i like the idea very much.

One small suggestion though:
Do you think you could give a distinct (and cool) name for this library?  There are several now and it gets confusing.  For example:

libcoin - Gronager
libccoin - jgarzik
libbitcoin - genjix
cbitcoin - MathhewLM

How about GarzikBitcoinLib?

+1

Seriously jgarzik, these names are getting more and more confusing.

How about calling it glibbitcoin, jgbitcoin or libbitcoin-jg ?


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: 2112 on November 28, 2012, 10:17:40 AM
Removing the GLib dependency is straightforward for any developer.  GLib is used for a few ADTs like variable length strings, hash tables and arrays.

Not sure you understand the problem space...  First, newlib is wholly different from GLib, and does not provide the ADTs that GLib provides.  Second, replacing GLib and OpenSSL would not change resource usage much at all.  ADTs, Big Integer support, SHA hashing and ECDSA are required regardless of lib chosen.  They would simply be replaced with...  code that did the same thing under name other than "OpenSSL" or "GLib."

If I don't understand the problem space then you don't understand the solution space. GLib's ADT heavily depend on glibc-style memory allocation and both depend on GCC extensions and cross-compiling with MinGW instead of Visual Studio or any other native Windows compiler.

Both GLib and OpenSSL are extremely resource intensive and the only way to get respectable performance out of them is by running on a host with paging MMU and TLB. This excludes majority of standalone / lightweight OSes.

The problem/solution space doesn't demand OpenSSL. It requires BigNums, ECDSA, SHA-256. Any multiuser OS is just a serious hindrance for an lightweight implementation on a lean hardware. All those tasks are best performed not on the main CPU but on the attached GPU or DSP array. When the GPU can be directly called without the overhead of going through OpenGL/OpenCL the whole new world of possibilities opens up. As far as I know currently only PowerVR GPUs have an open source toolchain. This means Texas Instruments OMAP chips. In a year or two probably someone will do the same for the Broadcom VideoCore chips in Raspberry Pi. The performance and power efficiency gained from MIMD implemenations will be immense.

It will soon be building on MacOSX and Windows.
The salient point for Windows is: is it going to be possible to build it using a native Windows compiler like Visual Studio, OpenWATCOM or Borland/Embarcadero? Because another MinGW-only deliverable will be impossible to integrate into the Windows ecosystem.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: grau on November 28, 2012, 01:27:08 PM
Source code URL:  https://github.com/jgarzik/picocoin/ (https://github.com/jgarzik/picocoin/)
Great to see a well written alternative for the thin client!

Follows the philosophy of "do, not hype."  This library is already far more secure and capable than other libraries hyped as the "future of bitcoin" by their authors.
Add do not FUD.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on November 28, 2012, 02:06:52 PM
Thanks to jrmithdobbs for helping in bringing up OpenBSD (and more FreeBSD help from denisx).

Here's the autogen.sh and configure magic he used for OpenBSD.  Some are specific to his system, but it is an illlustrative example:

Code:
AUTOMAKE_VERSION=1.11 AUTOCONF_VERSION=2.68 ./autogen.sh && \
CC=clang LD=llvm-ld CFLAGS='-O2 -Wall -g -I/usr/local/include/event2 \
-I/opt/OpenBSD/5.1/amd64/jansson-2.4/include -I/usr/local/include/event2' \
 LDFLAGS='-L/opt/OpenBSD/5.1/amd64/jansson-2.4/lib -L/usr/local/lib -L/usr/lib' \
      ./configure --prefix=/opt/OpenBSD/5.1/amd64/picocoin-0.0 && \
      make all check


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Peter Todd on November 28, 2012, 02:45:02 PM
As I've noted: I've forgot that you are one of the very few Bitcoin developers with regular access to big-endian hardware. I'm thinking that you will continue to be the only one developer who can test picocoin on the big-endian architecture. The mojibake-endian problem will continue to badly affect the future contributors/users of your library if you aren't going to use your own medicine (that is "sparse" and "__le32")

FWIW while I've never tried doing it QEMU can apparently emulate the big-endian PowerPC architecture. Here (http://dcreager.net/2010/05/13/powerpc-qemu-lucid/) is one guys notes about getting it working from two years ago, specifically to test endian issues.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: gmaxwell on November 28, 2012, 02:58:36 PM
As I've noted: I've forgot that you are one of the very few Bitcoin developers with regular access to big-endian hardware. I'm thinking that you will continue to be the only one developer who can test picocoin on the big-endian architecture. The mojibake-endian problem will continue to badly affect the future contributors/users of your library if you aren't going to use your own medicine (that is "sparse" and "__le32")

The frequent invocations of RedHat upthread are weird and misplaced.  _I_ did the initial BE testing for Jeff. Lots of people have access to BE systems and pretty much anyone who wants to can have access to one (for a couple bucks on ebay if nothing else).  ... but nothing Jeff is doing makes handling the endlessness hard as directly evidenced by the fact that he got it running fine on BE without using one.  Your 'solution' of pushing unions everywhere for that is an odd one and I'm happy to not work on your codebase.

Now that you've made your numerous complaints would you please cut it out? The tangent is going nowhere. If you want bitcoin software written to your particular, and somewhat odd, specifications feel free to do so yourself or to pay someone to do so.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on November 28, 2012, 03:50:20 PM
How about calling it glibbitcoin, jgbitcoin or libbitcoin-jg ?

Bleh :)  If someone actually does come up with a cool name I'd consider it :)

But anything directly related to (a) my name or (b) a dependency lib's name is right out.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: marcus_of_augustus on November 28, 2012, 07:24:17 PM
I think "libccoin" is fine ... clean and to the point like the code.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jimbobway on November 28, 2012, 08:42:39 PM
cbitcoinlib!

Sorta like Yahoo! but cbitcoinlib!
 
Sounds exciting when I read it.  :P

Search engines will also find it when people type "c bitcoin library"


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: wladston on November 29, 2012, 02:43:00 PM
Any ideas if this client would be suitable for RPC-like usage ? I'm currently running bitcoind on a server to power up my web service via RPC calls. But it eats a HUGE share of memory, and over the weeks it keeps getting slow, so I have to restart it often.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Mike Hearn on November 29, 2012, 03:04:42 PM
You could use an SPV client to reduce resource usage, but if you're running a web service that handles money, it's a really good idea to just pony up the resources for a full node. Security matters!


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: maaku on November 29, 2012, 03:30:10 PM
Quote
AES encryption is applied to the wallet.  Passphrase is specified via environment variable PICOCOIN_PASSPHRASE.

Is this secure? My understanding is that the environment isn't a safe place to store/transmit information.

That's temporary.  It will input via fd like GPG soon.

Please leave the environment-specified password as an option, however. In some environments (12-factor stacks like Heroku, for example) that remains the best available option.


I'm a little confused, does this project do full validation or not? Am I understanding it correctly that picocoin is and will remain a SPV thin-client, but libccoin implements (or will implement if it doesn't already) everything required to build a full validating node?


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on November 29, 2012, 05:45:57 PM
I'm a little confused, does this project do full validation or not? Am I understanding it correctly that picocoin is and will remain a SPV thin-client, but libccoin implements (or will implement if it doesn't already) everything required to build a full validating node?

Yes, libccoin implements everything needed to do full validation.

If you wanted to build a "full node" with libccoin, you could do that.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on November 30, 2012, 08:15:05 AM

picocoin's library successfully validates both main and testnet3 block chains, including scripts.  ("chain-verf" test in its testsuite)

It is working on Linux, MacOSX, FreeBSD and OpenBSD.  The library compiles on Windows, but the client requires additional Windows work to be functional.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: 2112 on December 01, 2012, 06:37:27 PM
The frequent invocations of RedHat upthread are weird and misplaced.  _I_ did the initial BE testing for Jeff. Lots of people have access to BE systems and pretty much anyone who wants to can have access to one (for a couple bucks on ebay if nothing else).  ... but nothing Jeff is doing makes handling the endlessness hard as directly evidenced by the fact that he got it running fine on BE without using one.  Your 'solution' of pushing unions everywhere for that is an odd one and I'm happy to not work on your codebase.

Now that you've made your numerous complaints would you please cut it out? The tangent is going nowhere. If you want bitcoin software written to your particular, and somewhat odd, specifications feel free to do so yourself or to pay someone to do so.
I apologize for misattributing anyone's work. I didn't actually read the commit log. From the way you wrote I have to assume that Jeff did 100% of work and you did 100% of testing. It doesn't really matter who did what, so long as it is getting done at a reasonable frequency.

The "somewhat odd" comment is really strange. It is the standard way to strenghten the C type system from structural equivalence to name equivalence. The type systems taxonomy is the standard curriculum of pretty much every CS or SE course at the university level. In fact the C++ code I posted above is patterned after the sample C/C++ code and a paper that was included with the "cfront" C++ compiler distribution from Bjarne Stroustrup that I've seen on some dusty VAX at my school. I didn't claim the autorship and in fact the requirement of not overusing type-cast-ing is one of the standard admonishments in the C/C++ project specifications.

On the other hand the 1995-1997 copyright dates on glib.h show that the base GTK/GNOME code was designed during the darkest days of GCC, predating the EGCS fork: http://en.wikipedia.org/wiki/GNU_Compiler_Collection#EGCS_fork . At that time GCC was widely used as a baseline: it would reliably produce largest and slowest code after taking the longest time to build it; so almost anything looked better in comparison. From what I've heard from one of the Linux distribution maintainers was that many sub-optimal choices in open source packages were motivated by the desire to avoid tripping up GCC to produce very slow or incorrect code. Those days are long gone.

It really doesn't take an advanced computer science education to understand why poking holes in C/C++ type system by using unsafe casts is not a good architectural choice. It may be a good short term choice. Open source projects thrive on finding a group of people who while "scratching their own itch" actually scratch in the same place. It may well be the truth that many programmers here enjoy lookin at every uint32 and ponder wheteher to use it directly or through GUINT32_{TO,FROM}_LE macros.

I also know from the past experience that overall such people are not a majority, and would consider the above task a software engineering equivalent of digging diteches and filling them back in. For every one person who posted here there is at least 10 who read this thread with understanding. Of those 10 at least 2-3 will copy/paste the example code I wrote for a future reference.

So, yes, go ahead and play with the following somewhat longer example C code. Modify it to make mistakes and see which way is better for yourself.
Code:
#include <stdio.h>
#ifdef _MSC_VER
#include <stdlib.h>
#define inline __inline
#define __builtin_bswap32 _byteswap_ulong
#endif
#ifdef __GNUC__
#define _byteswap_ulong __builtin_bswap32
#endif
/*
 * type safe little endian
 */
union le_int {
unsigned char b[4];
int i;
};
/* little endian setter on little endian host */
static inline union le_int h2l_int(int a)
{
union le_int t;

t.i = a;
return t;
}
/* little endian getter on little endian host */
static inline int l2h_int(union le_int a)
{
return a.i;
}
/*
 * type safe big endian
 */
union be_int {
unsigned char b[4];
int i;
};
/* big endian setter on little endian host */
static inline union be_int h2b_int(int a)
{
union be_int t;

t.i = _byteswap_ulong(a);
return t;
}
/* big endian getter on little endian host */
static inline int b2h_int(union be_int a)
{
return _byteswap_ulong(a.i);
}

/* excerpted from the Glib.h include tree */
typedef unsigned int guint32;
typedef signed int gint32;
#define GUINT32_SWAP_LE_BE(val) ((guint32) __builtin_bswap32 ((gint32) val))
#define GINT32_TO_LE(val) ((gint32) (val))
#define GINT32_TO_BE(val) ((gint32) GUINT32_SWAP_LE_BE (val))
#define GINT32_FROM_LE(val) (GINT32_TO_LE (val))
#define GINT32_FROM_BE(val) (GINT32_TO_BE (val))
/* end of Glib.h excerpt */

/* little endian type unsafe */
int lehmer1(int n)
{
int r = GINT32_TO_LE(1);

while (n--)
r = GINT32_TO_LE((16807 * GINT32_FROM_LE(r)) % 2147483647);
return GINT32_FROM_LE(r);
}
/* little endian type safe */
int lehmer2(int n)
{
union le_int r = h2l_int(1);

while (n--)
r = h2l_int((16807 * l2h_int(r)) % 2147483647);
return l2h_int(r);
}
/* big endian type unsafe */
int lehmer3(int n)
{
int r = GINT32_TO_BE(1);

while (n--)
r = GINT32_TO_BE((16807 * GINT32_FROM_BE(r)) % 2147483647);
return GINT32_FROM_BE(r);
}
/* big endian type safe */
int lehmer4(int n)
{
union be_int r = h2b_int(1);

while (n--)
r = h2b_int((16807 * b2h_int(r)) % 2147483647);
return b2h_int(r);
}

int main()
{
printf("%d\n",lehmer1(2112));
printf("%d\n",lehmer2(2112));
printf("%d\n",lehmer3(2112));
printf("%d\n",lehmer4(2112));
return 0;
}


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: 2112 on December 01, 2012, 07:03:41 PM
FWIW while I've never tried doing it QEMU can apparently emulate the big-endian PowerPC architecture. Here (http://dcreager.net/2010/05/13/powerpc-qemu-lucid/) is one guys notes about getting it working from two years ago, specifically to test endian issues.
Thanks. The other option for big-endian testing on Intel little-endian laptop is running an IBM mainframe Linux eg. Debian or RedHat S390 on the Hercules emulator.

http://www.hercules-390.org/


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on December 03, 2012, 10:28:59 PM
Recent progress:

  • libccoin now signs transactions.  Handling chain re-org is now the last hurdle for proper backend client support
  • Building on older Debian/Ubuntu Linux now supported



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: gmaxwell on December 04, 2012, 02:42:11 AM
The "somewhat odd" comment is really strange. It is the standard way to strenghten the C type system from structural equivalence to name equivalence. The type systems taxonomy is the standard curriculum of pretty much every CS or SE course at the university level. In fact the C++ code I posted above is patterned after the sample C/C++ code and a paper that was included with the "cfront" C++ compiler distribution from Bjarne Stroustrup that I've seen on some dusty VAX at my school. I didn't claim the autorship and in fact the requirement of not overusing type-cast-ing is one of the standard admonishments in the C/C++ project specifications.

Nice lecture, I suspect that you didn't get my argument or I misunderstood what you're insisting on.  I'm not saying that using the typesystem and avoiding typecasting all over the code is weird, I'm saying that endenanness is usually compartmentalized— you can handle it cleanly when serializing and deserializing and only at those boundaries, and in the manner specified for the kind of serialization being performed. Everywhere is just a single fundamental type, and there is no need to carry a union around and to pollute the whole codebase with type specifiers which are not only not useful but acutely dangerous— as adhoc serialization using the alternative references littered in the codebase may miss the required byteorder functions.

Programming is foremost an art of communicating between programmers. Forcing every machine integer into a specialized union type when the only conversions needed are at specialized serialization points weakens that communication and probably increases the frequency of errors. It certainly isn't the convention in the OSS C ecosystem.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: paybitcoin on December 05, 2012, 09:55:46 AM
Hey, thanks for libccoin!!! I used it for a brute forcer for Casascius's BIP38 contest and it was very clearly laid out (at least from what I used on the library side.)

I did find a possible bug, though:

When generating a public key and Bitcoin address from a private key, the EC key is using the compressed form, when I believe it should be using the uncompressed form. I added this workaround here, but I don't know if it is the right way to go.

https://github.com/notespace/picocoin/commit/736235acabf08f2eae8856673bfd68574c8600fe

I am generating the private key bytes from the outputs of the EC multiply operation required by BIP 38, then using it like this:

bp_key_init(&wallet)
bp_key_secret_set(&wallet,finalKey,32)
bp_pubkey_get(&wallet, ((void **) &pubKey), &pubKeylen);
btcAddress = bp_pubkey_get_address(&wallet, 0);

Also, you can see the whole code for the bip38 cracker here, all code is in main.c:

https://github.com/notespace/bip38-cracker


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on December 05, 2012, 05:17:01 PM
Hey, thanks for libccoin!!! I used it for a brute forcer for Casascius's BIP38 contest and it was very clearly laid out (at least from what I used on the library side.)

Thanks for giving it a try.

Quote
I did find a possible bug, though:

When generating a public key and Bitcoin address from a private key, the EC key is using the compressed form, when I believe it should be using the uncompressed form. I added this workaround here, but I don't know if it is the right way to go.

https://github.com/notespace/picocoin/commit/736235acabf08f2eae8856673bfd68574c8600fe

The developer consensus seems to be that uncompressed form just wastes bytes in the blockchain, which is true, so libccoin was coded to always used compressed form.

However, it might make sense to support uncompressed public keys for use cases such as yours, as long as the code still produces compressed form by default.

Such a change would involve adding "bool uncompressed" (default: false) to struct bp_key, and behaving accordingly based on that.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on December 05, 2012, 06:39:17 PM
A bit of a non-announcement, as it Just Worked(tm), but picocoin is happily talking to IPv6 nodes on the network.

Tor support (SOCKS5) should be added sometime soon.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: ffcitatos on December 07, 2012, 02:01:08 PM
A total lamer here: if I have a private key of a bitcoin address, that has 1 BTC in it, can I spend it with picocoin? Or does it only know about transactions that are broadcast to it? Thank you.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on December 07, 2012, 02:43:00 PM
A total lamer here: if I have a private key of a bitcoin address, that has 1 BTC in it, can I spend it with picocoin? Or does it only know about transactions that are broadcast to it? Thank you.

In the future, when picocoin is complete, yes, you can spend it with picocoin.

Right now, while picocoin is under construction, that is not yet implemented.

picocoin will provide everything a bitcoin client should provide: you can receive and spend bitcoins.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Nachtwind on December 22, 2012, 10:09:57 AM
Any progress on Picocoin? Been following this thread for a while ut it seems to be asleep for a few weeks now.. and if there is development. Any idea about compiling on windows?


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on December 22, 2012, 11:31:58 PM
Any progress on Picocoin? Been following this thread for a while ut it seems to be asleep for a few weeks now.. and if there is development. Any idea about compiling on windows?

Many devs, myself included, are on winter break ;p



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Nagato on December 30, 2012, 03:00:04 PM
Everything builds and tests pass on MacOS X 10.7.  Good job on this code, you've written the client Satoshi should have.

Satoshi could not have written a light client...


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: em3rgentOrdr on April 06, 2013, 10:49:22 AM
I downloaded the lastest code and was able to compile and run on both my x86-64 machine and my Raspberry PI arm marchine.  However, when I try running netsync, both machines fail and say all the ip addresses are unrechable:

Quote
socket connect 2a02:348:5e:5a29::1: Network is unreachable
net: failed to start connection to 2a02:348:5e:5a29::1
net: connect 69.164.196.4 failed: Connection refused

Regular bitcoin client works fine on my x86-64 machine, so I don't think my router blocking ports could be the problem.  Any ideas?

(I am wanting to turn my PI into a lightweight pay-to-download fileserver...so any suggestions are helpful.  I'd want to set it up so people can download a file after their bitcoin transaction gets a couple confirmations.)


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: rupy on May 03, 2013, 08:05:54 AM
Hey I'm onto this RPi with free colocation path aswell.

Bitcoind consumes to much memory and diskspace to be a viable option.

I'll try and compile this to see if it runs ok.

When will payment be added?

Edit: I'm a complete noob, how do I build picocoin?

sudo apt-get install libevent-dev libjansson-dev automake libglib2.0-dev


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Chaoskampf on June 26, 2013, 04:36:44 AM
Have there been any advances lately? It's a really awesome project and I hope it reaches a stable release sometime soon. Great work jgarzik!



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on June 26, 2013, 12:07:58 PM
Have there been any advances lately? It's a really awesome project and I hope it reaches a stable release sometime soon. Great work jgarzik!

Some slow advances.  The library is seeing use in various places, as the base for custom bitcoin apps that need to be really fast.

The two clients, "brd" (block relay daemon) and "picocoin" (SPV client) remain incomplete and have not seen much attention.

The library, libccoin, is largely feature complete and available for use today.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Aido on November 30, 2013, 08:45:54 PM
Is this project dead or stalled? I see that github hasn't had a commit in over 2 months now.

Also, any advice on whether incorporating Sophia (http://www.sphia.org/) is a good or a bad idea?


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on November 30, 2013, 09:09:20 PM
Is this project dead or stalled? I see that github hasn't had a commit in over 2 months now.

Stalled, though I am aware of at least two people working on embedded projects based on libccoin.

This Christmas I'm hoping to finish the client.

Quote
Also, any advice on whether incorporating Sophia (http://www.sphia.org/) is a good or a bad idea?

Seems worth checking out.  Most key/value databases cannot scale to the millions of records and data access patterns that bitcoin requires, so it requires careful evaluation.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Aido on December 02, 2013, 12:55:14 AM
Most key/value databases cannot scale to the millions of records and data access patterns that bitcoin requires, so it requires careful evaluation.

Indeed; but the Sophia benchmarks (http://www.sphia.org/benchmarks.html) compared to LevelDB look promising, if they are to be believed.

LMDB (http://symas.com/mdb/) looks interesting too. Not sure about the OpenLDAP license though.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Nummum on January 06, 2014, 09:03:26 PM
I am thinking of using the libccoin and picocoin code as the basis of an alternative currency I am developing.

More details here: https://bitcointalk.org/index.php?topic=399728.0

The main purpose of this alt currency is educational for me and to be a challenge to which I hope I can rise. I do not want to follow the same route as a lot of other alt currencies, cloning the bitcoin code and making a few minor changes. Not much can be learned from this.

Before I jump in and start writing code I would like an expert opinion. For the type of altcoin I intend to develop is libccoin/picocoin a good foundation to build upon or am I wasting my time? I know the picocoin code is not complete but that is part of the challenge I have set myself.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Aido on January 09, 2014, 09:18:19 PM
From the Bitcoin with Raspberry Pi (https://bitcointalk.org/index.php?topictopic=191729)thread:

I had bitcoind 0.8.1 running on a Raspberry Pi and performance was terrible.....very high CPU usage and too many r/w operations.

Jeff Garzik, one of the core bitcoin developers, is writing a lightweight C library called libccoin (https://github.com/jgarzik/picocoin) and a client called picocoin (https://github.com/jgarzik/picocoin).

When finished this should be suitable for running on low powered devices like a Raspberry Pi.

Does anyone know the status of picocoin? I can't wait for a stable version. It would help my project immensely.


There are a few people (i.e. me and someone else  :) ) interested in getting picocoin running on a Raspberry Pi. Has there been any further progress of late?



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on January 10, 2014, 12:35:35 AM
No recent progress, but any and all pull requests will be reviewed, tested and merged with lightning speed :)


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: thomas92911 on January 11, 2014, 06:58:38 AM
Nice work, thanks. :D


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on January 11, 2014, 07:33:04 PM
Minor update for a script parsing bug pushed to git.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Aido on January 11, 2014, 08:48:23 PM
A poor first attempt at writing some TX_MULTISIG code here:

https://github.com/aido/picocoin/commit/d6635879c1f6ab4812a0b123db3800d555c2993d

It's probably not worthy of a pull request but there may be something of use that can be copied and pasted into the main code.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on March 22, 2014, 01:53:39 PM
Added two new utilities, "txmod" and "blkstats"

Added some P2SH scripting ability, also.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on March 22, 2014, 01:55:41 PM
A poor first attempt at writing some TX_MULTISIG code here:

https://github.com/aido/picocoin/commit/d6635879c1f6ab4812a0b123db3800d555c2993d

It's probably not worthy of a pull request but there may be something of use that can be copied and pasted into the main code.

It's a good start.  The main bug there is that 'pubkeys' is plural.  You have to parse multiple entities out of the script to match a single "OP_PUBKEYS"  That requires a bit more reworking of the script parser, which fundamentally assumes there is a 1:1 ratio between opcodes and script data items.  With "OP_PUBKEYS", that assumption fails.

(I was looking at this last night :))


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: fbueller on March 22, 2014, 02:35:40 PM
Hey folks, wondering if anyone here could help me with this: https://bitcointalk.org/index.php?topic=526089

I was looking at libbitcoin, it seems pretty great, although I haven't looked at C++ in years.. Would this be easy to do: a parser for the blockchain to load every transaction into a mongodb database? I'd like to decode the transactions as they're read and log some information about each.

Once I have all the old transactions done I'd just write something simple in PHP to continuously add information as it happens, but there's no quick way to decode every single transaction if I'm asking bitcoind to decode the transaction every time.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Aido on March 22, 2014, 07:14:36 PM
Added two new utilities, "txmod" and "blkstats"

Added some P2SH scripting ability, also.


Always good to see picocoin getting some more TLC  :)




Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on March 22, 2014, 07:17:25 PM
A poor first attempt at writing some TX_MULTISIG code here:

https://github.com/aido/picocoin/commit/d6635879c1f6ab4812a0b123db3800d555c2993d

It's probably not worthy of a pull request but there may be something of use that can be copied and pasted into the main code.

Just pushed out a commit that implements this.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Aido on March 22, 2014, 07:44:55 PM
Also Jeff,

You have seen my attempts at swapping out the OpenSSL dependency in favour of PolarSSL to facilitate having picocoin run on low powered and embedded devices.

PolarSSL developers have indicated that they plan on releasing a new update early next week: https://github.com/polarssl/polarssl/issues/71

This new release should mean that PolarSSL now implements all bitcoin requirements. The PolarSSL guys have been most helpful on this.

Once this new PolarSSL release drops I will be able to finish up my PolarSSL pull request fairly quickly after that. A period of rigorous testing should follow as removing the OpenSSL dependency is a fundamental change.





Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on March 22, 2014, 10:01:24 PM
Also Jeff,

You have seen my attempts at swapping out the OpenSSL dependency in favour of PolarSSL to facilitate having picocoin run on low powered and embedded devices.

PolarSSL developers have indicated that they plan on releasing a new update early next week: https://github.com/polarssl/polarssl/issues/71

This new release should mean that PolarSSL now implements all bitcoin requirements. The PolarSSL guys have been most helpful on this.

Yep, looking forward to it.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Aido on March 22, 2014, 10:30:30 PM
I noticed that PolarSSL also provides some networking functionality like:

Code:
net_connect()
net_bind()
net_accept()
net_set_block()
net_set_nonblock()
net_usleep()
net_recv()
net_send()

I know picocoin's networking functionality works perfectly well and it's probably a case of "if it ain't broke, don't fix it". But I might investigate getting PolarSSL to do some of the networking heavy lifting if possible. It may turn out to be a quick and easy way of making picocoin IPv6 compatible (I haven't looked too closely at the networking code yet but I don't think it is currently).

Then again, I probably shouldn't be getting ahead of myself and should just wait until the PolarSSL crypto related stuff is working first.  :)


EDIT:

OK, a quick scan of the picocoin network code shows that picocoin may already be IPv6 compatible. Ignore above statements.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: hu_ciao on March 31, 2014, 11:07:21 AM
No recent progress, but any and all pull requests will be reviewed, tested and merged with lightning speed :)


I look forward to it. Thanks!


Been trying to compile picocoin on tinycore (http://tinycorelinux.net).

Complied jansson package ok. Then started with picocoin.

I did 'autoreconf -vi' and './configure'. All good.
Did not get to 'make DESTDIR=/mnt/sdb1/tmp/picocoin install-strip'.

Shame, as this looks like a great little package. I searched for and found it!
Now to test drive... I hope the SPV client is working (or soon)!


Fatal make errors are reported at 'make test' (and so 'make install' will also fail).

My c-coding is weak, so as to not make a mess of it I ask that someone take a look ???
Perhaps my method is obviously wrong?

Rather than register at github I posted here. OK?


Here is the end of 'make test' output:

<output>
Making check in lib
make[1]: Entering directory `/mnt/sdb1/picocoin-master/lib'
gcc -DHAVE_CONFIG_H -I. -I.. -I/usr/local/include/glib-2.0 -I/usr/local/lib/gli
b-2.0/include   -I../include    -march=i486 -mtune=i686 -Os -pipe -MT addr_matc
h.o -MD -MP -MF .deps/addr_match.Tpo -c -o addr_match.o addr_match.c
In file included from addr_match.c:11:
../include/ccoin/compat.h:26: error: static declaration of 'g_list_free_full' f
ollows non-static declaration
/usr/local/include/glib-2.0/glib/glist.h:53: note: previous declaration of 'g_l
ist_free_full' was here
make[2]: *** [addr_match.o] Error 1
make[2]: Leaving directory `/mnt/sdb1/picocoin-master/lib'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/mnt/sdb1/picocoin-master'
make: *** [all] Error 2
</output>

Seems these pages refer to something along the same lines.
--error: static declaration of 'foo' follows non-static declaration--


http://stackoverflow.com/questions/3148244/static-declaration-follows-non-static-declaration

https://www.linuxquestions.org/questions/programming-9/error-previous-declaration-of-xxxxx-was-here-627195/




Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on April 01, 2014, 03:47:30 AM
../include/ccoin/compat.h:26: error: static declaration of 'g_list_free_full' follows non-static declaration
/usr/local/include/glib-2.0/glib/glist.h:53: note: previous declaration of 'g_list_free_full' was here

Diagnosis:  Typically a system either (a) needs compat.h help due to older GLib, or (b) does not need such help.  All systems should be either "a" or "b".  Your system configured as both "a" and "b".

The most likely fix is to make sure you have GLib > 2.30, properly installed with pkg-config support.

Alternately, you may try manually editing compat.h to comment out the offending lines, and see if that works for you.
Code:
#if ((GLIB_MAJOR_VERSION == 2) && (GLIB_MINOR_VERSION < 30))
...
#endif /* GLIB_VERSION < 2.30 */


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: hu_ciao on April 04, 2014, 07:46:29 PM
@ jgarzik

Thanks for doing the diagnosis. It was my fault, of course.

Made no headway for quite a while tried changing... and many other fails. Gave up those routes.
Compiled new glib 2.40 from source.

Eventually it compiled very quickly ;)

Now I have it running. Test drive time.
I really like it. Keep up the good work!

Being able to move the existing bitcoid (etc.) wallet to picocon would encourage users to try it.
Maybe one should be able import the wallet-addr to picocon.wallet?

Is this possible?

It doesn't matter to me, as I have no stacks of coins to move or spend yet.
I'm still trying out the software and reading up. Learning.

Netsync still times out -- must get the right timeout value, I guess.
$ picocoin net.connect.timeout=240 netsync

You wrote on Github:
   
Quote
This is unfortunately normal -- you should try to addnode. Otherwise it takes a while to cycle through addresses, until it hits a live node.

The network peer manager code needs improvement.

Noob question: Which node/how do i find it?

Does picocoin connect through a proxy like polipo/privoxy or tor/SOCKS yet? This will be a popular feature.

Can we use nickname with 'picocoin new-address'? How about delete after address used/emptied of coins?

A secure way to "run it" on a MSWin system is from a quemu image, possibly on a removable drive.
Vbox (or portable vbox) is faster, but quemu doesn't require administrative privileges.
MSWin users can still get fork.

Of course TC and other small linux OSs can be booted from usb... without emulation or virtualization.

Maybe these are silly ideas. I'm sure you've heard them all.

I made a small script to get the passphrase and put it into the environmental var for the shell instance.
Saves me typing export... and reminds me not to use the window for anything else.

I wish I could help, but can not write -- read only  :-[



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: d_oilen on April 05, 2014, 11:41:34 AM
Hi,
i pulled down and tried to compile picocoin on my system.
Linux XXXXXXXXXXX 2.6.32-431.5.1.el6.i686 #1 SMP Tue Feb 11 21:56:33 UTC 2014 i686 i686 i386 GNU/Linux ( centos 6.5 final)
now with the latest and greatest i found the following small mistakes:

1. in brd.c you need to include socket.h and netinet/in.h in order to successfuly compile.
2. in blkscan.c and most probably in many other places you create a dependency on glib 2.28 to 2.30. if i might suggest:
 in blkscan.c "print_txout(bool show_from, unsigned int i, struct bp_txout *txout)" you can do the following:
 
Code:
                #if GLIB_CHECK_VERSION (2, 28, 0)
   g_list_free_full(addrs.pub, g_buffer_free);
   g_list_free_full(addrs.pubhash, g_buffer_free);
#else
  //addrs.pub cleanup
  g_list_foreach (addrs.pub, (GFunc) g_buffer_free, NULL);
  g_list_free (addrs.pub);
  //addrs.pubhash cleanup
  g_list_foreach (addrs.pubhash, (GFunc) g_buffer_free, NULL);
  g_list_free (addrs.pubhash);
#endif

3. similar as above in txmod.c "mutate_outputs(void)" you can:

Code:
if (!tx.vout){
#if GLIB_CHECK_VERSION (2, 30, 0)
tx.vout = g_ptr_array_new_full(8, g_bp_txout_free);
#else
tx.vout = g_ptr_array_sized_new (8);
g_ptr_array_set_free_func (tx.vout, (GDestroyNotify)g_bp_txout_free);
#endif
}


as a side note, be aware that on centos systems the stock libevent is not working ( it is at version is 1.4)  and you have to install libevent 2.x from source

regards,
JD


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: hu_ciao on April 11, 2014, 05:34:21 AM
Still getting timeout errors and no connection. My mistake?

Tried 'picocoin addnode=012.345.678.9 8333 netsync', but does not (yet) succeed. I used addresses from dns-seeds.

The best I can get is a long list of peers (wc -l picocoin.peers gives almost 2000); dns-seeds lists 97-131 each time.
Also got a few 'llnet: ... read EOF' results in among the 'net: ... connection timeout' results.

Anyone else succeeding/failing to get a picocoin.blkdb with the current version? Mine is still 0 KB.


Made a few icons and 'nix Bourne shell scripts to help run picocoin.
Icons made with ImageMagick on Debian.

Scripts should go in /usr/sbin/, or wherever your script bin is. Maybe they're useful for somebody?  
Extraxt with 'tar -xvjf buttn.tar.bz2'. Sorry, no scripts for msWin users (-_-)
...maybe Darwin (mac) can use the scripts? OK for Android, too.

I hope this encourages testing, exploration, and development of the project.

One script, pcoin.sh, loads a new CLI window and runs the second script, picocoin.sh. Aterm is called in pcoin.sh and a code example for rxvt is commented. One might rename it to the shorter pcoin, as there's no conflict with the program name (also p is the SI prefix for pico).
Edit the scripts as you like.

The picocoin.sh makes '.picocoin' directory, if not yet created, in users' home and changes to it.
User is asked for a passphrase, confirmed by retyping. That's exported to environmental variable in a new subshell.
The command prompt waits for picocoin commands.

When done there is no need to 'cd' back to original directory.
With command 'exit' the window closes if run from pcoin.sh or clears if picocoin.sh is run alone. Either way the passphrase is no longer in memory. Try 'echo $PICOCOIN_PASSPHRASE' to confirm.

I prefer to have the config and generated files in one place, uncluttered, hidden from view. We don't need regular access exept through the Bitcoin client. This also prevents the dropping of files all over the filesystem when picocoin is started from different curdirs.


Included with scripts are icons and a toy html page to show/play with the icons. The first 'button' has a radio option where down image is/not very small.
Click and hold down a second, release, and wait...

Eagerly awaiting developments.


Have fun :)


File: buttns.tar.bz2   51787 B (52 KB)
(P2P Torrent)

magnet:?xt=urn:btih:ee75467fd552464eeb266bf5bf46fd2581e46fa5&dn=buttns.tar.bz2

[edit removed trackers from torrent]


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on April 13, 2014, 12:39:53 AM
Hi,
i pulled down and tried to compile picocoin on my system.
Linux XXXXXXXXXXX 2.6.32-431.5.1.el6.i686 #1 SMP Tue Feb 11 21:56:33 UTC 2014 i686 i686 i386 GNU/Linux ( centos 6.5 final)
now with the latest and greatest i found the following small mistakes:

1. in brd.c you need to include socket.h and netinet/in.h in order to successfuly compile.
2. in blkscan.c and most probably in many other places you create a dependency on glib 2.28 to 2.30. if i might suggest:
 in blkscan.c "print_txout(bool show_from, unsigned int i, struct bp_txout *txout)" you can do the following:
 
Code:
                #if GLIB_CHECK_VERSION (2, 28, 0)
   g_list_free_full(addrs.pub, g_buffer_free);
   g_list_free_full(addrs.pubhash, g_buffer_free);
#else
  //addrs.pub cleanup
  g_list_foreach (addrs.pub, (GFunc) g_buffer_free, NULL);
  g_list_free (addrs.pub);
  //addrs.pubhash cleanup
  g_list_foreach (addrs.pubhash, (GFunc) g_buffer_free, NULL);
  g_list_free (addrs.pubhash);
#endif

3. similar as above in txmod.c "mutate_outputs(void)" you can:

Code:
if (!tx.vout){
#if GLIB_CHECK_VERSION (2, 30, 0)
tx.vout = g_ptr_array_new_full(8, g_bp_txout_free);
#else
tx.vout = g_ptr_array_sized_new (8);
g_ptr_array_set_free_func (tx.vout, (GDestroyNotify)g_bp_txout_free);
#endif
}


Thanks for pointing out several locations incompatible with older GLib versions.

In general, #ifdefs directly placed in the library code are to be avoided.  The pattern to follow is updating compat.h with

Code:
#if VERSION < X
  provide wrapper or full implementation of missing function
#endif

If there is a missing function, add the function to compat.h, inside a version check.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: hu_ciao on April 16, 2014, 08:21:57 PM
A bit more detail on test of picocoin.

picocoin debug netsync &> debug.txt

net: parent exec NC_START
net: have 46188/46188 peers
net: blkdb opened
net: gc'd 0 connections
net: open connections (have 0, want 8 more)
...     [ tries/connects to 8 ips ]
net: parent after NC_START
net: engine started. sleeping 10 minutes (cxn tmout 11 sec)
net: connected to 68.224.[scrubbed]
net: 68.224.[scrubbed] message version
net: 68.224.[scrubbed] version(70002, 0x1, 1397475165, To:[scrubbed], From:68.224.[scrubbed], /Satoshi:0.9.0/, 295782)
net: 68.224.[scrubbed] message verack
net: 68.224.[scrubbed] message alert
net: 68.224.[scrubbed] message inv
...     [ first timeout error reported - it's the first ip tried/connected ]
net: gc'd 1 connections
net: open connections (have 7, want 1 more)
...     [ keeps trying connections as previous connections timeout ]
net: 68.224.[scrubbed] message inv
net: 68.224.[scrubbed] message addr
net: 68.224.[scrubbed] addr(2 addresses, 0 old)
...     [ as before: new made, old dropped ]
...     [ a few ipv6 fails: '' ]
socket connect [scrubbed]:  Network is unreachable
net: failed to start connection to [scrubbed]
...
net: gc'd 1 connections
net: open connections (have 7, want 1 more)
net: connecting to [scrubbed]
net: 68.224.[scrubbed] message inv
net: stopping engine
peerman: 46707 peers to write

(Got 53982 peers last time tried.)


Looked at net connections: netstat -tp | grep picocoin  (ran in loop).

Shows only one ESTABLISHED others are SYN_SENT.

Not likely a firewall block, as picocoin talks to the one.
Connections being ignored?
Get SYN_SENT status only before timeouts; no further SYN-ACK or ACK steps in the connection handshake ...except for the one.

Can get other IPs to connect with block relay daemon (brd). Timeout stderr messages persist, though.

Adding an address which connected with brd doesn't increase number of connections made by picocoin; still speaks only to the one address.

Thanks for today!


(Not complaining, just saying.)


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on April 16, 2014, 09:52:40 PM
Because of stupid node selection, it takes a while to cycle through dead addresses passed to your node from other nodes.  You can use addnode etc.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Aido on May 11, 2014, 08:44:39 PM
Hi Jeff,

I'm trying to get my head around the brd.c code with a view to introducing LMDB (http://symas.com/mdb/) to the picocoin project.

There's a few things I don't yet get. It seems that brd no longer stores blocks in brd.blkdb (it's commented out in code) but uses brd.blocks instead. But, there are still attempts to write to brd.blkdb, like calling blkdb_add(), which I think will always fail.

So, can the blkdb.c code be removed or does it just need a tidy-up?


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on May 12, 2014, 11:54:41 AM
Hi Jeff,

I'm trying to get my head around the brd.c code with a view to introducing LMDB (http://symas.com/mdb/) to the picocoin project.

There's a few things I don't yet get. It seems that brd no longer stores blocks in brd.blkdb (it's commented out in code) but uses brd.blocks instead. But, there are still attempts to write to brd.blkdb, like calling blkdb_add(), which I think will always fail.

So, can the blkdb.c code be removed or does it just need a tidy-up?

blkdb is required and is always used.  It is the block header database.  It is kept in RAM, as bitcoind does.  It is optionally stored in a file, in addition to RAM.  It is optionally regenerated from the 17+GB blockchain file, if blkdb is missing.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Cryddit on May 12, 2014, 02:19:19 PM
What I don't see in these libraries, that I would very much like to see, is comments that say which things you cannot touch or what kind of changes you can make in the block structure, without breaking compatibility with existing ASIC miners. 

Anybody know? 

Anybody know who knows? 



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Aido on May 13, 2014, 10:51:14 PM
blkdb is required and is always used.  It is the block header database.  It is kept in RAM, as bitcoind does.  It is optionally stored in a file, in addition to RAM.  It is optionally regenerated from the 17+GB blockchain file, if blkdb is missing.

OK, thanks for the explanation.

Along with some other handy features (http://symas.com/mdb/#features) LMDB can have multiple sub-databases. I haven't thought things out fully yet but blkdb could possibly be stored as a sub-database with the blockchain as another sub-database or collection of sub-databases, performance permitting. Or maybe plagiarise the libbitcoin database layout (https://wiki.unsystem.net/index.php/Libbitcoin/Blockchain#Databases) a bit.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Aido on May 19, 2014, 11:35:20 PM
There has been a noticeable decline in the number of active bitcoin nodes over the past few months. The following website has some nice graphs showing this decline:

https://getaddr.bitnodes.io/dashboard/chart/?days=60

Maybe picocoin is a possible solution to this declining trend.

If picocoin (or more specifically brd) is lightweight enough it could be built as a DD-WRT and/or OpenWRT package. We would then potentially have tens if not hundreds of thousands routers in peoples homes working as bitcoin relays.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on May 19, 2014, 11:53:46 PM
If picocoin (or more specifically brd) is lightweight enough it could be built as a DD-WRT and/or OpenWRT package. We would then potentially have tens if not hundreds of thousands routers in peoples homes working as bitcoin relays.

It is more a question of whether or not a platform is heavyweight enough to support a full node.

At first blush, you need enough data storage to store the entire blockchain (though perhaps mount over NFS), enough CPU to verify incoming data, and enough bandwidth to send and receive data.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Aido on May 26, 2014, 08:28:10 PM
all pull requests will be reviewed, tested and merged with lightning speed :)


ahem......#25 (https://github.com/jgarzik/picocoin/pull/25) :)

On a more serious note, I mentioned previously that I was investigating introducing LMDB (http://symas.com/mdb/) to the picocoin project. Before delving too deeply into this should the OpenLDAP Public Licence (http://(http://www.openldap.org/doc/admin24/license.html) used by LMDB be a concern when used in a MIT/X11 software licensed (http://opensource.org/licenses/MIT) project like picocoin?





Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: KaChingCoinDev on May 27, 2014, 11:53:18 PM
This is interesting. How close to the regular bitcoin code is it?


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Aido on May 28, 2014, 06:32:49 PM
How close to the regular bitcoin code is it?

  • Supports all core data structures and network messages
  • Full script implementation
  • Passes hundreds of available reference client tests
  • Supports multiple block chains: main or testnet3
  • Written by bitcoin core developer



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Aido on June 08, 2014, 08:40:16 PM
In light of the current "51% attack" brouhaha raging on Reddit, how difficult would it be to add some p2pool mining functionality to the picocoin library?

All the core bitcoin and p2p functionality is there, ready to be built upon.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on June 10, 2014, 01:32:58 PM
In light of the current "51% attack" brouhaha raging on Reddit, how difficult would it be to add some p2pool mining functionality to the picocoin library?

It would take several days of work, but it's not complicated.  Basically, as you probably have guessed, P2Pool chain is simply another chain to manage.



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Bawaler on May 01, 2015, 05:10:56 PM
Gonna try this on my Mac Pro. Dual 970FX Big Endian, running Debian 8 smoothly. I've waited for this, just didn't notice this software earlier.

Aiming to have a pool running on the IBM Power BE architecture.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Wats0ns on May 01, 2015, 07:05:32 PM
Libccoin looks very useful, that's exactly what I was looking for my project, thanks a lot ;)
picocoin souds promising too, I will keep an eye on its developpement  :)


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on May 02, 2015, 07:46:02 PM
Some minor compile fixes pushed out to git.  Further OSX compile fixes coming soon.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: virtualx on May 03, 2015, 10:39:17 AM
I had some trouble locating all the packages. I was missing libjansson and libevent_core.
Is my repository out of date?


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on May 03, 2015, 04:49:46 PM
I had some trouble locating all the packages. I was missing libjansson and libevent_core.
Is my repository out of date?

jansson and libevent are both still needed.

This is what I have installed on Ubuntu:
Code:
ii  libevent-2.0-5:amd64                                        2.0.21-stable-1ubuntu1.14.04.1                      amd64        Asynchronous event notification library
ii  libevent-core-2.0-5:amd64                                   2.0.21-stable-1ubuntu1.14.04.1                      amd64        Asynchronous event notification library (core)
ii  libevent-dev                                                2.0.21-stable-1ubuntu1.14.04.1                      amd64        Asynchronous event notification library (development files)
ii  libevent-extra-2.0-5:amd64                                  2.0.21-stable-1ubuntu1.14.04.1                      amd64        Asynchronous event notification library (extra)
ii  libevent-openssl-2.0-5:amd64                                2.0.21-stable-1ubuntu1.14.04.1                      amd64        Asynchronous event notification library (openssl)
ii  libevent-pthreads-2.0-5:amd64                               2.0.21-stable-1ubuntu1.14.04.1                      amd64        Asynchronous event notification library (pthreads)
ii  libjansson-dev:amd64                                        2.5-2                                               amd64        C library for encoding, decoding and manipulating JSON data (dev)
ii  libjansson4:amd64                                           2.5-2                                               amd64        C library for encoding, decoding and manipulating JSON data

Not sure what's the process for OSX - macports?



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: jgarzik on May 06, 2015, 05:37:37 AM
The libccoin library no longer depends on GLib library.  That dependency has been completely removed.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: rupy on May 12, 2015, 06:10:58 PM
Could I use picocoin with JNI today to make payments with Java and use bitcoinj for the wallet parts?


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: 7queue on May 14, 2015, 05:51:00 PM
I had some trouble locating all the packages. I was missing libjansson and libevent_core.
Is my repository out of date?

jansson and libevent are both still needed.

This is what I have installed on Ubuntu:
Code:
ii  libevent-2.0-5:amd64                                        2.0.21-stable-1ubuntu1.14.04.1                      amd64        Asynchronous event notification library
ii  libevent-core-2.0-5:amd64                                   2.0.21-stable-1ubuntu1.14.04.1                      amd64        Asynchronous event notification library (core)
ii  libevent-dev                                                2.0.21-stable-1ubuntu1.14.04.1                      amd64        Asynchronous event notification library (development files)
ii  libevent-extra-2.0-5:amd64                                  2.0.21-stable-1ubuntu1.14.04.1                      amd64        Asynchronous event notification library (extra)
ii  libevent-openssl-2.0-5:amd64                                2.0.21-stable-1ubuntu1.14.04.1                      amd64        Asynchronous event notification library (openssl)
ii  libevent-pthreads-2.0-5:amd64                               2.0.21-stable-1ubuntu1.14.04.1                      amd64        Asynchronous event notification library (pthreads)
ii  libjansson-dev:amd64                                        2.5-2                                               amd64        C library for encoding, decoding and manipulating JSON data (dev)
ii  libjansson4:amd64                                           2.5-2                                               amd64        C library for encoding, decoding and manipulating JSON data

Not sure what's the process for OSX - macports?



Do you plan on staying with libevent? I would like to try this on *BSD/Solaris ...

Thanks!

8 )


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: polyclef on June 24, 2015, 09:57:19 AM

Not sure what's the process for OSX - macports?


These days, homebrew is the standard.  They do require a tagged release though.
"Formulae in the core repository must have a stable version tagged by the upstream project. Tarballs are preferred to git checkouts, and tarballs should include the version in the filename whenever possible."

If you do that, I'll make a formula and submit it.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: cloudboy on October 27, 2015, 06:08:27 PM
Is there a tutorial somewhere?

I can create addresses and start netsync, but how to check balance and use rawtx?

Why isn't there better documentation for this?


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: dako300 on December 26, 2015, 02:24:22 AM
Is there a tutorial somewhere?

I can create addresses and start netsync, but how to check balance and use rawtx?

Why isn't there better documentation for this?
I would love to see some documentation for this


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: 0bit on January 22, 2016, 04:54:47 AM
Any news on this project? I am trying to netsync although everything appears to be timing out or refusing. Even with no firewall. Is there any documentation someone can point me to?


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: buzztiaan on February 08, 2016, 03:38:45 PM
seems nobody is interested in adding documentation to this project :(

too bad, could have gotten some usage


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: cloudboy on February 16, 2016, 07:40:08 AM
seems nobody is interested in adding documentation to this project :(

too bad, could have gotten some usage

Yeah I agree, it's an interesting project.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Lucifer333 on April 19, 2016, 07:39:44 PM
I am trying to learn picocoin/bitcoin by looking at network communication and messaging between bitcoin clients.

picocoin netsync debug=true, doesnt show anything, why?


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Lucifer333 on April 19, 2016, 09:34:22 PM
HI i found this in brd.c

The "parr" suite of functions and parr structure is to replace the dependency on glibc??

Thanks

static void init_nci(struct net_child_info *nci)
{
        memset(nci, 0, sizeof(*nci));
        nci->read_fd = -1;
        nci->write_fd = -1;
        init_peers(nci);
        nci->conns = parr_new(NC_MAX_CONN, NULL); <----
        nci->eb = event_base_new();
}


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Aido on April 19, 2016, 09:43:01 PM
Hi Lucifer333,

This and other commits made to picocoin around this date may answer your question :


https://github.com/jgarzik/picocoin/commit/691534ece434ece9315aaca73dfb3974681970a4


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Lucifer333 on April 19, 2016, 09:45:10 PM
Hi Lucifer333,

This and other commits made to picocoin around this date may answer your question :


https://github.com/jgarzik/picocoin/commit/691534ece434ece9315aaca73dfb3974681970a4

Thanks) its very clear



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Lucifer333 on April 19, 2016, 10:58:55 PM
Hi , would it be possible to prevent the seeds from returning IPV6, My compu/ ISP fails all ipv6 addresses.

clist *bu_dns_lookup(clist *l, const char *seedname, unsigned int def_port)
{
        struct addrinfo hints, *res;

        memset(&hints, 0, sizeof(hints));
        /*hints.ai_family = AF_UNSPEC;*/   
        hints.ai_family = AF_INET;

        hints.ai_socktype = SOCK_STREAM;
        hints.ai_flags = AI_CANONNAME;

        if (getaddrinfo(seedname, NULL, &hints, &res))
                return l;

        struct addrinfo *rp;
        printf("canon-name:%s\n", res->ai_canonname);
        for (rp = res; rp != NULL; rp = rp->ai_next){
                l = add_seed_addr(l, rp, def_port);
        }

        freeaddrinfo(res);

        return l;
}


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Aido on April 19, 2016, 11:19:55 PM
Hi  Lucifer333,

Is it possible to enable IPv6 on your compu/ ISP?


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Lucifer333 on April 19, 2016, 11:21:17 PM
Hi  Lucifer333,

Is it possible to enable IPv6 on your compu/ ISP?

My adsl modem doesnt understand ipv6, I think many ppl still have the old hardware


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Lucifer333 on April 20, 2016, 12:28:57 AM
Hi  Lucifer333,

Is it possible to enable IPv6 on your compu/ ISP?

Maybe we can make it configurable with a command line option (default would be ivp6)?


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Aido on April 20, 2016, 12:36:49 AM
Yes, this would be doable.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Lucifer333 on April 20, 2016, 09:24:52 AM
Hi,

couple of things

1) I want to use argp in all executables (brd and picocoin doesnt use them). this would be nice because we can put documentation in   <exe> --help.
2) I want to put the origin of a peer (from which seeds it comes from, as defined in dns.c  "dns_seeds" array ) into the bp_address structure( char *pointer). Part of a bunch of statistics I want to reveil in my frontend.
   a) Peers only come from the dns_seeds and are not discovered by being forwarded through other peers ( aka bitorrent)

3) Make an option to only to get ipv4 addresses from bootstrap nodes; default ipv6

If you want i will do the work.


Regards,


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Lucifer333 on April 20, 2016, 09:45:44 AM
sorry, 2a is a question..... forget '?'  ))


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Lucifer333 on April 22, 2016, 03:10:25 AM
picocoin and brd are NOT BINARY executables?


#! /bin/sh

# picocoin - temporary wrapper script for .libs/picocoin
# Generated by libtool (GNU libtool) 2.4.2
#
# The picocoin program cannot be directly executed until all the libtool
# libraries that it depends on are installed.

#
# This wrapper script should never be moved out of the build directory.
# If it is, it will not operate correctly.

# Sed substitution that helps us do robust quoting.  It backslashifies
# metacharacters that are still active within double-quoted strings.



PS: I have libtool installed!!

sudo yum install libtool
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: centos.mirror.root.lu
 * epel: mirro[Suspicious link removed]t-systems.com
 * extras: centos.mirror.root.lu
 * ius: mirrors.ircam.fr
 * nux-dextop: mirror.li.nux.ro
 * rpmforge: miroir.univ-paris13.fr
 * updates: mirror.23media.de
Package libtool-2.4.2-21.el7_2.x86_64 already installed and latest version
Nothing to do


 



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Lucifer333 on April 22, 2016, 03:40:14 AM
comment on my prev question

normally I use static linkage  

noinst_LIBRARIES = xxxxx.a

xxxxx_a_SOURCES = ... list of sources


The picocoin/lib/makefile.am uses LT and it iseems to install the binaries in .libs

Not worked much with the LIBTOOL as i mostly build static libs

lib_LTLIBRARIES..mmm time to break out the autotools book)


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Lucifer333 on April 22, 2016, 10:15:23 PM
function "nc_conn_group_active" is a "dud" function?  I see it is called from "nc_conns_open"

Can someone explain the rational, you want to limit the number on a "group classification" basis?
.
.
   /* are we already connected to this network group? */
                if (nc_conn_group_active(nci, &conn->peer)) {
                        fprintf(plog, "net: already grouped to %s\n",
                                conn->addr_str);
                        goto err_loop;
                }

.
.


-- FUNCTION  nc_conn_group_active

static bool nc_conn_group_active(struct net_child_info *nci,
                                 const struct peer *peer)

{
        // FIXME
        return false;

        unsigned int group_len = peer->group_len;
        unsigned int i;
        for (i = 0; i < nci->conns->len; i++) {
                struct nc_conn *conn;

                conn = parr_idx(nci->conns, i);
                if ((group_len == conn->peer.group_len) &&
                    !memcmp(peer->group, conn->peer.group, group_len))
                        return true;
        }

        return false;
}


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Lucifer333 on April 22, 2016, 11:32:46 PM
questions questions questions

First i see that a new event (libevent) is created with when a "connect" is made with a socket, the event type is  EV_WRITE.  why not EV_READ aswell, ???

 /* add to our list of monitored event sources */
                conn->ev = event_new(nci->eb, conn->fd, EV_WRITE,
                                     nc_conn_evt_connected, conn);




Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Lucifer333 on April 23, 2016, 04:32:38 PM
Hi can some-one please confirm message sequence below?

Hi I am trying to understand the message sequence of BRD deamon, when initial connection is established

From the code i can see  (P1 initiates connection to P2)

P1 ---> connect--> P2
P1 --> sends 'version'  --> P2
P1 <--  reply 'version' -- P2   , in the code i see if the first received message is not "version" its an error
P1 --> sends verack -->P2   ,
P1 <-- reply verack <-- P2  , in code i see this must be the second message received
P1 -->request "getaddr" --> P2
P1 --> request "getblocks" --> P2

Is this correct so far? Thanks



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Lucifer333 on April 23, 2016, 08:30:56 PM
I added some logging to make the comminication more clear in brd
[20:27:53   GMT-23/04/2016][nc_conn_start][:818], connect start to:88.112.146.73
[20:27:54   GMT-23/04/2016][nc_conn_send][:315], attempting to send:[version] to 88.112.146.73
[20:27:55   GMT-23/04/2016][nc_conn_message][:665], version command received from 88.112.146.73.
[20:27:55   GMT-23/04/2016][nc_conn_send][:315], attempting to send:[verack] to 88.112.146.73
[20:27:56   GMT-23/04/2016][nc_conn_message][:678], verack command received from 88.112.146.73.
[20:27:56   GMT-23/04/2016][nc_conn_send][:315], attempting to send:[getaddr] to 88.112.146.73
[20:27:56   GMT-23/04/2016][nc_conn_send][:315], attempting to send:[getblocks] to 88.112.146.73
[20:27:56   GMT-23/04/2016][nc_conn_message][:691], addr command received from 88.112.146.73.
[20:27:57   GMT-23/04/2016][nc_conn_message][:697], inv command received from 88.112.146.73.
[20:28:03   GMT-23/04/2016][nc_conn_message][:697], inv command received from 88.112.146.73.
[20:28:03   GMT-23/04/2016][nc_conn_send][:315], attempting to send:[getdata] to 88.112.146.73
[20:28:03   GMT-23/04/2016][nc_conn_message][:697], inv command received from 88.112.146.73.
[20:28:04   GMT-23/04/2016][nc_conn_message][:697], inv command received from 88.112.146.73.
[20:28:04   GMT-23/04/2016][nc_conn_message][:697], inv command received from 88.112.146.73.
[20:28:16   GMT-23/04/2016][nc_conn_message][:691], addr command received from 88.112.146.73.
[20:28:16   GMT-23/04/2016][nc_conn_message][:697], inv command received from 88.112.146.73.
[20:28:16   GMT-23/04/2016][nc_conn_message][:697], inv command received from 88.112.146.73.
[20:28:16   GMT-23/04/2016][nc_conn_message][:697], inv command received from 88.112.146.73.
[20:28:16   GMT-23/04/2016][nc_conn_message][:703], block command received from 88.112.146.73.
brd: blkdb add fail  (havent figured this out why this happens)
blocks: process-block failed



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Lucifer333 on April 25, 2016, 08:03:29 AM
HI,

I see some seemingly contradiction in the code and documentation in the develope reference of bitcoin

https://bitcoin.org/en/developer-reference#data-messages

The currently-available type identifiers are:
Type IdentifierNameDescription
1MSG_TXThe hash is a TXID
2MSG_BLOCKThe hash is of a block header.
3MSG_FILTERED_BLOCKThe hash is of a block header; identical to MSG_BLOCK. When used in a getdata message, this indicates the response should be a merkleblock message rather than a block message (but this only works if a bloom filter was previously configured). Only for use in getdata messages.
Type identifier zero and type identifiers greater than three are reserved for future implementations. Bitcoin Core ignores all inventories with one of these unknown types.


Now if we look at code we see that type (3) is not mentioned in the code, when processing "inv" command.  (module brd.c)

 switch (inv->type) {
                case MSG_TX: strcpy(typestr, "tx"); break;
                case MSG_BLOCK: strcpy(typestr, "block"); break;
                default: sprintf(typestr, "unknown 0x%x", inv->type); break;
                }

Why does the brd.c not use MSG_FILTERED_BLOCK??  I am trying to understand the code with the bitcoin dev wiki.
Thanks for any answers you may give.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: marcopal on September 30, 2016, 03:06:11 PM
I am trying to run picocoin (bitsy) as an SPV client.
I create a wallet and then I run the netsync command.
I always get an empty picocoin.blkdb file.
am I doing anythin wrong?


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Aido on September 30, 2016, 05:02:51 PM
Hi marcopal,

 Maybe this helps.......

Hi Jeff,

I'm trying to get my head around the brd.c code with a view to introducing LMDB (http://symas.com/mdb/) to the picocoin project.

There's a few things I don't yet get. It seems that brd no longer stores blocks in brd.blkdb (it's commented out in code) but uses brd.blocks instead. But, there are still attempts to write to brd.blkdb, like calling blkdb_add(), which I think will always fail.

So, can the blkdb.c code be removed or does it just need a tidy-up?

blkdb is required and is always used.  It is the block header database.  It is kept in RAM, as bitcoind does.  It is optionally stored in a file, in addition to RAM.  It is optionally regenerated from the 17+GB blockchain file, if blkdb is missing

Actually, that reminds me. I never did have a go at adding LMDB to picocoin.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: marcopal on October 01, 2016, 08:02:37 AM
thank you for the replay Aido,

I read that post but I thought it was related to the brd client that, if I understand, should be implementing a full node.
picocoin executable should be implementing the SPV node instead.
am I correct?
I will give a more deep read to the code..


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: Aido on October 01, 2016, 12:09:49 PM
Hi marcopal,

If you look at the code you will see that both the client (picocoin) and the block relay daemon (brd) both have a init_blkdb() function. This creates the emplty blkdb file.

brd also has a block_process() function which uses blkdb_add() for adding blocks to the file. This is what my query to Jeff was about in the previous post.

The client (picocoin) does not have a block_process() function so nothing more is done to the blkdb file after it is created.


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: marcopal on October 04, 2016, 08:05:06 AM
thank you Aido.

how much effort, in your opinion, will take to have a full featured SPV node and HD wallet starting from the picocoin code?
do you know about any other SPV implementations in pure C?



Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: marcus_of_augustus on October 12, 2016, 10:37:46 AM
thank you Aido.

how much effort, in your opinion, will take to have a full featured SPV node and HD wallet starting from the picocoin code?
do you know about any other SPV implementations in pure C?


I don't think there are any SPV clients in pure C out there.

mSIGNA https://github.com/ciphrex/mSIGNA (https://github.com/ciphrex/mSIGNA) is a c++ SPV client that is feature-rich


Title: Re: [ANNOUNCE] picocoin and libccoin -- C-based bitcoin library and client
Post by: marcopal on October 12, 2016, 02:06:52 PM
thank you marcus_of_augustus