For starters, you'll have heck of a time to remove your dependency on glib.h from GTK. But there's more. You could actually try to compile your code from within the Linux kernel tree to see for yourself.

For a die-hard Linux developer like you I think the minimum definition of bare-metal would be uClinux. Full Linux doesn't count anymore for a true embedded development.

Both GLib and OpenSSL are heavy duty harvester combines. Removing the dependencies on them will at least halve the resource usage.

For a RedHat-ter at least make the code work with newlib instead of glib. The required knowledge should be no more than the internal phone call away for you. This includes a whole sermon on why you don't want to build a dependency on them.

As I've noted: I've forgot that you are one of the very few Bitcoin developers with regular access to big-endian hardware. I'm thinking that you will continue to be the only one developer who can test picocoin on the big-endian architecture. The mojibake-endian problem will continue to badly affect the future contributors/users of your library if you aren't going to use your own medicine (that is "sparse" and "__le32")

The performance loss for passing unions around may have been true for GCC 2.96. I'm not normally working with Linux kernel and/or GCC.

But I have a great, positive experience with code of the following style(C++ used for brevity, ... expands to long #ifdef trees using platform- and compiler-specific intrinsics)
The explicit strong-typing of endianness blocks the vast majority of possible endian errors and allows to produce meaningfull, endian-bug-free contributions from people who are exclusively using little-endian hardware. In my (rather extensive) experience it actually increases the performance because programmers can visualize the data structures better and can spend more time designing the algorithms instead of debugging trivial omissions or transpositions.

All in all, I apologise for jumping in into the battle between you and MatthewLM. Regretfully I can't easily build neither of yours code, because I'm almost exclusively working on the closed-source platforms like Windows, MacOS, etc. and standalone environments not relying on GCC/GLIB/autogen. But it is a shame that another C implementation of Bitcoin is useless for the developers of Bitcoin hardware wallets.

But even the definitions in your "ccoin" include files are deeply incompatible with any standalone environment, even if it uses GCC toolchain. Major rework would be required even now, when there's just couple of hundred lines of code.

The big-endian compatibility is already super-confusing. Istead of using uint32_t in bu256_t use an "union le_int32" to at least flag the mixing of little endian uint32_t with guint32_t.

From my extensive commercial experience of porting from Linux/Posix/Intel-only code to anything standalone or lightweight OS will unfortunatly go through the "first deny then rewrite from almost scratch" stages.

If I may suggest two things:

1) from the start build also on Mac Leopard or Snow Leopard to verify the endian-neutrality with "-arch ppc" with Rosetta on Intel CPU. Edit: Never mind, I forgot that you have access to the Linux on POWER and z/Arch through your association with RedHat.
2) obtain any embedded development/simulation environment. If you have any Xilinx Spartan6 boards (with memory) laying around you could use them to test using MicroBlaze CPU with Xilinx EDK/SDK.

Thank you for yor time.

Jeff, I have one comment that has value only very early in the design of your architecture. Since you've made a choice of C as a implementation language I suggest that you add one more target to the list of supported platforms: standalone a.k.a. bare metal.

It will be nearly impossible to convert your code later unless you very early make a commitment to supporting the event-driven programming like using lwIP-style network interface in addition to socket-style networking.

The same architectural choice needs to be made early to accomodate various lightweight flash-storage access methods instead of Unix-style open/lseek/read/write/close. Same for fork/wait/pthreads versus any RTOS-style explicit synchronization primitives.

One possible advantage of such choices would be that instead of relying purely on donations you may be able to obtain funds via cooperation with future manufacturers of the future Bitcoin devices that will greatly benefit from the "very low resource usage".

Are you sure? Are you really comparing MIPS16 with Thumb?

I've seen some numbers for an older project and the MIPS vs. MIPS16
and ARM vs. Thumb sizes were within about 10% of each other. Tests were made with commercial C compilers, not GCC though. Maybe some GCC oddity?

I'm sorry, but this is disinformation. I repeat: GPU boots first, then it starts the CPU. The "open source" driver is just a lean thunk layer to call back the real supervisor that runs on the GPU's undocumented instruction set. GPU also does things like power management.

Broadcom guys are very smart and very cynical. Turning upside down the central processor vs. peripheral processor architecture was a very smart, forward looking move.

Edit: Before anyone quotes the above: the term GPU is also a disinformation. The main processing unit in the Broadcom VideoCore architecture is a systolic array of relatively slow-clocked DSP processors. It works as a GPU when the DSP chips are executing graphic pipeline code. The secret architecture of the VideoCore processors was a serious drawback... unitil they came with a bright idea of tackling an ARM CPU running as an attached processor. This allows Broadcom to claim some sort of openness to bait the uninformed. It also allows them to markets using Android on ARM.

In Raspbery Pi a closed source GPU boots and controls the CPU. It is going to be a new frontier for hacking the gullible. Score for Broadcom.

Maybe somebody changed the "magic_quites_<whatever>" in php.ini?

Thank you for the writeup. I'm not really familiar with building clusters using USB, I always worked with real serial HDLC/RS-232/RS-422 controllers or with Ethernet multicast.

The only real USB experience I had was with FTDI USB controllers. Neither ngzhang nor Enterpoint bothered to route all available signals from the serial chip to the FPGAs, so the high-bandwidth low-latency modes of transmission couldn't be used with them.

Hopefully the ASIC controller designers won't make the same mistakes and will allow you to use isochronous or bulk modes when the bus utilization becomes non-neglible.

Thanks. I'm quoting this because it is a very nice reference for the state-of-the-art GPU/FPGA optimizations. I remembered the 4) on your list the most because it most clearly shows the shift-register structure inherent to the SHA-256.

Edit: Note to self: Kano is swapping the standard terminology: step vs. round. Using standard terminology first SHA-256 hash in Bitcoin consists of 2 steps of 64 rounds each.

At least for the chip discussed in this thread it appears that the block header structure is fixed:

I'm presuming that you had problems with the usbser.sys from Microsoft. Did you also had problems with the Prolific/FTDI drivers as well?

On the LPC1343 like ModMiner yes. ngzhang used hard serial-USB chips (Prolific or FTDI) in his designs. Same with Enterpoint (FTDI).

The 72 misconception is really getting boring.

FIPS-180-2 defines SHA-256 in terms of two arrays of 32-bit words:
H[8] and W[64]. 8+64=72. Yet a quick comparison with SHA-1 shows that the same "alternative implementation" can be used for SHA-256.

In case of SHA-1 the "original implementation" is H[5] and W[80]; while "alternative implementation" is H[5] and W[16]. Thus: 85 vs. 21.

In case of SHA-256 we have 72 vs. 24 (H[8] and W[16]).

The further observation is that the "arrays" or "circular queues" in the FIPS-180-2 definition aren't really accessed randomly or in any variable order. Therefore both H and W can be converted to 32-bit wide shift registers, but with unusual feedback functions.

The above is just for pure SHA-256, without any Bitcoin specific optimizations. At least two people claimed to be able to apply some unspecified optimizations to Bitcoin hash expressed as a binary function:

1) killerstorm
https://bitcointalk.org/index.php?topic=55888.0

2) Gareth (BitInstant)
https://bitcointalk.org/index.php?topic=10661.msg557579#msg557579

but nothing came out of it. By now pretty much everyone knows about the fact that one can shave 3 last rounds from the 2nd SHA-256 in Bitcoin: instead of looking for zero 32-bit word at the most significant position in H; take an advantage of the fact that H is a shift register and last 3 rounds simply shift the would-be-most-significant-word. So look for a negation of a specific constand value (0x3c6ef372?).
 
I'm puzzled by this part. I never mentioned CPU nor GPU. I tried to pattern my argument after the sort of arguments that were being made around 1980 during the http://en.wikipedia.org/wiki/Mead_%26_Conway_revolution . Perhaps you were mixing me with someone else?

EU-landia has since 2009 a law requiring that a cellphone sold there needs to be rechargable through the micro-USB port.  So maybe not every household does have micro-USB to USB cable, but the situation is on the way to where it becomes ubiquitous.

Hopefully the less progressive lands (like the USA) will join this standard sometime this century. Or maybe not. But there's no need for a device targeted for a global acceptance to specifically pander to the backward.

The argument for the sea-of-hashes design can be derived from the classic analysis made by Mead & Conway and contemporaries.

Consider a circular sea-of-gates big enough to implement many copies of the Bitcoin double-SHA256.

SHA256 is basically a pair of 32-bit wide shift registers with some somewhat convoluted feedback logic. The feedback logic is active (doing the actual computation) whereas D-type flip-flops and connections are passive (just shuffle the signal around). Let X be the average connection length in this design.

Now think about unrolling the above design over a plane. You'll need the values of the feedback terms from the neighbouring cells w-2,w-7,w-15,w-16. Your average connection length rises (2+7+15+16)/4 times or about 10*X . So the passive losses in the interconnect rose about an order of magnitude. You could compensate for this by removing some D-type flip-flop stages and slowing down the clock. By definition you can't really remove the active logic gates that compute the feedback terms. As an extreme you can have a purely combinatorial SHA-256 hasher doing everything in single cycle of a rather slow clock.

I'm not aware of any neat analytical solution for the above optimization problems. But the numerical experiments show that racing the combinatorial signals over vast expanses of silicon is a losing game. Speed of light in MOS transmission line is much less than the speed of light in vacuum. This analysis can be made without actual place-and-route, it is sufficient to have an estimated distribution of the inter-connection lenghts that create planar graph for the logic. I don't recall if sphere-of-gates instead of sea-of-gates is a win, but sphere-of-gates has an obvious termal problem even if we could somehow manufacture it.

In summary: wafer-scale integration was attempted several times in the past without an obvious win. Check out the history before you follow that trail.

The additional data that is not available from the blockchain:

1) true timestamps for transactions first (and last) seen over p2p
2) true timestamps for blocks first seen over p2p
3) non-volatile database of transactions seen over p2p (a.k.a. mempool, but non-volatile) pruned of the entries actually recorded in the blockchain.
4) a way to quickly locate only orphaned blocks without having to scan the whole table of blocks, really just an additional index to the blockchain storage, but explicitly indexing all the losing chains. This in theory is available from the blockchain, but under current architecture it is difficult to obtain this information. In effect the whitewash of the bitcoin transaction history is ongoing when the tools keep only the winning chain.

Thanks.

Are you talking about pure binary yes/no consistency check audit?

Or are you talking about open-ended auditing for the purpose of discovering normal vs. abnormal transaction patterns?

I've spoken a while back with an experienced forensic accounting professional and we discussed various things that would require keeping true time vs. block time stamp. It would also require keeping a separate database of the in-flight transactions, ones that aren't yet recorded in the blockchain.


The observation he had made is that the blockchain is a perfect whitewash: it leaves no trace of the attempted double-spends or other malarkey that would be of great interest to a forensic auditor, e.g.

a) list of addresses involved in the attempted double-spends
b) list of addresses involved in transactions recorded in the orphaned sub-chains but not recorded on the winning chain.

I'm not actually suggesting that you actually implement those auditing reports. But it would be nice if your architecture made possible generating such reports.

When I was young I thought so too. But now I understand this ritual.

It is like a secret handshake of the members of masonic lodge or some other gang. It allows to quickly recognize impostors and other newbies.

You can be sure that almost nobody will start their post: I'm a complete newbie to the computers, have very little idea how they work and understand only how to click with the mouse. I'm also slow with this "typing commands" thing. Listing the running processes in Task Manager/Resource Monitor (or equivalent method) is beyond me.

On the other hand people will freely post: "I clicked, black window popped up and disappeared." or "typed a command and nothing happened".

It declares the same informational content. In addition to the above one can frequently guess just from the writing style if the newbie is going to be a friendly, curious, inteligent person or a nasty, obnoxious dumbass. Or maybe somebody inbetween? Like a person who claims to be a writer but doesn't know which side of the postal stamp to lick?

As with many human social rituals: it works because it saves time overall for the society.

I'm fully with you on the use of ISO format for dates, but I disagree on the NTP for autoconfiguration.

NTP abuse is so rampant that it now has its own notable Wikipedia article:

http://en.wikipedia.org/wiki/NTP_server_misuse_and_abuse

And thats just a tip of iceberg. In many locales servers like "time.windows.com" are unreachable because of the rampant abuse by the various software licensing/subscription enforcement schemes. McAfee private-label products were implicated in DDoS-ing Swiss government NTP servers, if I recall correctly. And apparently the situation is slowly getting worse, not better.

Hardcore Americanism detected!  

The month-day-year versus day-month-year ambiguity with dates is an endless source of the errors that stick is thinking about. People in USA don't see is as often because most of the BIOSes and other low-level tools use the American layout. But it trips over many Europeans.

Wait till Americans start importing Czech-made wallets that use day-month-year order...

I sincerely wish you good luck with this. Now that I had a moment to think, I realize that most of the short-run die-casting manufacturers are working in the toy industry. They may not have the required strength information available at all. The toy industry is strictly form over function: the products are almost exclusively for display only. They will talk about surface smoothness or detail fidelity, not about internal voids or ultrasonic inspection or measuring the residual stress by X-ray diffraction.

I'm not mech-eng myself, but we used the mech-eng labs as a subsitute during the construction in the el-eng building. One can learn a lot by just passively staring at the cool experiments they make.

Actually this weirdness is as old as Unix, so it dates to mid-seventies of the last century. It allowed people to work efficiently with multiple programs in a single session over 300/1200bps modems.

And the "no news is good news" principle is even older than that.

If you are spooked by bitcoind running silently then use an argument "-printtoconsole" or put "printtoconsole=1" in "bitcoin.conf". You will get a nice, reassuring real-time spew scrolling in the window you used to start the bitcoin server.