OK, so actually you are dependent on the 3rd party utility for the integrity of your Windows installations. In that case probably all of them are corrupted in the same way somewhere in registry or low-level disk/controller drivers or other places that don't undergo integrity checking when cloning.
 
In the past we (my corp.) had site-license customers where the entire corporation was unable to successfully run our software. The thorough investigation had shown that all the failing machines were installed using some imaging software (Norton Ghost/Clone? for desktops, some vendor specific (Samsung Recovery Solution?) for laptops). Doing a clean install from the Microsoft distribution media miraculously cured all the problems.

I'm very suspect of the system maintenance practices in the Windows market segment since I learned the inside scoop about bankruptcy of 3dfx. Near the end all of their official corporate system images were having undetected corruption issues and the only the few developers who could do work were those who bough retail boxed Microsoft media. That was despite having spent literally millions on internal Windows IT management and consulting.

My corp. internally used Acronis and as far as I know they still haven't fixed the bugs I submitted to them years ago regarding imaging NTFS disks with files processed with some old Microsoft consumer photo editing software, which creates complex NTFS alternate streams. Edit: found it: https://en.wikipedia.org/wiki/Microsoft_Picture_It!
 

Are your Windows installations relatively clean (of 3rd party utilities) or are you using e.g. same antivirus on all installations? Bugs in online antivirus programs are the most frequent issue with database-heavy applications, not just Bitcoin Core. Some antivirus engines automatically recognize popular database engines (e.g. MSSQL,Oracle,Sybase,...) and self-disable themselves on database files.

Edit: The most common corruption is in the "chainstate" directory. You can save yourself a lot of reindexing time by doing a backup of those directories using e.g. 7zip in "synchronize files" mode. In case of corruption just restore the known good chainstate and the reindexing will pickup near the end and will finish quicker.

I don't buy your claim that your solutions "work beautifully". You are basically serving the market of "everyone knows the root password of the server". You can't properly provide elementary security like, e.g.

1) the web design subcontractor can't have the passwords to the accounting server
2) the are two independent web store fronts: the production one that is open 24*7 and the staging one for beta testing that is open intermittently and possibly managed by a different webmaster

It is also very telling that in your last paragraph you foresee closing down web front-end while having the accounting open. But the most real businesses operate in an opposite way: the web front end is open 24*7 but accounting is regularly being closed (weekly, monthly, quarterly) for periodic reporting and auditing.

I understand that there is a big market for "fly-by-nights" that essentially operate until the first audit by taxation authorities or first investigation by police/prosecution/regulators. I can't blame you for serving it, but I will stay away and always advise everyone else to stay away too.

There's a big disconnect between the two markets you mentioned: one-password shop with up to five people and multibillion corporation. The best example is what Microsoft targets with Server Essentials (a.k.a.) Small Business Server. The organizations of up to about 50 people who tend to subcontract various non-essential activities to the specialist firms. The people who know how to produce reports using Microsoft Access and why every commercial SQL database engine has some sort of per-column security.

Anyway, returning back to the ideas described in the opening post of this thread: it would be nice to have a Bitcoin engine that works somewhat like any other database engine. In the sense that you can start the engine itself to track the Bitcoin P2P network and then attach and detach wallets like one would attach and detach databases to the SQL engine.

But this isn't going to happen under current Bitcoin Core Development Team. In 2012 I mentioned something similar in the "What's the problem with getting Bitcoin compliant with GAAP???" and Gavin Andresen replied:
I think the current governance situation in the Core team could lead to the change in the general direction of the development. Time will show.

Ha, ha! The shrillest shills are the funniest.

Backstory to that cover is also on imgur: http://imgur.com/gallery/wV0UU

What you describe is acceptable only to one-person shops or similar setups.

Most organizations will have clear interdepartmental security delineations and will have a preexisting accounting system already based on the (accounting, securities, etc.) prevailing standards.

"walletnotify" and "blocknotify" are really hacks that cross the security domain in wrong directions: from the Internet front end to the accounting servers. Only desperados will approve such architecture. The other problem with event callbacks are:
1) is that they require both caller & callee systems both online at the same time
2) they have no indication of the backlog of pending callbacks.

While conceptually handling chain reorganizations isn't terribly difficult it is also the feature that most often goes nearly completely untested. It is one of the most common failure modes in various solutions mentioned on this forum.

The other problem is purported scalability of having one blockchain and multiple wallet. Again the practice had show that people handle they wallet-synchronization code with tight poll loops and other similar lameness, that in practice multiple copies of bitcoind take less CPU and bandwidth resources only more storage. I think Stratum & Electrum is a good example: Stratum protocol in theory handles asynchronous notifications well, but its practical implementation in Electrum is plagued by horrible inefficiencies.

The last thing is that even if you solve all the above problems you still don't have a clean interface to the industry-standard accounting tools: Fix protocol in securities and various "middleware/message queueing" (like e.g. ZeroMQ) in general accounting. On the other hand the raw bitcoind can be run with -privdb=0 (and minimal patches) and allow to fully integrate with accounting back-ends thanks to the support for X/Open distributed transaction monitoring in BerkeleyDB than handles the wallet.

In summary: individual problems aren't difficult by themselves, the complexity is in integrating all of them. The sad state is that various folks peddle incomplete solutions plagued with tight poll loops and with untested/incorrect handling of chain reorganizations. The only exception was bitsofproof but it is back to being closed source. Almost nothing has changed in this space since I joined this forum sometime mid-2011.

You seem to be an advocate of Synala. I have one question: did your solution survived at least one accounting audit conducted to the prevailing accounting standards in your country?

Well, on the positive side I can at least observe that folks ceased to propose "use built-in accounts in bitcoind". Before those got officially marked as obsoleted this was the closest thing to the willful accounting fraud.

On my computer there is nothing about UTXO indexing in the original post.

I'm going to quote both posts until I see the clarification.

Are you talking about that Electrum client that uses Stratum protocol (designed by slush) to talk to Electrum servers?

Those Electrum servers that couldn't keep up with the Bitcoin transaction stream during recent stress tests and were getting hours behind w.r.t. the live network?

Or are you talking about something else?

On Raspberry Pi there's absolutely nothing in it that does error detection. This is a piece of hardware that is marginal in every respect, including minimalist quality assurance of the shipped products.

Think of RPi as RAIC: Redundant Array of Inexpensive Computers. You really need several of them (at least 2) running in parallel and constantly compare results.

The simplest thing to do in your case is to replace USB SSD with an USB connected spinning disk drive with separate power supply. It will take a little bit longer to initially synchronize but it will exclude the problem of a weak USB power supply.

The other way is to hook up your USB SSD through a separately powered USB hub.

Anyway, you really need to realize that if you think of running a financial application on a computer without ECC RAM you are in the state of sin. It is a road to nowhere except how to learn troubleshooting and fault localization skills.

It is a complex project, I'm not aware of anything open source.

Also the RPC paradigm isn't really well suited for this concept: the communication between blockchain server component and wallet component must be two-way not one-way request-response.

A producer-subscriber paradigm (like e.g. FIX protocol) is much better way to implement this, and once correctly implemented it nearly completely obviates the need for one-way RPC.

There were very buggy attempts to implement what you're asking by effectively replacing subscription pattern by a busy-wait loop with very frequent polling. But this was always ending with coding disaster that wouldn't handle chain reorganizations correctly.

Make sure you stress simultaneous reading and writing to the same drive and that the drive is the bottleneck in both directions, not the network transfer.

My general (not bitcoin specific) recipe for triggering errors in disk controllers is as follows:

1) keep the source tree (e.g. ~/.bitcoin) on the local disk that is mounted read-only.
2) it is acceptable to have source over the net only if the net is gigabit
3) recursively compute MD5 checksum of the source tree
4) start filling out the target disk by doing the recursive copy, not with "cp -r" or "scp -r" but with piped tar "tar cf - . | tar xvf -" or cpio "find . -depth | cpio -o | cpio -i"
5) as soon as the first copy is made spawn running in parallel MD5 checksum verification of the sums computed in step (3)
6) continue running step (4) and (5) until the last copy fail with target disk nearly 100% full
7) remove exactly one target copy making room for exactly one more copy of source
8) keep doing the above over a weekend

I'm not up to speed about current SSD market, but in the past only Intel SSD drives survived this kind of torture. Only Intel from the "reasonably priced" segment, we also tested some models from "ridiculously expensive" segment.  Our test also involved doing very similar things running through commercial database engines over many drives in parallel (not as RAID but as FILEGROUP).

The simpler explanation is that "the authorities" already have a mole and simply continue to gather evidence. I know very little about Dash, but I remember what happened during the days of "Bitcoin Consultancy" operating in Europe and Bitomat.pl acquisition by Mt.Gox. At that time not even "the authorities" would gather evidence, the regular private gumshoes did, the circle of users was so small.

When both Bitcoin Consultancy and Bitomat disappeared the gathered information was used to close down and prosecute some controlled-substances distribution rings in Europe. Bitcoin acted as a sort of honey-pot, as it tended to attract businesspeople who weren't well-served by the regular banking transfer network SEPA.

Further back in history I remember that some early arrests of Al-Quaeda functionaries were thanks to the secret cooperation of the employees of Swisscom EasyROAM subsidiary.

Certain facts bear repeating, both now in this thread and in the past in the other threads.

You and other people tend to be missing the fact that most successful securities prosecutions in the USA were against promoters and sales agents. The courts spent very little time considering the nature of the underlying security. Most of the time was spent analyzing the "economic reality" of the promoter who sold "something" to the buyers and the promoter gained money while buyers lost money.

Erik Vorhees got of relatively easy because he astutely traded the Bitcoins raised by his issues and (after advice of his counsel) preempted prosecution by simply rebuying/repaying all the holders of his securities. Because the buyers had no loss to show the SEC had very weak case that basically consisted of "missing paperwork".

If you can design your "cryptographic protocol" in such a way that its users/buyers are highly unlikely to lose money and the managers/sellers are highly unlikely gain the money at the expense of the former, then it is the best defense against any prosecution in any jurisdiction.

One thing that you have in common with all the others successfully prosecuted in the USA is your plain and innate desire for attention and self-promotion. Conceivably you could be raising about the same money working quietly behind some figureheads (either persons or organizations). This is how the traditional securities industry is organized: the responsibility is spread over multiple layers underwriters, syndicators, sales brokers, etc. It seems like you abhor working with other professionals on the equal terms and always want to do it alone. In particular I clearly sense that you didn't even go to a friendly lunch with some securities lawyer. Your style of argumentation shows that you can't distinguish between "arguing the case" and "arguing with a person". Theoretically you could change and obtain a securities counsel and listen to his advice. But I feel that that would completely go against your personality and your goals in life.

TL;DR: Yup, it is the promoters that most often get prosecuted in the securities litigation.

Here's a short quote from a first Google result when searching for a properly spelled "Howey test":


Edit: Another relevant quote from the same article:

Hey, this discussion is getting really cool and entertaining. I'm going to quote your response in full so we can get back to it few years from now.

I think you clearly plan to promote and distribute a security in the USA but you plan to sidestep the registration requirements by claiming that your security is not a security.

In the nascent days of securities regulation in the USA one Charles Ponzi traded Universal Postal Union International Reply Coupons. UPU IRC clearly weren't securities, yet Ponzi was successfully prosecuted.

In the nascent days of Bitcoin regulation in the USA one Erik Vorhees launched several Bitcoin-based securities while simultaneously running diversionary campaigns like "Let's end one debate: Commodity vs Money" https://bitcointalk.org/index.php?topic=47111.0 . He settled with SEC by disgorging all profits (repurchased all his securities) and paying some fine.

In the criminal defense there is a maxim that anyone who defends himself pro se had hired a fool for his defense counsel. I have a feeling that you've also hired a fool for your securities counsel.

IANAL, but I really like to talk with them and I will be watching this space for further developments. The past experience with Bitcointalk was that no real IAAL will post in this thread, but I would love to be proven wrong.

Also could you do all the readers a favor and start spelling Howey correctly? You keep spelling it "Howe test" which makes it hard to search for the actual precedents.

Are you seriously trying to cite an "invitation for comments" from some law firm as a legal precedent having valid legal standing in USA law?

I mentioned "pork bellies" as a good example. Although pork bellies are not traded on Chicago Mercantile Exchange since 2011, the legal analysis of "When and how a pig can become a security subject to regulation?" is still a valid exam problem for the law students.

---

And my points are:

a) you have no legal training
b) you seem to lack even amateur's appreciation of https://en.wikipedia.org/wiki/Adversarial_system and how it applies to a common law land like USA
c) you are misapplying reasoning rooted in https://en.wikipedia.org/wiki/Inquisitorial_system to an old case from 1946 https://en.wikipedia.org/wiki/SEC_v._W._J._Howey_Co.

d) you seem to be searching for a https://en.wikipedia.org/wiki/Affirmative_defense loophole to https://en.wikipedia.org/wiki/Securities_Act_of_1933#Regulation_S

While I understand and commiserate with your general position about societal over-lawyering I also observe that your legalistic divagations are of very low quality.

Much better divagations were posted by John Nagle in 2011:

https://bitcointalk.org/index.php?topic=46486.0

but regretfully many of the links are no longer working. The AdSurfDaily case he mentioned ended in imprisonment of Bowdoin in 2012. It would be a much better source for a precedent involving cryptocurrencies in 2015 than some old cases from the 1st half of the 20th century.

False. "Pork belly" is not a security. "Pork belly futures/options/participation shares" were securities subject to the regulation and oversight.

Miscommunication. You seem to be thinking of USA exclusively. I'm writing to a worldwide audience because this forum is not limited to the "American" readers.

https://en.wikipedia.org/wiki/List_of_national_legal_systems
https://en.wikipedia.org/wiki/Common_law
https://en.wikipedia.org/wiki/Civil_law_(legal_system)

IANAL, but I spent a lot of time on the meetings with lawyers, including securities lawyers (but not related to cryptocoins).


A) under common law systems:

Your idea of "managerial control" is way to narrow. You've completely neglected the promotion and sales of securities and "managerial control" of said sales and promotion.

Therefore most of your analysis in invalid. One could probably come up with a case where the "security" itself is valid, but was promoted and sold in an invalid way.

B) under civil law systems:

The story is completely different. Under those systems the securities law tends to root in the Roman law concept of "depositum irregulare".

One more thing you seem to be neglecting is that the current prevailing common law definition of "conspiracy" doesn't require a proof of communication between the distributed co-conspirators. It is sufficient for prosecution to show that all "conspirators" acted in furtherance of their common goal.

It was discussed earlier on this forum under the term "licensed mining".
By "earlier" I mean couple of years ago.

I'll just quote this in the entirety for the posterity. I was actually expecting the visits and comments from the competing hustlers.

Anyway: everyone knows that the proof of the cook is in the pudding. Proof of the EE is in that his design doesn't suffer from the problems that are taught at the undergraduate levels. I don't have an easy way to dig through my textbooks, so I'll just cite what I remember off the top of my head:

Intel 8080 was from 1974 and required careful sequencing of power up to work properly and avoid damage.

Intel 8085 was from 1975 and was free of power sequencing requirements.

Zilog Z80 was from 1976 and was not only free from power sequencing requirements but had a patented way of powering up and resetting to enable special debug mode.

(From Wikipedia) In 1977 engineers at Hughes Aircraft invented general circuit for protecting CMOS IC's from latch-up.

I'm not in the mood for devising of the wall jokes, but this situation really calls for some satire.

How about:

Spondoolies: the honey badger of SHA-256 mining: it comes up correctly no matter in which order you plug in its subsystems.

Maybe someone has or can cheaply find an undergraduate digital logic design textbook from 1980 (or later) and send one for the benefit of the designers at BitMainTech?

Edit: Also, for somebody who's a native English speaker or a decent poet: can someone come up with a version of "Timex: takes on lickin' and keeps on tickin'". Something like "Spondoolies: takes on plugin' and keeps on hashin'".
 

Well, I do know a lot about hardware and I can recognize real technical problems from the cargo-cult and voodoo-chicken-blood inspired pseudo-solutions.
I wouldn't be surprised if the engineering staff at that Chinese company doesn't have a real engineering degree but some sort of "work experience" degree after serving in one of those remote technical help call centers.

Edit: https://en.wikipedia.org/wiki/Latch-up is something discussed in every electronic engineering course and any competent student would know how to deal with and avoid this problem.
End edit.

It is very easy for me to pontificate here because I have all the experience required:

1) I helped many very non-technical artists help set up and maintain their "render farm" which is the finishing stage of ray tracing in computer animation.
2) I worked for many years in hardware design and I know how to recognize common hardware faults and properly fix them.

The "home mining" market is based on pandering to the lowest common denominators of the users, the ones who even have to be explained where on the wall they have to plug in their device.

I had a good fortune on having worked with non-technical people in other markets (animation artists and medical doctors) where such pandering is neither expected nor provided. I really do know what kind of technical work can be expected from people without technical education but with common sense and self-respect.