Whenever any cites that Val Aurora article, please follow-up and link to this newer, better, article:

https://z.cash/technology/history-of-hash-function-attacks.html

blog post about this article when we posted it:

https://z.cash/blog/hash-functions.html

Sincerely,

Zooko

testing image:

testing image:

FWIW we evaluated Cuckoo as well for Zcash, and it was a strong second-place contender. There wasn't really anything wrong with it — it just didn't seem to have quite as much of a rigorous scientific analysis as Equihash. However, that is a very subjective thing for me to say. You could argue (and Cuckoo's author, John Tromp, does argue persuasively) that Cuckoo's history of analysis and refinement is better than Equihash's.

I doubt that this is true. Can you point me to where you heard that?

You should probably think carefully and make up your mind whether decentralization or 51%-attack-resistance are more important to you:

https://bitcoinmagazine.com/articles/zcash-creator-on-the-upcoming-zcash-launch-privacy-and-the-unfinished-internet-revolution-1472568389/

(Search in text for "fundamental trade-off".)

If you're looking for a new Proof-of-Work, I can recommend the Equihash Proof-of-Work that we selected for Zcash.

We studied Proof-of-Work functions for a long time, chose Equihash (https://z.cash/blog/why-equihash.html), hired a legendary hacker to study it and write his evaluation (https://z.cash/blog/the-zcash-equihash-analysis.html), gave out $30,000 dollars in bounties to make the best implementations for CPU and GPU (https://z.cash/blog/announcing-miner-contest.html), got lots of good open source implementations (https://zcashminers.org/submissions), launched the network (https://z.cash/blog/zcash-begins.html), and mining has been working very well at scale ever since (http://www.coinwarz.com/network-hashrate-charts/zcash-network-hashrate-chart).

The reasons we chose Equihash are:

* it is memory-oriented rather than computation-oriented which makes it less cost-efficient to implement in ASIC,
* it is asymmetric, meaning that verifying a solution is much cheaper than generating that solution (_even_ starting from the nonce that generates that solution); In particular it requires substantial RAM (hundreds of MB, depending on parameter choices) to generate a solution (or an attempted solution), but it does not require that much RAM to verify a solution. This may be useful for constrained implementations such as SPV wallets in constrained hardware, implementation inside the Ethereum VM, etc.
* it has a good level of scientific investigation behind it, which makes me think it relatively unlikely than an algorithmic breakthrough would enable someone to find solutions much cheaper than the competition. This has been born out by the experience of live Zcash mining, where developers have made tremendous progress on micro-optimization, but as far as we know no algorithmic breakthroughs.

A reason to choose Equihash is that you benefit from the all of the research and implementation work described above. There are many well-tested implementations available, both open source and proprietary.

Downloading only a subset of a file is already implemented, but there isn't a command implemented that says "pick a random segment of this file, download it from that server, and let me know if it passed integrity checks". You can approximate it with the current Tahoe-LAFS client, like this (the following lines that begin with "$" is me typing in stuff as though I were using a bash prompt):

1. Pick a random spot in the file. Let's say the file size is 10 MB:

$ FILESIZE=29153345
$ python -c "import random;print random.randrange(0, $FILESIZE)"
2451799

2. Fetch the segment that contains that point. Segments are (unless you've tweaked the configuration in a way that nobody does) 128 KiB in size, so this will download the 128 KiB of the file that contain byte number 2451799 and check the integrity of all 128 KiB:

$ curl --range 2451799-2451799 http://localhost/uri/URI:CHK:jwq3f6lkcioyxeuxlt3exlulqe:sccvpp27agfz32lqjghq2djaxetcuo7luko5dhrpdgs7bfidbasa:1:1:29153345 | hexdump -C
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100     1    0     1    0     0      0      0 --:--:--  0:00:01 --:--:--     0
00000000  a9                                                |.|
00000001

As you can see it took only a second to download and emitted only one byte to stdout, but it downloaded and verified the integrity of the 128 KiB segment that contained that byte.

If you are using multiple servers, using Tahoe-LAFS's awesome erasure coding feature to spread out the data among multiple servers, then this will download the data from the 3 fastest servers (unless you've changed the default setting from "3" to some other number). There is no good way to force it to download the data from specific servers in order to test them -- it always picks the fastest servers. You can see which server(s) it used by looking at the "Recent Uploads and Downloads" page on the web user interface, which will also tell you a bunch of performance statistics about this download.

In short, this feature is *almost* there. We just need someone to write some code to do this automatically in the client (which is written in Python) instead of as a series of bash commands. Also this code should download one (randomly chosen) block from every server it can find instead of from just the three fastest servers, and it should print out a useful summary of what it tried and which servers had good shares.

Oh, there is a different function which does print out a useful summary of results -- the "verify" feature. But, that downloads and tests every block instead of just one randomly chosen block. Another way to implement this is to add an option to that to indicate how many blocks it should try:

$ time tahoe check --verify URI:CHK:jwq3f6lkcioyxeuxlt3exlulqe:sccvpp27agfz32lqjghq2djaxetcuo7luko5dhrpdgs7bfidbasa:1:1:29153345
Summary: Healthy
 storage index: 7qhuoagk4z4ugsjkjgjcre6sx4
 good-shares: 1 (encoding is 1-of-1)
 wrong-shares: 0
real    1m2.705s
user    0m0.570s
sys     0m0.060s

It looked like the first post was a little out of date with regard to BFL Single. Here's a G+ "blog" post about it by me:

https://plus.google.com/u/0/108313527900507320366/posts/2ztAhLnXQKm

Right, to win the bounty you have to submit a patch that passes quality standard for the Tahoe-LAFS project and gets accepted into trunk. Then, when Brian (the dev) uses the new version of trunk, it has to be 5X as fast on his laptop.

Regards,

Zooko

For the record, the 50ⓑ that I'm putting up is coming out of own wallet. If I wanted to use the donations that people have generously contributed to the Tahoe-LAFS project, I would have to get the rest of the Tahoe-LAFS Software Foundation to come to consensus on whether we should spend it this way. Instead, I just spontaneously decided "screw it, if anybody can make the tests that much faster it will be worth my precious money". Bitcoin allows me to impulse-spend on a wide array of new goods and services. :-/


So why not? Serious question -- I'm sure there are good reasons why you haven't, and I'd like to know what the number one reason is.

Regards,

Zooko

I contribute to an open source project to build a decentralized secure storage system: https://tahoe-lafs.org .

Our comprehensive unit tests are invaluable for finding bugs, and we run them every time we commit a patch, on all supported platforms, but they take too long and this interferes with development flow.

This is especially a problem on the laptop of one of the lead developers, Brian Warner, who is using OS X with some sort of full-disk encryption that makes his disk I/O slow. But it is also a problem on everyone else's development machine and on the buildbots.

Therefore, I'm offering a 50ⓑ bounty to the first person who submits a patch which causes the tests to take only 4 minutes 30 seconds (approximate) instead of 24 minutes (approximate) on Brian's laptop.

More info here:

https://tahoe-lafs.org/pipermail/tahoe-dev/2012-April/007314.html

The fact that Tradehill moved the ⓑ from the original address to two other addresses should be considered a positive sign rather than a negative one (now that they've responded to your concern). This suggests that they have some internal separation between private keys, which might be able to limit the damage in case of accident or malice in which one of the private keys is stolen or destroyed. Or maybe they don't have any effective firewalls internally between private keys, but they might.

In contrast Mtgox is obviously not doing that, since they are using a private key which currently controls almost half a million ⓑ to sign small transfers:

https://bitcointalk.org/index.php?topic=53848.0;all

That just seems unnecessarily risky to me.

There appears to still be a technical issue. When I click on Instant Buy, enter an amount, and click "Get Quote", it pops up an alert (with a delightfully scary red pattern) that says "Socket not connected

Sorry, our notification socket isn't connected yet. Try again in a few seconds!".

I've tried half-a-dozen times over the course of ten or so minutes -- always the same result.

Regards,

Zooko

Could you please tell me what happened? A post to your blog about what went wrong would go a long way to reassuring me that you're still engaged in maintaining the service.

Regards,

Zooko

I withdrew a non-negligible amount of USD from TradeHill by simply filling out a form on the site and clicking submit. The resulting page said that they would put a check in the mail that same business day. Apparently they did! Because that was only last Thursday, if I recall correctly, and the check arrived today.

So this is just a report for anyone who wants to know that there was no delay, confusion or extra steps.

Here's my TradeHill referral code. So far I've made about $0.50 in referral bonuses, so it isn't so much that I hope to make a lot of money by passing out my referral code, but it does give you 10% off of their normal fees if you sign up with it:

https://tradehill.com/?r=TH-R11149

Regards,

Zooko

I use "ⓑ".

Pshaw, 2008 is nothing—we have open tickets from 2007!

http://tahoe-lafs.org/trac/tahoe-lafs/ticket/20

You can see recent activity by looking at the trac timeline or the tahoe-dev mailing list, or the tahoe-lafs-weekly-news mailing list.

Regards,

Zooko

We Tahoe-LAFS hackers are actively working on "accounting", which means enabling storage servers and storage clients to discriminate among one another and to track which ones are used for how much storage. There is extensive technical detail available, although some of it may be for an earlier design than the current effort:

http://tahoe-lafs.org/trac/tahoe-lafs/query?status=!closed&keywords=~accounting&order=priority

http://tahoe-lafs.org/trac/tahoe-lafs/wiki/QuotaManagement

http://tahoe-lafs.org/trac/tahoe-lafs/wiki/AccountingDesign

http://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/docs/proposed/accounting-overview.txt

A digital payment scheme such a BitCoin would probably fit into Tahoe-LAFS pretty easily once we have accounting.

Nobody has written it down in great detail, but several of the Tahoe-LAFS developers have a lot of ideas about it. There is significant interest, for example Brian Warner—who is currently the leader in "most lines of code in the current version of Tahoe-LAFS"—went so far as to reimplement some BC functionality in Python a couple of months ago.

The basic idea is merely that a storage server agrees to store and serve your data only if pay it. There are a large number of details that would need to be worked out beyond that basic idea of course. I currently think a long-term relationship would probably work better than a per-transaction micropayment, so for example you could deliver a payment to each of your storage servers once per month in return for continued service.


Tahoe-LAFS is great at what it is designed for—reliable (secure) distributed storage. If that's what you want, then I encourage you to use or extend Tahoe-LAFS. If you want a substantially different sort of behavior then I would like to learn more about your use case.

What didn't work when you built it without MingW? MingW really should be optional and if it was required for you then I'd like to know why. It probably means there is a 3rd party library that for some reason isn't hosted here:

http://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-eggs/README.html

But I really need to know which library was missing or which other error was encountered in order to fix it.


Could you please update the wiki page to fix it so the next person won't hit these two issues?

What was the error message? Or what step didn't work?

Thanks.

Zooko

Folks:

I just posted this note over on the Trade forum:

http://bitcointalk.org/index.php?topic=30.msg10592#msg10592

There are a few hackers who would be pretty interested in contributing patches to integrate BitCoin payments directly into the Tahoe-LAFS storage protocol. It would be a big job! It would probably take a year of effort before we would be ready to deploy it in a new stable release. Of course, the more people pitch in the faster the work goes.

Regards,

Zooko

community organizer, Tahoe-LAFS project

I'm part of an open source project developing a secure, decentralized storage system. We just started accepting BitCoin donations along with PayPal and Flattr donations. I hope to add acceptance e-gold donations as well.

http://tahoe-lafs.org

The donations are mostly used to defray the costs of running servers and to send special "thank you gifts" when people contribute to our project, such as when they find a security hole:

http://tahoe-lafs.org/hacktahoelafs

We actually have a long-term plan to integrate digital cash payments directly into the storage protocol. :-) Tahoe-LAFS is a grandson of a p2p file-sharing application named "Mojo Nation". (BitTorrent is another grandson of Mojo Nation, so Tahoe-LAFS and BitTorrent are first cousins.) Mojo Nation had blinded Chaumian ecash baked into the protocols. It was a great idea! :-) But it tried to do too many different things and it never worked right. :-( Nowadays Tahoe-LAFS tried to do fewer things: just the secure distributed storage part. It turns out even just that part takes a lot of careful engineering to make it work. But it does work, so now we have one solid component that we can build on. Hopefully in the future we can marry it to another solid component that does digital payments. :-)

Regards,

Zooko Wilcox-O'Hearn
community organizer, Tahoe-LAFS project