Bitcoin Forum

Having setup a Tahoe-LAFS node. Zooko (one of the developers on this forum) gave the following tip:


So, with this topic I'm starting to setup and coordinate a Tahoe-LAFS grid among the bitcoin users. It's goal:

- Setup a grid that everybody can join and use (e.g. for backing up your wallet.dat and other files).
- Encourage the Tahoe-LAFS developers to put some more effort in a bitcoin integration for Tahoe-LAFS and help with ideas of how it should be implemented.
- Use the grid as a test bed for the bitcoin integration.

What do we need:
- Storage nodes, the more the better. Can be desktop systems.
- Introducer node(s), to connect the storage nodes. I think we need a few systems that are permanent on for this.
- (Optional) Helper node, to have things uploaded more quickly for users with limited bandwidth. Also a system that is permanent on and has a large bandwidth.

What we need to discuss before we start:
- The storage encoding for the network.

Documentation for starters and how to set things up:
About: http://tahoe-lafs.org/source/tahoe/trunk/docs/about.html
Install: http://tahoe-lafs.org/trac/tahoe-lafs/wiki/AdvancedInstall
Gentoo: https://bugs.gentoo.org/show_bug.cgi?id=220018

About the storage encoding:

The storage encoding contains two values, often written down as a, for example, "3-of-10" value.

In this the 3 means the "split". A file is split in 3 shares (a 100 MB file would each have 33.3MB per share).
The 10 value means the redundancy. Put each share on 10 nodes (doesn't need to be the same).
If one node contains one share of your file and it goes off-line you have 9 nodes left that still have the share.
3 shares (on 30 nodes at most) are needed to rebuild your file.

Also note that the 3-of-10 value gives a total file size of 300MB.

3-of-10 is a quite conserved value. Personally I think it will be better to have some other values, because of the nature
of our grid: many users with limited availability of shares.

How about 4-of-20? (100MB file will be 500MB total)

I will be initially providing a node with around 10GB of space (partially on-line).

just a proposal: why not using https://vekja.net servers as one or more nodes. They accept bitcoins as a payment and bitcoin community can put their btc together and buy space and bandwidth for the Grid. I can donate 100 btc.

I'd like to run a node, but last time i checked (about a month ago) i couldn't get it to work on windows.

any news about that?
or do i have to install a linux-VM to get it to work?

@jorgen, Good idea, but I think it's better to have a introducer or helper node then instead of a storage node. The amount of disk space would be too limited to be useful I think (25btc per week for 100MB?). They might even be able to collaborate, they could use the grid to add redundancy for their servers (they use raid1 now) or as a service for their customers in return for a permanent introducer node.

@Bit-Lex, I wouldn't install a linux-VM if you don't need to (you could better use the disk space for the grid itself ;) ). If you scroll a bit further down on the install page there are instructions on how to install tahoe-lafs manually. You could try that.

that's what i did, i just couldn't get it to work.


diskspace wouldn't be a problem at all, got a few hundred gigs left anyway and hdds arent that expensive nowadays,
bandwidth might (or will) be though (sooner or later), but i'll happily donate as much as possible of it.
so far i can't and i guess i'm not the only one.

Well, I had a check on a Windows system today. You have to go through some hoops to get it working (the description is not completely correct in some places and doing some things slightly different might help a lot).

Here is how I got it working:
No problem here. I've used Python 2.6.6, which is out now. Install it in the default location (C:\Python26).
Just install, no problems here.
3. Now install MingW as described in the "What if that doesn't work?" part. Install it in C:\MingW This is not optional as the wiki might suggest.
I used file: http://sourceforge.net/projects/mingw/files/Automated%20MinGW%20Installer/mingw-get-inst/mingw-get-inst-20100909/mingw-get-inst-20100909.exe/download It is important to check the c++ compiler, base system, development system and the build system when you get that option to select it. This is different then what is told on the wiki.
4. Then continue with:
I installed this initially in C:\Program Files\Tahoe-LAFS. This does not work. The space in "Program Files" causes problems, so install it in C:\Tahoe-LAFS
After the cd c:\Tahoe-LAFS there is a directory below that ("allmydata" or something) cd into that dir also, the wiki doesn't say to do that.

From this point on you can continue with the normal installation (point 6) with one modification. The wiki says that the tahoe script is in C:\Python26\Scripts, well, it isn't. It is in the bin directory in where you did the setup.py build command C:\Tahoe-LAFS\allmydata-something\bin\

I know this is written down a bit confusing, I hope to provide a more clear "Howto" later, when I have access to a windows system again.

Well, using a cloud filesystem storage would be great for wallet backup (after encryption, of course).

But I checked the availibilty of tahoe on my debian system, and "apt-cache seach tahoe" returned nothing.

I still have to check out the website of this stuff, though.

As soon as I got a node running and finish fixing my system clock, I'll pledge 5 BTC for whoever integrated bitcoins into the tahoe network.

No need for encryption, tahoe does that for you. That is why tahoe is so great.

For debian you need to add a repository to your sources.list, see the install page.

I got this error. Help?

Google says that setuptools should be installed.

Googling doesn't seem to turn up a solution. I try easy_install the .egg but that doesn't exactly work. Perhaps, I'll need to install easy_install too.

Strange, easy_install seems to be part of setuptools. Could be different with your distro though.

I pulled directly from darcs.

I'm normally not a big fan of using development code from a cvs, but anyway.

Did you already try to do the following?:
Change to /home/kiba/projects/tahoe-lafs/
and run

Someone is trying to do something against the difficult installation of Tahoe-LAFS:

http://tahoe-lafs.org/pipermail/tahoe-dev/2010-October/005415.html

Someone might want to try out his tree.

I did that already.

Do you have setuptools installed? Did you try a reinstall of setuptools? And what version?

It seems version 1.8.1 will be released within 2 weeks that will have binary packages of all dependencies
with it bundled. Might make a few things more easy.

Is anybody still experimenting actively with a Tahoe-LAFS grid?

I am setting aside some time to experiment with Tahoe-LAFS. If all goes well, I will fulfill my promise to pledge 5 BTC for a bitcoin payment solution.

Someone should do a dropbox replacement using tahoe lafs.

Nobody is going to do it for free....and even if they, they're probably occupied with an open source project.

I've been following Tahoe-LAFS for some time now, and IMHO it is not suited for the thing I wanted to use it for. If people go off-line for some reason and the node doesn't say that to the introducer (e.g. network split, someone pulls his network cable, wifi?) the whole grid will become unresponsive due to the fact that clients try to contact that node. This is especially the case for small grids, on larger grids, this might not be such a big problem. But I haven't tested the latter. A people posted about this on the mailing list last few weeks.

I got it running, but I don't know how to use introducer. I think this tahoe-lafs thing is more difficult than it needs to be.

Yes, I'm using it almost year very intensively (for backups ~50GB). I even made binding for pyfilesystem https://code.google.com/p/pyfilesystem (https://code.google.com/p/pyfilesystem) so anybody can mount Tahoe-LAFS as normal drive (Windows, Linux, Mac).

Unfortunately there is no real effort for accounting, I think joining Tahoe and Bitcoin would be great. Unfortunately everybody who says that running Tahoe-LAFS is too much complicated is right. There is effort to simplify installation on all platforms, but IMHO it is not ready for end user yet. On the other side, TahoeLAFS is great piece of software!

Question to stability - there were made massive improvements and I can say it is pretty stable now. But I don't know which type of connection uses peers in my grid.

The simplest way is to join public grid http://pubgrid.tahoe-lafs.org/ (http://pubgrid.tahoe-lafs.org/).
Second way is to run own introducer by running 'tahoe create-introducer' command.
Last way, if you are interested, you may ask to access to volunteergrid.

In any case, you will get introducer's strange URL started with pb:// which you have to enter to ~/.tahoe/tahoe.cfg under "introducer.furl". Then start your node and everything will work!

It said I have zero peers after I wrote the stuff for my introducer node or whatever.

What was the error message? Or what step didn't work?

Thanks.

Zooko

What didn't work when you built it without MingW? MingW really should be optional and if it was required for you then I'd like to know why. It probably means there is a 3rd party library that for some reason isn't hosted here:

http://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-eggs/README.html

But I really need to know which library was missing or which other error was encountered in order to fix it.


Could you please update the wiki page to fix it so the next person won't hit these two issues?

So erm, I have no peers.

I figured out how to use it, with some difficulty and some heart jumping moment.(Do not post URI to private data)

It was some time ago when I did this, hope I remember it correctly. I was doing  an install from source. The "optional part" that I was talking about, was the location of MingW. The wiki says you could put it anywhere, but spaces in the Windows filesystem aren't respected, causing problems (I initially installed it in C:\Program Files\MingW).


Could you please update the wiki page to fix it so the next person won't hit these two issues?
[/quote]

I'll put it on my ToDo list. I don't have access to a MS Windows system atm. If someone here does a fresh install with the new version on MS Windows, feel free if this is still an issue and adjust the wiki as accordingly.

P.S. There is also a multiple-introducer, people might want to try that.

http://bitcointalk.org/index.php?topic=2236.0

For those who want bitcoin/tahoe-lafs integration, pledge your bitcoin where your mouth is.

I am more interested in technical details.  How would bitcoin be integrated?  Has anyone even sketched out such a scenario with sufficient technical detail?

Do the maintainers of Tahoe-LAFS plan bitcoin integration, or is this simply something bitcoiners think would be a good idea?

If bitcoin integration was not planned from the beginning, maybe we can come up with a better system with better incentives.

We Tahoe-LAFS hackers are actively working on "accounting", which means enabling storage servers and storage clients to discriminate among one another and to track which ones are used for how much storage. There is extensive technical detail available, although some of it may be for an earlier design than the current effort:

http://tahoe-lafs.org/trac/tahoe-lafs/query?status=!closed&keywords=~accounting&order=priority (http://tahoe-lafs.org/trac/tahoe-lafs/query?status=!closed&keywords=~accounting&order=priority)

http://tahoe-lafs.org/trac/tahoe-lafs/wiki/QuotaManagement (http://tahoe-lafs.org/trac/tahoe-lafs/wiki/QuotaManagement)

http://tahoe-lafs.org/trac/tahoe-lafs/wiki/AccountingDesign (http://tahoe-lafs.org/trac/tahoe-lafs/wiki/AccountingDesign)

http://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/docs/proposed/accounting-overview.txt (http://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/docs/proposed/accounting-overview.txt)

A digital payment scheme such a BitCoin would probably fit into Tahoe-LAFS pretty easily once we have accounting.

Nobody has written it down in great detail, but several of the Tahoe-LAFS developers have a lot of ideas about it. There is significant interest, for example Brian Warner—who is currently the leader in "most lines of code in the current version of Tahoe-LAFS"—went so far as to reimplement some BC functionality in Python a couple of months ago.

The basic idea is merely that a storage server agrees to store and serve your data only if pay it. There are a large number of details that would need to be worked out beyond that basic idea of course. I currently think a long-term relationship would probably work better than a per-transaction micropayment, so for example you could deliver a payment to each of your storage servers once per month in return for continued service.


Tahoe-LAFS is great at what it is designed for—reliable (secure) distributed storage. If that's what you want, then I encourage you to use or extend Tahoe-LAFS. If you want a substantially different sort of behavior then I would like to learn more about your use case.

Looking around the Tahoe site, I see a lot of todo items with dates going back to 2008. Does anyone have a sense for how active the development is on this project?

Pshaw, 2008 is nothing—we have open tickets from 2007!

http://tahoe-lafs.org/trac/tahoe-lafs/ticket/20

You can see recent activity by looking at the trac timeline or the tahoe-dev mailing list, or the tahoe-lafs-weekly-news mailing list.

Regards,

Zooko

Is it possible to integrate different transport protocols into Tahoe? I'm especially interested in the Storage server <---> gateway transport, as this would be really cool if it could be done via BitTorrent (announced on DHT).

Btw. a really nice sum of BTC you got in your donation account there! Maybe it's time to set some bounties here in the forum to get some work done? ;)