The Tahoe-LAFS Bitcoin Grid - topic

[BioMike]

Having setup a Tahoe-LAFS node. Zooko (one of the developers on this forum) gave the following tip:

I have an idea! Get together with some
other BitCoin users and set up a shared Tahoe-LAFS grid among you.
Please let me know how it goes. Eventually, as you may be aware, we
want to integrate micropayments directly into the Tahoe-LAFS protocol
so that anyone who pays a BitCoin would automatically get access to
your storage...

So, with this topic I'm starting to setup and coordinate a Tahoe-LAFS grid among the bitcoin users. It's goal:

- Setup a grid that everybody can join and use (e.g. for backing up your wallet.dat and other files).
- Encourage the Tahoe-LAFS developers to put some more effort in a bitcoin integration for Tahoe-LAFS and help with ideas of how it should be implemented.
- Use the grid as a test bed for the bitcoin integration.

What do we need:
- Storage nodes, the more the better. Can be desktop systems.
- Introducer node(s), to connect the storage nodes. I think we need a few systems that are permanent on for this.
- (Optional) Helper node, to have things uploaded more quickly for users with limited bandwidth. Also a system that is permanent on and has a large bandwidth.

What we need to discuss before we start:
- The storage encoding for the network.

Documentation for starters and how to set things up:
About: http://tahoe-lafs.org/source/tahoe/trunk/docs/about.html
Install: http://tahoe-lafs.org/trac/tahoe-lafs/wiki/AdvancedInstall
Gentoo: https://bugs.gentoo.org/show_bug.cgi?id=220018

[1714115555]

1714115555

[1714115555]

1714115555

[BioMike]

About the storage encoding:

The storage encoding contains two values, often written down as a, for example, "3-of-10" value.

In this the 3 means the "split". A file is split in 3 shares (a 100 MB file would each have 33.3MB per share).
The 10 value means the redundancy. Put each share on 10 nodes (doesn't need to be the same).
If one node contains one share of your file and it goes off-line you have 9 nodes left that still have the share.
3 shares (on 30 nodes at most) are needed to rebuild your file.

Also note that the 3-of-10 value gives a total file size of 300MB.

3-of-10 is a quite conserved value. Personally I think it will be better to have some other values, because of the nature
of our grid: many users with limited availability of shares.

How about 4-of-20? (100MB file will be 500MB total)

I will be initially providing a node with around 10GB of space (partially on-line).

[jorgen]

just a proposal: why not using https://vekja.net servers as one or more nodes. They accept bitcoins as a payment and bitcoin community can put their btc together and buy space and bandwidth for the Grid. I can donate 100 btc.

[BitLex]

I'd like to run a node, but last time i checked (about a month ago) i couldn't get it to work on windows.

These installers can be downloaded from (TODO: they're built, but where are they downloadable?).

any news about that?
or do i have to install a linux-VM to get it to work?

[BioMike]

@jorgen, Good idea, but I think it's better to have a introducer or helper node then instead of a storage node. The amount of disk space would be too limited to be useful I think (25btc per week for 100MB?). They might even be able to collaborate, they could use the grid to add redundancy for their servers (they use raid1 now) or as a service for their customers in return for a permanent introducer node.

@Bit-Lex, I wouldn't install a linux-VM if you don't need to (you could better use the disk space for the grid itself ). If you scroll a bit further down on the install page there are instructions on how to install tahoe-lafs manually. You could try that.

[BitLex]

that's what i did, i just couldn't get it to work.


diskspace wouldn't be a problem at all, got a few hundred gigs left anyway and hdds arent that expensive nowadays,
bandwidth might (or will) be though (sooner or later), but i'll happily donate as much as possible of it.
so far i can't and i guess i'm not the only one.

[BioMike]

that's what i did, i just couldn't get it to work.


diskspace wouldn't be a problem at all, got a few hundred gigs left anyway and hdds arent that expensive nowadays,
bandwidth might (or will) be though (sooner or later), but i'll happily donate as much as possible of it.
so far i can't and i guess i'm not the only one.

Well, I had a check on a Windows system today. You have to go through some hoops to get it working (the description is not completely correct in some places and doing some things slightly different might help a lot).

Here is how I got it working:

1. Download and install Python 2.6.5 from  http://www.python.org/ftp/python/2.6.5/python-2.6.5.msi.

No problem here. I've used Python 2.6.6, which is out now. Install it in the default location (C:\Python26).

2. Download and install pywin32 from  http://sourceforge.net/projects/pywin32/files/pywin32/Build%20214/pywin32-214.win32-py2.6.exe/download

Just install, no problems here.
3. Now install MingW as described in the "What if that doesn't work?" part. Install it in C:\MingW This is not optional as the wiki might suggest.
I used file: http://sourceforge.net/projects/mingw/files/Automated%20MinGW%20Installer/mingw-get-inst/mingw-get-inst-20100909/mingw-get-inst-20100909.exe/download It is important to check the c++ compiler, base system, development system and the build system when you get that option to select it. This is different then what is told on the wiki.
4. Then continue with:

Download the latest Tahoe-LAFS release from http://tahoe-lafs.org/source/tahoe-lafs/releases/. Unpack it in a convenient place, such as C:\tahoe-lafs.

I installed this initially in C:\Program Files\Tahoe-LAFS. This does not work. The space in "Program Files" causes problems, so install it in C:\Tahoe-LAFS

5. Open a command prompt and cd to the top of the Tahoe-LAFS tree (e.g. cd \tahoe-lafs).
Run "C:\Python26\python setup.py build". Wait a bit until it stops working. Note that you need a working network connection because it will download various dependencies. Ignore any warnings.

After the cd c:\Tahoe-LAFS there is a directory below that ("allmydata" or something) cd into that dir also, the wiki doesn't say to do that.

From this point on you can continue with the normal installation (point 6) with one modification. The wiki says that the tahoe script is in C:\Python26\Scripts, well, it isn't. It is in the bin directory in where you did the setup.py build command C:\Tahoe-LAFS\allmydata-something\bin\

I know this is written down a bit confusing, I hope to provide a more clear "Howto" later, when I have access to a windows system again.

[grondilu]

Well, using a cloud filesystem storage would be great for wallet backup (after encryption, of course).

But I checked the availibilty of tahoe on my debian system, and "apt-cache seach tahoe" returned nothing.

I still have to check out the website of this stuff, though.

[kiba]

As soon as I got a node running and finish fixing my system clock, I'll pledge 5 BTC for whoever integrated bitcoins into the tahoe network.

[BioMike]

Well, using a cloud filesystem storage would be great for wallet backup (after encryption, of course).

No need for encryption, tahoe does that for you. That is why tahoe is so great.

For debian you need to add a repository to your sources.list, see the install page.

[kiba]

Code:

  File "/home/kiba/projects/tahoe-lafs/support/bin/tahoe", line 6, in <module>
    from pkg_resources import load_entry_point
ImportError: No module named pkg_resources

I got this error. Help?

[BioMike]

Code:

  File "/home/kiba/projects/tahoe-lafs/support/bin/tahoe", line 6, in <module>
    from pkg_resources import load_entry_point
ImportError: No module named pkg_resources

I got this error. Help?

Google says that setuptools should be installed.

[kiba]

Code:

  File "/home/kiba/projects/tahoe-lafs/support/bin/tahoe", line 6, in <module>
    from pkg_resources import load_entry_point
ImportError: No module named pkg_resources

I got this error. Help?

Google says that setuptools should be installed.

Googling doesn't seem to turn up a solution. I try easy_install the .egg but that doesn't exactly work. Perhaps, I'll need to install easy_install too.

[BioMike]

Code:

  File "/home/kiba/projects/tahoe-lafs/support/bin/tahoe", line 6, in <module>
    from pkg_resources import load_entry_point
ImportError: No module named pkg_resources

I got this error. Help?

Google says that setuptools should be installed.

Googling doesn't seem to turn up a solution. I try easy_install the .egg but that doesn't exactly work. Perhaps, I'll need to install easy_install too.

Strange, easy_install seems to be part of setuptools. Could be different with your distro though.

[kiba]

Code:

  File "/home/kiba/projects/tahoe-lafs/support/bin/tahoe", line 6, in <module>
    from pkg_resources import load_entry_point
ImportError: No module named pkg_resources

I got this error. Help?

Google says that setuptools should be installed.

Googling doesn't seem to turn up a solution. I try easy_install the .egg but that doesn't exactly work. Perhaps, I'll need to install easy_install too.

Strange, easy_install seems to be part of setuptools. Could be different with your distro though.

I pulled directly from darcs.

[BioMike]

I pulled directly from darcs.

I'm normally not a big fan of using development code from a cvs, but anyway.

Did you already try to do the following?:
Change to /home/kiba/projects/tahoe-lafs/
and run

Code:

python setup.py build

[BioMike]

Someone is trying to do something against the difficult installation of Tahoe-LAFS:

http://tahoe-lafs.org/pipermail/tahoe-dev/2010-October/005415.html

Someone might want to try out his tree.

[kiba]

I pulled directly from darcs.

I'm normally not a big fan of using development code from a cvs, but anyway.

Did you already try to do the following?:
Change to /home/kiba/projects/tahoe-lafs/
and run

Code:

python setup.py build

I did that already.

[BioMike]

Do you have setuptools installed? Did you try a reinstall of setuptools? And what version?

It seems version 1.8.1 will be released within 2 weeks that will have binary packages of all dependencies
with it bundled. Might make a few things more easy.

[jgarzik]

Is anybody still experimenting actively with a Tahoe-LAFS grid?