Bitcoin Forum
November 23, 2017, 04:38:05 AM *
News: Latest stable version of Bitcoin Core: 0.15.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: Time for an anonymous BTC based storage network?  (Read 1123 times)
iamsomeoneelse
Newbie
*
Offline Offline

Activity: 10


View Profile
January 22, 2013, 02:34:08 AM
 #1

If Mega won't take on Bitcoin maybe Bitcoin should take (on) Mega!  Smiley

I think that this has been briefly discussed previously but its time to consider it seriously.  It seems to me like a distributed p2p file storage network that uses BTC as the other side of the exchange could be very valuable.  Even if the majority of the storage/bandwidth providers end up being dedicated hosting companies/machines, the existence of a liquid, frictionless, fine-grained market for these services is very valuable.  Just look at ebay... at this point there are very few actual individuals selling as opposed to small companies.  But just like Ebay at first there would be a lot of individuals offering storage on their own PC.  And this could be an easy way to earn (and spend) BTC, as opposed to mining or dealing with an exchange.

Key markets:
  Backup
  Sharing
  Content delivery
  Streaming
 
Each market has interesting issues to solve.  For example, a backup is rarely accessed and so needs some kind of periodic verification to ensure that the provider has not deleted the data but is still taking the BTC.

Of course, a strong focus on privacy and encryption will ensure that untrusted hosting sites cannot access content.  This simultaneously protects these sites from hosting objectionable content (Dotcom's argument).  Is it necessary (possible) to obfuscate the source of the data in a manner similar to TOR?  Or the storage network recommend TOR be used for those who care?

Here's a use case example that is only possible because of the low friction:

To host files for Mega you've got to call them and do a deal which will result in you dedicating a certain # of machines, bandwidth and disk to Mega.  So there a lot of lawyers involved in hammering this out, etc.

Imagine instead that a provider of hosted services (VMs) also runs the BTC storage network nodes on its machines.  Whatever portion of the machine is not providing hosted services (I'm assuming that that would be higher margin) is allocated to the storage network and receiving BTC for services provided.  So the machine is 100% utilized all the time, resulting in a quicker ROI on the hardware.


We need to store the blockchain somewhere!  Grin


Join ICO Now Coinlancer is Disrupting the Freelance marketplace!
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1511411885
Hero Member
*
Offline Offline

Posts: 1511411885

View Profile Personal Message (Offline)

Ignore
1511411885
Reply with quote  #2

1511411885
Report to moderator
1511411885
Hero Member
*
Offline Offline

Posts: 1511411885

View Profile Personal Message (Offline)

Ignore
1511411885
Reply with quote  #2

1511411885
Report to moderator
1511411885
Hero Member
*
Offline Offline

Posts: 1511411885

View Profile Personal Message (Offline)

Ignore
1511411885
Reply with quote  #2

1511411885
Report to moderator
BlackLilac Jordan
Full Member
***
Offline Offline

Activity: 163



View Profile
January 22, 2013, 09:36:12 AM
 #2

Go for it.   Smiley
bonker
Hero Member
*****
Offline Offline

Activity: 560



View Profile
January 22, 2013, 09:47:10 AM
 #3

Go for it.   Smiley
+1 it's a great idea

Pies
kangasbros
Hero Member
*****
Offline Offline

Activity: 812



View Profile
January 22, 2013, 10:20:37 AM
 #4

This idea has revolved in my idea for a long time. Just that for it it to succeed, I would recommend starting with something really simple, and building from there onwards.

2weiX
Legendary
*
Offline Offline

Activity: 1834

bitcoincommodities.com


View Profile
January 22, 2013, 11:43:08 AM
 #5

awesome idea.

could also implement that, for example, streaming is more valuable than just backupping, for instance.
so if you as the service provider have sufficient bandwidth, you can adjust to hosting streams over backups.

the system will self-regulate in sich a way that streaming-hosters that prove to consistently deliver get chosen over those who don't.

incentives, baby!

Sukrim
Legendary
*
Offline Offline

Activity: 2198


View Profile
January 22, 2013, 01:34:02 PM
 #6

Well, you could check out the code on Tahoe-LAFS and help them implement the accounting backend (as far as I know they plan on integrating BTC in there as well)...

They are doing encrypted secure file storage with an Open Source solution already for years.

https://bitfinex.com <-- leveraged trading of BTCUSD, LTCUSD and LTCBTC (long and short) - 10% discount on fees for the first 30 days with this refcode: x5K9YtL3Zb
Mail me at Bitmessage: BM-BbiHiVv5qh858ULsyRDtpRrG9WjXN3xf
meowmeowbrowncow
Sr. Member
****
Offline Offline

Activity: 322



View Profile
January 22, 2013, 02:13:05 PM
 #7


Yep, this is gooooood.


There have been very interesting threads involving p2p storage and BTC.  I'll try to find them.


edit:

Building a fully functional CDN, including robust load balancing, leveraging p2p storage is tantalizing.


Wonders if p2p CDN could be the enabler for DDOS resistance.  hmmmm.....

"Bitcoin has been an amazing ride, but the most fascinating part to me is the seemingly universal tendency of libertarians to immediately become authoritarians the very moment they are given any measure of power to silence the dissent of others."  - The Bible
iamsomeoneelse
Newbie
*
Offline Offline

Activity: 10


View Profile
January 23, 2013, 04:12:38 AM
 #8

Let's start at the beginning, but please forgive me if this posting is not well organised.  Right now it is just a bunch of ideas on implementation issues.

Clearly we need a basic protocol for P2P discovery. 

This protocol can use public/private address grabbed directly from Bitcoin source code to uniquely identify entities.  Let's use the terminology "provider" and "subscriber" to distinguish who is offering and who is buying the storage, although the same entity may do both.

It is not (necessarily) useful to force files into standard block sizes because this system is highly abstracted from the underlying block device.  So let's define a block as an arbitrary-sized encrypted chunk of bytes identified by the SHA-256 hash, after encryption.  So an entire file can be stored in a block.  Or a file could comprise of a list of blocks (this may help access times).  Is my assumption that SHA-256 collisions would be essentially impossible acceptable?  So a subscriber does not modify a block; it simply uploads a new block.  For efficiency we can have a protocol message to delete old blocks, but ultimately the blocks are cleaned because the subscriber stops paying maintenance for them.

Of course, the subscriber encrypts/decrypts the blocks.  The provider cannot access the cleartext data.  But just like a bitcoin wallet, any subscriber with the private key can decrypt blocks so files can be shared simply by creating a new private key and giving it to your other devices and/or friends.  This becomes a "group share" key.  The subscriber is welcome to send the blocks to multiple providers to implement data redundancy, or content delivery.  And providers could act as a subscriber, request popular blocks from other providers and then compete on delivery price.


Layered on that, let's add a protocol for service requests and advertisements.

Very simply, this protocol allows a provider to advertise (and a client to request) capacity, bandwidth, access protocols (whether it willing to act as a gateway for protocols like http & streaming, or whether is will only pass encrypted blocks within the native storage protocol), privacy (will it divulge IP addr/location for CDN calculations), and the cost of the services.  Service cost splits out storage from access bandwidth so subscribers can choose the optimum provider for their needs. 

So the first issue is verification that the provider meets its advertised services.  Obviously, it would be unfortunate for a provider to just turn off his computer permanently.
A distributed keepalive mechanism could calculate every provider's uptime.
Subscribers can issue challenges of the form "calculate the SHA-256 of this portion of a block" to a provider to ensure that the provider is retaining the data.
How about a trust network?

This meta-data should be universally accessible and associated with the provider and subscriber's public key.

How shall all the trust attacks be defeated?  A POW blockchain is the obvious answer but that requires CPU.  I am wondering if we cannot rely directly on the Bitcoin blockchain either by requiring transaction fees to modify this meta-data or by determining some level of trust by an analysis of the revenue received by the provider's address.  To defeat fake transactions, this analysis would have to examine the sending addresses and determine that they are generally involved in other providers as well.  I'd love some of the devs opinions on this question!!!

But I do not see this as an app-killer even if unresolved.  We will also allow providers to connect their identity to a real-life identity by piggy-backing on the efforts to do the same with Bitcoin (ultimately by having the public key squirreled away in a https request that is verified by a cert authority, I'd imagine).  So in the worst case subscribers with strong data preservation requirements only deal with identified provider companies that are committed to availability.

How can payments be made?  Perhaps via an automatic 2-of-3 escrow.  I wonder if a proof-of-work network could act as the escrow arbiter.  Or, since we are talking about small transactions here, can failed escrow bitcoin somehow count against both parties and then be raked into transaction fee style fund?  It is important that some mechanism exist to trickle-pay for the maintenance of blocks even if the subscriber is offline.  The best option would be a POW network as a whole acts as an escrow agent, but in the worst case redundant 2-of-3 escrow agents could be picked randomly out of entities that are advertising this service.

Anyway, these are some ideas... I'm interested in your thoughts.
jl2012
Legendary
*
Offline Offline

Activity: 1722


View Profile
January 23, 2013, 05:00:47 AM
 #9

Just a rough idea

It is a P2P network. A node (A) wants to offer storage space will broadcast messages like "I have 2TB of free space. I will host your files at 0.001BTC/MB/month". B reads the message and sends 10MB of data to A, and sends 0.01BTC to escrow. In the following month, B will send some random request to A, asking for some random bytes of the data (This is to make sure A is really holding all data). B will release the escrow if and only if A gives correct answers. On the other hand, if A does not receive payment after giving correct answers, it will drop B's data.

Donation address: 374iXxS4BuqFHsEwwxUuH3nvJ69Y7Hqur3
PGP: D3CC 1772 8600 5BB8 FF67 3294 C524 2A1A B393 6517
iamsomeoneelse
Newbie
*
Offline Offline

Activity: 10


View Profile
January 23, 2013, 01:36:58 PM
 #10

Just a rough idea

It is a P2P network. A node (A) wants to offer storage space will broadcast messages like "I have 2TB of free space. I will host your files at 0.001BTC/MB/month". B reads the message and sends 10MB of data to A, and sends 0.01BTC to escrow. In the following month, B will send some random request to A, asking for some random bytes of the data (This is to make sure A is really holding all data). B will release the escrow if and only if A gives correct answers. On the other hand, if A does not receive payment after giving correct answers, it will drop B's data.

yes!  that's exactly what I'm talking about in the above message but I was suggesting a SHA-256 checksum over a broader swatch of data.  But in retrospect I can't think of why this would be more effective then just asking for random bytes as you propose.  Also, I'm thinking that it might be desireable to delegate this task (periodically dropping coins in the machine) to some 3rd party "agent" that has greater uptime then the subscriber.

A quick look at Tahoe-LAFS is making me think that it over-structures the data (it pulls issues like redundancy into the cluster, whereas I do not see why those issues cannot be handled at the client) compared to what I want.  Too much structure is going to limit the applicability.  And it seems to have no market, whereas in this proposal the market is just as important as the storage.
Fiyasko
Legendary
*
Offline Offline

Activity: 1428


Okey Dokey Lokey


View Profile
January 23, 2013, 02:00:27 PM
 #11

I was about to walk in screaming "HOW STUPID ARE YOU!?!!?" And then i started to read, Saving myself from being an idiot.
a P2P bitcoin Trading Market is what we need, None of this MT.Gox and stuff. They can be shutdown!

Its not a P2PStorage network we need, Its a P2P Buy/Sell market that we need

http://bitcoin-otc.com/viewratingdetail.php?nick=DingoRabiit&sign=ANY&type=RECV <-My Ratings
https://bitcointalk.org/index.php?topic=857670.0 GAWminers and associated things are not to be trusted, Especially the "mineral" exchange
TraderTimm
Legendary
*
Offline Offline

Activity: 2002



View Profile
January 23, 2013, 03:04:55 PM
 #12

This is a very interesting idea - how about combine aspects of both? (Storage and peer-to-peer decentralized trading)

One thing that came to mind is this for the front-end:

https://en.wikipedia.org/wiki/Osiris_%28Serverless_Portal_System%29

And combining aspects of this for filesharing via the web portal above:

https://en.wikipedia.org/wiki/OneSwarm

Got me thinking here... I really like the idea of offering up bandwidth/storage for BTC, assuming of course inbound/outbound is encrypted and I have no way to decrypt the files. (Common carrier defense.)

As for accepting files for bitcoins - I think you could use a system where waiting for at least three confirmations to start file transfer, checking for the final confirm to retain the file in storage.

fortitudinem multis - catenum regit omnia
jl2012
Legendary
*
Offline Offline

Activity: 1722


View Profile
January 23, 2013, 03:33:55 PM
 #13

Just a rough idea

It is a P2P network. A node (A) wants to offer storage space will broadcast messages like "I have 2TB of free space. I will host your files at 0.001BTC/MB/month". B reads the message and sends 10MB of data to A, and sends 0.01BTC to escrow. In the following month, B will send some random request to A, asking for some random bytes of the data (This is to make sure A is really holding all data). B will release the escrow if and only if A gives correct answers. On the other hand, if A does not receive payment after giving correct answers, it will drop B's data.

yes!  that's exactly what I'm talking about in the above message but I was suggesting a SHA-256 checksum over a broader swatch of data.  But in retrospect I can't think of why this would be more effective then just asking for random bytes as you propose.  Also, I'm thinking that it might be desireable to delegate this task (periodically dropping coins in the machine) to some 3rd party "agent" that has greater uptime then the subscriber.

A quick look at Tahoe-LAFS is making me think that it over-structures the data (it pulls issues like redundancy into the cluster, whereas I do not see why those issues cannot be handled at the client) compared to what I want.  Too much structure is going to limit the applicability.  And it seems to have no market, whereas in this proposal the market is just as important as the storage.


You may ask for SHA-256 of random fractions of the data, just to prevent the storage nodes from cheating. That could be more efficient because no matter how big the fractions are, the checksum is always 256bits.

You may use something like the so called RAID-96 system: http://www.symform.com/blog/tag/raid-96/

People may also exchange storage space in 1:1 ratio, so they don't need to pay each other

Donation address: 374iXxS4BuqFHsEwwxUuH3nvJ69Y7Hqur3
PGP: D3CC 1772 8600 5BB8 FF67 3294 C524 2A1A B393 6517
ahdinosaur
Jr. Member
*
Offline Offline

Activity: 38

hi, i'm dinosaur. :)


View Profile WWW
January 24, 2013, 09:20:22 AM
 #14

+1 for purchasing tahoe-lafs storage with bitcoin. https://tahoe-lafs.org/trac/tahoe-lafs/wiki/Ostrom
is this bounty still active? https://bitcointalk.org/index.php?topic=2236.20

1FRW9Sd6EBedduZ8oZWdkJBdRHYCvLxrYe
profiles: bitcoin-otc
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!