[FEEDBACK REQUEST] Bitcoin cloud services

[tgerring]

Trying to figure out if developers would make use of bitcoin cloud services:

• pre-configured bitcoind VM
  
  • up-to-date blockchain
  • txindex=1 for full querying
  • optionally enable an authenticated REST interface to interact with bitcoind, including "spend from address" functionality
  • optionally enable Abe for blockchain viewing
• block/transaction notifications delivered via Webhooks
• high-availability, low-latency, geo-distributed REST-queryable blockchain cache

The idea being to make it easier for developer to code something new without having to deal with the pain of setting up the client. The services would be designed to complement the model being done by other cloud providers, so you can string together services with some REST calls and Webhooks.

Feedback requested, whether positive, negative, or otherwise.

Thanks!

[tgerring]

Bump for formatting update. Looking for all feedback, positive or negative! Lots of views, but no comments?

[🏰 TradeFortress 🏰]

Personally, I would never host Bitcoins on a machine I don't own the metal to.

[tgerring]

Personally, I would never host Bitcoins on a machine I don't own the metal to.

I know trust is a concern many people in this community have, but it's a sliding scale with level of service/amount of trust. Sure, it's feasible to host your own metal somewhere, but you're still trusting the network providers, hosting company, hardware manufacturers, etc.

Would you be more interested if the machines were dedicated versus virtual? Or is "renting" a service completely out of the question for you, personally?

[greBit]

This would interest me providing I could do some sort of split private key arrangement.

[tgerring]

This would interest me providing I could do some sort of split private key arrangement.

Is related to split keys a la vanitygen pools?

My idea for ensuring that I don't have any access to your sensitive data would be to encrypt the partition containing the datadir using a customer-supplied key. It would be similar (the same?) as using a keypair for login, just as Amazon handles it. I'm not 100% sure on the implementation, but it's part of the reason why I'm here gathering feedback. I want to build something people feel comfortable using.

[greBit]

Sorry I wasn't particularly clear with my mini reply

Basically yes this would be of interest to me, a VM can be under my control so even it having my private key wouldnt concern me too much.

But if there was a way to have a secure-mode, whereby the server never sees my private key, that would be awesome. Perhaps the API could expose a mechanism whereby transactions are given to the client to do the signing, instead of the signing taking place at server end

But in any case an open-source VM with a RESTful API to bitcoind (with some nice blockchain querying) would be pretty cool.

[greBit]

My idea for ensuring that I don't have any access to your sensitive data would be to encrypt the partition containing the datadir using a customer-supplied key. It would be similar (the same?) as using a keypair for login, just as Amazon handles it. I'm not 100% sure on the implementation, but it's part of the reason why I'm here gathering feedback. I want to build something people feel comfortable using.

Basically if at any point my private-key touches the memory of some server I do not have complete control over, I would not use it.

[tgerring]

My idea for ensuring that I don't have any access to your sensitive data would be to encrypt the partition containing the datadir using a customer-supplied key. It would be similar (the same?) as using a keypair for login, just as Amazon handles it. I'm not 100% sure on the implementation, but it's part of the reason why I'm here gathering feedback. I want to build something people feel comfortable using.

Basically if at any point my private-key touches the memory of some server I do not have complete control over, I would not use it.

And just to be clear, having a VM authenticated by your own keypair would be sufficient "complete control"? My thought was to have all the binaries pre-built and ready to use, but also leave the source on the system for you to re-build and replicate the final setup at any point. Additional features could be added to give the VM operator maximum flexibility like automatic encrypted wallet backup, scheduled sweep to address, wallet reset, etc.

My goal is to make it secure & easy for developers to use so they can decide how to manage their funds. Does anyone else feel that a secured, preloaded, source-available VM would be beneficial to the development community? Or is everyone happy custom coding against the raw bitcoind API?

[Abdussamad]

Basically you are offering a pre-configured VPS for bitcoin related developers? IMO the market is too small for this. There aren't that many developers.

Also if you provide them with something like this they would also expect to be able to turn to you for future tech support. Would you be willing to provide tech support for bitcoind? Even if you manage to price that in you would still struggle to do it because bitcoind is very much beta software.

[greBit]

And just to be clear, having a VM authenticated by your own keypair would be sufficient "complete control"? My thought was to have all the binaries pre-built and ready to use, but also leave the source on the system for you to re-build and replicate the final setup at any point. Additional features could be added to give the VM operator maximum flexibility like automatic encrypted wallet backup, scheduled sweep to address, wallet reset, etc.

I think its quite reasonable to say that "I never ever want my private key to be seen by any other person."

To be fairly sure of this I must require that I be the sole person to have access to any physical machine that ever puts my PK in its memory.

Therefore, I would not dream of giving my key to a VM in the cloud somewhere, unless the key unlocks only trivial sums of money.

So if you published a VM building script (perhaps VagrantUp ?), I would build it from source and run it one of my physically-secured computers.

[tgerring]

Basically you are offering a pre-configured VPS for bitcoin related developers? IMO the market is too small for this. There aren't that many developers.

I see so many posts of people with NEXT BIG IDEA(tm). You don't feel these people would be interested in trying the idea if it wasn't so cumbersome to get bitcoind working? Even if you do manage to get it running, eventually you run into the "spend from address" problem

The goal not just to make bitcoind instantly-available through a VM, but rather to make it easier to develop against through logic abstraction on the REST layer. I'm working on adapting contrib/sprendfrom that Gavin contributed to exist as a standalone library including additional functionality and would like to open-source it to be developed further.

[tgerring]

Perhaps the API could expose a mechanism whereby transactions are given to the client to do the signing, instead of the signing taking place at server end

This wouldn't be too difficult to implement architecturally, but what would be an easy way for the customer to handle signing of every single transaction in an automated way? Perhaps set up a service they host themselves... but where? Maybe a cross-platform open-source client could be developed to proxy these requests? It might be of interest to some... happy to hear from those people

The point of the machines wouldn't necessarily be to host vast amounts of value--in fact, I'd like to provide automatic sweep functionality to external addresses. How the developer manages the funds on their VM is up to them, but it's perfectly conceivable that they could have their own "minimum balance" script monitoring the wallet of the environment and sending funds there to replenish as needed. If you can spin up fully-loaded bitcoind VMs at a moment's notice, I would expect there's some interesting things people could do by sending balances between them with a few calls to cURL. I'm not advocating to host mixing services, but I am trying to point out the vast possibilities with bitcoind "in the could".

[btcmind]

Good idea, yes. But solving this issue technically is very hard. Putting "some thing" on the blockchain doesn't solve anything.

It does not make sense to ship any code to an unknown location. If I can be "certain" then one might be able to ship code to a "brokered-cloud", similar to the electric grid. And this with potentially very advanced capabilities. I think you should have a look at docker and trusted computing + containers. The worlds second biggest exchange (and by far the most sophisticated) is currently working on a cloudexchange. Overall this might take 5 or 20 years to build. But it makes a lot of sense if you think about it. But there are many legal layers which allow exchanges to work in the first place. For example: why is it today not possible to have one world-wide global common law? Definitions of contracts are not at all the same in different countries. And of course there is no such thing as a global court, which enforces global contracts. By way of analogy, Bitcoin is a bit like TCP/IP and many problems are really on a much higher level (HTTP, "SOAP", ecommerce). Just because there is a website amazon.com based in Seattle, doesn't mean they can sell things to everyone in the world. Amazon has 88'000 employees, and most work is probably dealing with local issues.

I think you have to be clearer on what a transaction is, and for what transactions Bitcoin is good for and why.

[tgerring]

Good idea, yes. But solving this issue technically is very hard. Putting "some thing" on the blockchain doesn't solve anything.

It does not make sense to ship any code to an unknown location. If I can be "certain" then one might be able to ship code to a "brokered-cloud", similar to the electric grid. And this with potentially very advanced capabilities. I think you should have a look at docker and trusted computing + containers. The worlds second biggest exchange (and by far the most sophisticated) is currently working on a cloudexchange. Overall this might take 5 or 20 years to build. But it makes a lot of sense if you think about it. But there are many legal layers which allow exchanges to work in the first place. For example: why is it today not possible to have one world-wide global common law? Definitions of contracts are not at all the same in different countries. And of course there is no such thing as a global court, which enforces global contracts. By way of analogy, Bitcoin is a bit like TCP/IP and many problems are really on a much higher level (HTTP, "SOAP", ecommerce). Just because there is a website amazon.com based in Seattle, doesn't mean they can sell things to everyone in the world. Amazon has 88'000 employees, and most work is probably dealing with local issues.

I think you have to be clearer on what a transaction is, and for what transactions Bitcoin is good for and why.

I have no idea what you're trying to say here 

The initial idea is a (probably open-source) VM that can be instantly spun-up in the cloud with up-to-date blockchain. It would be secured by your own keypair--so the host has no access to the machine--and 100% be yours so tinker with. The value is that it's "instantly" available to spin-up and the RESTful layer would be simple enough for even armchair programmers to control.

• Need a full node for PoS transaction security? Spin up a VM
• Want to develop a new application against testnet? Spin up a VM--I'll have that blockchain up-to-date too
• Want a temporary wallet in the cloud while you travel? Spin up a VM and load some money while you're away from home. Safer than storing it on your phone and easy to refill your phone from anywhere

There are a whole other slew of use cases and they could probably all be built off these VMs with service layers on top. It's unreasonable to expect every developer to interact with raw bitcoind just to try a new idea.

[btcmind]

In general some good thoughts. I agree there are some use-cases somewhere.

But for the specific idea of cloud services: I can do that for 5$/month already at digitalocean (Linode starts at 10$). I certainly wouldn't use such a machine in production, let alone for sensitive data. Monitoring service levels of machines is very hard. Perhaps I'm misunderstanding something.

[tgerring]

Monitoring service levels of machines is very hard. Perhaps I'm misunderstanding something.

When you abstract away the difficulty of replicating the VMs, I think monitoring becomes trivial:

* Each VM has a base OS on the main volume that is "instantly" cloneable
* A secondary user volume is created, formatted, and mounted at ~/.bitcoind/
* Necessary user configuration copied into place including up-to-date blockchain data files copied from internal datastore
* bitcoind is started and is set to start on boot
* RESTful layer is started and set to start on boot
* VM is available for SSH authenticated by user key pair

End state is that a virtual machine can be spun up within a few minutes. Given the components above, it'd be trivial to move the user partition to a new instance if the first dies or needs to be killed. Monitoring metrics would have to be put in place to determine when a machine is killed/etc., but it's not an insurmountable task. And it's something that the theoretical service would be responsible for, obviously.

This is obviously in much more technical detail than I expect the average developer will want/need to know, but I'm hoping it makes things more clear. In the end, the daemon itself doesn't matter much. If you wanted some custom build of bitcoind to work in the same way, it'd be possible, but the workflow would be a little different.

Better? Worse? Happy to hear from anyone else that has an opinion!

[btcmind]

Unfortunately I don't understand. Are you talking about services for webapplication-developers? If so, what would be the advantage over AWS or other existing providers?

[tgerring]

Unfortunately I don't understand. Are you talking about services for webapplication-developers? If so, what would be the advantage over AWS or other existing providers?

Ease of use. How long does it take a novice to start coding against bitcoind JSON-RPC for the first time? Blockchain download alone is days at a minimum. Then learn about & download testnet. Then start working with the interface. Discover that you need txindex=1 for full querability. Discover the way that accounts and coin selection works isn't how you expect.

Hurdle after hurdle. Bitcoin isn't easy to code for right now. What would fix that?

[battani]

I love your idea and it's one that I've had for a while.

Currently, the market for bitcoin developers may be small, but it will only keep growing in the future. As you said, the market may be small because there are high barriers to entry, e.g. there is demand but developers are waiting for an easy and simple way to configure and deploy their bitcoin app. Abstracting bitcoind calls with a REST API could be one way to go. You could have bits of code for the most popular web and mobile frameworks that developers add to their app to bring up and connect to an instance of bitcoind.

Obviously the big question here is security, but if you provide enough value to developers and military-grade security, they will come to trust you. Coinbase has 300K users, and all their bitcoins are stored on servers customers don't own.

It's a big undertaking, but I can tell you there's definitely (gonna be) a market for this, as long as you focus on great service and security.