[ANN alfa] Full node wallet in a clouds, auto setup, start, stop. Sends as IFTTT

[emotional-engineering]

Own full node wallet in Amazon Clouds, open source, fully automatic, with start of server in reaction for outgoing transaction, sent from the service such as IFTTT.

** commit from the first of August **

The text below is not very relevant, because the project has progress with sending transactions from Lambda* functions, directly to the bitcoin network, using the ip addresses of the servers which were previously connected to the common full node server.
Module Duo Mobile Security also adapted for work from Lambda, so starting the server for each transaction not necessary.

* AWS Lambda is a compute service that runs your code in response to events

Now you can add "fast wallet", and put some money to them from "classic server wallet", which can be turned on later when you want to check incoming balance.
It is still own full node, but it use advanced cloud technology to increasing the speed of operations and decreasing resource consumption.

Also with new AWS service "API gateway", in combination with Lambda Functions, it is possible subscribes to real-time apis of some social services, without any servers and almost free.
For example, you can connect own installed facebook application on own cloud resources through real-time api.
This allows to send a transaction between two connected facebook pages:

https://www.facebook.com/walletinclouds

http://aws.amazon.com/api-gateway

Now presented only Amazon Clouds, but elements can be combined or expanded from analogs on another clouds, if such exist or when they will be.


Hello.

I create an open source system, built using Amazon Compute Cloud API and their services.
Now it's more or less works, link to the repository at the end of this text, and I'm interested in feedback in early stage.
It includes one fully automated script to integrate the system in your AWS account, which you can run from your computer.

In general, you will have your own full bitcoin node, but the server will start on a reaction to the outgoing transaction, and most of the time can be switched off.
Each time for the first transaction in an hour will start a new server with default OS, with all closed ports, including 22 ssh, excluding 8333 with smart control of traffic cost, and with private ssh key that not stored anywhere.
After starting the server, AWS Lamba scripts will mount EBS storage with your wallet and your copy of blockchain to the new server. (http://aws.amazon.com/lambda)
At the end of each billing hour, server will be automatically shutdown, if no active transactions.
The system runs on EC2 spot servers, which is usually several times cheaper then common servers, now it near 0.009$/hour in the US regions (m3.medium with 4GbRam, [upd] you can change it in config file. http://aws.amazon.com/ec2/purchasing-options/spot-instances).
The price of the server may be increased depending on the load on the cloud (http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances-history.html).
The server has a communication with the outside world only through the Amazon Simple Notification Service, Amazon Simple Queue Service, and authentification services like DUO mobile, which can be connected like modules.

Speaking in simple language - EC2 servers have excellent external firewall which call "Security Groups", with control by api without connecting to the server.
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html

If you use only "fast Lambda wallet", you will pay only for Elastic Block Storage with your copy of blockchain:
50GB of blockchain + 10GB extra = 60GB * 0.05$ GB/month for magnetic hard disk in N. Virginia region = 3$/month
For new users 30GB/month of them during the first year will be free: http://aws.amazon.com/free.
The Lambda functions free tier includes 1M free requests per month.

The system uses the Amazon Simple Notification Service, as a simple interface endpoint for outgoing transactions, and AWS lambda functions for managing the transactions and prepare the server. (http://aws.amazon.com/sns/)

I connect to them via zapier.com and can send commands from a lot of social services. You can create a lot of variants of schemes.
I hope when this will be finished, other services like IFTTT will also support the SNS, because in zapier you need premium account.
Now with "AWS API Gateway" and "Webhooks by Zapier" in some cases premium account is no needed.

After you import this system into your AWS account, you will have few streams to operate your wallet:

https://s3.amazonaws.com/bitforumscreenshots/second_ed.png

1) SNS stream for outgoing transactions.

After connection with services like zapier you can send simple command from any source: email, sms, trello or any you want.
The command such as:
18ozPxUtzyKgFGZ94PTxZUDfpiCqCzdwYm 1.764236

I'm use Trello, and create a new card in a special board for the new outgoing transactions.

https://s3.amazonaws.com/bitforumscreenshots/first.png

Also possible add outgoing stream, based on S3 automatically uploaded files. If you know some services which allow it, please write a comment.

2) multi-factor transactions authentification.

I'm doing a modular system with the ability to connect a few different or identical services running in series or in parallel.

Now it's 2 modules that are executed in parallel:

- Built-in system based on SNS and SQS streams: you receive a message with the random code and should send answer with the confirmation. Sources you connect in zapier: email, sms and other.

- Duo security with push authentication requests for mobile: https://www.duosecurity.com/product/methods/duo-mobile

In combination of sms and sns authorization, you can control the wallet without an internet connection.

It is possible to restrict access to SNS streams, and this allows to receive payment requests from untrusted applications.

3) Status stream.
The stream which will be receive status of transactions and of server.

Plans and what in the process:
 
1) Create a simple application for entering a password and pin code.
You can get a link such as "https://54.140.50.120:random_port/unique_string", via sms from AWS for example.
Each session will have different ip, port can be open only during the authorization for a limited time, for example 60 seconds.
Now password protected wallets are not supported.

2) Add stream to check the balance, get a new addresses and QR codes from the wallet. It can be run without starting the server.

3) Open port 8333 optional.

Port opened.

You can set how much you can spend for traffic per month.
Each time when the server is starting, system checking traffic bill.
If it not exceed the limit, system will check increased or decreased the number of full nodes in the world lately, using http://getaddr.bitnodes.io, and if it decreased - opens the incomming port.

4) Automatically change the size of the hard drive, when a blockchain becomes larger.

5) Change DynamoDB table to table based on S3 files. DynamoDB not neccesary and is several times more expensive then other services all together.

6) Fast Lambda Wallet in progress.

The server save IPs of connected nodes. Lambda functions using this IPs to send raw transactions.
Like I'm read in the docs, this functions ready to execute the code within 100ms after the occurrence event.

Thanks to the "bitcore" javascript library - http://bitcore.io

Scripts for integration in AWS account in progress.

7) Direct integration with social services.

https://www.facebook.com/walletinclouds

Scripts for integration in AWS account in progress.

8 ) "Fast server setup" in progress.

Each time you can use different configuration with your node. This can be used for first blockchain synchronization.
On the server with 16 or 40 CPUs and syncing all blockchain in RAM memory, the whole cycle takes 3.5 hours.
After server start, daemon was connected to the 500 servers from biggest data centers, taken from current snapshot of getaddr.bitnodes.io

9) Add other clouds.


This PRE-ALFA VERSION, NOT STABLE AND NOT TESTED, DON'T USE IT NOW, if you are not an nodejs and AWS expert.
If you have this knowledge, you can test it on a very small wallet and empty AWS account, and you must be absolutely sure that you will do.
Only for tests, please be very careful.

https://github.com/emotional-engineering/cloud-castle

If it can be useful, I will be happy to give ownership of the repository to any organization for the code control in the master branch.
Maybe there are ways to make a secure repository.
It may be better to use BitBucket, and make push to master branch after a few people approve changes.
I alone can't do it, I don't have the relevant experience and you have no reason to trust my code, so without doing the above items this not secure and can't be used.

It is important to know that if your wallet is not password protected, security will consists from the safety of your AWS account.
Having access to it can be easily to mount EBS storage with wallet to new not protected virtual machine.
This can be useful: http://docs.aws.amazon.com/IAM/latest/UserGuide/Using_ManagingMFA.html
Also, this can be avoided by using the common instances and disk encryption.

Any advices, issues or commits are welcome.

Also welcome any donations: 17GoS8cWoCmZMmGkW9SguGtUWMzekkYHDo

[2112]

Why not using the "Previous generation" instances with local ephemeral storage for the blockchain?

m1.small has 160GB that is doubly free (no per GB charges and no per IOPS charges)
m1.medium has 410GB doubly-free

They are both cheaper than m3.medium.

So what is the point of forcing everyone to double pay for EBS (per GB and per IOPS) to primarily store a copy of the shared blockchain which will be exactly the same for everyone (if everything works right)?

I'm really curious what is the technical point of all this rigmarole, aside from shilling for AWS EC2 and earning sales commissions?

Edit:

I just did a quick refresh of my AWS knowledge, and it seems like the my long-time observations still hold:

1) Bitcoin blockchain should be a shared, no-per-user charge, dataset hosted on S3 (one copy per region, 3 to cover USA and 10 to cover the whole world). You'll need to come up with some schedule on how to update them, e.g. one region update per day. This way the blockchain backup will be no more than 10 days old (if you cover the whole world) or 3 days old (if you just target USA).

2) rapidly restore the blocks and chainstate directory from that shared set before starting the bitcoind for the first time in the particular instance.

3) This should save money for the users, I don't know what will be the effect for the commission for the shill.

4) One can recognize good Amazon shills by how they provide a really differentiated, high-level service instead of relying on brute force multiplication of charges for storing duplicate data in each client's private storage.

[emotional-engineering]

normal m1.small    cost $0.044/hour, but i'm not sure that it must work with full node in production.
normal m1.medium cost $0.087/hour
spot   m3.medium cost $0.009/hour and the price may vary.

http://aws.amazon.com/ec2/purchasing-options/spot-instances

The type of server is doesn't matter here, and it's just microeconomics. Add support of normal instances is very easy, it is much easier then do the work with spot instances, that's why I added them first.

Magnetic hard disk not used IOPS, so you do not need to pay for it.

Very surprised to hear about my commission from Amazon.

[2112]

normal m1.small    cost $0.044/hour, but i'm not sure that it must work with full node in production.
normal m1.medium cost $0.087/hour
spot   m3.medium cost $0.009/hour and the price may vary.

http://aws.amazon.com/ec2/purchasing-options/spot-instances

The type of server is doesn't matter here, and it's just microeconomics. Add support of normal instances is very easy, it is much easier then do the work with spot instances, that's why I added them first.

Magnetic hard disk not used IOPS.

Very surprised to hear about my commission from Amazon, I don't know how they can understand that you store copy of blockchain and that they must pay commision.

Are you trying to pretend to be a retard that compares spot prices with normal prices? Why?

Current prices as of now in US East:

m1.small     $0.0070
m1.medium $0.0080
m3.medium $0.0081

Actually, it doesn't seem to be worthwhile to refute your claims one by one.

My current diagnosis: some panicked shill at an Amazon AWS Value Added Partner who spend time developing an expensive and complex overlay on top of Bitcoin Core. But his design was lame and couldn't handle chain reorganizations even in theory. Recent chain reorg made this very obvious and the shill isn't going to make a his shilling quota for the quarter and therefore goes into the panic mode to create some semblance of interest in his Rube-Goldberg-ian contraption.

[emotional-engineering]

The main thing about instances you advised:

"The local instance store volumes that are available to the instance. The data in an instance store is not permanent - it persists only during the lifetime of the instance."
It mean that if you shutdown the server, all data will be lost.

[emotional-engineering]

Are you trying to pretend to be a retard that compares spot prices with normal prices? Why?

I thought that m1.small and m1.medium is not available for spot request, now i'm found it.
And i think that you recommend to use normal instances for storage, spot instances allways fully deleted after shutdown.
I don't understand what you mean, because one type of spot instance you can change to another type of spot instance by design in config file.

Ok, one more time:
Instance type you can edit in config file.
Big local hard drive you can't use in both types of instances, because they are temporary.

[2112]

Dude!

Who cares that his copy of blockchain was erased? It is supposed to be the same all over the world.

The main thing about instances you advised:

"The local instance store volumes that are available to the instance. The data in an instance store is not permanent - it persists only during the lifetime of the instance."
It mean that if you shutdown the server, all data will be lost.

Are you trying to pretend to be a retard that compares spot prices with normal prices? Why?

I thought that m1.small and m1.medium is not available for spot request, now i'm found it.
And i think that you recommend to use normal instances for storage, spot instances allways fully deleted after shutdown.
I don't understand what you mean, because one type of spot instance you can change to another type of spot instance by design in config file.

Ok, one more time:
Instance type you can edit in config file.
Big local hard drive you can't use in both types of instances, because they are temporary.

I keep calling you are shill, because you keep treating blockchain like somehow unique and valuable data resource. It isn't. The real valuable data can be stored on an EBS volume of minimal size or in S3 snapshots. All else is just overhead and paying to keep it stored is just plain dumb.

If you want to do something valuable devise a way of storing blockchain in the Amazon RDS or DynamoDB and a way to interface them with the Bitcoin-Core. This way the blockchain will be stored only once per AWS region and will be treated like true shared resource it is.

That would be what makes the cloud computing shine technically: shared data is truly shared.

[emotional-engineering]

Who needs all these servers, if you already have Lambda functions, which can immediately send the transaction directly to the bitcoin network.

Direct connection to Facebook and sending transactions between 2 connected pages are also available:
https://www.facebook.com/walletinclouds

The first topic was updated.

[emotional-engineering]

We have the installation github page, which makes full setup using only browser javascript and aws api:

https://emotional-engineering.github.io/cloud-castle

Now it will try to install the new version, which is almost ready and will be very soon.

[RGBKey]

I don't like the idea of connecting a bitcoin wallet with IFTTT. So much could go wrong there. Maybe if it had only read-only access.