Bitcoin Forum
April 23, 2014, 11:09:31 PM *
News: Due to the OpenSSL heartbleed bug, changing your forum password is recommended.
 
   Home   Help Search Donate Login Register  
Pages: [1] 2  All
  Print  
Author Topic: Is it possible to run Bitcoin on Google App Engine? / alternatives?  (Read 2913 times)
stevenroose
Member
**
Offline Offline

Activity: 63



View Profile WWW

Ignore
June 21, 2012, 09:23:38 PM
 #1

Can anyone please move this topic to the Development section?
This question is cross-posted on bitcoin.stackexchange, stackoverflow and bitcointalks.



I'm planning to build an application on Google App Engine that will heavily make use of Bitcoin trading. I've been Googling along a little but I couldn't find whether it is possible to run Bitcoin itself on App Engine (with Java). I have some experience with App Engine, but limited to a pure web-app centered usage.
I've read about a few people that have made applications using Bitcoin with App Engine as well that are hosting Bitcoin separately on an Amazon EC2 instance.

So, does anyone here either has experience with running Bitcoin in App Engine for Java or would anyone have an idea how this could possibly be done?
I know there are a lot of Bitcoin applications out there, I'd like to know how these manage their Bitcoin traffic.

I'm trying to avoid needing a separate Amazon service running all the time next to App Engine.

In fact, receiving Bitcoin can easily be done by using passive APIs like blockexplorer or blockchain, so I'm considering to find a reliable API to handle my outgoing payments. But this approach causes extreme dependency on this API service, which I actually want to avoid as much as possible.

There is one thing I will never understand about Bitcoin.
Why didn't Satoshi Nakamoto start with a 100 BTC bounty per block instead of 50. 100 is so much more logical.
But especially, 42 million is a tremendously fancier number for the maximum amount of bitcoins in circulation than the current 21 million.
1398294571
Hero Member
*
Offline Offline

Posts: 1398294571

View Profile Personal Message (Offline)

Ignore
1398294571
Reply with quote  #2

1398294571
Report to moderator
1398294571
Hero Member
*
Offline Offline

Posts: 1398294571

View Profile Personal Message (Offline)

Ignore
1398294571
Reply with quote  #2

1398294571
Report to moderator
Unbeatable Service & Product Support
Grab Your Miners at GAWMiners.com
Order Before April 25th to receive
Double your Hashing Power for 1 week!

Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1398294571
Hero Member
*
Offline Offline

Posts: 1398294571

View Profile Personal Message (Offline)

Ignore
1398294571
Reply with quote  #2

1398294571
Report to moderator
1398294571
Hero Member
*
Offline Offline

Posts: 1398294571

View Profile Personal Message (Offline)

Ignore
1398294571
Reply with quote  #2

1398294571
Report to moderator
Stephen Gornick
Hero Member
*****
Offline Offline

Activity: 1246



View Profile WWW

Ignore
June 21, 2012, 09:34:32 PM
 #2

\So, does anyone here either has experience with running Bitcoin in App Engine for Java or would anyone have an idea how this could possibly be done?
I know there are a lot of Bitcoin applications out there, I'd like to know how these manage their Bitcoin traffic.

I'm trying to avoid needing a separate Amazon service running all the time next to App Engine.

If what you are after is the blockchain there might be a way to replicate the data into App Engine's Google Cloud SQL (from libbitcoin's MySQL data, perhaps).  

But if you are wanting to put a wallet on App Engine, here's an idea: Why don't you give me half the bitcoins you were thinking of storing, then we'll go out back, I'll kick you in the nuts and we'll call it a day.



 - http://www.youtube.com/watch?v=CEeqCbEFIJw (length: 10s)
 - http://www.youtube.com/watch?v=lvDu3IysKiM (length: 60s)

i.e.,  If there is any lesson to be learned from the Bitcoinica mess is that shared systems and cloud infrastructure cannot be secured.

stevenroose
Member
**
Offline Offline

Activity: 63



View Profile WWW

Ignore
June 21, 2012, 09:41:01 PM
 #3

But if you are wanting to put a wallet on App Engine, why don't you give me half the bitcoins you were thinking of storing, then we'll go out back, I'll kick you in the nuts and we'll call it a day.

i.e.,  If there is any lesson to be learned from the Bitcoinica mess is that shared systems cannot be secured.

What's so less secure on running Bitcoin in App Engine compared to running it on an Amazon instance? In the latter option you have to make a secure connection between App Engine and this instance, which in my eyes leads to the same, if not even more, security issues?

There is one thing I will never understand about Bitcoin.
Why didn't Satoshi Nakamoto start with a 100 BTC bounty per block instead of 50. 100 is so much more logical.
But especially, 42 million is a tremendously fancier number for the maximum amount of bitcoins in circulation than the current 21 million.
Stephen Gornick
Hero Member
*****
Offline Offline

Activity: 1246



View Profile WWW

Ignore
June 21, 2012, 09:47:36 PM
 #4

What's so less secure on running Bitcoin in App Engine compared to running it on an Amazon instance? In the latter option you have to make a secure connection between App Engine and this instance, which in my eyes leads to the same, if not even more, security issues?

Neither can be secured.

This is some useful reading:
 - http://bitcointalk.org/index.php?topic=81045.msg894870#msg894870

stevenroose
Member
**
Offline Offline

Activity: 63



View Profile WWW

Ignore
June 21, 2012, 09:56:24 PM
 #5

Of course we are already working with strong caution and we know that hosting all your coins on the web is dumb.
We were thinking about a daily routine checking what the average amount of bitcoins is required per day. Have this amount + some buffer available for the service and just have all the rest sitting somewhere on an offline address.
This would inevitably mean we have to manually refill the online reserve when it got lower than the calculated required amount, but we are aware of the safety issues of not doing this.

That's also the big reason why I wanted to host on App Engine and not on a self-managed Amazon instance. I trust Google when it comes to server security. I trust them way more than myself, since I'm not a security expert, and also trust them more than I trust Amazon. So I thought App Engine would be the safest solution.

But as ThePiachu pointed out on the Bitcoin StackExchange, hosting a bitcoin client on App Engine is not possible.

There is one thing I will never understand about Bitcoin.
Why didn't Satoshi Nakamoto start with a 100 BTC bounty per block instead of 50. 100 is so much more logical.
But especially, 42 million is a tremendously fancier number for the maximum amount of bitcoins in circulation than the current 21 million.
gweedo
Hero Member
*****
Offline Offline

Activity: 728


Java, PHP, HTML/CSS Programmer for Hire!


View Profile WWW

Ignore
June 21, 2012, 11:24:01 PM
 #6

Why don't you get a small VPS, put bitcoind on it, lock it down so only request from and to GAE are possible using the linux firewall, you know lockdown SSH. And you should be good to go.

http://www.apicoin.io - making the blockchain rest | Donations: 1GweedoZJYb5CNLfSaBgBBYS2y7BMVb2Wo| Tox: 8814C1F411BBF9FCCFA3338DEB23E4DCCAEC2BF7E89C0048F2C23389651C071959F0EB5F4A58 |Escrow (0.5% FEE), just PM | PGP key |The Mark Cuban of this Forum
Bitcoin Oz
Hero Member
*****
Offline Offline

Activity: 672


Wat


View Profile WWW

Ignore
June 21, 2012, 11:36:42 PM
 #7

Dedicated server in a locked cage is the best choice for financial sites.

stevenroose
Member
**
Offline Offline

Activity: 63



View Profile WWW

Ignore
June 24, 2012, 12:56:06 PM
 #8

Ok,  thanks!

There is one thing I will never understand about Bitcoin.
Why didn't Satoshi Nakamoto start with a 100 BTC bounty per block instead of 50. 100 is so much more logical.
But especially, 42 million is a tremendously fancier number for the maximum amount of bitcoins in circulation than the current 21 million.
MoneyIsDebt
Full Member
***
Offline Offline

Activity: 179



View Profile

Ignore
June 24, 2012, 03:50:54 PM
 #9

I don't know about your VPS, but mine can't even fit the blockchain anymore.

Sig for sale
stevenroose
Member
**
Offline Offline

Activity: 63



View Profile WWW

Ignore
June 24, 2012, 04:17:22 PM
 #10

I don't have anything running yet, so can't tell.

Hmm, yeah the blockchain seems to be growing exponentially, quite frightening...
How much memory does it take to host it? Is it equal to the size listen here https://blockchain.info/charts/blocks-size or is it more?

There is one thing I will never understand about Bitcoin.
Why didn't Satoshi Nakamoto start with a 100 BTC bounty per block instead of 50. 100 is so much more logical.
But especially, 42 million is a tremendously fancier number for the maximum amount of bitcoins in circulation than the current 21 million.
John (John K.)
Global Troll-buster and
Hero Member
*****
Offline Offline

Activity: 798



View Profile

Ignore
June 24, 2012, 04:20:02 PM
 #11

You're whitelisted, so feel free to post at the Development subforum.

My BTC Tip Jar: 1NB1KFnFqnP3WSDZQrWV3pfmph5fWRyadz , GPG ID: B3AAEEB0 ,OTC ID: johnthedong
stevenroose
Member
**
Offline Offline

Activity: 63



View Profile WWW

Ignore
June 24, 2012, 04:48:04 PM
 #12

Thanks!

I have another question about VPS services.

If a VPS is setup only for your Bitcoin transactions, the Blockchain doesn't have to be loaded into RAM, right?
Most simple VPS services offer at least 10GB in their most basic setup, so that can't be a problem?

I'm not sure and this quote makes me even less sure Smiley

I don't know about your VPS, but mine can't even fit the blockchain anymore.

Can anyone confirm / correct?

There is one thing I will never understand about Bitcoin.
Why didn't Satoshi Nakamoto start with a 100 BTC bounty per block instead of 50. 100 is so much more logical.
But especially, 42 million is a tremendously fancier number for the maximum amount of bitcoins in circulation than the current 21 million.
gweedo
Hero Member
*****
Offline Offline

Activity: 728


Java, PHP, HTML/CSS Programmer for Hire!


View Profile WWW

Ignore
June 24, 2012, 04:49:32 PM
 #13

Thanks!

I have another question about VPS services.

If a VPS is setup only for your Bitcoin transactions, the Blockchain doesn't have to be loaded into RAM, right?
Most simple VPS services offer at least 10GB in their most basic setup, so that can't be a problem?

I'm not sure and this quote makes me even less sure Smiley

I don't know about your VPS, but mine can't even fit the blockchain anymore.

Can anyone confirm / correct?
no 10gb should be good for now, but keep an eye on it and make sure your not going to run out of space in the future.
and if you get 512mb for the ram and only run bitcoind on it you should be good.

http://www.apicoin.io - making the blockchain rest | Donations: 1GweedoZJYb5CNLfSaBgBBYS2y7BMVb2Wo| Tox: 8814C1F411BBF9FCCFA3338DEB23E4DCCAEC2BF7E89C0048F2C23389651C071959F0EB5F4A58 |Escrow (0.5% FEE), just PM | PGP key |The Mark Cuban of this Forum
Mike Hearn
Hero Member
*****
expert
Offline Offline

Activity: 1232


View Profile

Ignore
June 25, 2012, 09:25:46 PM
 #14

I work for Google. AppEngine servers aren't going to get hacked. However, your admin account might well do. If you want to do that, you should at the very least activate 2-step verification. Even then it's not unhackable - people have managed to access such accounts by taking over the backing phone numbers (eg, by tricking the phone company or starting a number port). So it's not a panacea.

AppEngine is designed to load and unload servers on demand, it's not really designed for a long-running P2P process. But it can still make sense for your app - as mentioned earlier, you can run bitcoind on a locked down machine that only accepts connections from your service, have some Bitcoin related software running there, and run the rest of your webapp logic on AppEngine. The services it provides can still be very useful.

You may find bitcoinj a useful Java library. I wrote it Wink Just be aware, that you will need to keep up to date if you use that, as it still is changing quite fast and will continue to do so for the forseeable future. It might make it easier for you, if you prefer Java - you can just use the class library and point it at a normal bitcoind you run on a hosted machine (or even your own). When events happen, like your balance changing, you can then upload the relevant data to your app and do whatever you want with it there.

12LMm82ZgAzf7yNDpPydEYxEr4Ap7XtSSK
stevenroose
Member
**
Offline Offline

Activity: 63



View Profile WWW

Ignore
June 25, 2012, 09:49:50 PM
 #15

Oh, nice, thanks!

I can find bitcoinj by googling it I suppose? On Google Code or GitHub somewhere Smiley

Yeah we don't have anything ready, but I think we will start working on it in about a month or 2, so thank you!
(I suppose bitcoinj is completely compatible with AppEngine Smiley )

Oh btw, You say it can connect to any bitcoind instance running on a vps or so, do I need specific software on that machine? I'm not familiar with running software on a VPS, I usually work in systems like AppEngine that are managed by others...

There is one thing I will never understand about Bitcoin.
Why didn't Satoshi Nakamoto start with a 100 BTC bounty per block instead of 50. 100 is so much more logical.
But especially, 42 million is a tremendously fancier number for the maximum amount of bitcoins in circulation than the current 21 million.
Mike Hearn
Hero Member
*****
expert
Offline Offline

Activity: 1232


View Profile

Ignore
June 25, 2012, 10:09:12 PM
 #16

You can find it at www.bitcoinj.org

bitcoinj is a set of classes that help you use the Bitcoin protocol. It isn't designed for AppEngine specifically. Some classes, like the P2P networking classes and the block stores, won't work with AppEngine. Others just parse data and let you manipulate it.

Knowing more details about what you want to do would help.

If we assume you want to have a site that both sends and receives money automatically (the most dangerous/risky type!), the way I'd do it is to have a regular Linux box - ideally, one you physically own and is in a nearby datacenter so you can drive over to it - that runs normal bitcoind as downloaded from bitcoin.org, and a Java app you wrote using bitcoinj. The Java app connects to your local node and manages a wallet. You'd program it to generate batches of keys and POST the addreses to your appengine app, which would then stuff them into a data store.

Now when you want to receive a payment from a user, you take an unused address from the data store and give it to the user. They send the money, which is noticed by the app on your physically owned machine. It then POSTs a notification to your AE app which can then do whatever it needs to do, having received the payment. For the reverse, your AE app would POST to the dedicated server (your java app can both talk to bitcoin and be a web app).

If you aren't familiar with running your own machines, I'd think twice about hosting wallets. The moment you have significant funds on a machine, you get a massive set of cross-hairs painted on your back.  If you aren't really sharp when it comes to computer security, you'll get robbed. Outsourcing it to other companies is a technique that's failed several times now ...... most hosting operations are not intended for high security applications like Bitcoin financial services. Not even AppEngine.

12LMm82ZgAzf7yNDpPydEYxEr4Ap7XtSSK
gweedo
Hero Member
*****
Offline Offline

Activity: 728


Java, PHP, HTML/CSS Programmer for Hire!


View Profile WWW

Ignore
June 25, 2012, 10:12:57 PM
 #17

Oh btw, You say it can connect to any bitcoind instance running on a vps or so, do I need specific software on that machine? I'm not familiar with running software on a VPS, I usually work in systems like AppEngine that are managed by others...

I highly advise against you running a VPS with a hot wallet, if you have never managed a regular VPS, you need to learn about firewalls, Users, and how to security talk to the VPS thru GAE. Start with a VM hosted on your computer to play with before doing any VPS work.

http://www.apicoin.io - making the blockchain rest | Donations: 1GweedoZJYb5CNLfSaBgBBYS2y7BMVb2Wo| Tox: 8814C1F411BBF9FCCFA3338DEB23E4DCCAEC2BF7E89C0048F2C23389651C071959F0EB5F4A58 |Escrow (0.5% FEE), just PM | PGP key |The Mark Cuban of this Forum
stevenroose
Member
**
Offline Offline

Activity: 63



View Profile WWW

Ignore
June 25, 2012, 10:49:57 PM
 #18

That's why I was considering AppEngine, since they handle security for me.

Aren't there VPS services that just let me run those 2 programs and fix the security themselves? (Preferably not too expensive ofc...)

@Mike, What I want to do is indeed what you suggested. But, I think the traffic of Bitcoin in and out our service would not be soo high (spoken as fraction of the total amount we'd store). So we were thinking just about hosting just the estimated amount needed for one or two days on the server and have the rest send to an offline address. I think most input will cover the output so when we just always have like twice the average output volume hosted it would be ok.
Also, unlike services like wallets we can f.e. say that withdrawing a greater number of coins could take 24 hrs. I think people won't bother if they know their coins are stored more securely that way.

Although I still have to find out how to produce safe offline transactions, I think that can't be that hard. (I suppose it's possible to put the blockchain on an USB stick, put into an offline machine, produce transactions and broadcast them to the network.

About the outsourcing. Yes I think it's really weird that nobody offers a good reliable Bitcoin API for these purposes. Ofc it's quite dangerous to trust them with your money, but some should be reliable enough by now... I considered Blockchain.info's API, they host your private keys encrypted. But I doubted they could handle large amounts of requests.

There is one thing I will never understand about Bitcoin.
Why didn't Satoshi Nakamoto start with a 100 BTC bounty per block instead of 50. 100 is so much more logical.
But especially, 42 million is a tremendously fancier number for the maximum amount of bitcoins in circulation than the current 21 million.
Mike Hearn
Hero Member
*****
expert
Offline Offline

Activity: 1232


View Profile

Ignore
June 25, 2012, 10:52:49 PM
 #19

It looks like AE has got better since I last looked at it. Now you can have "backends" that are basically long-running Java programs that only occasionally get shut down for load balancing, etc.

bitcoinj still isn't totally suitable for this environment because it assumes it can store the wallet into a file. Nothing stops you from writing code to store it in a database instead, of course, it's just not done for you out of the box.

I think the integration between all these parts will get better over time. You're right that having an API that makes your use case easy would be a good thing, though maybe it shouldn't be too easy to throw other peoples money around :-)

12LMm82ZgAzf7yNDpPydEYxEr4Ap7XtSSK
stevenroose
Member
**
Offline Offline

Activity: 63



View Profile WWW

Ignore
June 25, 2012, 11:35:11 PM
 #20

It would of course be nice if you as a Googler could provide AE support for bitcoinj SmileySmiley

Albeit, I think hosting the complete Blockchain on EA including a separate CPU-consuming instance that has to be awake 100% of the time could cost a dime Smiley


BTW, receiving coins is no problem anyway, public APIs like blockexplorer and blockchain are really great and reliable when used together (the one as backup for the other), so the only functionality still absent is a sending API :p I would pay for one, wouldn't that be business? Tongue It's a tough business though.

Anyway I go to sleep, I have a few more months to find out. In the worst case we can just let app engine generate a transaction list each day which we create offline and broadcast every day Cheesy But that'd be too much of a hassle Smiley

There is one thing I will never understand about Bitcoin.
Why didn't Satoshi Nakamoto start with a 100 BTC bounty per block instead of 50. 100 is so much more logical.
But especially, 42 million is a tremendously fancier number for the maximum amount of bitcoins in circulation than the current 21 million.
Pages: [1] 2  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!