Bitcoin Forum
April 27, 2024, 03:56:25 AM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
   Home   Help Search Login Register More  
Poll
Question: Where you able to run Datacoin Local HTTP server yourself?
Yes - 17 (37%)
No, I don't understand this post - 22 (47.8%)
No, it doesn't work on my PC - 2 (4.3%)
No, but I'll try tomorrow - 5 (10.9%)
Total Voters: 46

Pages: [1] 2 3 4  All
  Print  
Author Topic: [ANN][Datacoin] Next step: web applications inside blockchain (demo)  (Read 12067 times)
oocook5u (OP)
Newbie
*
Offline Offline

Activity: 36
Merit: 0


View Profile
January 09, 2014, 12:22:30 AM
Last edit: January 09, 2014, 12:49:52 AM by oocook5u
 #1

Hello!

Datacoin blockchain started about two months ago. Thank you to all of you supporting Datacoin. It exists because of your support.

Now we can start developing blockchain-based web applications. Note that this feature doesn't touch sources of Datacoin daemon or qt-wallet: if you are not interested in this it you can still use Datacoins and without understanding this post. Blockchain-based web applications require a http-server that redirects HTTP requests from browser to daemon. This is implemented now with Perl (dtc_http_server.pl). Yes, I know I would have to do it using C++. I'll rewrite it later.

Why web-applications inside blockchain are good?

1. they are protected from being deleted or censored
2. they work locally and don't require a live Internet connection
3. web interface is convenient to users
4. no new technologies are required for developement: HTTP, AJAX and JSON RPC are simply linked together Wink

Datacoin HTTP server is very simple and it is intended to work locally on end-user's machine.

Types of requests supported by Datacoin local HTTP server

NB: NET can be 'dtc' or 'dtc-testnet'

1. Get data from blockchain
http://localhost:PORT/NET/get/TXHASH -> data from tx with TXHASH will be sent back (data is assumed to be in Envelope format - see envelope.proto file)

Example:
http://localhost:8080/dtc/get/40a3eb9624528008a87d9119dbed4041d45c9f6483ecd52a7b24f9c8b4fdf548

2. Call daemon's method
http://localhost:PORT/NET/rpc/METHOD(/ARGS) -> json-rpc call to daemon's METHOD will be performed and resulting json will be sent back to browser

Several arguments are possible in form arg1/arg2/arg3/...

Example:
http://localhost:8080/dtc/rpc/getblockhash/1000

Code:
{
 "result" : "ce3ca3728ee050bf14d8f9a059f13c0062ca303cf2c982796c7dec2ce10f13c6",
 "error" : null,
 "id" : null
}

NB: due to security reasons only 'getinfo', 'getmininginfo', 'getblock', 'getblockhash' and 'getrawtransaction' are supported now. Later all reading methods will be added. 'sendtoaddress' and other money-spending methods probably will be added also but some security issues are to be considered before in order to protect users from theft.

3. Call http local server's method
http://localhost:PORT/NET/lsrpc/METHOD(/ARGS) -> http server will handle this method itself (or translate to several requests to daemon).

There is only one example of lsrpc methods now: 'getenvelope'. It parses tx.data field of a given transaction as an Envelope and returns envelope in json format (without 'Data'). This method is needed to get metadata easily.

Example:
http://localhost:8080/dtc/lsrpc/getenvelope/40a3eb9624528008a87d9119dbed4041d45c9f6483ecd52a7b24f9c8b4fdf548

Code:
{
  "result":
  {
    "FileName":"blockexplorer.html",
    "ContentType":"text/html",
    "Compression":"1",
    "PublicKey" : "MIGJA... a lot of base64 letters ...AE=",
  }
}

Demo
In order to demonstrate this in action I've sent 'Block explorer' application in transaction 40a3eb9624528008a87d9119dbed4041d45c9f6483ecd52a7b24f9c8b4fdf548. Here is an instruction on how to access it from your own computer with local HTTP server (actually this will be too difficult for 99% of users, I need to rewrite server using C++ and deliver it as a precompiled binary):

Before you start: DON'T do this on the machine with many coins!!! Setup a different user account on Windows (or Linux) or a virtual machine. There is a lot of possible bugs in this code that can lead to loss of money.

Now you are warned. Let's continue.

1. backup yout wallet.dat and protect it with password

2. git clone https://github.com/foo1inge/datacoin-browser.git

3. install all required CPAN modules (the way to do this depends from your Perl distribution, ask Perl guru if you don't know how to do this). Here is list of modules:

Code:
HTTP::Server::Simple::CGI
Google::ProtocolBuffers
MIME::Base64
IO::Compress::Bzip2
IO::Uncompress::Bunzip2
Crypt::OpenSSL::RSA
JSON::RPC::Client
File::HomeDir
Data::Dumper

4. start Datacoin daemon ('getdata' RPC method is required. It was commited two weeks ago and probably you have to update your daemon).

5. cd datacoin-browser/perl/

6. perl dtc_http_server.pl --http-port=8080
This will start local HTTP server on port 8080. In order to see Datacoin testnet add "--testnet=1" option - it will connect to daemon working in testnet.

7. go to http://localhost:8080/dtc/get/40a3eb9624528008a87d9119dbed4041d45c9f6483ecd52a7b24f9c8b4fdf548

8. enjoy

You can deploy this on any machine. Here is this application deployed on Amazon EC2 (micro instance, very slow):
http://ec2-54-201-144-165.us-west-2.compute.amazonaws.com/dtc/get/40a3eb9624528008a87d9119dbed4041d45c9f6483ecd52a7b24f9c8b4fdf548

Blockexplorer application consists of three transactions:

40a3eb9624528008a87d9119dbed4041d45c9f6483ecd52a7b24f9c8b4fdf548 - blockexplorer.html
465ca75b6dedf70bc85db24548ee99bab776028f9295116afba4fa1e4838ff5f - blockexplorer.js
73b7c7165a51f604040e17681705a39145392691a0447266b9074a2ad9140327 - datacoin-logo.png


All these files are accessed by TXID (not by names!) the following way (snippets from blockexplorer.html):

Code:
<script
  src="/dtc/get/465ca75b6dedf70bc85db24548ee99bab776028f9295116afba4fa1e4838ff5f"
  type="text/javascript"
  encoding="UTF-8">
</script>

Code:
<td style="padding-right:6px">
<img
  src="73b7c7165a51f604040e17681705a39145392691a0447266b9074a2ad9140327"
  alt="Datacoin logo"/>
</td>

Blockexplorer does following steps while starting:

1. getinfo RPC call (to get current height)
2. getblockhash / getblock for each block in cycle
3. getrawtransaction for each transaction
4. getenvelope for each transaction with data

All these calls are coded as ordinary AJAX in blockexplorer.html.

Important issues for web developers on how to develop applications for Datacoin

1. Start development in testnet

Testnet block explorer: http://ec2-54-201-144-165.us-west-2.compute.amazonaws.com:81/dtc-testnet/get/7fead7473c448b2d69d0e62354dedd59895a1ab099c1835155d83f7327c5156a

2. Never use specific IP or PORT in URLs - only relative URLs are meaningful (user can start local HTTP server on any IP:PORT).
In your HTML code instead of doing

Code:
<a href="http://localhost:8080/dtc/get/40a3eb9624528008a87d9119dbed4041d45c9f6483ecd52a7b24f9c8b4fdf548">...</a>

use relative URL:

Code:
<a href="/dtc/get/40a3eb9624528008a87d9119dbed4041d45c9f6483ecd52a7b24f9c8b4fdf548">...</a>

3. Use "--content-type" argument of dtc_put_file.pl script to set a "Content-type" HTTP attribute for your files.

Here is an example:
Code:
perl dtc_put_file.pl --content-type="text/html" blockexplorer.html

4. Use "--add_key" argument of dtc_put_file.pl script and save private key (it will be printed on screen) in a secure place in order to upate your application later.

Here is a full example:
Code:
perl dtc_put_file.pl --add_key=true --content-type="text/html" blockexplorer.html

NB: update feature isn't implemented yet Sad I'm working on it now.

Datacoin development plans regarding blockchain-based web-applications:
(I don't mention here issues not related to the topic)

1. bugfixing

2. Update feature (see detailed description of update scenario in envelope.proto file)

3. Big files (more 128Kb) feature

4. C++ version of local HTTP server

5. "senddata" from local HTTP server

6. Access to P2P file sharing networks from local HTTP server (something like localhost:8080/kademlia/HASH). Needs some investigation.
1714190185
Hero Member
*
Offline Offline

Posts: 1714190185

View Profile Personal Message (Offline)

Ignore
1714190185
Reply with quote  #2

1714190185
Report to moderator
1714190185
Hero Member
*
Offline Offline

Posts: 1714190185

View Profile Personal Message (Offline)

Ignore
1714190185
Reply with quote  #2

1714190185
Report to moderator
1714190185
Hero Member
*
Offline Offline

Posts: 1714190185

View Profile Personal Message (Offline)

Ignore
1714190185
Reply with quote  #2

1714190185
Report to moderator
"If you don't want people to know you're a scumbag then don't be a scumbag." -- margaritahuyan
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1714190185
Hero Member
*
Offline Offline

Posts: 1714190185

View Profile Personal Message (Offline)

Ignore
1714190185
Reply with quote  #2

1714190185
Report to moderator
1714190185
Hero Member
*
Offline Offline

Posts: 1714190185

View Profile Personal Message (Offline)

Ignore
1714190185
Reply with quote  #2

1714190185
Report to moderator
GreekBitcoin
Legendary
*
Offline Offline

Activity: 1428
Merit: 1001


getmonero.org


View Profile WWW
January 09, 2014, 01:08:24 AM
 #2

Very nice to see you here! I am reading this right now and trying to understand as much as i can! Already posted the link of this on the other thread so others can see it too cause with all the spam coins it will get buried.

Edit: Very good news! I dont understand it and i would not be able to run Datacoin Local HTTP server myself but i suppose in a few days it will be more explained and i will. I lack skills.
oocook5u (OP)
Newbie
*
Offline Offline

Activity: 36
Merit: 0


View Profile
January 09, 2014, 01:28:48 AM
 #3

Very nice to see you here! I am reading this right now and trying to understand as much as i can! Already posted the link of this on the other thread so others can see it too cause with all the spam coins it will get buried.

Edit: Very good news! I dont understand it and i would not be able to run Datacoin Local HTTP server myself but i suppose in a few days it will be more explained and i will. I lack skills.

Wink

This link doesn't need any installation procedures: http://ec2-54-201-144-165.us-west-2.compute.amazonaws.com/dtc/get/40a3eb9624528008a87d9119dbed4041d45c9f6483ecd52a7b24f9c8b4fdf548
GreekBitcoin
Legendary
*
Offline Offline

Activity: 1428
Merit: 1001


getmonero.org


View Profile WWW
January 09, 2014, 01:35:34 AM
 #4

The thing is i dont understand exackly what this do! What i thing is that we now can find files stored immediately from the browser just with a link. If i am correct what do i do wrong:

There is a pic on block 131513. I got there and i see this : data   Browse data in another tab

I press it but i cant see the pic or the base64 of it.



P.S. Try to read the other thread. There are many questions to be answered. Maybe already answered but yeah. For example  people are asking about the fee being too big Wink
Crackmacs
Full Member
***
Offline Offline

Activity: 154
Merit: 100



View Profile
January 09, 2014, 01:43:37 AM
 #5

cryptrol
Hero Member
*****
Offline Offline

Activity: 637
Merit: 500


View Profile
January 09, 2014, 08:17:38 AM
 #6

Nice, this starts to show the real possibilities of the project and how disruptive it can be.
Essentially, you can have a self contained Internet inside Datacoin.

Regarding the envelope protocol (the one defined with Google Protocol Buffers), are you planning to make any changes ? I am currently implementing the big file functionality for a website (see my sig).

I am doing all these with an external HTTP server and using JSON-RPC calls to the daemon, not the Perl browser you just posted.

On a side note, can we agree, or at least discuss about, lowering the fees for data transactions ? Since this does not require a hard fork it would be easily implemented. My motivations are as follows :
- With the current block reward model the cost of storage will go "to the moon". And this is bad for adoption (where we currently are).
- No hard fork and people with large amounts of DTC will not complain.
- Will drive adoption and awareness UP.
- Miners will receive more since (hopefully) people will spend more DTC, you can see right now the usage of the data field is minimal.

What do you think about it?
maxsolnc
Sr. Member
****
Offline Offline

Activity: 350
Merit: 250


DTC unofficial team


View Profile WWW
January 09, 2014, 08:25:45 AM
 #7

oocook5u , I'm really happy to see you and double more happy to see such great updates! Smiley I'm following you on git and reading your commits carefully (I know perl and a bit of c\c++). I'll clone HTTP server to more powerful machines and publish their addresses soon. It will be more comfortable for those devs who just want to try their own existing apps together with Datacoin - they don't need to install all this on their machines 'just to try', they will use external interface, and it will work much faster - t1.micro is a pain sometimes.

That's great that you're planning to add 'update' feature, and P2P linking idea is awesome. Really, I'm very excited about that, it seems like some kind of revolution both in cryptos and in storage systems.

tl;dr for those who didn't understand the whole OP post:
Now you have a possibility to get resources from Datacoin blockchain like they are hosted on your Web hosting.
You want to go to http://yoursite.com/blog.html
If your blog.html page is already stored in Datacoin blockchain and TX ID is qwertyqwerty1, you can go to http://yoursite.com/dtc/get/qwertyqwerty1 and see your page. It is higher abstraction layer built on top of Datacoin network - it was announced during release and is being implemented now.

I'll notify community when I install this server on bigger machines - it should happen today or at least tomorrow.

edited: I also heard this thing about lowering fees from many people who come to me with questions, it seems to be one of the most important Qs just now.

DTC: DMcKNp47fNtgM7sritK9GfJEQ1DzME5nwk
BTC: 1FgUGra685ZwkrX5VnRvfaYp4bHJhC7x4H
panonym
Sr. Member
****
Offline Offline

Activity: 266
Merit: 250

Help and Love one another ♥


View Profile
January 09, 2014, 08:45:09 AM
 #8


This.

So...
A: that allow you to easily reach your data inside the blockchain on common website.
B: you did - or are trying - to create a kind of subnet?
A whole new Internet with permanent archive hosted on your computer and futurely thousands, where you no longer rely on mainstream http/tor/i2p but use the blockchain itself as a new local/p2p hosting/browsing thing?

Which one is it? (something else than A & B?)
maxsolnc
Sr. Member
****
Offline Offline

Activity: 350
Merit: 250


DTC unofficial team


View Profile WWW
January 09, 2014, 08:55:04 AM
 #9


This.

So...
A: that allow you to easily reach your data inside the blockchain on common website.
B: you did - or are trying - to create a kind of subnet?
A whole new Internet with permanent archive hosted on your computer and futurely thousands, where you no longer rely on mainstream http/tor/i2p but use the blockchain itself as a new local/p2p hosting/browsing thing?

Which one is it? (something else than A & B?)

both A and B. Now you can easily reach your data inside blockchain, and it can be (and will be) whole new Internet stored locally and being updated every minute.

DTC: DMcKNp47fNtgM7sritK9GfJEQ1DzME5nwk
BTC: 1FgUGra685ZwkrX5VnRvfaYp4bHJhC7x4H
panonym
Sr. Member
****
Offline Offline

Activity: 266
Merit: 250

Help and Love one another ♥


View Profile
January 09, 2014, 09:45:49 AM
Last edit: January 09, 2014, 10:21:12 AM by panonym
 #10

So is an intern http-like browser already functional?
Do you have a basic working example?
Just something like a hello world.block5872 this is my hello kitty pict, click here for comments.

Might buy in now if that's the case.

Any possibilities in future to go beyond static website?
Like, archive will be from start to now. Might get massive with time (but 2Tio HD is quite cheap).
But is a intern evolutive DB possible?

(Damn I've to stop watching wolf of wall street and go buy if we're up there yet)

PS: I've been distantly watching Datacoin for a while now.
Main problem is that it'is way too complex.
I managed to burn BTC using counterpartyd software for example, that's where my low level is at.
But I barely understand anything about uploading file into DTC blockchain and retrieve them.
Any centralized doc trying to become more user friendly?

Otherwise most people may just buy the minimum they need to upload what they want, but may have low appeal to invest into DTC.

PPS: yet one more question (sorry for the length)
Do you already have a decentralized DNS system NMC.bit like? (major for B-subnet adoption)

edit: not gonna invest for now. Reason = same that last time research : get mindfucked >_<
+ leader dev trying to sell his project + graph strong bear trend
Go back to my movie. There is potential, but not ready for mass adoption.
GreekBitcoin
Legendary
*
Offline Offline

Activity: 1428
Merit: 1001


getmonero.org


View Profile WWW
January 09, 2014, 11:12:16 AM
 #11

Yes i would like to se an example and maybe a guide of how to do something like this example.

Also can someone answer to my question up in this thread about the pic on that block? Can i see the pic somehow?
maxsolnc
Sr. Member
****
Offline Offline

Activity: 350
Merit: 250


DTC unofficial team


View Profile WWW
January 09, 2014, 11:13:23 AM
 #12

So is an intern http-like browser already functional?
Do you have a basic working example?
Just something like a hello world.block5872 this is my hello kitty pict, click here for comments.

Might buy in now if that's the case.

Any possibilities in future to go beyond static website?
Like, archive will be from start to now. Might get massive with time (but 2Tio HD is quite cheap).
But is a intern evolutive DB possible?

(Damn I've to stop watching wolf of wall street and go buy if we're up there yet)

PS: I've been distantly watching Datacoin for a while now.
Main problem is that it'is way too complex.
I managed to burn BTC using counterpartyd software for example, that's where my low level is at.
But I barely understand anything about uploading file into DTC blockchain and retrieve them.
Any centralized doc trying to become more user friendly?

Otherwise most people may just buy the minimum they need to upload what they want, but may have low appeal to invest into DTC.

PPS: yet one more question (sorry for the length)
Do you already have a decentralized DNS system NMC.bit like? (major for B-subnet adoption)

Datacoin logo is available instead of kittens Smiley
http://ec2-54-201-144-165.us-west-2.compute.amazonaws.com/dtc/get/73b7c7165a51f604040e17681705a39145392691a0447266b9074a2ad9140327

What do you mean 'to go beyond static website'? When oocook5u implements 'update' feature, it will be possible to update web pages and Datacoin-based sites.

Just now desktop GUI wallet doesn't have possibility to upload data, but it is pretty easy to go to https://krypte.net and do anything you want with your data.

Questions are good, please ask as much as you want - it can help other people to find answers! Smiley
No, Datacoin is not intended to have own DNS system - DTC is not trying to be 'all-in-one', but it can collaborate with Namecoin for DNS.

DTC: DMcKNp47fNtgM7sritK9GfJEQ1DzME5nwk
BTC: 1FgUGra685ZwkrX5VnRvfaYp4bHJhC7x4H
extro24
Sr. Member
****
Offline Offline

Activity: 481
Merit: 252


View Profile
January 09, 2014, 11:19:44 AM
 #13

One integrated ecosystem. Bitcoin, Namecoin, Datacoin. A platform, not just a currency.  

So file browsing will also be done via Internet.  With a coin built right in.

When the personal chains arrive the files will be farmed out to the nodes.

maxsolnc
Sr. Member
****
Offline Offline

Activity: 350
Merit: 250


DTC unofficial team


View Profile WWW
January 09, 2014, 11:24:26 AM
 #14

One integrated ecosystem. Bitcoin, Namecoin, Datacoin. A platform, not just a currency.

Exactly. As I've said above, no need for trying to implement ALL features in one coin - Bitcoin can be used as 'general money', Namecoin as coordinating system (DNS), Datacoin as nosql db. I feel like we can build our own network that is like 'ordinary' Internet, maybe not so big in sense of content-size, but extremely important for technicians and scientists.

And remember: Information is money, Datacoin implements these words Wink This 'Internet' - BTC, NMC, DTC, possibly some new coins to be created soon - just imagine, it's really 'money as information'. If all goes well, you guys will see the history making here and now.

When the personal chains arrive the files will be farmed out to the nodes.

Personal chains or some kind of it can be used for content storage, i.e. main chain will contain only text data (html pages, keys, magnet links, etc), and personal chains can contain photos, voice recordings, maybe short videos and so on.

DTC: DMcKNp47fNtgM7sritK9GfJEQ1DzME5nwk
BTC: 1FgUGra685ZwkrX5VnRvfaYp4bHJhC7x4H
rayvellest
Member
**
Offline Offline

Activity: 90
Merit: 10


View Profile WWW
January 09, 2014, 11:36:37 AM
 #15

Now we can start developing blockchain-based web applications. Note that this feature doesn't touch sources of Datacoin daemon or qt-wallet: if you are not interested in this it you can still use Datacoins and without understanding this post. Blockchain-based web applications require a http-server that redirects HTTP requests from browser to daemon. This is implemented now with Perl (dtc_http_server.pl). Yes, I know I would have to do it using C++. I'll rewrite it later.

Wow! I would never thought of accomplish something like that. It seems DTC has more potential than I though at first. I wonder what else is coming! Smiley

cryptrol
Hero Member
*****
Offline Offline

Activity: 637
Merit: 500


View Profile
January 09, 2014, 11:53:53 AM
 #16

Just another question, does your perl get file implementation supports both raw data and data embedded into the envelope protocol ?
maxsolnc
Sr. Member
****
Offline Offline

Activity: 350
Merit: 250


DTC unofficial team


View Profile WWW
January 09, 2014, 12:04:58 PM
 #17

Just another question, does your perl get file implementation supports both raw data and data embedded into the envelope protocol ?

Quick answer (I think oocook5u will give more detailed info): just now - no, if you want to get raw data stored by krypte.net, you should either use command-line with the daemon, or use krypte.net for downloading this data. I've just checked this. But this envelope proto really makes sense when we're talking about building 'Internet' on top of Datacoin.
Question for you: can you build these perl scripts for upload\download into kryptenet? Or maybe write your own implementation... just to avoid 'data format hell' in future.

DTC: DMcKNp47fNtgM7sritK9GfJEQ1DzME5nwk
BTC: 1FgUGra685ZwkrX5VnRvfaYp4bHJhC7x4H
cryptrol
Hero Member
*****
Offline Offline

Activity: 637
Merit: 500


View Profile
January 09, 2014, 01:16:16 PM
 #18

Question for you: can you build these perl scripts for upload\download into kryptenet? Or maybe write your own implementation... just to avoid 'data format hell' in future.
Absolutely, in fact I am already testing a working version, just wanted to know if we (as a community) should drop the raw data and use only the envelope protocol. I personally don't have a problem keeping both formats.
maxsolnc
Sr. Member
****
Offline Offline

Activity: 350
Merit: 250


DTC unofficial team


View Profile WWW
January 09, 2014, 01:20:35 PM
 #19

Question for you: can you build these perl scripts for upload\download into kryptenet? Or maybe write your own implementation... just to avoid 'data format hell' in future.
Absolutely, in fact I am already testing a working version, just wanted to know if we (as a community) should drop the raw data and use only the envelope protocol. I personally don't have a problem keeping both formats.
I'm sure that you understand importance of format standardization. In fact, krypte.net is the only working resource which fully uses get\send data function just now. Other projects are in development, so all of them should use this standard. Oocook5u was absolutely right to release this proto and tools for it as first priority, when coin is so young and there are no many services what need to change their protocols.

DTC: DMcKNp47fNtgM7sritK9GfJEQ1DzME5nwk
BTC: 1FgUGra685ZwkrX5VnRvfaYp4bHJhC7x4H
bidji29
Sr. Member
****
Offline Offline

Activity: 392
Merit: 250


View Profile
January 09, 2014, 01:22:11 PM
 #20

Great work!

This blockchain explorer is really good : http://ec2-54-201-144-165.us-west-2.compute.amazonaws.com/dtc/get/40a3eb9624528008a87d9119dbed4041d45c9f6483ecd52a7b24f9c8b4fdf548

oocooku, If you want i can provide you with a better AWS instance

http://www.freebieservers.com/  100% FREE GAME SERVERS
Pages: [1] 2 3 4  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!