Bitcoin Forum

Bitcoin => Bitcoin Discussion => Topic started by: theymos on June 27, 2011, 07:14:18 PM



Title: Need mirror for blockexplorer.com [Found one]
Post by: theymos on June 27, 2011, 07:14:18 PM
Edit: I no longer need a mirror.

I need someone to run a permanent mirror of blockexplorer.com. I will switch to the mirror while doing maintenance, and I will share load with it in times of heavy load.

You will need to agree to these terms:
- Your mirror will only be accessible via IP address when it is not blockexplorer.com
- Your mirror will be excluded with robots.txt when it is not blockexplorer.com
- You may not allow the code to be released
- You may not base any software off of my code
- You may not automatically publish any data gotten from the BBE database

You can, however, use the local BBE database to gather information that you use privately or manually publish. You can publish the IP address for times when BBE is unexpectedly down.

Requirements:
- PostgreSQL
- PHP with sockets and bcmath
- Some web server able to alias all requests to one file (I use Apache)
- The ability to compile Bitcoin with my modified version of getblock

Typical load is about 800,000 requests per day, with a burst of perhaps 1.5 million requests per day. Most of these are quite small, however. Average upload is 60 KB/s. My server has a Pentium D 3.4GHz CPU with 3GB of memory, and it's doing alright for now.

You must be extremely trustworthy, as I will share the HTTPS private key.

PM me if you are interested.


Title: Re: Need mirror for blockexplorer.com
Post by: error on June 27, 2011, 07:39:52 PM
You probably need more than one someone. :)


Title: Re: Need mirror for blockexplorer.com
Post by: Chick on June 27, 2011, 07:43:04 PM
Why can't you bother to pay like, 30 dollars a month for a Linode?


Title: Re: Need mirror for blockexplorer.com
Post by: lemonginger on June 27, 2011, 08:18:40 PM
I would happily chip in to get blockexplorer on a VPS somewhere.


Title: Re: Need mirror for blockexplorer.com
Post by: theymos on June 27, 2011, 10:55:26 PM
Why can't you bother to pay like, 30 dollars a month for a Linode?

To get the performance that BBE currently has (which is not perfect), I give PostgreSQL almost my entire 3 GB of memory. Linode charges $160/month for 4GB. Less memory will equal less performance unless disk speed is much better (which I doubt will be the case with any VPS).


Title: Re: Need mirror for blockexplorer.com
Post by: JoelKatz on June 27, 2011, 11:07:26 PM
Would you prefer someone supply you with a machine or VPS that you control? That might actually be easier to get because it doesn't require any expertise or effort on the part of the person offering it to you.

And are you prepared to offer something in return? Normally that would be something like a banner on the page, but that doesn't work well with block explorer. I'm honestly not sure what the right thing would be.


Title: Re: Need mirror for blockexplorer.com
Post by: hamdi on June 27, 2011, 11:12:35 PM
why not link to an instance of ABE while your downtimes?


i have a dedicated server you could use for some weeks. germany located, 500 gigs webspace, 100mbit unmetered, dont know exact ram. pm me.


Title: Re: Need mirror for blockexplorer.com
Post by: andes on June 27, 2011, 11:19:56 PM
I could send you one or two AMD Athlon 3200+ machines with 4 GB Ram and 80 GB HD, if you need them (for free). MB is DFI NF4-GF. Dont know if this would be helpful. Where are you located?


Title: Re: Need mirror for blockexplorer.com
Post by: joepie91 on June 27, 2011, 11:34:39 PM
Why can't you bother to pay like, 30 dollars a month for a Linode?

To get the performance that BBE currently has (which is not perfect), I give PostgreSQL almost my entire 3 GB of memory. Linode charges $160/month for 4GB. Less memory will equal less performance unless disk speed is much better (which I doubt will be the case with any VPS).
Have you looked at Ramhost (http://www.ramhost.us/)? They offer 2GB guaranteed, 3GB RAM burst for $30/month (http://www.ramhost.us/?page=vps/linux-openvz-atlanta-east), and from my experience they are very good.


Title: Re: Need mirror for blockexplorer.com
Post by: Ryland R. Taylor-Almanza on June 27, 2011, 11:50:23 PM
PM sent. I have multiple unused 4GB ram linodes, and I've been wanting to contribute more to the bitcoin community somehow.


Title: Re: Need mirror for blockexplorer.com
Post by: jgarzik on June 27, 2011, 11:52:30 PM
I need someone to run a permanent mirror of blockexplorer.com. I will switch to the mirror while doing maintenance, and I will share load with it in times of heavy load.

You will need to agree to these terms:
- Your mirror will only be accessible via IP address when it is not blockexplorer.com
- Your mirror will be excluded with robots.txt when it is not blockexplorer.com
- You may not allow the code to be released
- You may not base any software off of my code
- You may not automatically publish any data gotten from the BBE database

You can, however, use the local BBE database to gather information that you use privately or manually publish. You can publish the IP address for times when BBE is unexpectedly down.

Requirements:
- PostgreSQL
- PHP with sockets and bcmath
- Some web server able to alias all requests to one file (I use Apache)
- The ability to compile Bitcoin with my modified version of getblock


These requirements seem quite heavyweight.

Consider asking for reverse [caching] proxy volunteers.  Just need to install one piece of software on each mirror, presuming that the main blockexplorer server provide useful last-modified and/or etag headers, and potentially other cache-control headers.

A reverse proxy with a reasonable disk cache will only touch the main blockexplorer server the first time a new request is seen.



Title: Re: Need mirror for blockexplorer.com
Post by: JoelKatz on June 27, 2011, 11:57:33 PM
Consider asking for reverse [caching] proxy volunteers.  Just need to install one piece of software on each mirror, presuming that the main blockexplorer server provide useful last-modified and/or etag headers, and potentially other cache-control headers.

A reverse proxy with a reasonable disk cache will only touch the main blockexplorer server the first time a new request is seen.
That's a great idea to reduce his load, but it won't solve the problem of getting live data when he's down. The reverse proxies would have to be trusted with his SSL key. But an awful lot of his data is static.

He may have to modify his code to add appropriate headers to indicate the suitable caching for his data.


Title: Re: Need mirror for blockexplorer.com
Post by: phantomcircuit on June 28, 2011, 12:00:05 AM
We'd be willing a throw a server at it.

(you can find us on irc ofc)


Title: Re: Need mirror for blockexplorer.com
Post by: theymos on June 28, 2011, 01:33:41 AM
Thanks a lot for all the offers! I will consider them all tomorrow.

Would you prefer someone supply you with a machine or VPS that you control?

I would prefer that someone else maintain it, though I can do it.

I won't accept physical machines sent to me, as one of the main things this is meant to solve is situations when my power or Internet is offline.

Quote from: hamdi
why not link to an instance of ABE while your downtimes?

ABE has reduced functionality.

Quote from: joepie91
Have you looked at Ramhost? They offer 2GB guaranteed, 3GB RAM burst for $30/month, and from my experience they are very good.

I'll probably need more than 2GB in the long-term.

Quote from: jgarzik
These requirements seem quite heavyweight.

Consider asking for reverse [caching] proxy volunteers.  Just need to install one piece of software on each mirror, presuming that the main blockexplorer server provide useful last-modified and/or etag headers, and potentially other cache-control headers.

A reverse proxy with a reasonable disk cache will only touch the main blockexplorer server the first time a new request is seen.

I will consider this if load becomes the main problem, but I'm really trying to solve downtime right now. This request was prompted by the fact that my Internet access will be down for a day or two in the near future.


Title: Re: Need mirror for blockexplorer.com
Post by: error on June 28, 2011, 01:36:17 AM
I don't think the spare VPS I have will be up to the task of taking over your entire load, though I'm certainly willing to test it and to help you optimize the code, if possible.


Title: Re: Need mirror for blockexplorer.com
Post by: mizerydearia on June 28, 2011, 02:37:24 AM
Alternatively, feel free to collaboratively establish an open source alternative for anyone to mirror?


Title: Re: Need mirror for blockexplorer.com
Post by: BitcoinPorn on June 28, 2011, 02:48:17 AM
Alternatively, feel free to collaboratively establish an open source alternative for anyone to mirror?

Speaking of.. [ANNOUNCE] ABE: Open Source Block Explorer Knockoff (http://forum.bitcoin.org/index.php?topic=22785.0;topicseen)  http://forum.bitcoin.org/index.php?topic=22785.0;topicseen


Title: Re: Need mirror for blockexplorer.com
Post by: Stephen Gornick on June 28, 2011, 08:38:36 AM
Speaking of..

And don't forget:
 - https://github.com/bitcoinjs/node-bitcoin-explorer


Title: Re: Need mirror for blockexplorer.com
Post by: theymos on July 02, 2011, 03:14:26 AM
Thanks to rasengan and coderrr of MtGox Live (http://mtgoxlive.com) for hosting the mirror. It is available directly at http://109.123.116.245 . Tell me if you see any bugs unique to the mirror.


Title: Re: Need mirror for blockexplorer.com [Found one]
Post by: romanticon on December 21, 2011, 11:32:34 AM
Hey, what is going on with the bitcoinexplorer
on test environment, it behaves strange,
sometimes it gives me old results.

Thanks in advance.


P.S: didn't find a better place to ask it.


Title: Re: Need mirror for blockexplorer.com [Found one]
Post by: theymos on December 21, 2011, 12:08:54 PM
Hey, what is going on with the bitcoinexplorer
on test environment, it behaves strange,
sometimes it gives me old results.

Use http://99.188.19.113/testnet for now. I'll fix the other version in a few days.


Title: Re: Need mirror for blockexplorer.com [Found one]
Post by: romanticon on December 21, 2011, 12:11:12 PM
Great, thanks , now I see 43215, 43214 -
the first blocks I've created !!!


Title: Re: Need mirror for blockexplorer.com [Found one]
Post by: phelix on June 07, 2012, 09:00:50 AM
the API seems to be broken...

for example: http://blockexplorer.com/q/estimate says ERROR: unknown


can I somehow access the mirror if the main page is not working?


Title: Re: Need mirror for blockexplorer.com [Found one]
Post by: Stephen Gornick on June 07, 2012, 09:13:20 AM
the API seems to be broken...

for example: http://blockexplorer.com/q/estimate says ERROR: unknown


can I somehow access the mirror if the main page is not working?

Blockchain.info has an api that is (nearly) identical.  estimate is one that is different.

 - http://blockchain.info/q/eta
 - http://blockchain.info/q


Title: Re: Need mirror for blockexplorer.com [Found one]
Post by: phelix on June 07, 2012, 09:35:48 AM
the API seems to be broken...

for example: http://blockexplorer.com/q/estimate says ERROR: unknown


can I somehow access the mirror if the main page is not working?

Blockchain.info has an api that is (nearly) identical.  estimate is one that is different.

 - http://blockchain.info/q/eta
 - http://blockchain.info/q

thanks, I was just looking into that.

there is another bitcoin difficulty estimate here (from Khal) http://dot-bit.org/tools/difficulty_bitcoin.txt
seems to be a little off, though, maybe it is also based on blockexplorer...

it would really be nice to have an eta and difficulty estimate that would average accross retargets...



Title: Re: Need mirror for blockexplorer.com [Found one]
Post by: phelix on June 08, 2012, 10:34:34 AM
hey theymos, I noticed you did a rescan or something and most things work fine again - thanks!

one thing is still odd, though:
http://blockexplorer.com/q/nethash/2016 does not seem to be quite up to date, stuck somewhere shortly before the last retargeting.