Bitcoin Forum

Bitcoin => Mining => Topic started by: Artefact2 on July 29, 2011, 10:06:58 PM



Title: (Experimental) pident — a pool-biased blockchain representation
Post by: Artefact2 on July 29, 2011, 10:06:58 PM
Hello there. This thread is about pident, a pool-aware, opensource Blockexplorer-like tool.

Main features


Other features

  • Factoids: not-so-useful interesting statistics about the blockchain, like the block with the biggest fees, biggest transaction, average size of a block, etc.
  • Opensource: pident is released under the Do What The Fuck You Want license. It's basically less legally ambiguous public domain. You can run your own pident instance locally, play with the code, … You're free to do anything you want, for any purpose!
  • Blockexplorer-compatible URIs: most of the URIs are compatible with Blockexplorer, so you can compare both tools by only changing the domain name.

Caveats

  • This tool is still experimental, it may break at any moment. Although I do most of my breakage on the testing version, it's better for you to be warned. Please don't use pident if you need a reliable service (for example, links in your pool statistics).
  • Performance may be poor: I am running this at home with my poor upload bandwidth. Please don't hammer the server with lots of automated requests. If you need to, I'd rather you run your own pident instance locally (and I may even help you do so).

Try it !

  • Try the regular (decimal) version: pident.artefact2.com (http://pident.artefact2.com)
  • Try the tonal version: tonal.pident.artefact2.com (http://tonal.pident.artefact2.com)
  • Browse the source code on GitHub: https://github.com/Artefact2/pident (https://github.com/Artefact2/pident)

If you need support, or have comments, you can post on this thread or send me an email (artefact2@gmail.com). Thanks!


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: 1bitc0inplz on July 29, 2011, 11:43:42 PM
Great job on pident!

We've already started linking to you instead of block explorer. The UI alone makes it worth it  :)


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Sukrim on July 31, 2011, 12:39:35 AM
Full tonal compatibility: pident can operate in decimal or in tonal mode. You choose which one you preffer.
Hehe, tonal geekiness knows no borders, hm?

How exactly do you identify which blocks belong to which pools, where do I have to look in the code?


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Artefact2 on July 31, 2011, 11:25:10 AM
How exactly do you identify which blocks belong to which pools, where do I have to look in the code?

I try to use their API if they have some, or I just parse their statistics page. Some pools are trickier than others but it's not a big obstacle anyway (MtRed, BTCMP for example)…
The interesting code that does this is here: https://github.com/Artefact2/pident/blob/master/lib/inc.pools.php#L62


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: doldgigger on August 04, 2011, 12:38:54 PM
How exactly do you identify which blocks belong to which pools, where do I have to look in the code?

I try to use their API if they have some, or I just parse their statistics page. Some pools are trickier than others but it's not a big obstacle anyway (MtRed, BTCMP for example)…
The interesting code that does this is here: https://github.com/Artefact2/pident/blob/master/lib/inc.pools.php#L62
Great work! This looks very much like what I was looking for in this thread: https://bitcointalk.org/index.php?topic=28309.0 . Basically, my idea is that miners should be able to easily verify the transaction fee policies of the pools they mine for, so they can have their share of control over how this is handled in the bitcoin network.

Maybe it is possible to promote some kind of standard interface for pools to serve this data (consisting of a pident plugin and some PHP code for pool owners), so we an reduce the number of blocks where the pool remains unidentified.


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Graet on August 05, 2011, 09:53:03 AM
I picked up the link in irc and have ben using it for a while it also inspired me to add some more stats to my pool :)
- awesome job Artefact2 :D
Thanks,
Graet


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Artefact2 on August 10, 2011, 04:26:24 PM
Added BitMinter support. If I'm missing other pools, don't hesitate to nag me about it. (Or even better, add the support yourself and make a pull request!)


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Reckman on August 11, 2011, 07:40:52 PM
Wont pool hoppers use/abuse this sort of thing? 


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Artefact2 on August 11, 2011, 08:21:38 PM
Wont pool hoppers use/abuse this sort of thing? 

Maybe. That's not my problem.


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Graet on August 14, 2011, 12:46:44 AM
Wont pool hoppers use/abuse this sort of thing? 
in what way? they already use pools APIs to know when to hop.


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: miscreanity on August 17, 2011, 08:06:11 PM
Added BitMinter support. If I'm missing other pools, don't hesitate to nag me about it. (Or even better, add the support yourself and make a pull request!)

If I had the time...

BitClockers?


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Artefact2 on August 17, 2011, 09:39:11 PM
Added BitMinter support. If I'm missing other pools, don't hesitate to nag me about it. (Or even better, add the support yourself and make a pull request!)

If I had the time...

BitClockers?

Yeah, unfortunately it's not as easy as it sounds. They don't put block hashes or block numbers, only a date. I tried to correlate the dates they give with the time headers of blocks, unsuccessfully. I tried every time zone, so I think they deliberately obfuscate their data. Until they do something about it, I can't really do much.


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: teukon on August 19, 2011, 08:01:28 PM
Do you think it would be difficult to add simplecoin.us (http://simplecoin.us/index.php)?  It's not a pool which is trying to hide information and the site is being actively developed but I don't know the API situation.  It's been at 50 Gh/s recently so is big enough to be worth considering.

I offer 1 BTC for the addition of this pool.

The developers can be easily contacted here (https://bitcointalk.org/index.php?topic=11186.0).


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: darnoth on August 19, 2011, 08:14:56 PM
I tried running this in the past, but whenever I run updateblocks it fails to import, repeatedly spewing out:

PHP Warning:  pg_fetch_row() expects parameter 1 to be resource, boolean given in /root/pident/lib/inc.update.php on line 34
PHP Warning:  pg_query(): Query failed: ERROR:  current transaction is aborted, commands ignored until end of transaction block in /root/pident/lib/inc.update.php on line 100
000015/141657 processing... (00000000b3322c8c3ef7d2cf6da009a776e6a99ee65ec5a32f3f345712238473)PHP Warning:  pg_query(): Query failed: ERROR:  current transaction is aborted, commands ignored until end of transaction block in /root/pident/lib/inc.update.php on line 33

Can you tell me what version of postgresql you are using?

Thanks.


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Artefact2 on August 20, 2011, 11:24:11 AM
I tried running this in the past, but whenever I run updateblocks it fails to import, repeatedly spewing out:

PHP Warning:  pg_fetch_row() expects parameter 1 to be resource, boolean given in /root/pident/lib/inc.update.php on line 34
PHP Warning:  pg_query(): Query failed: ERROR:  current transaction is aborted, commands ignored until end of transaction block in /root/pident/lib/inc.update.php on line 100
000015/141657 processing... (00000000b3322c8c3ef7d2cf6da009a776e6a99ee65ec5a32f3f345712238473)PHP Warning:  pg_query(): Query failed: ERROR:  current transaction is aborted, commands ignored until end of transaction block in /root/pident/lib/inc.update.php on line 33

Can you tell me what version of postgresql you are using?

Thanks.

This was solved on IRC, but the problem was using PostgreSQL 8.x (needs version 9 at least).


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Artefact2 on August 20, 2011, 11:44:42 AM
Do you think it would be difficult to add simplecoin.us (http://simplecoin.us/index.php)?  It's not a pool which is trying to hide information and the site is being actively developed but I don't know the API situation.  It's been at 50 Gh/s recently so is big enough to be worth considering.

I offer 1 BTC for the addition of this pool.

The developers can be easily contacted here (https://bitcointalk.org/index.php?topic=11186.0).


Okay, done. Now I don't mind you donating 1 BTC for pident developement, but to be honest, I don't think a 5-line commit (https://github.com/Artefact2/pident/commit/f04ba372bd614699a908aac96c8e8dbbbd4a887b) is worth this :P


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: teukon on August 20, 2011, 01:00:26 PM
Do you think it would be difficult to add simplecoin.us (http://simplecoin.us/index.php)?  It's not a pool which is trying to hide information and the site is being actively developed but I don't know the API situation.  It's been at 50 Gh/s recently so is big enough to be worth considering.

I offer 1 BTC for the addition of this pool.

The developers can be easily contacted here (https://bitcointalk.org/index.php?topic=11186.0).


Okay, done. Now I don't mind you donating 1 BTC for pident developement, but to be honest, I don't think a 5-line commit (https://github.com/Artefact2/pident/commit/f04ba372bd614699a908aac96c8e8dbbbd4a887b) is worth this :P

Damn! I didn't know it would be so easy.

Ah well, I'm a man of my word so 1 BTC for you.  Enjoy! :)


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Artefact2 on September 02, 2011, 01:27:39 PM
Okay, the recent changes are enough for a version bump (because why not?). There is now a cool SVG graph of pool sizes, pools are now correctly sorted by descending size. Added support for BTCServ and readded RFCPool (yes, they repoened!).

I'd like to add BitClockers, ABCPool and NinjaCoin to the list, but I can't… If you want them too, try to convince the operators to publish their blocks!

(Also, sorry for the downtime Monday & Tuesday, had a big power outage just when I left the house for 2 days…)


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: leveer on September 07, 2011, 10:59:55 PM
Only three confirmed block owners since block #144,352, and none since #144,364. Did something radical change in the last 8 hours?


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: mb300sd on September 09, 2011, 12:02:53 AM
Completely down now???


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: leveer on September 09, 2011, 12:59:48 AM
Completely down now???
Each block is still being scored, just no verified owners...


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Artefact2 on September 09, 2011, 02:43:36 PM
Hello everyone. Sorry, I moved out and I only got my internet connection back right now… I'm still quite content to see it worked fine except for the "pools not showing" problem.

It should be fixed now. Email me or reply if you have more issues :)


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: bz on October 01, 2011, 09:05:37 PM
Seems stuck. It is not showing any updates since:

12 hours and 13 minutes ago    147,603   00000000000001b77b706b389df3ddfa9f161b66402092abc941e4dcd3f61cbc    4532 bytes    DeepBit    Scores


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: wknight on October 01, 2011, 10:03:14 PM
Ok who broke the server!


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Artefact2 on October 03, 2011, 06:15:24 PM
Ok who broke the server!

Turns out I did. The patched "getblockbycount" branch does not compile with the newer miniupnpc versions, so I'm basically stuck here. If anyone can fix it, it's cool. Since I didn't get any feedback about pident in the last few weeks, I thought nobody was really using it anyway…

(And, of course, merging the old getblockbyhash branch in the new tree fails. And I'm not that great with C++.)


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Litt on October 03, 2011, 06:32:41 PM
so it is down for good?  


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Artefact2 on October 03, 2011, 07:41:49 PM
so it is down for good?  

Until someone can manage to get a working bitcoind with the getblockbycount patch, pretty much. Good news is, you can still toy with it, the source is still out there!


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Luke-Jr on October 30, 2011, 01:10:42 PM
Didn't see this till today... do you still want that fixed patch, or have you abandoned pident entirely in the meantime?


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Artefact2 on October 30, 2011, 01:15:12 PM
Didn't see this till today... do you still want that fixed patch, or have you abandoned pident entirely in the meantime?

Well, I don't think I will run a pident instance here on the foreseeable future, although it would be nice to actually have it in a runnable state, if anyone else wants to host it :)


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Raoul Duke on October 30, 2011, 02:01:37 PM
I wouldn't mind hosting it. Have an empty VPS I could use for it. But fixing it is out of my capacity as I'm not a programmer ;P

What would be the minimum requirements for the server?


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Aexoden on October 31, 2011, 12:01:23 AM
I've been running my local bitcoind with the getblockbycount patch just fine. The patch has to be somewhat manually applied because there are conflicts since the last time that branch was rebased or merged. Someone should either update that branch or put an updated version somewhere else. Or even better, get the patch merged into bitcoin.

psy: In what experience I had, the biggest barrier is doing the initial load of the blockchain. It was fairly I/O intensive and wrought havoc with my VPS. I eventually had to give up (as it would have taken several days to finish), and run it instead on my local machine. However, the particular host machine my VPS is on has had a history of high I/O load from other users, so it's possible it might work better elsewhere.


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Artefact2 on October 31, 2011, 08:28:27 AM
psy: In what experience I had, the biggest barrier is doing the initial load of the blockchain. It was fairly I/O intensive and wrought havoc with my VPS. I eventually had to give up (as it would have taken several days to finish), and run it instead on my local machine. However, the particular host machine my VPS is on has had a history of high I/O load from other users, so it's possible it might work better elsewhere.

Yeah, it took several hours for me too, the first time. It was really frustrating actually when debugging it, when the integrity breaks near the end and you have to restart everything just to debug it! ;D

But, if you can manage to get a somewhat recent dump of the db, inserting the new missing blocks is faster. The latest backup I have is from late august and is about 420M.


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Raoul Duke on October 31, 2011, 03:25:56 PM
The first 120k blocks are on sourceforge, zipped and ready to download


Title: Re: (Experimental) pident — a pool-biased blockchain representation
Post by: Dusty on December 26, 2011, 08:56:57 PM
Great, work, thanks!