Title: [BOUNTY] Pushpool question Post by: davout on June 01, 2011, 11:38:40 AM TL;DR : Share solutions, how do they work ?
I've installed jgarzik's pushpool and it's successfully running against a bitcoin client running on testnet, Pointed a miner at it, generated a few blocks, now I need to track them, and link them to the share that provided the right solution. So I have a share where pushpool's result is "Y", upstream result is "Y", seems like it generated the block http://blockexplorer.com/testnet/block/000000000055825d01acc3dcc80d02671c2e75ab5b692940446e9b8615ea3824 I do own the address and the logs show a share submission time that is the same as the block timestamp. The recorded solution for the share is Code: 0000000150c549978a44e3ef671a4ed3f5b922195ebde0fbbccfdb5900e6827000000000c8a03dfb42fa04d47e9fbfd3e6afcb3d0df2b516eb86f4e64d09fe53cc397b6c4de61f4c1c069652a228e13d000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000 and the block hash is Code: 000000000055825d01acc3dcc80d02671c2e75ab5b692940446e9b8615ea3824 So my question is, how do I get to the block hash from the submitted share solution so I can track these properly ? Title: Re: Pushpool question Post by: davout on June 01, 2011, 12:58:40 PM No one ?
Title: Re: Pushpool question Post by: Pieter Wuille on June 01, 2011, 01:27:10 PM That looks like a hex-encoded block header, including padding done by the sha256 hasher.
Take the first 160 characters of it (80 bytes), decode each group of 2 characters as a hex to a byte, and feed those 80 bytes to a regular sha256 hasher, twice. That should give you the block's id. Title: Re: Pushpool question Post by: davout on June 01, 2011, 01:44:26 PM That looks like a hex-encoded block header, including padding done by the sha256 hasher. Take the first 160 characters of it (80 bytes), decode each group of 2 characters as a hex to a byte, and feed those 80 bytes to a regular sha256 hasher, twice. That should give you the block's id. Thanks a lot! I'll try that and post the results. Title: Re: Pushpool question Post by: davout on June 01, 2011, 02:28:42 PM Maybe I'm doing something wrong but it doesn't really seem to work ...
The original solution : Code: irb(main):038:0> str I strip it down to the 160 first chars Code: irb(main):039:0> stripped = str[0,160] I pack it to a binary string : Code: irb(main):042:0> binary_data = stripped.to_a.pack("H*") Hash it a first time Code: irb(main):043:0> first_pass = (Digest::SHA2.new << binary_data).digest Then a second time Code: irb(main):044:0> second_pass = (Digest::SHA2.new << first_pass).digest And then finally unpack the result back to hex Code: irb(main):045:0> second_pass.unpack("H*") Doesn't match the block hash... I feel like a total clueless noob on this one :( Any ideas ? Title: Re: [BOUNTY] Pushpool question Post by: davout on June 01, 2011, 02:56:50 PM This is pretty frustrating,
I'll pay 3 BTC to the person providing a Ruby solution (or that helps me code a working one :D) EDIT : Maybe there's something to do with endianness ? Title: Re: [BOUNTY] Pushpool question Post by: xf2_org on June 01, 2011, 03:45:42 PM It is as sipa said... This is not a custom format. pushpool is recording the 'getwork' data sent from the miner client, verbatim. That is standard across all of bitcoin. The first 160 hex chars (80 bytes) are the block header that successfully found a solution.
To link that to a block, the block header includes the 'prevhash' field that tells you precisely block X-1, for solved block X. Title: Re: [BOUNTY] Pushpool question Post by: keine-ahnung on June 01, 2011, 04:35:44 PM http://pastebin.com/Ya3604J0
I found this code which takes some information and calculates the checksum of the blockheader. It should be possible to use this code to read the hash of the previous block. Title: Re: [BOUNTY] Pushpool question Post by: davout on June 01, 2011, 08:03:58 PM Hey !
thanks for all the input ! http://pastebin.com/Ya3604J0 I found this code which takes some information and calculates the checksum of the blockheader. It should be possible to use this code to read the hash of the previous block. I found this piece of code, it's been really useful, now I manage to decode the block header and extract the different fields, I don't think it's a necessary step to retrieve the 2xSHA256 checksum but I just felt it would help my understanding. Thing is I'm doing something that shouldn't work, yet it somehow works... I can't get my head around it. So for example, to retrieve the previous block hash from the block header I thought I would have to do this : - Chop the string between position 8 and 8 + 64 to get a hex representation of the previous hash in little endian - Swap bytes to retrieve big endian representation So, to swap bytes I first tried to reverse the string with chunks of two hex chars, "ab cd ef gh ij kl mn op" becoming "op mn kl ij gh ef cd ab". Didn't work out well. But for some reason I don't understand, doing that in chunks of 4 bytes (8 chars) yielded correct results ("ab cd ef gh ij kl mn op" -> "ij kl mn op ab cd ef gh") Looks pretty straightforward in the example code (which yields correct results btw when I hardcode my data sample) : - Take previous block hash in big endian - Swap bytes without chunks - Hash along with the rest of header data My code (missing the checksum part, and mysteriously succeeding at extracting merkle root and previous block hash) : http://pastebin.com/yVvvwRUd Title: Re: [BOUNTY] Pushpool question Post by: xf2_org on June 01, 2011, 08:09:59 PM You may also find python hash checking source code, well documented for analysis:
https://github.com/jgarzik/pyminer/blob/master/pyminer.py Miner.work() http://yyz.us/bitcoin/poold.py checkwork() Title: Re: [BOUNTY] Pushpool question Post by: davout on June 01, 2011, 08:14:32 PM You may also find python hash checking source code, well documented for analysis: Thank you very muchhttps://github.com/jgarzik/pyminer/blob/master/pyminer.py Miner.work() http://yyz.us/bitcoin/poold.py checkwork() Title: Re: [BOUNTY] Pushpool question Post by: davout on June 01, 2011, 09:53:36 PM Here we go, if anyone is interested in the ruby BlockHeader class.
Not very elegant, but it does the job : - Extracts block header fields - Computes block hash http://pastebin.com/y0sMZSbi Title: Re: [BOUNTY] Pushpool question Post by: Wuked on June 09, 2011, 04:44:53 PM Thanks for this, it's excellent.
Can I ask what method you are using to convert your solution, or your block_hash into the block ID? Title: Re: [BOUNTY] Pushpool question Post by: Wuked on June 09, 2011, 04:59:04 PM Obviously you can get it from : http://blockexplorer.com/testnet/rawblock/<HASH>
But, is there a better way ? Title: Re: [BOUNTY] Pushpool question Post by: davout on June 10, 2011, 08:19:43 AM Obviously you can get it from : http://blockexplorer.com/testnet/rawblock/<HASH> Certainly not, the block hash is derived directly from the share solution, see the class I pasted.Basically you initialize a BlockHeader object, feed it the share solution, from there it gives you accessors on the various block header properties (prev. block hash, timestamp, nonce etc.) and an accessor on the block hash. You can do : Code: b = BlockHeader.new(solution) And this doesn't hit any external service Title: Re: [BOUNTY] Pushpool question Post by: Wuked on June 10, 2011, 08:19:24 PM Yes, I understand that part.
Sorry, I think I was confusing - I'm new to this and was just wondering how you personally going about getting the block number. I've since worked out that different people seem to have different "counts" on the block number - some people starting from 0 and some from 1. When I say block number, I don't mean the hash. I mean the number. Title: Re: [BOUNTY] Pushpool question Post by: davout on June 10, 2011, 09:25:04 PM Oh I see, that makes sense.
The answer is that I don't compute the block number. I don't think there is something like a block ID, there can be multiple blocks at a point of time with the same height (same previous block) and (theoretically) multiple blocks with the same hash. Theoretically it's also possible, even if tremendously unlikely, to have different blocks, at the same height, and having the very same hash. I wondered about how finding about it though, didn't find a satisfying solution and ended up dropping this issue altogether since I don't really need it in my context. Please someone correct me if I'm wrong but I think the only way to get the block number without relying on an external service is to simply maintain a linked list of the whole block chain's headers starting from the genesis block and count the items of the list. Title: Re: [BOUNTY] Pushpool question Post by: redshark1802 on June 19, 2011, 04:25:24 PM Hello,
could some please provide a php solution to this problem, or tell where i need to put the share result in the ruby programm. I couldn't figure it out. regards, redshark1802 Title: Re: [BOUNTY] Pushpool question Post by: redshark1802 on June 21, 2011, 04:32:41 PM thats sad :/
Title: Re: [BOUNTY] Pushpool question Post by: davout on June 21, 2011, 04:40:45 PM thats sad :/ What is your question exactly ? If you need someone to feed you an already made PHP solution I think you're out of luck :If what you want is use my code you just need a ruby interpreter Title: Re: [BOUNTY] Pushpool question Post by: redshark1802 on June 21, 2011, 04:57:30 PM Hi,
Sadly I have no idea of ruby, but is no problem runnind it in shell. My question exactly is: what command do I have to use, in order to retrieve the (previous)blokchash. I tried already bh = BlockHeader.new(shareValue) bh.inspect puts bh I get everytime a new hexvalue. thank you Title: Re: [BOUNTY] Pushpool question Post by: davout on June 21, 2011, 05:30:01 PM Hope this helps, don't hesitate.
Code: irb(main):005:0> bh = BlockHeader.new('0000000150c549978a44e3ef671a4ed3f5b922195ebde0fbbccfdb5900e6827000000000c8a03dfb42fa04d47e9fbfd3e6afcb3d0df2b516eb86f4e64d09fe53cc397b6c4de61f4c1c069652a228e13d000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000') Title: Re: [BOUNTY] Pushpool question Post by: redshark1802 on June 21, 2011, 08:13:38 PM Thanks a lot :D
Title: Re: [BOUNTY] Pushpool question Post by: davout on June 21, 2011, 08:18:44 PM Thanks a lot :D I'm happy it helps you !Title: Re: [BOUNTY] Pushpool question Post by: AndyRossy on July 04, 2011, 05:01:18 PM Here's a Java port of the ruby code, for anyone who may need it.
Quote /* * Java code to Parse Solution */ package com.kokade.utils; import java.util.Date; /** * * @author Andrew P Ross - apr at kent.ac.uk */ public class Block { public Block(String solution) { //Creates a block from a solution; not sure if i'll make this an entity soon this.solution = solution; construct(); } public static void main(String args[]) { Block b= new Block(args[0]); System.out.println(b); } private String solution; private long version; private String header; private String previousBlockHash; private String merkleRoot; private Date timeStamp; //To change to date pl0iz. private long bits; private long nonce; private void construct() { String raw = solution.substring(0,160); version = Long.parseLong( raw.substring(0,8), 16);// @version ||= @raw[0, 8].hex previousBlockHash = switchEnd(raw.substring(8,8+64)); // @previous_block ||= switch_endianness(@raw[8, 64]) merkleRoot = switchEnd(raw.substring(72,72+64)); //@merkle_root ||= switch_endianness(@raw[72, 64]) timeStamp = new Date(Long.parseLong( raw.substring(136,136+8),16 )); // @timestamp ||= @raw[136, 8].hex bits = Long.parseLong( raw.substring(144,144+8), 16); // @bits ||= @raw[144, 8].hex nonce = Long.parseLong(raw.substring(152,152+8), 16); // @nonce ||= @raw[152, 8].hex } private String switchEnd(String string) { String str = string; String result = ""; while(str.length() > 0) { result += str.substring(str.length()-8, str.length()); str = str.substring(0, str.length() - 8 ); } return result; } @Override public String toString() { return String.format("Block [solution=%s, version=%s, header=%s, previousBlockHash=%s, merkleRoot=%s, timeStamp=%s, bits=%s, nonce=%s]", solution, version, header, previousBlockHash, merkleRoot, ((timeStamp==null)?"null":timeStamp.getTime()), bits, nonce); } public long getBits() { return bits; } public String getHeader() { return header; } public String getMerkleRoot() { return merkleRoot; } public long getNonce() { return nonce; } public String getPreviousBlockHash() { return previousBlockHash; } public String getSolution() { return solution; } public Date getTimeStamp() { return timeStamp; } public long getVersion() { return version; } } Quote java Block 00000001d219003dfd0ec774541ac59c6b566c2b11a64e99d745a2e70028f3e100000000fc415a3 869208702e88db306d9473f116b0412fd76739780b29953fea82d83d24e11de451c00824f8c5b32 da00000000000000000000000000000000000000000000000000000000000000000000000000000 0000000000000000000 Block [solution=00000001d219003dfd0ec774541ac59c6b566c2b11a64e99d745a2e70028f3e100000000fc415a3869208702e88db306d9473f116b0412fd76739780b29953fea82d83d24e11de451c00824f8c5b32da000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000, version=1, header=null, previousBlockHash=000000000028f3e1d745a2e711a64e996b566c2b541ac59cfd0ec774d219003d, merkleRoot=a82d83d2b29953fe767397806b0412fdd9473f11e88db30669208702fc415a38, timeStamp=1309793861, bits=469795407, nonce=2354787034] http://blockexplorer.com/testnet/block/00000000005784f943f091bb8383aebd97333f129510f6327c084f58de25a1e9 Obviously, remove the "main" to use it in other code as a bean or whatnot. Title: Re: [BOUNTY] Pushpool question Post by: Furyan on July 19, 2011, 09:57:21 PM And a port to C#, for anyone who may want/need it.
This isn't exactly the most efficient way to do it but will serve the purpose. I've also done a couple of things to make the returned values slightly more convenient for consumption purposes. A .NET pushpool will need this, which is why I bothered to take the time... Code: using System; Using is easy: Code: Block b = new Block("solution string"); Title: Re: [BOUNTY] Pushpool question Post by: Orodben on July 20, 2011, 02:50:40 PM For testnet solution "000000010b362ab3ec365c55dce00249fb087cb90ebf2208c4e97aa60022678a00000000616eb86 4ec5d0265277b564b916219818bad92cce0d52af9e3e748306af986384e206c1a1c00824f2a78f6 a600000000000000000000000000000000000000000000000000000000000000000000000000000 0000000000000000000"
Download the ruby code from http://pastebin.com/y0sMZSbi and save it as 'block_header.rb' make sure ruby and irb (interactive ruby shell) are installed. Code: irb I've found ruby mechanize (http://mechanize.rubyforge.org/) to be a good way to access web pages (such as block explorer) your block info will be at http://blockexplorer.com/testnet/block/00000000001b2ff1c57142a9d90592ea3be28fe747da10a889c90afc03ab6e7b scRUBYt! (http://www.justinspradlin.com/programming/ruby-screen-scraping-with-scrubyt/) makes a good screen scraper for pulling transaction info (txid) from block explorer. Lastly, you will need a json-rpc library (such as http://json-rpc.rubyforge.org/ - see the client side section) to pull transaction data from bitcoind. Now you have all of the pieces!
Title: Re: [BOUNTY] Pushpool question Post by: davout on July 20, 2011, 03:05:08 PM go to block explorer (via something like mechanize) to find information about the block (specifically the txid) better, trust your local data (and abide by block explorer's ToS), use the getblockbyhash patch to pull the generation tx id in order to track confirmations for the block and possible orphan status, see https://github.com/davout/bitcoin-pool for the complete implementationTitle: Re: [BOUNTY] Pushpool question Post by: Orodben on July 20, 2011, 06:22:56 PM Davout:
I appreciate the heads-up about blockexplorer. It's a great service and I do not want to end up spamming it. The only TOS I was able to find was "This server is up more than 99% of the time, but anything that pulls data from here should still be prepared for failure." on the real time stats page. Is there another TOS I've missed? The link you provided to the "getblockbyhash" patch went to a rails bitcoin-pool. Is the patch here or at another location? Title: Re: [BOUNTY] Pushpool question Post by: TeraPool on July 20, 2011, 07:03:04 PM Perhaps he is talking about this post? (http://forum.bitcoin.org/index.php?topic=724.msg382043#msg382043)
I am looking for help if anybody can tell me how to perform that patch :S (Never patched anything before!) Title: Re: [BOUNTY] Pushpool question Post by: davout on July 21, 2011, 08:06:19 AM The only TOS I was able to find was "This server is up more than 99% of the time, but anything that pulls data from here should still be prepared for failure." on the real time stats page. Is there another TOS I've missed? IIRC theymos stated something around the lines of this, I might be completely wrong here though.The link you provided to the "getblockbyhash" patch went to a rails bitcoin-pool. Is the patch here or at another location? I found it on Jeff Garzik's github bitcoin fork, it's funny, I went to check on the repo and I can't find the branch I used to compile bitcoin with the getblock RPC call built-in, maybe it's been included upstream. You might want to look this up because it's a pretty convenient and reliable way to get info about blocks :)Ok, found it, it's on the "getblockbycount" branch and hasn't been merged upstream. https://github.com/jgarzik/bitcoin/tree/getblockbycount Title: Re: [BOUNTY] Pushpool question Post by: Furyan on July 23, 2011, 09:13:31 AM Speaking of which, where might I find code to do this in C/C++?
Thinking about adding this to my version of pushpool. Title: Re: [BOUNTY] Pushpool question Post by: davout on July 25, 2011, 06:53:52 AM Speaking of which, where might I find code to do this in C/C++? Baiscally, any C miner has this code.EDIT : Here -> http://pastebin.com/Ya3604J0 Title: Re: [BOUNTY] Pushpool question Post by: Dusty on July 25, 2011, 08:13:36 AM btw: I can't find any documentation about the extraNonce: where is it recorded?
why I can't see it on blockexplorer and such? Title: Re: [BOUNTY] Pushpool question Post by: Furyan on July 25, 2011, 04:59:23 PM Speaking of which, where might I find code to do this in C/C++? Baiscally, any C miner has this code.EDIT : Here -> http://pastebin.com/Ya3604J0 = awesome. Thanks! Title: Re: [BOUNTY] Pushpool question Post by: MarKusRomanus on November 29, 2013, 06:03:00 PM I came across this thread after looking for info to convert the solution that pushpoold reports into the blockhash. It anyone is interested, I've written php code for this.
Title: Re: [BOUNTY] Pushpool question Post by: greatbotboy on February 06, 2014, 05:53:35 PM Deleted... I figured it out.
|