Title: pushpool's client developing issues
Post by: elmigranto on June 29, 2011, 09:13:06 AM
Developing pushpool clientHello, I'm really interested in jgarzik's pushpool (http://forum.bitcoin.org/index.php?topic=8707.0) (it's binary protocol in particular) and writing Qt-based client. Problem is that I can't make it through LOGIN step. As far as I undestand from forum's threads and (mostly) pushpool's source code, I need to send ubbp_header with BC_OP_LOGIN operation code and message's length in bytes. Message is SHA256 of compressed json updated with userpass following compressed json. Json looks something like this (protocol is always 1, config is always null): {"protocol":1, "user":"username", "config":null}And I'm doing following (higlighted version (http://pastebin.com/snGT48tJ)): ubbp_header header;
strcpy((char*)header.magic, PUSHPOOL_UBBP_MAGIC);
QSocket socket;
socket.connectToHost("my.pushpool.host", 8336);
socket.waitForConnected();
// add compressed json to data
dataToSend = json.toUtf8();
dataToSend = (qCompress(dataToSend, 9));
// Since qCompress() adding length (first 4 bytes in dataToSend) in Big-Endian,
// but server uses le32toh() to get it, change endianness to Little-Endian:
dataToSend.remove(0, 4);
uint32_t len = json.length();
dataToSend.insert(0, (char*)&len;, sizeof(uint32_t));
// add compressed json digest
char pass[] = "test";
uchar md[SHA256_DIGEST_LENGTH];
SHA256_CTX ctx;
SHA256_Init(&ctx;);
SHA256_Update(&ctx;, dataToSend.data(), dataToSend.size());
SHA256_Update(&ctx;, pass, strlen(pass));
SHA256_Final(md, &ctx;);
dataToSend.append((char*)md, SHA256_DIGEST_LENGTH);
// Send operation code & message size
header.op_size = UBBP_OP_SIZE(BC_OP_LOGIN, dataToSend.size());
socket.write((char*)&header;, sizeof(ubbp_header));
socket.waitForBytesWritten();
// Send message
socket.write(dataToSend);
socket.waitForBytesWritten();
// Here's a problem:
// after message sent, I get
// QAbstractSocket::error() with 'Remote host closed connection' My assumptions: - JSON compressed by qCompress() cannot be uncompressed by zlib (even after my manipulation)
- I'm calculating wrong SHA256
- Something wrong with json (i.e. config must not be null)
After puzzling for almost whole day, I'm writing here. Guys, please help.
Title: Re: pushpool's client developing issues
Post by: elmigranto on June 29, 2011, 04:02:11 PM
By printing out some debug info I've find out that's problem is uncompressing on server-side. I'm trying to do the following using Qt-built-in zlib (highlighted version (http://pastebin.com/SHPSmmPL)): char json[] = "{\"version\":1,\"user\":\"test\"}";
std::auto_ptr<Bytef> message(new Bytef[ // allocate memory for:
sizeof(ubbp_header) // + ubbp_header
+ sizeof(uLongf) // + uncompressed length
+ strlen(json) // + message
+ SHA256_DIGEST_LENGTH // + sha256
+ 64]); // + reserve (if compressed size > uncompressed size)
uLongf unc_len = strlen(json);
uLongf comp_len = unc_len + 64;
Bytef* pHeader = message.get(); // header
Bytef* pLen = pHeader + sizeof(ubbp_header); // length of uncompressed data
Bytef* pCompDat = pLen + sizeof(uLongf); // compressed data
Bytef* pSha256; // digest
if (Z_OK != compress2(pCompDat, &comp_len, (Bytef*)json, unc_len, 9))
{
qDebug("Compression failed");
return false;
}
/* calculating message digest */
char pass[] = "test";
uchar md[SHA256_DIGEST_LENGTH];
SHA256_CTX ctx;
SHA256_Init(&ctx);
SHA256_Update(&ctx, pLen, comp_len + sizeof(uLongf));
SHA256_Update(&ctx, pass, strlen(pass));
SHA256_Final(md, &ctx);
/* building message */
uLongf msg_len = sizeof(ubbp_header) + sizeof(uLongf) + comp_len + SHA256_DIGEST_LENGTH;
header.op_size = UBBP_OP_SIZE(BC_OP_LOGIN, msg_len - sizeof(ubbp_header));
pSha256 = pCompDat + comp_len;
memcpy(pHeader, &header, sizeof(ubbp_header));
memcpy(pLen, &unc_len, sizeof(uLongf));
memcpy(pSha256, &md, SHA256_DIGEST_LENGTH);
/* sending message */
socket.write((char*)message.get(), msg_len);
socket.waitForBytesWritten();And still server-side uncompress() (located in cjson_decode()) doesn't returns Z_OK. Where's mistake?
Title: Re: pushpool's client developing issues
Post by: elmigranto on June 29, 2011, 05:30:01 PM
uncompress() actually returns Z_BUF_ERROR indicating the output buffer was too small.
I don't understand how it can be since server allocating uncompressed_length + 1 bytes which obviously enough.
I'm pretty sure that sent uncompressed_length is correct.
Title: Re: pushpool's client developing issues
Post by: elmigranto on June 30, 2011, 12:52:54 PM
Turns out it was server-side bug. In cjson_decode() function located in server.c unpacking code contains following lines: 176 /* decompress buffer (excluding first 32 bits) */
177 comp_p = buf + 4;
178 if (uncompress(obj_unc, &dest_len, comp_p, buflen - 4) != Z_OK)
179 goto out;
180 if (dest_len != unc_len)
181 goto out;
182 memcpy(obj_unc + unc_len, &zero, 1); /* null terminate */ This call of uncompress() isn't completely correct according to zlib docs (http://zlib.net/manual.html). dest_len should contain actual size of buffer for uncompressed data before calling uncompress(), but it's never initialized. This should be corrected (e.g. dest_len = unc_len + 1).
Title: Re: pushpool's client developing issues
Post by: ius on June 30, 2011, 02:40:48 PM
You seem to be right. I have submitted a pull request for Jeff: https://github.com/jgarzik/pushpool/pull/39
(I passed unc_len instead of unc_len + 1, as the last byte is used for null termination and not part of the compressed data)
Title: Re: pushpool's client developing issues
Post by: elmigranto on June 30, 2011, 04:26:30 PM
You seem to be right. I have submitted a pull request for Jeff Could you submit another one (I'm pretty new to CVS, and have sadly too little free time to dig in)? cjson_encode(), server.c: 233 /* fill in compressed length */
234 *obj_clen = htole32(comp_len); It's little mismatched with whole idea. Instead of comp_len, unc_len should be there. Must be too late in night, when this part was written.
Title: Re: pushpool's client developing issues
Post by: ius on July 01, 2011, 01:48:18 PM
Really looks like you're the only one to have ever used the binary protocol - anyway, submitted as well. https://github.com/jgarzik/pushpool/pull/40
Thanks.
|