Syn - MultiMiner protocol server/backend

[sakkaku]

https://github.com/sakkaku/syn-multiminer/

This is something I have been working on for "fun".  If you can consider fun smashing your head repeatedly into your desk.  I wrote a small networking library that used json, saw the multiminer protocol specification and thought "hmm I could modify this slightly and..." BAM you have this monstrocity.

Aside from a slightly modified version of the WorkUnit class from the original multiminer server this is all new code.  It uses gevent so additional connections cost next to nothing with lightweight threads or "greenlets".  For a baseline on my dev machine the server consumes about 10-12MB of ram and less then 0.5% of a cpu core with ~30 connections from phoenix from my other desktop.

So far I have done:
- Notification on block change with new work, reset the accepted workunits so shares for the old block are invalid, etc.
- Checks to make sure duplicate shares are not submitted.
- Stubs for login, share accepted/rejected, meta variables and block_solved in config, so you could add your own database interaction.

If I could get someone to go over the encodings for when it sends out work and submits it to bitcoind and make sure I didn't fuck something up badly I would be very happy.  It hasn't solved a block yet, but I only have 800MH/sec so it will take awhile to see if it will  Connected to testnet and everything works great.

It features fairly nice debug output including all send/receives with the ability to disable it.  I <3 output so it is a bit much but makes debugging easy as pie.

Example after a block announce w/ 2 clients + some meta rate updates

Code:

[BROADCAST] <= 'BLOCK 132931'
[192.168.1.3:52737] <= 'WORK 000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa200000000e90b41897d0e334a00a5c4ac6393229154fcf69256d412d5be44076507a305b94e03ddc71a13218500000000 32'
[192.168.1.3:52738] <= 'WORK 000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa20000000095552674e0fe76d6c0dbf7beffaa3cc9115601ffcb087b52149311c51e9dffdc4e03ddc71a13218500000000 32'
[192.168.1.3:52738] => "('more', [])"
[192.168.1.3:52738] <= 'WORK 000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa2000000000347be4614686e62a5764a97395a0f06181050c6203cef990627e8e4694471934e03ddc71a13218500000000 32'
[192.168.1.3:52737] => "('more', [])"
[192.168.1.3:52737] <= 'WORK 000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa200000000e32d590c0f569e2c853c581d1090c6db0762390028119c970dec1bf1958424ec4e03ddc71a13218500000000 32'
[192.168.1.3:52738] => "('more', [])"
[192.168.1.3:52738] <= 'WORK 000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa2000000005690863bf1d400840ba325ad2cdc8b02d582ee9f1dee4f4200e5ed784fd4b0434e03ddc81a13218500000000 32'
[192.168.1.3:52738] => "('result', ['000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa20000000095552674e0fe76d6c0dbf7beffaa3cc9115601ffcb087b52149311c51e9dffdc4e03ddc71a132185ddbb7673'])"
[192.168.1.3:52738] <= 'ACCEPTED 000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa20000000095552674e0fe76d6c0dbf7beffaa3cc9115601ffcb087b52149311c51e9dffdc4e03ddc71a132185ddbb7673'
[192.168.1.3:52738] => "('more', [])"
[192.168.1.3:52738] <= 'WORK 000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa20000000071c68581dafa6c9404a65642889cfd7ac082020e1cd79cd0e3d4646694ae2ead4e03ddc81a13218500000000 32'
[192.168.1.3:52738] => "('result', ['000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa2000000005690863bf1d400840ba325ad2cdc8b02d582ee9f1dee4f4200e5ed784fd4b0434e03ddc81a1321856bef3f2f'])"
[192.168.1.3:52738] <= 'ACCEPTED 000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa2000000005690863bf1d400840ba325ad2cdc8b02d582ee9f1dee4f4200e5ed784fd4b0434e03ddc81a1321856bef3f2f'
[192.168.1.3:52737] => "('more', [])"
[192.168.1.3:52737] <= 'WORK 000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa200000000b23a3e901fa82004b459cbe05a25d5fa89f56079682086786e0abc242cb0f5fa4e03ddc81a13218500000000 32'
[192.168.1.3:52738] => "('meta', ['rate', '362750'])"
[192.168.1.3:52737] => "('meta', ['rate', '296942'])"
[192.168.1.3:52738] => "('result', ['000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa2000000005690863bf1d400840ba325ad2cdc8b02d582ee9f1dee4f4200e5ed784fd4b0434e03ddc81a132185ddf755a1'])"
[192.168.1.3:52738] <= 'ACCEPTED 000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa2000000005690863bf1d400840ba325ad2cdc8b02d582ee9f1dee4f4200e5ed784fd4b0434e03ddc81a132185ddf755a1'
[192.168.1.3:52738] => "('more', [])"
[192.168.1.3:52738] <= 'WORK 000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa200000000ce546d6c2e6324fec4c6a6286aa21a48c4b660bfbf9f035e244e2224d1a688434e03ddd41a13218500000000 32'
[192.168.1.3:52737] => "('result', ['000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa200000000e32d590c0f569e2c853c581d1090c6db0762390028119c970dec1bf1958424ec4e03ddc71a1321852e4827e1'])"
[192.168.1.3:52737] <= 'ACCEPTED 000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa200000000e32d590c0f569e2c853c581d1090c6db0762390028119c970dec1bf1958424ec4e03ddc71a1321852e4827e1'
[192.168.1.3:52738] => "('result', ['000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa20000000071c68581dafa6c9404a65642889cfd7ac082020e1cd79cd0e3d4646694ae2ead4e03ddc81a13218567eb715a'])"
[192.168.1.3:52738] <= 'ACCEPTED 000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa20000000071c68581dafa6c9404a65642889cfd7ac082020e1cd79cd0e3d4646694ae2ead4e03ddc81a13218567eb715a'
[192.168.1.3:52737] => "('more', [])"
[192.168.1.3:52737] <= 'WORK 000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa2000000005b7919b7770d9745538fde4ac2596aeca134aacc9c63c0759e6b7f617b5780804e03ddd61a13218500000000 32'
[192.168.1.3:52738] => "('result', ['000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa20000000071c68581dafa6c9404a65642889cfd7ac082020e1cd79cd0e3d4646694ae2ead4e03ddc81a132185c592fc7d'])"
[192.168.1.3:52738] <= 'ACCEPTED 000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa20000000071c68581dafa6c9404a65642889cfd7ac082020e1cd79cd0e3d4646694ae2ead4e03ddc81a132185c592fc7d'
[192.168.1.3:52737] => "('result', ['000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa200000000b23a3e901fa82004b459cbe05a25d5fa89f56079682086786e0abc242cb0f5fa4e03ddc81a13218552e9132f'])"
[192.168.1.3:52737] <= 'ACCEPTED 000000015323dbf15be2a4ecf4b106bcc4fda0370267e7a263349ae900000fa200000000b23a3e901fa82004b459cbe05a25d5fa89f56079682086786e0abc242cb0f5fa4e03ddc81a13218552e9132f'
[192.168.1.3:52738] => "('more', [])"

Client logins:

Code:

[192.168.1.3:52737] => "('login', ['default', 'default '])"
[192.168.1.3:52737] <= 'TIME 120'
[192.168.1.3:52737] <= 'TARGET ffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000'
[192.168.1.3:52737] <= 'WORK 000000011b9cbbf492d0ff14840647461e460d2bfee30340c4defc0f000011ee00000000f09dad9427cf7856e88e4a7636003f632be7bb1de3bf3920739ff7f6a277199f4e03d9a71a13218500000000 32'
[192.168.1.3:52737] => "('meta', ['kernel', 'phatk r95'])"
[192.168.1.3:52737] => "('meta', ['rate', '295633'])"
[192.168.1.3:52737] => "('meta', ['version', 'Phoenix Miner v1.50'])"
[192.168.1.3:52737] => "('meta', ['device', 'Cypress '])"
[192.168.1.3:52737] => "('meta', ['cores', '18'])"
[192.168.1.3:52737] => "('meta', ['os', 'Windows 6.1.7601'])"
[192.168.1.3:52738] => "('login', ['default', 'default '])"
[192.168.1.3:52738] <= 'TIME 120'
[192.168.1.3:52738] <= 'TARGET ffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000'
[192.168.1.3:52738] <= 'WORK 000000011b9cbbf492d0ff14840647461e460d2bfee30340c4defc0f000011ee00000000aaec5665f1b0e0014be4c6bc97b44a3e98052891cb6165d395200244dad6e1fc4e03d9a81a13218500000000 32'
[192.168.1.3:52738] => "('meta', ['kernel', 'phatk r95'])"
[192.168.1.3:52738] => "('meta', ['rate', '364722'])"
[192.168.1.3:52738] => "('meta', ['version', 'Phoenix Miner v1.50'])"
[192.168.1.3:52738] => "('meta', ['device', 'Cypress '])"
[192.168.1.3:52738] => "('meta', ['cores', '20'])"
[192.168.1.3:52738] => "('meta', ['os', 'Windows 6.1.7601'])"

I am going to go take a nap.  Such a long week >_<

[1715037947]

1715037947

[1715037947]

1715037947

[sakkaku]

Finally got it to connect to testnet.  I only meant to generate 1 or 2 blocks but it sorta took off on its own...

Sorry testnet.  Hopefully I didn't kill the difficulty >_<

[sakkaku]

I have added an experimental plugin that uses the same schema as pushpool I think.  I really don't feel like setting up a pushpool server to verify things match up 1:1 though.  If someone doesn't mind dumping their shares table to get better sample data I can make sure it is behaving as expected.  It doesn't support the "reason" column because I would need to modify WorkUnit a bit and a lot of the signaling I use rather then just onReject or onAccept.  I got one exception where the data/work was cut off but it should be 256 characters long.  Eh.

The plugins use storm for the ORM (to connect to mysql you need MySQL-python).  I haven't used it before so it was interesting and hacky with omg unicode requirement.

I need to finish up my schema backend as it should consume much, much less space/resources SQL wise then pushpool.  But I was a bit uncertain how to maintain parity with some of the pools like like displaying Mh/s.  Because I want to only sync shares every few minutes.  But it is not guaranteed it would be every x minutes.  Also there is the trick of counting shares delta time.  Do we opt just to resume and keep incrementing shares and update the table entry, or do we insert the shares for the given work into a new row.  Do we need to track rounds in the shares?  Ughghg

Oh well I will sleep on it a bit.  Also this post made no sense whatsoever but I am bored and chatty.