[ANN] Stratum mining protocol - ASIC ready

[slush]

luke-jr's code is intentionally checking far too often, trying to make stratum look worse than GBT. Enough of his crap, don't let his influence ruin stratum too.

Stepping out of holy war, do we have some solution how to prevent miners to abuse these calls? Make them non-free?

I think bfgminer users will be a bit surprised when they'll see charges of few hundreds satoshis every 30 seconds :-D.

[1713949310]

1713949310

[-ck]

luke-jr's code is intentionally checking far too often, trying to make stratum look worse than GBT. Enough of his crap, don't let his influence ruin stratum too.

Stepping out of holy war, do we have some solution how to prevent miners to abuse these calls? Make them non-free?

I think bfgminer users will be a bit surprised when they'll see charges of few hundreds satoshis every 30 seconds :-D.

Well this is what happens with his current approach to unsuspecting miners:
https://bitcointalk.org/index.php?topic=16385.msg1376838#msg1376838

[Graet]

luke-jr's code is intentionally checking far too often, trying to make stratum look worse than GBT. Enough of his crap, don't let his influence ruin stratum too.

Stepping out of holy war, do we have some solution how to prevent miners to abuse these calls? Make them non-free?

I think bfgminer users will be a bit surprised when they'll see charges of few hundreds satoshis every 30 seconds :-D.

Slush,
holy war and associated FUD aside

what attracted me to stratum was low bandwidth minimal pool software which is giving miners great results re stales and bandwidth savings (especially with VarDiff), where is the demand from miners to include this extra "functionality"- I see only 1 individual calling for this
Leave the author of GBT to fix his software and you focus on stratum - rather than changing stratum to suit him
Please do not bog down stratum with extra unneeded and unwanted code - if miners don't trust the pool they are mining on to do the right thing - why do they mine there at all?

Making a BIP for this is also a diversionary measure to waste your time, please, take a moment, step back and consider what is actually needed and will improve the protocol you have designed
Thanks for your time and effort creating a viable mining protocol and sharing it with us all
Graeme

[-ck]

Stepping out of holy war, do we have some solution how to prevent miners to abuse these calls? Make them non-free?

Specifically I think it's inappropriate to ask for more than one set of transactions for each stratum notify push. I don't know what else you'd want to do to prevent them using this for every single stratum push.

[slush]

Ok, let's focus to important things and don't solve something, what's not a problem. I'm going to disable get_transactions() on my pool, at least until Luke stops abusing it in holyminer.

[generalfault]

I've been working on my stratum-mining fork (which is pretty far along at this point.) But I've come to a conundrum that I'd like opinions on.

So let's start out with two important points:
there are 3 DB Libraries (Sqlite, Postgresql, and MySql) that I'm maintaining
I do NOT want to double that number (6) (In fact I WILL not... so I have to make a choice)

Currently they are using the native libraries (sqlite3,psycopg2,and MySQLdb)
These libraries are VERY fast (since they are basically just C libs)

I have ported the sqlite DB library interface over to sqlalchemy for testing and to get my footing on what it takes. (quite a bit of work, but doable)

However they do not have connection pooling (not a big deal for this application) and do not auto-reconnect (as far as I know.)

So to get these two features, the only real "popular" option seems to be SQLAlchemy (Core, not ORM)
I've done some profiling and SqlAlchemy bulk loading (after optimizations) seems to be half the speed of the native libraries. It's understandable... I'm not complaining mind you.

This brings me to the question, Is it worth the slowdown (and extra work) for the benefits (connection pooling and reconnects)

[Thralen]

Okay, stupid question here. I'm sure there is a simple answer but I'm not pulling it right now. Here is my problem:

I'm running BAMT on a headless, caseless setup. I'm trying to install Stratum, I have it installed and working. I have two problems though:

1. Since I'm headless, if I open a terminal window and putty in to the BAMT box I need to keep the computer I am logging in from turned on and the terminal window open, or Stratum stops. I know I can probably use nohup to get around this but since it automatically redirects to a log, and BAMT runs off of a flash drive, I need to figure a way for it to not be logging or I'm going to wear out my flashdrive mighty quickly (not to mention filling it to overflow first probably).

2. Same issue, Bamt, headless, caseless, etc... I want to set Stratum to start in the startup sequence so if I have to power down the BAMT box Stratum automatically starts up before the mining software does. I know where to put it, in theory, to do so but I need the same issue resolved. If the actual solution to my question #1 won't work in a startup sequence, then I need to know how to do the same thing in the startup sequence.

I keep my BAMT box on 24/7 except for reboots. I do not keep the windows machine I am using to log into the BAMT box on 24/7 so I'm trying to figure out how to resolve these issues so I can use Stratum all the time. Sorry if they are newbieish questions but I haven't been doing much with linux in more than a decade (except minimal experimentation to get BAMT going).

Thank you for any help in advance.
Thralen

[Graet]

Okay, stupid question here. I'm sure there is a simple answer but I'm not pulling it right now. Here is my problem:

I'm running BAMT on a headless, caseless setup. I'm trying to install Stratum, I have it installed and working. I have two problems though:

1. Since I'm headless, if I open a terminal window and putty in to the BAMT box I need to keep the computer I am logging in from turned on and the terminal window open, or Stratum stops. I know I can probably use nohup to get around this but since it automatically redirects to a log, and BAMT runs off of a flash drive, I need to figure a way for it to not be logging or I'm going to wear out my flashdrive mighty quickly (not to mention filling it to overflow first probably).

2. Same issue, Bamt, headless, caseless, etc... I want to set Stratum to start in the startup sequence so if I have to power down the BAMT box Stratum automatically starts up before the mining software does. I know where to put it, in theory, to do so but I need the same issue resolved. If the actual solution to my question #1 won't work in a startup sequence, then I need to know how to do the same thing in the startup sequence.

I keep my BAMT box on 24/7 except for reboots. I do not keep the windows machine I am using to log into the BAMT box on 24/7 so I'm trying to figure out how to resolve these issues so I can use Stratum all the time. Sorry if they are newbieish questions but I haven't been doing much with linux in more than a decade (except minimal experimentation to get BAMT going).

Thank you for any help in advance.
Thralen

screen is your friend
open a screen session (type screen -help after ssh to headless linux box) do as you would usually the screen session, when you are finished screen -d CTRL-a dto detach out of screen (programs keep running) then turn your pc off
 next time you want to check it ssh in screen -r and it is all still there waiting for you

[Thralen]

Thanks Graet, that looks like it'll be a solution for problem #1. There probably isn't any way to integrate that to problem #2 though, is there? Sorry to be a pain and as I said, thank you for the solution. That should at least let me be shutting my windows box off overnight. So long as I don't have an unexpected reboot on the BAMT box it'll work and I can always set it to roll over to non-Stratum proxy mining if it does reboot on me.

Thralen

[Inaba]

screen -d?  That's pretty rude!

Just CTRL-a d out of the screen session.  screen -d is kind of like kill -9... it's useful but not something you want to do if you don't have to.

[Graet]

screen -d?  That's pretty rude!

Just CTRL-a d out of the screen session.  screen -d is kind of like kill -9... it's useful but not something you want to do if you don't have to.

my bad, I rarely detach
I'll edit post - cheers

[crazyates]

I heard of other people using byobu, so I tried it and it works pretty well. Similar to screen, but it's one command to get into it, and I press F6 to exit back to the SSH.

[slush]

2. Same issue, Bamt, headless, caseless, etc... I want to set Stratum to start in the startup sequence so if I have to power down the BAMT box Stratum automatically starts up before the mining software does. I know where to put it, in theory, to do so but I need the same issue resolved. If the actual solution to my question #1 won't work in a startup sequence, then I need to know how to do the same thing in the startup sequence.

Put something like this to /etc/rc.local

Code:

screen -AmdS proxy /home/youruser/proxydir/mining_proxy.py

Then you can attach console by "screen -x proxy" and detach it (without stopping it) by Ctrl-a,d (double combo)

[Thralen]

Thank you Slush, I knew there had to be a way to do it just hadn't been able to figure it out yet.

'screen' did the trick, I had the BAMT box running overnight with my windows box shut down. Stales were under 0.2%, Considering my stales on Slush's pool were always in the 1% - 1.5% range before (don't know why but was willing to accept that to use the pool) it is a massive improvement.

I'll give that entry a shot Slush and see how it works, thank you.

Thralen

[PetrovMichael]

hello Slush

is it very stupid idea ? to add feature to your stratum mining_proxy.exe  additional feature to save stats data per worker and per block to local computer ?

for example "mining_proxy -f" means store block stats data to file to local comp where mining_proxy running

i think a many files will better than 1 file, the file per block (with the worker shares ) will reasonable ?
new block started - new file created? block found - file closed.

csv (or json) format for the stored data will great

[antirack]

I am calling stratum_proxy.py from a bash wrapper script (inside a GNU screen session) that restarts the proxy and notifies me by email in case it unexpectedly exists. This doesn't happen very often but once in a while.

A couple of hours ago it "crashed" and along with it apparently screen/bash/python as none of them where still running when I found out. It didn't get restarted nor did I get an email.

Code:

Dec  9 15:44:30 miner kernel: [364310.319247] show_signal_msg: 39 callbacks suppressed
Dec  9 15:44:30 miner kernel: [364310.319257] python[3275]: segfault at fffffffffffffffe ip 000000000049c2c9 sp 00007fff96908620 error 4 in python2.7[400000+271000]

Any idea how I could catch this easily in the future? I am running Ubuntu 12.04.

[slush]

Strange, I never had segfault on any of my machines (including ARM boxes etc). Do you have some unusual machine configuration? Are other applications stable?

[antirack]

It's all stable normally. It's a Zotac ID15 with Atom CPU - not very unusual I guess. If I am not wrong it happened the first time. But what bothers me a bit is that the wrapper script was supposed to catch unexpected exits such as this. Any other command line magic that comes to mind? ps aux via crontab every few mins?

[slush]

ps aux via crontab every few mins?

That will work, because crash of the proxy won't affect the watchdog itself.

[slush]

i think a many files will better than 1 file, the file per block (with the worker shares ) will reasonable ?
new block started - new file created? block found - file closed.

csv (or json) format for the stored data will great

This is completely outside of the project scope. It's more like job for miner itself. Please check documentation of your miner, some of them (cgminer) has some statistics and reporting already implemented.