Bitcoin Forum
April 25, 2024, 05:07:32 PM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
  Home Help Search Login Register More  
  Show Posts
Pages: [1]
1  Economy / Service Announcements / Re: [ANN] NiceHash.com - innovative professional cryptocurrency cloud mining service on: July 14, 2014, 06:12:22 PM
Python stratum-mining doesn't have an option to decide how it processes packets.  It runs under twisted and all of the json code is handled by the stratum package that was written directly by slush himself.

If a miner can connect fine, and then nicehash puts something in between the miner and the pool that makes things stop working, it seems to me it's nicehash's problem and responsibility to fix this and if he needs to involve the pool owner he should be respectful and not try to shift blame by calling out the pool in public like he has.  To my knowledge there is no contract between nicehash and ipominer and if nicehash does not give the user a choice in pools then they are responsible when things don't work.

I doubt the issue is inside slushs code, but must be related to recent addition of variable diffs by Ipominer.

I am sorrry, but it looks like you don't have full understanding of how our system works exactly. Did someone ask you to post in this thread? Because you started posting just yesterday.

All other pools except Ipominer works fine.

Cgminer/sgminer is not god. I will remind you of the bugs that we discovered - extranonce in cgminer 3.7.2 and idlebug was present in even most recent cgminer and sgminer. Guess with what was idlebug related? Networking! So, we simply cannot afford to take another publicly available code and say - this code is 100% reliable, we have to build on top of that. No, the only valid document we take are specifications (yes, these are poor) written by slush himself and found at various places around the internet.

I found a post by slush regarding a similar problem, but with regards to the pool rather than miner:

I have read the discussion about separating target/difficulty and work data. I have to say I think this was a mistake as it opens possibilities for abuse. I think something about these issues should go in the docs. Otherwise someone is going to create a naive implementation with some nasty issues.

Well, after discussion with Kano I slightly changed my mind. As I described few posts ago, with slightly modified meaning of "set difficulty" message, everything should be fixed.

Instead of meaning "stop sending lower difficulty shares than X immediately", set_difficulty should mean "all next jobs are using difficulty X". This still keeps the concept of separated difficulty message from the job itself, it doesn't require sending both messages together, fixes the problem of roundtrips (sending some bad-diff shares by the client because of latency) etc. It also gives the possibility to choose if pool wants to change difficulty in aggressive or in lazy way; by just sending "set difficulty" message without new job with clean_jobs=True, clients will slowly adopt higher difficulty as miners will start using new jobs, but pool can enforce higher difficulty with immediate effect by sending clean_jobs=True, as it is now.

This change in protocol is also backward compatible, because pools using current meaning of set_difficulty don't need to change anything, they just *may* stop sending clean_jobs job, if they want. Because of this, it is win-win solution for me. I described this already, but the idea didn't get any significant attention yet.

Quote
Is it ok to pipeline commands to the client? That is, send the difficulty change and new work (2 RPC calls) in quick succession without waiting for a reply to the first.

Yes, commands (notifications) can be pipelined. set_difficulty() and notify() don't expect any response from the client. Just sending one packet with both commands serialized is possible.

One interesting part is:
Quote
...set_difficulty should mean "all next jobs are using difficulty X".
Which, to me, means that the pool and miner should both be using the difficulty which was *sent most recently*. If this is the case, Ipominer has the issue.

Quote
...doesn't require sending both messages together...
Sure, this is talking about pool to miner, but the protocol is (from my understanding) supposed to be pretty similar from both the pool and miner's perspectives (aka "bi-directional" as per slush's specs). He says that sending both messages together isn't "required", which implies that it isn't "prohibited" either. If we can assume that it works the other way around (from miner to pool), then it seems that the pool should allow messages to be sent "together", because that is not specifically prohibited (unless I missed some part of the specification, which wouldn't be difficult to do). However, as has been mentioned, the time that those particular bytes arrived shouldn't be used to parse them apart.

Slush says notification commands can be pipelined, and then qualifies that allowance by stating
Quote
set_difficulty() and notify() don't expect any response from the client.
However, the miner *can* expect a response from its subscribe request, so I think that same qualification doesn't apply in this situaton.

Then he says
Quote
Just sending one packet with both commands serialized is possible.
Again this was intended to be regarding the pool->miner direction, but if we can safely assume that the protocol is truly "bi-directional" then the same can be said about the miner->pool direction, as we are talking about here.

Therefore, it is still my conclusion that Ipominer is the one originally at fault and should fix their implementation to allow for multiple commands per packet. NiceHash seems to be using a method which is new, yet is not specifically prohibited, and therefore pools need to allow for it to occur.

This service abstracts the pool it's mining on away from the end user and out of the user's control, correct?
To my knowledge there is no contract between nicehash and ipominer and if nicehash does not give the user a choice in pools then they are responsible when things don't work.
There are two types of "users" on NiceHash - buyers and sellers. The sellers (miners) are the ones who don't know where their shares are being directed at. The buyers are the ones who do the directing. So in that sense, users (buyers) do have a choice in pools.

Now, Ipominer's site says it uses a "modified Proportional scoring system". If this system was paying out based on the higher difficulty (the same one that miners on NiceHash were using), then there shouldn't be a problem other than the aforementioned hashrate display issue. If instead, however, their system was basing share payout value on the lower diff, or not at all for those shares, then there's a problem. The solution would be to recalculate submitted share value and pay their users a compensation for the difference.

2  Economy / Service Announcements / Re: [ANN] NiceHash.com - innovative professional cryptocurrency cloud mining service on: July 14, 2014, 02:40:31 AM
No, this is just example I put together to get wanted result. We don't send together, we have several sends. In fact, after each JSON string, we have separate send call for \n. And in real case scenario I observed that first JSON string is sent without \n, then second JSON string has \n pre-appended and one \n at the end. Even though in code, it is being done 2x send+send. This is TCP protocol and you have no control over how data is being split among packets - this means you should never rely on PACKETS in TCP, but read it as a STREAM! If your software reads them as packets, then this is very very wrong.

Your software should not parse JSON line until \n is received. If we follow this logic, then first line is parsed with some result (in this case, diff adjusted + reply + mining notify sent), then it is time to parse next line, which is authorization. Your software does something terribly wrong, because as it seems, it parses both lines at the same time (maybe due to multithreading?) or even worse - it parses line that comes second first. I can't judge that, because I have no idea what kind of software you use.

So in summary, let me see if I understand correctly:

ipominer used code that works with existing mining software and called it good. Since it worked with 100% of clients 100% of the time, there was no reason to do further testing, right? From my understanding, there's really only a small handful of different mining software out there. Sure there are many versions of sgminer, but the original neworking code is probably all the same from the first version (no, I didn't check this myself - that's just a guess). It just so happened that (nearly?) all software out there sends the initial subscribe and authorize requests in the same way - probably ending up in two separate packets.

nicehashdev used an implementation that was slightly different than most (all?) pre-existing miner applications. Instead of those initial subscribe/authorize requests being sent over the wire in two packets, they were sent in a single packet (as is the case in that C# example). I know nicehashdev said they use separate sends, but it seems whatever they're doing ends up combining messages as in the example. In any case it doesn't really matter, and I'll focus on the example given.

nicehashdev is correct that networking communications data must be treated as a stream and not as individual packets. Data could be split/combined in ways you can't control over the internet - there's no guarantee that data will arrive in the same 'chunks' they were sent. There IS a guarantee, however, that the bytes arrive in the same order as they were sent (this was mentioned earlier).

I tried out nicehashdev's C# example (both subscribe & authorize sent in a single packet) and sure enough the response received was the low diff, then the high diff (received all in a single packet  Wink). However, splitting the initial subscribe/authorize Send() into two different sends (no pause in-between) resulted in a response with the high diff, then the low diff. Exactly the same data was sent, but splitting the data into two send events (and therefore packets) resulted in the 'proper' response.

I've dealt with this kind of problem before in my own experience with network coding. The issue is always in how I read the data - not how it was sent. I always have to remind myself that the receiving end needs to treat the incoming bytes as a stream - never as a 'packet'.

Therefore it is my conclusion that nicehashdev is correct and that ipominer's implementation does not correctly parse the received data. This also explains why other pools don't have this issue with NiceHash.

The situation isn't helped by the fact that the stratum protocol documentation is kinda poorly organized. Seems like programmers are sort of forced to test their code by comparing it to how other preexisting implementations handle things, rather than coding according to the documentation (hard to do since it's seemingly spread out in an unorganized fashion). Frankly I'm surprised these sorts of problems don't pop up more often.
3  Alternate cryptocurrencies / Announcements (Altcoins) / Re: [ANN][DRK] DarkCoin | First Anonymous Coin | First X11 | First DGW | ASIC Resistant on: May 19, 2014, 01:27:54 PM
Did anyone else notice that in the Windows Binaries ZIP (Windows 1.3 avx-aes: http://download.darkcoin.fr/darkCoin-cpuminer-1.3-avx-aes-windows-binaries.zip) both
minerd-core-avx-i.exe and minerd-corei7-avx.exe are exactly the same files? A checksum comparison gives the same result. I'm trying to write a little setup script for my buddies to set up the miner on their computers and noticed this.

4  Alternate cryptocurrencies / Pools (Altcoins) / Re: [ANN][POOL] Profit switching pool - wafflepool.com on: March 27, 2014, 11:09:04 AM
phzi -

Maybe I'm misunderstanding your posts, but it seems you think network switches forward packets to all physical ports? I think you mean "hubs". Google "switch vs hub" and you'll see what I mean.
Pages: [1]
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!