[4+ EH] Slush Pool (slushpool.com); Overt AsicBoost; World First Mining Pool

[puddinpop]

The Remote mining in the official bitcoin release has no "send back" - function but we are searching a pooling-miner like puddinpops.

Is there a particular reason that the pooled miner I created is not sufficient?  It has the basics there already.  It checks the best hash from each client to make sure the clients are doing the work they say, and it distributes the generated bitcoins based on the calculated hash rate of each connected client.  I just updated this thread with the latest source and binaries for pooled mining.

[farmer_boy]

How can the server create winning hashes? Winning hashes are by definition hard to compute objects that are all shared as soon as they are known. In fact, you can cheat by simply having a fake miner on a bunch of different IPs (via ipv6 you can have millions) with one real miner. The real miner remembers all the previously hard to compute objects and does hard work so has the same information as the server. If the server sends a unit of work to a fake miner, the fake miner forwards the request to the real miner and otherwise does nothing.

Your assumption about IP addresses is also invalid, btw. People still have dynamic ip addresses.

I would be interested in the concept, but I don't think there is a solution.

[grondilu]

To me it seems that cooperative mining is a tough task, because the honnesty of participants has to be checked.  What's preventing someone to run a modified version of the client, that would just keep generated bitcoin for himself, while receiving bitcoins from others ?

I wonder if it wouldn't be simpler to just rent processing power.  For instance by renting a ssh access to each machine.

The price could be directly proportionnal to the total number of computed hashes.

[FreeMoney]

The Remote mining in the official bitcoin release has no "send back" - function but we are searching a pooling-miner like puddinpops.

Is there a particular reason that the pooled miner I created is not sufficient?  It has the basics there already.  It checks the best hash from each client to make sure the clients are doing the work they say, and it distributes the generated bitcoins based on the calculated hash rate of each connected client.  I just updated this thread with the latest source and binaries for pooled mining.

Can't they just send all except the winning ones? Maybe I don't understand though.

[ribuck]

To me it seems that cooperative mining is a tough task, because the honnesty of participants has to be checked.  What's preventing someone to run a modified version of the client, that would just keep generated bitcoin for himself, while receiving bitcoins from others ?

<sigh>

Either I haven't been very good at explaining why there's no possibility to cheat, or I'm wrong. But if I'm wrong, no-one has posted a specific objection. So I'll try to explain it again, by presenting a specific design to show that a dishonest client cannot cheat.

Suppose I operate a pooled mining server, and I recruit some clients who wish to pool their mining.

My server asks each client to do some hashing for it. It asks each client to submit any hashes they find that are above a certain threshold of difficulty. The server chooses a difficulty that is one-fortieth (1/40th) of the current "official" difficulty level.

My server gets a constant trickle of candidate hashes sent back by the remote mining clients. Every now and then, one of those hashes meets the official difficulty level and my server can generate a block, which earns my server 50 bitcoins.

I now distribute bitcoins to the remote mining clients, at the rate of one bitcoin for each hash that was submitted for the current block that was at or above 1/40th of the official difficulty level.

In the long run, I would expect to distribute 40 coins out of every 50 that my server generates, although there will be some fluctuation from block to block. Nothing in this scheme requires the clients to be honest, because there is no way that a dishonest client can cheat!

The client is calculating hashes that will generate 50 BTC for my server. Those same hashes are not of any use to a dishonest client. They cannot be used to generate 50 BTC for the dishonest client, because a different hash code is needed to encode the payment of the generated bitcoins to someone else. And if the dishonest client tries to cheat by generating hashes that will pay the generated bitcoins to themselves, then the hash codes they submit won't validate at my server and I won't distribute any share of the payouts to them.

So this scheme requires absolutely no trust of the client.

This scheme also does not require the mining client to have faith that the server is honest. If the server advertises that it is paying out 1BTC for each hash that is at least 1/40th of the official difficulty level, then every client that submits an "easy" hash for a block that was generated can check that they received their bitcoin. Any fraud would show up immediately.

[grondilu]

Either I haven't been very good at explaining why there's no possibility to cheat, or I'm wrong. But if I'm wrong, no-one has posted a specific objection. So I'll try to explain it again, by presenting a specific design to show that a dishonest client cannot cheat.

Sorry, your system seems quite well thought, indeed.  I hadn't been paying enough attention.

Still, I keep thinking that renting ssh access would be much simpler a solution.

[FreeMoney]

Ribuk, thank you for explaining again. I get it now. It wasn't sinking in that the miners were working on a block for which they couldn't take the reward, seems obvious now.

[QuantumMechanic]

It sounds like pooled mining would either flood the network with tiny transactions, or be unfeasible due to transaction fees.  So the server would therefore have to keep accounts for users which they could withdraw from once their balance gets large enough.

I guess it would probably be best then if the server required the miners to have MyBitcoin accounts, or something similar.  It'd be nice if MyBitcoin offered a pooled mining service; they've got all the accounting infrastructure in place, and it could be a nice additional source of revenue for them.

Very elegant solution, Ribuck.

[RHorning]

My server asks each client to do some hashing for it. It asks each client to submit any hashes they find that are above a certain threshold of difficulty. The server chooses a difficulty that is one-fortieth (1/40th) of the current "official" difficulty level.

One of the potential problems I can see is that you may have to tweak the "reduced" difficulty level in some way so that it really is about a 1 in 40 chance of getting a successful hash.  It would seem to me that a reduction of even a couple of bits in the expected hash would have major consequences in terms of how many "reduced difficulty hashes" are created and it is possible that reducing the difficulty to 1/40th of the "official" difficulty level may create many more "successful" reduced difficulty hashes than 40, perhaps even to a higher order of magnitude.  This is something to test with a successful experiment of this idea to see just how it might turn out.

Furthermore, it is also going to be random in terms of how many reduced hashes are going to be created per successful hash cycle, as you are dealing with probability here and not mathematical certainty in terms of how many are going to be generated by the "farm" even if it does produce in this example on average about 40 "reduced difficulty hashes".  It may still be worth the effort if it turns out that on average that is all you have to spend for an increase in successful blocks created, but it will be some blocks where the hash farm server makes a bunch of coins for the server operator and other times it may be at least temporarily a loss.

Another potential problem I also see is if you happen to be requesting hashes where you've received a couple dozen reduced hashes and then the network has a huge spike in difficulty.  This is a potential liability issue so far as the "owner" of the hash farm server is concerned, as it would substantially impact the "profit margin".  Think of it this way:  If the current difficulty is at say 800 and you've received 30 hashes with a difficulty level of 20 (1/40th of the difficulty, as you suggested here), then the difficulty increases to 1600, it is very likely that you may have to plow through another 30-40 hashes at the new higher difficulty level of 40 before you get a successful block into the chain mining the coins.  If in this example you are still paying out 1 BTC per reduced hash, it could get very costly for the person running the hash farm in this case.  It may wipe out any profit earned and certainly is another potential liability issue to think about.  I'd have to think through the statistical probability on an increase of difficulty some more in terms of giving some hard numbers, but it is something that should be accounted for.

Then again, difficulty could drop brining in at least statistically more money to the server operator.  The current tendency is for increases of difficulty, however.

Otherwise, I think the basic concept as outlined here is pretty sound and would tend to work rather well.  Almost too well as I could see this being put into a botnet and having the computers in the botnet be sending back potential hashes without having to worry about any payments.  Network bandwidth would be trivial and almost unnoticed, and you could even "throttle" the CPU bandwidth or throw it onto the GPU instead.  I have no doubt that eventually somebody is going to build one of these botnets if it hasn't already been built for Bitcoins.

[ribuck]

... if you happen to be requesting hashes where you've received a couple dozen reduced hashes and then the network has a huge spike in difficulty.  This is a potential liability issue so far as the "owner" of the hash farm server is concerned, as it would substantially impact the "profit margin"

Sorry, you misunderstand. The server operator only pays out to easy hashes that were submitted for a block that is won. There's no payout for easy hashes that were submitted for a block that someone else generates.

The difficulty can't change in the middle of a block. It always changes at a known time, and you always know exactly what the difficulty is, so there's no risk here.

...you are dealing with probability here and not mathematical certainty...

That's absolutely true, but over time probability tends towards mathematical certainty. If the server operator is getting a few percent of the generated bitcoins, it doesn't take long to become virtually guaranteed profit.

[asdf]

This is brilliant!

In the future, when bock sizes are huge, this will save a lot of bandwidth and storage. Only one "server" node needs the blockchain and the fat bandwidth required to receive the flood of transactions. Many clients can then contribute to bitcoin without the need to pay for all that infrastructure multiple times. This means that small players can stay in the market and kind of solves the whole scaling problem. Awesome!

[grondilu]

Am I the only one who thinks it would be way more simple to just rent distant shell access ??

[FreeMoney]

This is brilliant!

In the future, when bock sizes are huge, this will save a lot of bandwidth and storage. Only one "server" node needs the blockchain and the fat bandwidth required to receive the flood of transactions. Many clients can then contribute to bitcoin without the need to pay for all that infrastructure multiple times. This means that small players can stay in the market and kind of solves the whole scaling problem. Awesome!

Oh, I didn't even realize this. But it is correct I think. Only the header needs to be hashed. So this scheme has benefits beyond reducing the variance in generating. Very nice.

[farmer_boy]

I think I initially misunderstood the blocks in that the hashes that we compute are independent of the address. Now, I believe that we are looking for something like sha256 (bitcoint address + amount (I am not sure whether the value is implicit, so this value might not be there)  + <answer>) with some desired properties (of which the number of final zeros is an approximation), where + means some concatenation operator of bits.

If the computation is like that, _then_ it might work, but I don't see why someone cannot just only send you the worthless hashes (for you) and don't send those that are useful to you. By this mechanism they can bankrupt you. Anyway, I think it is a bad way to communicate with natural language only about these topics. Just write out the computations that you intend to do on the server and the client and then we can see whether it is a brilliant idea or a flawed one. Annotations on the computations as to which goal they intend to achieve is also useful for communication.

Forcing cooperation out of nodes is not easy.

[satoshi]

ribuck's description is spot on.

Pool operators can modify their getwork to take one additional parameter, the address to send your share to.

The easy way for the pool operator would be to wait until the next block is found and divy it up proportionally as:
user's near-hits/total near-hits from everyone

That would be easier and safer to start up.  It also has the advantage that multiple hits from the same user can be combined into one transaction.  A lot of your hits will usually be from the same people.

The instant gratification way would be to pay a fixed amount for each near-hit immediately, and the operator takes the risk from randomness of having more or less near-hits before a block is found.

Either way, the user who submits the hit that solves the block should get an extra amount off the top, like 10 BTC.

New users wouldn't really even need the Bitcoin software.  They could download a miner, create an account on mtgox or mybitcoin, enter their deposit address into the miner and point it at anyone's pool server.  When the miner says it found something, a while later a few coins show up in their account.

Miner writers better make sure they never false-positive near-hits.  Users will depend on that to check if the pool operator is cheating them.  If the miner wrongly says it found something, users will look in their account, not find anything, and get mad at the pool operator.

[RHorning]

... if you happen to be requesting hashes where you've received a couple dozen reduced hashes and then the network has a huge spike in difficulty.  This is a potential liability issue so far as the "owner" of the hash farm server is concerned, as it would substantially impact the "profit margin"

Sorry, you misunderstand. The server operator only pays out to easy hashes that were submitted for a block that is won. There's no payout for easy hashes that were submitted for a block that someone else generates.

The difficulty can't change in the middle of a block. It always changes at a known time, and you always know exactly what the difficulty is, so there's no risk here.

I guess I do have a misunderstanding here, although there is another side effect in terms of hashes being made for blocks that aren't won.  Clearly that is a form of "proof of work" and at least represents effort being contributed to the pool and I guess I'm not convinced that "good faith efforts" to perform a hash ought to go unrewarded, but that clearly would be a part of the "rules" of the pool.  As long as everybody is agreed ahead of time and this point is clear, I don't see huge problems in that regard.

However, excluding potentially successful hashes would have the tendency of driving out weaker participants.  I also think it would introduce a significant bias statistically toward the server operator as the number of successful blocks "won" compared to successful partial hashes for the current block would be considerably less than the 1/40 ratio.  I envision a situation for pooling groups like this with more mundane clients where it would be common for only one person to get rewarded for a successful block hash using these rules.  Or to put it in another context, on average there would be no benefit at all for somebody to join into such a pool with such rules where they would simply be more effective generating coins with the network itself.

Again, I'd like to see some statistics of this in practice, as all I'm making here is a conjecture based upon my own understanding of statistics.

New users wouldn't really even need the Bitcoin software.  They could download a miner, create an account on mtgox or mybitcoin, enter their deposit address into the miner and point it at anyone's pool server.  When the miner says it found something, a while later a few coins show up in their account.

Miner writers better make sure they never false-positive near-hits.  Users will depend on that to check if the pool operator is cheating them.  If the miner wrongly says it found something, users will look in their account, not find anything, and get mad at the pool operator.

In terms of verifying a "near hit", it would be verified against current rules but for a reduced difficulty.  There is a statistical probability that such a near hit may even be a successful hash.   Some "client" of the coin farm who is consistently reporting "false positive" on a hit would be almost instantly verified as such and be considered a troll where certainly they would not be rewarded or even recognized, and it would be in the interest of the server operator to simply blacklist those clients altogether or at least dump the connection.

Still, I get what you are saying here Satoshi in terms of the miner writers not producing something which produces these false-positives.  This is something that ordinary economic theory would resolve so far as those mining clients are concerned.  Somebody who comes up with a software package which only contains the communication protocol but doesn't produce results would be quickly discarded by its users.

I love the idea here that somebody could install a miner on their computer without even installing the Bitcoins software itself.  This implies that perhaps you could even put a miner on a cell phone or some other device if you cared (but of course chewing up battery life as a consequence).  It does open up a whole bunch of potential ways to mine coins and expand the pool of potential CPUs engaged in this activity.

[slush]

Oh, thank you ribuck, I definitely catched all your ideas. I'm happy that transaction to pool operator is already included in hash data, so cheating is practically impossible in that way. Now I have to solve only situations with false positives.

I think I definitely have enough information to knock up pool software now. Give me a time, I'm working on ;-).

[slush]

Cooperative miner is almost ready. It is already working and pretty fast. Tested on 'testnet', my Intel Atom can manage clients >500Mhash/s without any significant load. There is just need for some web UI for users and a lot of code cleanup.

It is working with jgarzik's cpuminer, m0mchil's python miner and Diablo's java miner pretty well. But I found one problem in comparing targets. As I understand their codes well, they don't fully compare found hashes with target. So if I set difficulty pretty low, because I want many low-difficulty blocks for fair accounting, miners simply does not return correct hashes (hashes corresponding current target).

For example DiabloMiner. Even if I set target to 'ffff..ffff00', miner does not return hashes with less than eight zeros on the end. Am I wrong or miners really need patches to work correctly with low-difficulty targets?

When I set target with eight zeros on the end (so all tested miners work correctly with that target), computer with ~2mhash/s gives around 2 blocks per hour, which is IMHO too little for fair short-term distribution.

[jgarzik]

It is working with jgarzik's cpuminer, m0mchil's python miner and Diablo's java miner pretty well. But I found one problem in comparing targets. As I understand their codes well, they don't fully compare found hashes with target. So if I set difficulty pretty low, because I want many low-difficulty blocks for fair accounting, miners simply does not return correct hashes (hashes corresponding current target).

It's too slow to do a full 256-bit comparison for each hash generated.  Taking the lead from mainline bitcoin, miners tend to "cheat" and simply look for a certain number of zero bits in the area that, when byte-swapped, holds the high-order bits of a 256-bit number.  My cpuminer stops if the top 32 bits are all zero.  Mainline bitcoin stops scanning for hashes if 16 bits are zero:

Code:

        if (((unsigned short*)phash)[14] == 0)
            return nNonce;

Ideally, the miners would all do something like

Code:

    if (test for certain number of zero bits)
        if (perform full target comparison)
            // found a solution
            return nNonce;

but we programmers can be a lazy bunch sometimes.   

[lfm]

It's too slow to do a full 256-bit comparison for each hash generated.  Taking the lead from mainline bitcoin, miners tend to "cheat" and simply look for a certain number of zero bits in the area that, when byte-swapped, holds the high-order bits of a 256-bit number.  My cpuminer stops if the top 32 bits are all zero.  Mainline bitcoin stops scanning for hashes if 16 bits are zero:

The reason the mainline bitcoin only tests 15 bits is to support testnet where difficulty can be less than 1.00. 32 zero bits corresponds to difficulty 1.00.