Bitcoin Forum

The current system goes something like this:

1. Register for a pool
2. Plug username/pass into miner
3. Miner gets block template from pool, tries random nonces.
4. Miner finds share, sends it to pool. If the share turns out to be a valid block, pool distributes winnings.

But a far better way would be:

1. Register for a pool.
2. Get Bitcoin address for the pool.
3. You run miner on your own local bitcoind.
4. Miner calls getwork on your bitcoind, gets block template YOU create locally! However, it gets the difficulty and generation address from the pool (to allow share-based mining, and to make sure the pool gets paid.)
5. Miner tries random nonces.
6. Miner finds share! Sends it to pool. If the share turns out to be a valid block, pool distributes winnings.

Ta-da. Now, all block generation is done by miners, not by pools, as Satoshi intended. In other words, the pool has /no/ control over the content of blocks! But pools still get block/share based mining, as pools want.

Don't thank me, send me BTC.

I came.

Holy fuck.

NICE!

Interesting, but wouldn't that method allow for miners to withhold valid blocks from the pool?

As I understand it, that can't happen today because the miner doesn't have all of the information about the block. In this case, they would, no?

They can withhold them, but they don't get the coins in them if they do.

This.
You could under this situation publish your 50BTC blocks on a local daemon and then still reap the benefit of those publishing them to the pool.

I don't know much about the Crypto, but someone please explain what is wrong with this :D

Ok, I think I see...

The miner could substitute their own public key during the generation process, but the pool would not recognize those shares. If they find a valid block and substitute their own public key, it's no longer a valid block.

Yup!

great idea.
pool allocates a new address to each participant, and participant uses that address in the coinbase (so not possible to 'steal' coins).

The benefits i see:

1. there's no more getwork network overhead for the pool (therefore fewer resources needed to run a pool). all we send are completed diff1 (or greater) shares that the pool needs to verify.

2. no more pools being able to decide what transactions to include or not (see eligius and its policy of charging fees for all tx), or to try to create blockchain forks for double spending if they get too powerful (this hasn't happened yet, but it /could/, theoretically.)

The only thing needed is to write some code for sending the shares to the pool.

As i see it, this scheme does not introduce any downsides.  miners cannot 'steal' blocks, if they withhold a good block, they don't get the coins, since the coinbase address used is one belonging to pool owner.

Brilliant idea.

In the far future, these pools might not be competitive though due to the costs from every miner having to verify signatures. Maybe there is a solution for that?

Still, that's a ways away and for now it sounds like a great idea.

Well, the pool could also provide transactions to the miner; that still stops double-spending.

All work is done locally ...

1) If you're doing all the work local then why bother sending it to a pool?
2) Why should the pool trust the number of shares you worked on? Hack the bitcoin program so that your Pentium III reports the same shares processed as the guy with 5 GH/s.

For the same reasons people use pools in the first place (insurance).
You still send each share to the pool.

The pool should give you N addresses:

One for D=1 work, one for D=6 work, one for D=12 work. etc.

D=6 work pays 6 shares. Etc.  The reason for this is because your scheme uses a lot more bandwidth to transmit shares, but this is trivially corrected by increasing difficulty. But that would increase variance to unacceptable levels for slow miners.  By changing the address they pre-commit to a target difficulty and the shares will be credited accordingly.

The miner software can then bet setup so that it picks the diff that gets it close to 1 share per minute...which should end up being less bandwidth than currently used.

Also, while it would be possible to buffer shares while the pool was down and the pool could choose to pay for stales, I think thats actually a bad idea: it would allow you to get shares on network-offline hosts that aren't really contributing to the success of the pool... plus it would require more coding and storage for the shares.  Instead,  when the pool isn't reachable to accept shares you simply switch to using your own address for mining until it comes back.

Annnd... as we mentioned on IRC pools could still enforce sane behavior on miners (e.g. don't send 1MB blocks of crap transactions) by simply refusing to pay for shares that look like that. So the pool acts as a check on miner behavior, but it can't enforce secret rules since the miners will need to know them in order to conform.  This somewhat invalidates nanotube's #2 re-fee rules, but I think thats a good thing.  Pools don't get unilateral control but they get some influence and I think thats good.


Also from IRC, the logical place to put all this would be in bitcoind, not in a miner client. A simple modification to bitcoind would let you change the payout address... then you use normal miners against it.


(1) in order to get pooled payments, silly.

(2) Because you still submit 'shares' (though now with more data) in order to prove that you're working for the pool, same as now.

Brilliant.
Good point, probably right.
I also agree with this.

Yup, probably right.

If I'm understanding everything correctly, this is fantastic.

This won't be popular in the long-term, since running a full node will be expensive.

It has all the flaws of solo mining (except with the benefit of pooled mining, that you get a fairly steady payout.) That said, you can connect to some hosted bitcoind to do this.

Too many pool members would be randomly working on the very same shares ... no good.  That is why it is sent out the way it is.

What problem does it solve exactly?

If you're still communicating with a remote server and doing shared work and the server goes down, how is it any different than the current scheme? You won't be able to communicate with the server to submit work and all client will basically be orphaned.

Nope, different BTC addresses for each user.

The fact that a pool operator with > 50% of hashing power can double-spend.

No, they wouldn't. They'd all have different seeds randomly given to them by their own bitcoind.

Precisely ... and the miner must get the block from the pool.  The miner therefor is not building the block.  Basically, you can't solo mine, find the result and submit it to a pool.  That is the gist of what you seem to be proposing.

This is a really great idea!  I think it's a good step even considering the longer term costs of transaction signature verification and maintaining the full block chain.  In the future, miners could pick and choose how they want to operate...they could have their own TX policies, but they could also be given transactions by the pool or even by another third party.  They could opt to do transaction verification themselves, or purchase that service from someone else (who would get economies of scale by having many customers requesting verification of the same transactions).  They could maintain a full block chain themselves, or use the services of someone else that charges a fee for it.  And the pool itself could also bundle these services.  But what this does right now is make it so that pools can operate without forcing their TX acceptance policies on their members (although, I suppose the pool could still set some rules by rejecting blocks that violate their rules...but at least the miners have a larger say in the matter).

Noo.
The miner only gets the difficulty and the address from the pool.

read the OP.

I did, but the only reason you are part of the pool is to reduce variability and get a more constant payout [or with low hash rates, ever get a payout].  How is that handled if all shares are not submitted back to the pool?  Sorry, this just comes across as seriously flawed to me.  I like the idea, but it seems to be missing the key cog.

All shares are submitted back to the pool.

Could someone be kind enough to explain me (or point me at some page) to what is composed a share by, exactly?

Thanks, and sorry for still being a n00b...

You have found a valid block when the hash of the block header is less than the current target.

You have found a valid share when the hash of the block header is less than the maximum target.

The current difficulty is (something like) maximum target divided by current target.

I finally managed to understand the bit I was missing!

Thank you very much, BitterTea.

@Dusty: When you're mining, you're trying to find a 256-bit hash that is less than or equal to a certain value, the 'target'. The maximum target is hardcoded into the client as 0x00000000ffff0000000000000000000000000000000000000000000000000000, and the difficulty is actually maxTarget / currentTarget.

A 'share' is a hash that is less than or equal to maxTarget. It's not actually useful to the pool, except as proof that you're really working. They're also useful for calculations: at difficulty 244139.48158254, you should find that the pool generates blocks at an average rate of one per 244139.48158254 shares submitted.

Wise man once say - man who thinks everything ok, doesn't understand anything.

Man who stand on toilet, high on pot.

So far it looks sound. Good find.

One question: In the long term, how do you manage transaction inclusion rules?

Technically, miners only take the pool's address and build up the block header on their own, so they can include any transaction they want.

Pick one.

Currently, the miners just issue a getwork request, which includes the merkle hash of the transactions (among other things)...hence it is the pool that determines which transactions are included in a block.

That's fucking genius!


Doing both: Thanks!

Unlike now where pools can use any rules they want and their miners wouldn't easily know, this would REQUIRE the pool to publish their rules. The pool can then simply reject any shares that don't follow their rules.

A VERY nice side-effect of this publishing requirement is that bitcoin clients can directly fetch these rules from large mining pools and use the information to more accurately predict what fees are needed to get a certain transaction in a block within a certain timeframe if a majority of the network's hashing power is coming from these pools.

Two birds, one stone.

Please can you tell step by step.
I have already registered with a name & password in slush pool. I also created workers there & gave bitcoin address & mining & getting coins.
Now according to the method you saying, where & what changes i have to do exactly. If you give step by step, it will be useful. Coz, programming is headache to me.
In bitcoin.conf
what rpc user, password, ip or url , address have to make it work?

I really like the idea. This keeps the system more democratic, as it makes the pools less like aristocrats that collect their serfs work and submit it under their own name. The miners decide what goes in the blocks, not the pool owner.

If someone made a pool like this I'd certainly join it, and would accept paying a larger share for it than current pools.

You've misunderstood. This is a suggested protocol for future mining pools. It's not something you can do with existing pools.

damn, now has to wait for pool owners to change protocol.
But in theory good one.

Worse than that.  The mainline client needs to be changed, plus all of the pools, plus all of the miners.

Excellent idea, so I'm sure it'll happen.  But it will take a little while.

This is not new idea, I thought about that months before, too.

There is one huge problem - performance. It does not look so on first view, but client need transfer _complete block data_ to central server for _every share_, because pool have to validate that share is correct (by user intention or by bug). So sending short hash to server with filled nonce is NOT enough.

I calculated this and needed performance (for pool with tens or hundreds Ghash/s) would be enormous. Even worse - it is rising with more transactions in the block. So this scale even worse than standard centralized pool. In case that bitcoin will grow massively, this is show stopper.

Maaybe.
Other options:

Just send transaction IDs to the pool for verification, along with the header.

Better, but not good, as load is not driven by pool, but by miners. I see thousands of CPU miners on my pool even with current difficulty, which is - from economical point of view - nonsense. So how to solve problem where hundreds CPU miners can shut down your pool with sending 1diff blocks, 1MB in size each? :)

With rising difficulty (expect diff over milion soon!), one share will be amost worthless. By increasing basic difficulty, you can make it better, but will people accept minimum difficulty @ 1000? :)

Btw it's not only transfer problem, calculating complete block for every share is pretty hard, too. Forgot that pool can calculate tens of hundreds shares per second...

Basically I like the idea, but those are reasons why I leaved it long time ago.

If you set a minimum difficulty of 1000 for your pool, and they want to participate, that pretty much means they'll accept it.

If people really can't let go of CPU mining, they can run their own mini-pool that handles difficulty 1 blocks before sending those that meet the pool criteria up to the big pool.  Maybe meta-pools will pring up.

Just because you can't imagine how to scale things doesn't mean that no one can.

I see your point, I'm just extrapolating in the future. Right now the priority is this fix.

Is there some sort of a .conf file you can feed to bitcoind for inclusion rules?


Have a new function in the pool software, that stores the Merkle root + timestamp per account. On the miners side, each times you add a transaction or update the timestamp, you call that function, upload your new Merkle root and timestamp once. Then whenever you have a share to submit you only send in your nonces. Pool is waiting there with the header precalculated, waiting to add the nonce to hash it, so the server side you will be fine with CPU time. You'll need something like an extra 1 kb memory per account (doesn't have to be per worker since they getwork() from bitcoind).

When your miners hit a full difficulty solution, you upload the whole thing to the pool, transactions and all.

Goatpig, I see what you mean; not _so_ bad idea, but still worse from side of performance and scalability. I'll think about it little more.


Pool still need to know all transactions to validate single share, so those information must be known by pool in time of share submit.

So just send the TX ids.

Or assuming most people will integrate the totality of available transactions, send in the IDs of the ones that are not integrated, saves Bandwitdh and memory server side.

? if slush is correct and all this extra communication is needed ... wouldn't that simply take out the mega-HUGE groups and force everyone to create lots of individual and mini-groups (de-centralized) the way Bitcoin was meant to be?  isn't that a good thing?

YES. Frankly, the pools should do exactly 1 thing: provide insurance. Nothing else.

So you're saying that a small set of pool systems scales better than the idle CPUs of thousands of miners? Thats silly. As the txload rises miners can simply prioritize transactions based on their hash distance from some random value, allowing TX validation to scale far beyond what the pool could support.

Today the responses to take about 181 bytes on the wire.  Blocks are frequently about 4k, so at the moment difficulty would need to be 22 to send the whole block and use the same amount of traffic.  If it were compressed by only sending the TX ids, it would be 354 bytes/share for 10tx shares, or less then double now.

Someday in the future when blocks are 1MB (the largest size clients will accept today) the 'compressed' size will be 128032 bytes/share. Share difficulty would need to be ~750 to get to the _same_ traffic levels we have now.

This could all be further reduced by miners only sending incremental updates. So basically in that case it would only take resending each TX, along with one extra per per new block (~6/hour) to setup the root. Done this way it should do no more than 2x the current bandwidth, though it would take more software to track the incremental work per miner.  

But even ignoring all the things that can be done to make it more efficient: at current bulk internet transit prices ($2/mbit/sec/month) full 1MB shares would each cost the pool $0.0000064 each.

Assuming 2MH/J, $0.05, and an exchange rate of $6/BTC  GPU mining won't be power profitable past difficulty 10,000,000, even for people with cheap (but not stolen) power, assuming the current exchange rates.  At a share difficulty of only 12 this would be bandwidth costs of only about $5.33 block solved while sending full 1MB shares without any efficiency measures.  If you can't figure out how to make that work then I'll welcome your more efficient replacements.

(the formula for breakeven profitability is
diff = (719989013671875*exc*mhj)/(17179869184*kwh)
where diff is difficulty, exc is the exchange rate in $/BTC, MHJ is the number of MH per joule, and kwh is $/KWH)

As I write this deepbit is down and the network has gone 30 minutes without confirming a transaction.  This is nuts. I don't think the bitcoin community should continue to tolerate the reliability problems created by large pools.  You're free not to participate in an operating practice, but the network is also free to ignore blocks mined by pools which actively sabotage the stability and security of the network.

so .. ?? deepbit goes down for 30 minutes and the network dies ... and you all want this  ???? ... 1 HUGE mining groups that dictates ....

I'd prefer lots of small ones .. {no single point of failure}  ... if Bitcoin continues with big groups .. I don't see any reason people will want to use it in the future if it is to easy to kill it by crashing one single group.

just my opinion .. but you can clearly see its death coming from dictatorship.

It's quite clear that the people in the deepbit pool do not care about the health of the network. Most probably don't understand it. Unfortunately, there is no cure for stupidity. Many of us have been calling for miners to balance the pools, but our arguments fall on deaf ears.

Apparently you misunderstand the point of this thread.

Ok .. never mind .. I'm outy  :)


EDIT: /??? just asking but  ..... if one MEGA goes down and the network can not adapt immediately .. then you get delays and a back log of transactions ... so the 'dictatorship' I was referring to was the MEGA that controls the network ... if it glitches, goes down, etc .. and everyone hurts ... ??? so .. am I still wrong about forcing the break up of MEGAs and creating smaller groups being better????

No! You are wrong because you suggested that we want large pools.


This thread is an example of a way to reduce the control a large pool has over the network. The people posting in this thread want many small pools, the same as you. This is why I said you misunderstand the point of this thread.

Ah .. ok ... thank you .. I thought some of you wanted big ones !! made no sense to me ..
much better .. :) thank you.

Well, some do want big pools, otherwise they wouldn't join deepbit. They like low variance (not realizing the highest fee costs them more in the long run). And I suspect many of them come from F@H, where you want to be big. I also suspect many of them only care about making easy money, not the health of the Bitcoin network (which is also stupid because the Bitcoin network is what is allowing them to earn easy money).  >:(

So when does the new 'client' come out with the additions/modifications?   and please forgive this .. but what's F@H???

Whenever someone writes the code and tests it. F@H is Folding at Home, a distributed computing project. The various groups that "fold" compete to see who can do the most computational work, and the competition is good because it's for a good cause. But that competition doesn't translate well to Bitcoin.

While I like the idea of the first post, it seems like the only difference between this type of pool-mining setup and the existing ones is that the BTC generated at the user and then the pool decides how it's divided up, is that correct?

is anyone communicating with pool managers to depoy such changes ?

No.

The pool picks the share difficulty and creates the coinbase TX.

The user does the rest.

mmm unless something changed in the past 3 pages :), my understanding is quite the contrary.

well, the pool is still in charge of user registration, and assigning addresses to be used in the coinbase by the clients. the creation of coinbase tx, the monitoring of the network and inclusion of transactions into the block, etc, is done client-side.

but then, the distribution of the bounty is based on number of shares submitted by the clients.

I figured doing the coinbase TX on server-side'd make it easier to check.

This could create strategic problems for pool operators. With the current "plan", it is impossible for the often-foreseen future where merchants can pay a flat monthly "priority fee" to pools and such. So any system which allows the end-miner to control block creation would probably need some kind of way for the pool to say "must include txids <x>, <y>, <z>" as well...

The pool can reject shares not containing those TXs. However, it /must/ publicize which ones it requires, which is a good thing IMHO.

What about a .conf file capability added to bitcoind that allows to script TX inclusion rules. It doesn't even have to be added to bitcoind. Let's say you add a class that the miner calls, which will include TX according to the .conf file rule, then call the core bitcoin API to create the block header. Then a pool operator can propagate these to his clients, assuming most users won't just be douches and ignore it, since it is directly related to their profits in the long term.

Also, doesn't the network already order the TX by fee? The system described here implies the miners have to inform the pool of the TX they are including, by ID to save bandwidth obviously. The ID linking means the miners are referring to the network propagated list of pending TX, which, if they are ordered by fee, will smooth this whole system a lot.

The pool can check that the transactions fit the inclusion rules, and reject shares where they don't.

Sure that too.

I tried this a long time ago....it doesn't work.  If you getwork from your local bitcoind, you can't send it to anyone else's bitcoind, and visa-versa.

Sorry to burst your bubble.

Oh yes, no need to radically modify the mining platform, let's just use vanilla bitcoind!

:/

maybe it is simple enough - have the block be sent to your local bitcoind - and your local bitcoind will then broadcast your block to everyone - including the pool node.

of course, everyone but the pool node will reject your difficulty-1 blocks, but the pool node will see them and give you credit for them.

so it seems that modifications necessary to bitcoind are actually quite minimal. (1) configurability to let it set getwork difficulty (2) configurability which nodes to send blocks to (ideally, to prevent network spam, it can be configured to send diff-1 blocks only to the pool's node, not everyone else.)

Don't mod bitcoind. Build in a class that your miner will getwork() from. Have some simple pool type detection code. If the pool is classic, the class will bypass its main functions and just deal directly with the pool. If the pool is modded, the class will request the getwork() off of localhost bitcoind with the pool's address, and then upload whatever data it needs to to the pool.

This way both bitcoind and number crunching side of miners will remain untouched. No need for several versions of the same miner either.

You still need a way to get work at a specific difficulty and for a specific public key

You'd be feeding that work to yourself. The public key and difficulty are broadcasted by the pool.

But currently there is no way to tell bitcoind to give you work of a specific difficulty and for a specific public key.

This is why it would be better to come up with a class independent of bitcoind, that will create the block header and provide the work on its own. This prevents modifications to the core elements while it allows to:

1) Skip bitcoind entirely for pool miners as it is done right now. Keeping things simple is always a plus. It also ensures miners aren't nodes in the network, as it is now.
2) Simply have the miners getwork() from that class, let's say as a COM object. Then as said before, the class can determine pool version and outright bypass its main functions if it is dealing with a classic pool, or provide the work as a modded pool would need it to. This allows miner developers to easily implement the change, while keeping it independent from further improvement related to pure number crunching.

then who collects the transactions?

The pool, who else? The header is created with the pool's address.

/sigh/
the whole point of this is to allow TX selection (and headers to an extent) by individual miners.

Here are the things that need to be done to get this idea working in the real world:

1) The standard client must be modified to accept an address for the generation transaction.
2) Mining clients (and/or the flex proxy) must be modified to send another copy of each found block (share) to a second server.
3) New pool software (or a mod of an existing pool) to accept these block copies.

1) The pool posts an address to be used for generation.  This could be on the website, or through RPC.
2) A node takes that address, and forms a block using it as the generation address.
3) The miner gets that work from the local node and starts working.
4) The miner gets a candidate, and sends it to the local node, plus another copy to the pool server.
5) The pool verifies that the generation address is the one published, and assigns credit to the miner.
6) When the block is found, it goes out through the local node as usual, and the pool notices it, and pays the miners.

I thought you were referring to the 50BTC reward.

The independent class makes the transaction list upload to the pool simpler since you have dedicated code to maintain and access this information. Naturally the pool needs to know which transactions the users have included in their block in order to verify the hash. The TX ID communication thing will work fine for this. The pool needs extra memory to maintain this transaction list, but it can be reduced to a certain extent:

1) only maintaining TX per ID, and per non included TX.
2) the class can send a "worker" group ID to the pool to put all the workers running from the same COM object (same machine or even same IP if you link your miners to one IP within your own LAN) under the same list in order to maintain a single TX list per machine and/or IP.

Lastly using IDs ensures the TX have been broadcasted to the network before hand. No shady business that way.

It sadly is going to be necessary to modify the bitcoin client to return enough of the block under construction - right now, there is no way for a pool to verify that the pool's payment address is being specified unless the necessary information from the block is provided rather than just midpoint hash, etc.

Proposal: https://docs.google.com/document/d/1ciKH3M8WYS49ywz08beXtvpCm2wVGdzU7waKwcn_uaU/edit?hl=en_US&authkey=CJTqyOMF#

Why they can't now?

Any progress on this? Seems to be one of the most important outstanding issues.

I've got other stuff on the works right now, but if by the time I've got some time no one has implemented it, I'll give it a try.

I am waiting for permission from my employer before I proceed. Nearly have it, but bureaucracy bureaucracy.

I wish I had the skills to do this, myself.


Just curious, but do you mind sharing who your employer is?

you need your employer to work on a private project? wtf?

First comment and a new user to bitcoins.. I'm very intrigued by this pool solution, hope it gets solved eventually.

I think she works for Google, and is planning on working on this project during her 20% time.

Correct. I actually was planning to do it outside work originally, but if Google is trying to get copyright on it anyways, I may as well use 20% time rather than my non-work time :)

http://www.leginfo.ca.gov/cgi-bin/displaycode?section=lab&group=02001-03000&file=2870-2872

Some things to consider:

Transaction fees would also go to the pool, right?

How about the "I find a solution but don't post it" attack, aka. "withholding winning shares"? As miners might have to be adapted for this pooling scheme anyways, please also implement "oblivious shares"!

Can the pool require it's miners to always include some special transactions (for free), like payouts?

Sounds awesome, if anyone gets this going I am willing to test.

Google...... Perhaps deepbit with 75% controll would be better. Would have to see details but by default I do not trust them.

If I'm allowed to develop, source will be open, you'll be welcome to inspect it for yourself, I'll accept patches.

Lastly, I don't plan to be the only distributor. Anyone can start a distributor simply by publicizing a bitcoin address for their distributor and convincing people to start using their new distributor address; they'd obviously be responsible for actually running the distributor after each found block to disburse funds. It just would require having enough trust such that people would be willing to point their miners at your distributor.

When I get this off the ground as the first distributor, I plan to escrow 50 btc with a well known community member as a show of good faith should I fail to pay out.

Correct.

Doing so just hurts you, and someone else will come up with a winning share eventually. This is a generic problem with pools and I can't fix that today. Additionally, oblivious shares are tricky to get right and require centralized trust, which is something this system is intended to *avoid*.

Yes. There could be published policies about only accepting proposed shares/solutions that contain specific transactions; participants in the DHT could refuse to grant credit for shares without those transactions, and the distributor's double-check could verify this.

Not if I want for example to manipulate the pool down.

Imagine Tycho directing his entire pool (or even just parts of it) to a pool like this, creating insane amounts of (valid) shares but just filtering out ones that are >= current difficulty to artificially lower your income.

He would even gain BTC on payouts(!), meaning he'll save money for servers while making sure your pool gets less attactive (as payout is less than in other pools). It then only depends if it would be chaper to buy the remaining BTC (or pay them out of his own pocket/fees) or not to sustain a "leeching attack" like this. (nothing against Tycho here, he just stands for $random_huge_pool_operator - and I don't suspect him in any way to do something like that!)

In the end you'd pay a pool with hashrate X but get BTC for a pool with hashrate X - Leechers. As this attack is not really statistically detectable, people would then start to accuse you of stealing/manipulating stats etc.

In the long run it might even pay off to become the biggest fish in the pond (there is no logical reason atm. for example why you should mine exclusively at deepbit, still people are doing it  - nearly 50% of people!) to sacrifice a little income if you can eliminate competitors.
This is also why I think this attack is more serious than pool hopping - the latter is a statistical attack and can be countered with rating shares. This one can only be countered with oblivious shares, and as you said they require rewrites and might be likely to go wrong.

Yes, I acknowledge this is a threat. Seeing if anyone has come up with a good solution.

This is actually not a problem. The distributed pool design is explicitly designed to be independently verifiable by as many people as wish to run distributors or mock-distributors and who can verify or sign off on the payout before it's sent out. Thus, they can examine all the proofs of work.

Okay maybe this is too simple but why not just make a the largest mining pool ineligable for generating the next block with an exception for holding say <%20 or something of the network.  It'd come down to shares & odds against mining profitability to figure a viable cap to avoid the skip but it'd also prevent the finny weakness to by never allowing a single miner or pool to gain majority and keep it.

It would also force more and smaller pools.  The figure could even vary based on mining population (in terms of power, not individual participants) to allow larger nodes to be formed when populations go higher.

Am I missing something or could the finny attackers just pool hop between their own pools?  Seems to me there might be something here but I admittedly don't know enough about what I'm really talking about yet.  Maybe someone who does could turn it into something better. :P

I don't understand all of the nuance of the hashing system, but I think I have a possible solution for the high bandwidth problem. As I understand it, currently the pool gives you the block information once, you add nonce and hash, and when you get a share you send the nonce:

Current System (low bandwidth)

1. Pool gives block to you once
2. You add nonce and hash
3. If share, send nonce to pool for verification
4. Repeat at 2, when valid block found start back at 1

This results in low enough bandwidth to be a sustainable system. Now people have said that the new system would require to much data to be sent because it would look like this:

New system (high bandwidth)

1. You generate your own block
2. You add nonce and hash
3. If share, send whole block for verification
4. Repeat at 2, when valid block found start back at 1

This results in bandwidth too high because you send the entire block with each share. But this seems thoroughly unnecessary to me. Why not as below:

New system (lower bandwidth)

1. You generate your own block
2. Send block to pool once
3. Add nonce and hash
4. If share, send nonce to pool for verification
5. Repeat at 3, when valid block found start back at 1

This would be almost the exact same bandwidth as the old system. In the old system the pool sent the block data to each member once, in the new system each member sends the block data to the pool once. The pool would have to do a little more work storing the blocks that each member is working on, but I don't think it is really a show stopper.

Is there a reason why this doesn't work?

That means every bitcoin client would need a lot more connections to the network and/or have built in long polling to make sure they are working on the current block.

Depending on how many workers/clients are connected, it could mean a lot of storage and processing.

This is incorrect. The pool sends the partially completed hash, you never see the block you are working on. You only add the nonce and hash.

This is much higher bandwidth, because the central pool master now has to process the entire block from each client every time the current block in progress changes instead of handing out precomputed data. This results in a DoS as all the clients supporting LP hit it at the exact same time with their new draft blocks.

Oh, ok I see now. I did not realize that the pool gave you partially precomputed data. Now it all makes much more sense. Could the pool member send the same sort of precomputed data back to the pool? This way the pool doesn't have to process every block itself? Then when a proper solution for the difficulty is found the pool could hash the entire block for verification. Is there some reason the pool would need to "pre-hash" every block from every user itself? Can the member not be trusted to send a valid precomputed value for the block?

Most of the entire block must be sent; otherwise, you could mine for yourself or for another pool and submit block hashes as 'proof of work' without actually working on solutions that pay the pool's payment address.

Edit: oh, I see, you could verify over time rather than all at once by just caching the values and computing later. Sure, that's a possibility, but this still doesn't address the 'single point of failure for recording work' problem that my proposal addresses.

So, if I understand this right, and I'm not sure I do, the main problem is that there is no way to insure that the "generation address" is in fact the pool's address unless the pool hashes the block itself? So if the pool doesn't generate the hash for each block, you could potentially trick the pool into thinking that you are working on pool hashes when you aren't. When you actually hit a hash at the right difficulty, you claim it for yourself because you were actually using your address. At the same time you could submit the same work to a different pool that also can't confirm that you are using the correct address. Is this why you can't trust a pool member to provide the precomputed block data?

Correct.

So it seems to be that the issue is that the transaction data and the generation address get hashed, and then the nonce is added and it is all hashed again like this:

{generation address}+{transaction data} -> {block hash}

{block hash}+{nonce} -> {final hash}

The pool currently gives you the block hash, you return the nonce of a share and the pool only has to check the final hash. I suppose what we need to do is change the entire system (everything) so that the pool can verify the generation address. Something like this.

{transaction data} -> {transaction hash}

{generation address}+{transaction hash}+{nonce} -> {final hash}

This way a pool member can send its own transaction hash once and the nonce of each share, but the pool still gets to verify the generation address. Is this the upshot of your proposition?

This change is very major and not in scope. What I am proposing to do is verify the entire block upon each share, but to have a distributed network rather than a single master verifying receipt of share.

I'm confused, if shares aren't signed how do you collect?

You send your shares to several other peers who verify and commit to their distributed hash tables. When it's time to pay out, the distributor retrieves your shares from the DHT, verifies them, and generates the proportional share to give you.

I had a different solution in mind, but maybe it might be merged into this one as well.

cuddlefish's solution requires running bitcoind on the miner side. What if miners who don't want to run it could opt to get work from trusted work streams?

• Pool keeps a list of authorized channels that it gets work from.
• Each channel supplies the server with signed block data, address set to one of pool's addresses.
• Miner picks as many of these channels as it likes, pool sends work from any one of them at random, signed by the channel owner.

This way, attacks would require everyone to choose channel(s) controlled by a single entity (unfeasible I think), traffic is low and miners wouldn't have to run bitcoind.

What do you think?

(As a further step, we could separate work servers from pools.)

Under my proposal, you are welcome to use someone else's local pool manager instead of your own. I'm imagining most people but not all will run a local pool manager, and only a small subset of people will run a DHT node, but there will be enough to keep the system resilient.