Bitcoin Forum
July 16, 2018, 05:17:31 PM *
News: Latest stable version of Bitcoin Core: 0.16.1  [Torrent]. (New!)
 
   Home   Help Search Donate Login Register  
Pages: 1 2 3 4 5 6 7 8 9 10 11 [All]
  Print  
Author Topic: Decentralized mining protocol standard: getblocktemplate (ASIC ready!)  (Read 31853 times)
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
September 12, 2012, 12:06:49 AM
 #1

Since Eleuthria and slush have both recently announced their own developed-behind-closed-doors mining protocols, I felt it appropriate to make a thread here for end-user discussion of the "getblocktemplate" standard which they intend to compete with. As a poolserver author, a pool operator, a miner author, and a contributor to the reference client, my life is not improved by having yet another protocol to support, and what's bad for the authors of the infrastructure tools is also bad for all their users.

Getblocktemplate Features

  • Decentralized: the miners decide what they will mine, and make their own blocks - no more blindly giving up mining control to pool operators
  • ASIC-ready: miners can produce as much work as they need with a single request
  • Usable yesterday: Eligius has supported it since February, and more pools real soon
  • Solo mining: just point your miner to bitcoind 0.7 and mine by yourself (no bitcoind load issues)
  • Simple: the basic implementation requires only minimal block building
  • Extensible: extensions are available today for miners who wish to take more control over their miners, and future improvements can easily be added later

Documentation

See the new Bitcoin Wiki page for getblocktemplate, with lots of useful resources including end user and developer documentation.

Technical specifications:
Supported by... (post or PM me to get yours added)

Poolservers (providing getblocktemplate to miners):
Mining software:
Libraries for mining software:
Mining pools:
History/Background

Getblocktemplate's origins trace back to forrestv's getmemorypool JSON-RPC method for bitcoind. He created it so that his pool (P2Pool) could piggy-back on bitcoind so as to avoid harming the Bitcoin network (up to this point, it mined only empty blocks that never confirmed transactions). Having been fighting with scalability problems in pushpool/bitcoind for months on my pool (Eligius), I set out to implement a fast pool server using getmemorypool to do its own block production (this became Eloipool, this first open source makes-its-own-blocks poolserver). Other poolservers also implemented block creation using getmemorypool over the months following.

At about the same time, interest in decentralizing pooled mining became a hot topic. While BitPenny had initially released its own decentralized mining proxy months prior, P2Pool's implementation became rapidly popular. Anyone involved in Bitcoin mining protocols could see the need to move control of block creation back into the hands of the miners. Unfortunately, both BitPenny and P2Pool had used very pool-specific proprietary protocols to implement their decentralization. On the other hand, getmemorypool was almost a perfect fit for the task - it just lacked support for the now-ubiquitous pooled mining extensions that had developed around the getwork protocol over time.

In February, I implemented and deployed a first draft of getmemorypool mining support in Eloipool (and on Eligius) along with a proof-of-concept getwork proxy (now known as gmp-proxy), adding revisions as needed to function as a general-purpose decentralized mining protocol. After I had confirmed it was working, I documented and proposed it on the Bitcoin development mailing list for review on February 28th, where discussion began on what was missing and what needed to be changed or clarified. During the following few months, a number of others, both developers and testers, provided constructive criticism and suggestions, which were integrated into the standard. I also actively encouraged participation in the development of the standard among pool operators and poolserver authors, especially as it became necessary to move forward into the ASIC "mining generation". Eventually, it was decided it would be best to rename it to the more appropriate "getblocktemplate" name and drop backward compatibility with getmemorypool for simplicity. The standard was split into two pieces and the technical specification can be found in BIP 22 and BIP 23.

Support for getblocktemplate mining was announced for BFGMiner a week ago, and is currently functional (implementing the basics is quite easy) though not entirely optimized yet. A number of other software developers and pools have told me they intend to add getblocktemplate support soon.

(side note: all of this occurred before Eleuthria or slush made their protocols public, and neither of them chose to participate in the open development of the getblocktemplate standard)

Contribute

All that being said, there is always room for improvement. Reception of the proprietary proposals strongly suggests getblocktemplate is in desperate need of good miner-side example documentation. If you have the time to help bring getblocktemplate to everyone sooner in any way, please get in touch Smiley

Thanks to... (in no particular order)
  • Forrest Voight
  • Pieter Wuille
  • Jeff Garzik
  • Gregory Maxwell
  • Geir Harald Hansen
  • Gavin Andresen
  • Stefan Thomas
  • Michael Grønager
  • ...anyone else I forgot to mention

1531761451
Hero Member
*
Offline Offline

Posts: 1531761451

View Profile Personal Message (Offline)

Ignore
1531761451
Reply with quote  #2

1531761451
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1531761451
Hero Member
*
Offline Offline

Posts: 1531761451

View Profile Personal Message (Offline)

Ignore
1531761451
Reply with quote  #2

1531761451
Report to moderator
1531761451
Hero Member
*
Offline Offline

Posts: 1531761451

View Profile Personal Message (Offline)

Ignore
1531761451
Reply with quote  #2

1531761451
Report to moderator
gmaxwell
Moderator
Legendary
*
Offline Offline

Activity: 2450
Merit: 1201



View Profile
September 12, 2012, 12:12:14 AM
 #2

I haven't yet read about about these other protocols, but I'm really disappointed to see more protocol proliferation. Getblocktemplate is well established, peer reviewed, and is good for bitcoin's decentralized security. I hope GBT gains whatever features its missing so it can subsume these things.  Having yet another protocol to cope with is a bummer.

Bitcoin will not be compromised
Graet
VIP
Legendary
*
Offline Offline

Activity: 980
Merit: 1000



View Profile WWW
September 12, 2012, 12:35:28 AM
 #3

I haven't yet read about about these other protocols, but I'm really disappointed to see more protocol proliferation. Getblocktemplate is well established, peer reviewed, and is good for bitcoin's decentralized security. I hope GBT gains whatever features its missing so it can subsume these things.  Having yet another protocol to cope with is a bummer.
It shows me interest from varied developers is strong - is this not good for Bitcoin?
what is wrong with proposing a new way for miners to communicate with pools, it isn't like it is changing a Bitcoin protocol it is a mining change.
there is no single pool software everyone uses, there is no single mining software everyone uses - I for one am interested at looking at options.

Thanks to all the developers taking interest in this topic and offering some options to consider Smiley

https://bitcointalk.org/index.php?topic=108533.0

| Ozcoin Pooled Mining Pty Ltd https://ozcoin.net Double Geometric Reward System https://lc.ozcoin.net for Litecoin mining DGM| https://crowncloud.net VPS and Dedicated Servers for the BTC community
eleuthria
Legendary
*
Offline Offline

Activity: 1750
Merit: 1001



View Profile
September 12, 2012, 12:42:11 AM
 #4

I started my development privately because I wasn't sure it was ever going to be completed.  I did know one thing.  The problem that slush and I were both addressing separately (though we wound up doing almost the exact same thing) was that POOLS will not be able to cope with ever increasing speeds, because pooled mining is using the standard getwork protocol that is used in bitcoin.

Our protocols were designed to be specific for pooled mining.  The primary Bitcoin project does not need to be polluted with new protocols and RPC commands for mining related purposes.  Bitcoin is not about mining, and developer time should be spent on more important features that matter.

RIP BTC Guild, April 2011 - June 2015
slush
Legendary
*
Offline Offline

Activity: 1372
Merit: 1019



View Profile WWW
September 12, 2012, 12:46:49 AM
 #5

Well, I don't want to start flamewar and personal fights, really. I did the Stratum just because of my requirements, not because I wanted to compete with anybody. Let me explain it:

Actually I see the difference between Stratum and GMP as Electrum client and Satoshi client. You can achieve similar results with both of them, but every of such solution has pros and cons.

Two days ago I summarized my reasons for Stratum here: http://mining.bitcoin.cz/stratum-mining

I was comparing it mostly with the getwork, because it's the only used protocol by real miners now (voting by the hashrate). Maybe I should write up longer comparsion between Stratum and GMP. Back to the topic.

THere's really no need to have one super-protocol which fits all needs. By the way, I'm using getblocktemplate in the Stratum implementation myself and I consider it as a great way how to relocate pool logic outside the bitcoind. GBT is surely very complete protocol and you can do anything what you want, *if* you know how. I see GBT as very complex and very hard to implement full specs without breaking anything. Luke will say the oposite, but GBT has simply unnecesary compexity for the environment of pooled mining. You know, many optional features etc. But pooled miners don't need that. Period.

In the oposite we proposed complete stack for pooled mining, which *is* very lightweight and straightforward to implement. Simply said, there's not a *single* optional feature, which is possibly the biggest benefit of whole concept. Miners don't want tons of options, they need simple and clear algorithm how to talk with the server. It includes also exact specification of used transports and mechanisms for real-time broadcasting new jobs to the client. Current BIPs don't care about this anyhow, because they don't need to. Both are just for different use cases.

Luke told me yesterday, that GBT is not bounded to HTTP, that it can be implement with any transport. By mistake he expressed the exact reason why it isn't suitable for pooled mining. It gives miners/pool ops degree of freedom which don't help anything, just mess the source code of miners and pools, because interpretation of the current protocol "as is" is ambiguous and cannot be used by some additonal specification about transports etc.

By the way, pool servers must be super-optimalized for handle high loads. Current Stratum protocol implementing coinbase templates is definitely the most optimized solution for given purpose and I doubt that any getblocktemplate pool would be more optimized that this.

Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
September 12, 2012, 01:22:56 AM
 #6

what is wrong with proposing a new way for miners to communicate with pools, it isn't like it is changing a Bitcoin protocol it is a mining change.
there is no single pool software everyone uses, there is no single mining software everyone uses - I for one am interested at looking at options.
The problem is more about when people don't work together with others. Everyone was (and is) welcome to contribute to the development of GBT, so why quietly reinvent an incompatible protocol on the side? GBT vs StratumMP is like W3c HTML/CSS vs Microsoft Internet Explorer "quirks" HTML/CSS.

The problem that slush and I were both addressing separately (though we wound up doing almost the exact same thing) was that POOLS will not be able to cope with ever increasing speeds, because pooled mining is using the standard getwork protocol that is used in bitcoin.

Our protocols were designed to be specific for pooled mining.  The primary Bitcoin project does not need to be polluted with new protocols and RPC commands for mining related purposes.  Bitcoin is not about mining, and developer time should be spent on more important features that matter.
GBT is primarily a pooled mining protocol, and does address the problem you mention perfectly, while also addressing other (arguably) more serious problems (centralized pool control).

I did the Stratum just because of my requirements, not because I wanted to compete with anybody. Let me explain it:

Actually I see the difference between Stratum and GMP as Electrum client and Satoshi client. You can achieve similar results with both of them, but every of such solution has pros and cons.
This analogy doesn't really make sense. Electrum and Satoshi implement things very differently. The only thing StratumMP really does differently from GBT is the protocol/communication syntax.

THere's really no need to have one super-protocol which fits all needs. ... I see GBT as very complex and very hard to implement full specs without breaking anything. Luke will say the oposite, but GBT has simply unnecesary compexity for the environment of pooled mining. You know, many optional features etc. But pooled miners don't need that. Period.
Pooled miners already need those features enough that they've been hacked on top of getwork for a long time. Probably no miner or pool implements full specs for all the getwork extensions, and that's fine for GBT as well. The important thing is that in cases where those features are needed, there is a consistent protocol for it rather than every pool/miner doing the same thing 5 different ways.

slush
Legendary
*
Offline Offline

Activity: 1372
Merit: 1019



View Profile WWW
September 12, 2012, 01:43:09 AM
 #7

The problem is more about when people don't work together with others. Everyone was (and is) welcome to contribute to the development of GBT

It's the same like designing the universal car. We should have just one model which will fit small city car, family car, a van and the crane. We definitely can design one machine which will fit all these needs just by mounting some special equipment over it. But we just don't want to, because it will lead to overcomplicated monster with many exceptions in the *real* implementation.

Look around you at this time. Miner software cannot follow even such dumb protocol as current getwork is. I don't want to show you my pool source, because I'm shy for tons of hacks involved because of crappy miner support. This is real-world lesson and we should learn from it. Please don't repeat same design mistakes with the full featured protocol which is ten-times complicated than getwork.

Quote
The only thing StratumMP really does differently from GBT is the protocol/communication syntax.

Then please show me exact payload done by GBT and providing exactly the same behaviour like Stratum does in the documentation example. And please don't use submitting whole coinbase. It's impossible to do full validation of coinbase for every share.

Quote
Pooled miners already need those features

Which features they *really* need from GBT except?

Quote
Probably no miner or pool implements full specs for all the getwork extensions, and that's fine for GBT as well. The important thing is that in cases where those features are needed, there is a consistent protocol for it rather than every pool/miner doing the same thing 5 different ways.

For the god sake, how you can manage miner implementation if every pool operator will have full freedom in implement any of BIP option?

Graet
VIP
Legendary
*
Offline Offline

Activity: 980
Merit: 1000



View Profile WWW
September 12, 2012, 02:02:43 AM
 #8

what is wrong with proposing a new way for miners to communicate with pools, it isn't like it is changing a Bitcoin protocol it is a mining change.
there is no single pool software everyone uses, there is no single mining software everyone uses - I for one am interested at looking at options.
The problem is more about when people don't work together with others.
Indeed a lot of people have had issues working with you before this, the history is spread through this forum :/ the alternative proposals were so close together the 2 developers involved decided to work together - not sure of your point here
Everyone was (and is) welcome to contribute to the development of GBT
but no-one is required to? right?,
so why quietly reinvent an incompatible protocol on the side? GBT vs StratumMP is like W3c HTML/CSS vs Microsoft Internet Explorer "quirks" HTML/CSS.


Both of these guys announced in their pool threads that they were working on ideas to cope with upcoming changes to mining hardware and most pools have said they are looking at options, either quietly or loudly....

I'm probably wrong again but looking around Bitcoin there is no monopoly on anything, there are even alternatives to the satoshi client
I think a lot of Bitcoin's growth has come from the different ideas that have been developed by a lot of people both on the main dev team and outside by other interested people, both foss and closed source

I welcome choices Smiley


| Ozcoin Pooled Mining Pty Ltd https://ozcoin.net Double Geometric Reward System https://lc.ozcoin.net for Litecoin mining DGM| https://crowncloud.net VPS and Dedicated Servers for the BTC community
P_Shep
Legendary
*
Offline Offline

Activity: 1176
Merit: 1000


View Profile
September 12, 2012, 02:05:13 AM
 #9



I welcome choices Smiley



And may the strongest survive Smiley
zvs
Legendary
*
Offline Offline

Activity: 1596
Merit: 1000


House Nogleg


View Profile WWW
September 12, 2012, 01:33:42 PM
 #10

For the god sake
ahh, reminds me of Adam's Apples subtitles, 'for the sake of fuck', one of my favorites

ok, carry on

optimator
Sr. Member
****
Offline Offline

Activity: 350
Merit: 250



View Profile WWW
September 12, 2012, 10:47:27 PM
 #11

miners decide what they will mine, and make their own blocks - no more blindly giving up mining control to pool operators

I thought one of the benefits of a pool was to determine the efficiency of including transactions in a block. It is the pool that could decide whether to include or exclude a transaction based on the transaction fees.

Doesn't having the miner take responsibility for the transactions undermine some of the benefit of the pool?

If the miner is building the transaction why doesn't the miner submit a block when his hash is less than the difficulty?


Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
September 13, 2012, 12:45:35 AM
 #12

miners decide what they will mine, and make their own blocks - no more blindly giving up mining control to pool operators
I thought one of the benefits of a pool was to determine the efficiency of including transactions in a block. It is the pool that could decide whether to include or exclude a transaction based on the transaction fees.

Doesn't having the miner take responsibility for the transactions undermine some of the benefit of the pool?
No, the benefit of pooled mining is mostly cooperation/economics: you're saying "if you guys split the block with me if you find it, I'll split it with you if I do". Centralized control over transaction acceptance is one of the flaws of the original pool design, as it compromises Bitcoin's security if any one person has too much influence. On the other hand, one benefit of pools is that there is an entity to arrange a bulk fee contract with for transactions; GBT preserves this ability by allowing pools to flag those transactions as "required", while not weakening the control of the individual miner who can still make an informed decision not to mine for the pool (with centralized mining, the miner is in the dark about what exactly he is working on).

If the miner is building the transaction why doesn't the miner submit a block when his hash is less than the difficulty?
I don't understand this question.

optimator
Sr. Member
****
Offline Offline

Activity: 350
Merit: 250



View Profile WWW
September 13, 2012, 01:57:01 AM
 #13


If the miner is building the transaction why doesn't the miner submit a block when his hash is less than the difficulty?
I don't understand this question.

Thanks - I missed the part about flagging transactions. Why can't a miner just submit the block header and the transactions when he finds it? Doesn't the miner have everything he needs to make a block? or am i missing something?

Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
September 13, 2012, 02:27:31 AM
 #14

If the miner is building the transaction why doesn't the miner submit a block when his hash is less than the difficulty?
I don't understand this question.
Thanks - I missed the part about flagging transactions. Why can't a miner just submit the block header and the transactions when he finds it? Doesn't the miner have everything he needs to make a block? or am i missing something?
He can, that's an added optimization. However, that only works for full blocks, not shares.

optimator
Sr. Member
****
Offline Offline

Activity: 350
Merit: 250



View Profile WWW
September 13, 2012, 02:34:25 AM
 #15

But then how do the other miners get credit for the completed block? Doesn't this allow dishonest miners to usurp the block from the pool?

Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
September 13, 2012, 02:36:50 AM
 #16

But then how do the other miners get credit for the completed block? Doesn't this allow dishonest miners to usurp the block from the pool?
No, the block rewards can't be changed after mining the block.

Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
September 15, 2012, 09:55:12 PM
 #17

BFGMiner 2.8.0 released with integrated getblocktemplate support.

Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
September 17, 2012, 09:22:20 PM
 #18

Question off-topic in another thread, pertaining to bitcoind GBT support:
luke-jr: Can you give me a reason why it was felt necessary to remove getmemorypool, instead of having it exist alongside GBT?
The existing solution breaks many of the pool packages out there. Backwards compatibility is normally a concern for developers.
The GBT patch for bitcoind was originally fully backward compatible, but was changed to replaced GMP instead per the request of the rest of the bitcoind development team.

makomk
Hero Member
*****
Offline Offline

Activity: 686
Merit: 500


View Profile
September 17, 2012, 11:07:56 PM
 #19

I haven't yet read about about these other protocols, but I'm really disappointed to see more protocol proliferation. Getblocktemplate is well established, peer reviewed, and is good for bitcoin's decentralized security. I hope GBT gains whatever features its missing so it can subsume these things.  Having yet another protocol to cope with is a bummer.
The problem with GBT is not so much what features it's missing, from what I can tell. Everything that can be done with Stratum can be done with GBT, it'd just far require more bandwidth and every piece of mining software would need code to parse and serialize Bitcoin transactions and blocks and to implement a whole bunch of different cases. Plus, I suspect most mining software would end up requiring a different subset of the GBT protocol than the one bitcoind supports so it's not like using the same protocol for both would even help solo miners.

Quad XC6SLX150 Board: 860 MHash/s or so.
SIGS ABOUT BUTTERFLY LABS ARE PAID ADS
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
September 18, 2012, 04:06:52 AM
 #20

I've mostly finished a more friendly writeup on getblocktemplate on the Bitcoin Wiki, covering (almost?) everything StratumMP's page covered. Take a look and let me know what you think, or how it might be improved! Feel free to edit it directly too, of course Smiley

Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
September 18, 2012, 04:16:24 AM
 #21

it'd just far require more bandwidth and every piece of mining software would need code to parse and serialize Bitcoin transactions and blocks and to implement a whole bunch of different cases.
This isn't correct.

Plus, I suspect most mining software would end up requiring a different subset of the GBT protocol than the one bitcoind supports
This isn't a problem.

eleuthria
Legendary
*
Offline Offline

Activity: 1750
Merit: 1001



View Profile
September 18, 2012, 05:54:30 AM
 #22

it'd just far require more bandwidth and every piece of mining software would need code to parse and serialize Bitcoin transactions and blocks and to implement a whole bunch of different cases.
This isn't correct.

An empty (coinbase-only block) StratumMP work payload is 399 bytes.  An empty GBT work payload based on your example is 737 bytes.

StratumMP work payloads increase 67 bytes per power of 2 transactions.  GBT work payloads increase by the size of the transactions being included in a block (roughly 250 bytes for a normal tx with 1 input, 1 payout address, and 1 change address).

For StratumMP, a block containing 257-512 transactions would require an extra 603 bytes, for a total of 1002 bytes.  GBT would require approximately 128,000 bytes.
Using a more common example, lets say each block has 100 transactions on average.  StratumMP would be 868 bytes per work payload.  The GBT equivalent would be approximately 26,000 bytes.

Similarly, work submission payloads are quite different.  GBT would require 501 bytes per share submission in your example.  StratumMP requires only 109.

None of the above factors in the added overhead of wrapping GBT into HTTP Requests.


The biggest hurdle on implementing StratumMP is changing from constant HTTP connections and inefficient longpolling, to a static TCP connection.  Beyond that change, StratumMP is a simpler protocol to work with and implement.  GBT is more full featured, but your arguments imply that most mining software wouldn't bother implementing the full feature set to begin with.

RIP BTC Guild, April 2011 - June 2015
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
September 18, 2012, 06:09:04 AM
 #23

StratumMP work payloads increase 67 bytes per power of 2 transactions.  GBT work payloads increase by the size of the transactions being included in a block (roughly 250 bytes for a normal tx with 1 input, 1 payout address, and 1 change address).
Yes, I forgot the exponential factor involved here. However, with a local bitcoind, GBT can shrink far smaller than StratumMP. If bandwidth is a real practical concern for BTCGuild (or anyone else), someone (possibly me, if someone needs and plans to use it) could easily write a "Centralized Mining" extension for GBT providing the same bandwidth savings. Of course, keep in mind that GBT is already going to use far less bandwidth than getwork, so this really shouldn't be a problem. Remember that centralized mining moves the miner's authority blindly to the pool operator, and thus creates a greater risk to the Bitcoin network. Health of the Bitcoin network is IMO far more important than saving a few KB of bandwidth here and there. So... as long as there isn't a practical problem involved, it would seem a shame to keep centralized mining going unnecessarily.

The biggest hurdle on implementing StratumMP is changing from constant HTTP connections and inefficient longpolling, to a static TCP connection.  Beyond that change, StratumMP is a simpler protocol to work with and implement.
Yes, the biggest hurdle is rewriting all the miner code actually relevant to the protocol at all.

GBT is more full featured, but your arguments imply that most mining software wouldn't bother implementing the full feature set to begin with.
Hopefully mining software will grow closer to full-featured over time. It's pretty important that mining become decentralized, at least.

hahahafr
Hero Member
*****
Offline Offline

Activity: 658
Merit: 500



View Profile
September 18, 2012, 02:01:51 PM
 #24

The decentralization is important, I agree with you on that Luke. The single-line, TCP connection of slush is still an elegant solution from a miner's point of view. I didn't read much into the technicalities of the GBT protocol, but eventually it's the effective implementations, and the stales rates that come with which ones that is going to determine what protocol is gonna be widely adopted once ASICs are shipped.

        ▄▄█████████▄▄
     ▄███████ ░████████▄
   ▄█████ ░██  ██  ██████▄
 ░███████  ██  ██ ░████████
░████  ██  ██  █▌ ▐█████████
▐████  ██  ██  █▌ ▐█████████▌
█████░ ▐█  ██  █  ████  █████
█████▌ ▐█████████████  ██████
█████▌ ▐████████████▌ ▐██████
██████  ████████████ ░███████
▐█████  ███████████░ ▐██████▌
 ██████  ████████▀░ ▄███████
  ▀█████░  ▀███▀  ▄███████▀
    ▀█████▄░   ░▄███████▀
       ▀▀███████████▀▀

▄███▄                                                                  ▄███▄
█████░                                                                 █████
█████░                                                                 █████
█████░                                                                 █████
███████████▄▄       ▄███████▄    ▄███▄     ░▄███▄     ░▄██▄░   ▄▄███████████    ▄███████▄░      ▄▄██████▄▄
██████████████▌  ░▄███████████▄░ █████     ▒█████     ▒████▒ ▐██████████████ ░▄███████████▄░  ▄████████████▄
███████████████▌░█████▀  ▀██████ █████     ▒█████     ▒████▒▐█████▀  ▀██████░▐█████▀  ▀█████░▐█████▀  ▀█████▌░
█████▀░   ▀█████▒████      █████ █████     ▒█████     ▒████▒█████░   ░░█████░█████░     ████▒█████      █████
█████░    ░█████▒████      █████ █████░   ░▐█████▌    ▓████▒█████░    ░█████░█████      ████▒█████      █████
█████░    ░█████░█████▄  ▄██████ ██████▄░▄████████▄░▄██████░▐█████▄░░▄█████▌░▐█████▄  ▄█████░▐█████▄  ▄█████▌
█████░    ░█████ ░█████████████░ ░████████████████████████░  ▐████████████▒  ░▀████████████░  ▀████████████▀
▀███▀░    ░▀███▀   ▀▀███████▀▀     ▀▀████████▀ ▀███████▀▀     ░▀████████▀       ▀███████▀▀     ░▀████████▀
  First Decentralized Social Ecosystem
  A SOCIAL NETWORK BUILT FOR THE BLOCKCHAINWORLD






───    APPLY FOR WHITELIST    ───
WEBSITE    WHITEPAPER   TELEGRAM
FACEBOOK  TWITTER         ANN
slush
Legendary
*
Offline Offline

Activity: 1372
Merit: 1019



View Profile WWW
September 18, 2012, 02:07:35 PM
 #25

The decentralization is important

I agree on this as well. Currently there are 20+ pools + P2Pools and switching over them is the matter of few seconds, and miners already rotates around backup pools. There's really no reason to think that individual miners know better than poolop what they're doing. And if miner is suspicious about poolop activity, he can just switch the pool to another.

I mean - there's no reason to make mining protocol less effective (from any point of view, including CPU resources and used bandwidth, and more difficult implementation in the miners) just because decentralization itself. Market of mining pools is great example of free market and miners can achieve the same level of "decentralization" just by switching to the pool which fits their needs.

Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
September 18, 2012, 06:23:22 PM
 #26

...and the stales rates...
FWIW, after 3+ days of use, I can safely say the stale rate is not significantly higher using GBT.

...And if miner is suspicious about poolop activity, he can just switch the pool to another.

I mean - there's no reason to make mining protocol less effective (from any point of view, including CPU resources and used bandwidth, and more difficult implementation in the miners) just because decentralization itself. Market of mining pools is great example of free market and miners can achieve the same level of "decentralization" just by switching to the pool which fits their needs.
Except miners can't viably choose to switch when they are deprived of the information needed to know when to act.

hahahafr
Hero Member
*****
Offline Offline

Activity: 658
Merit: 500



View Profile
September 18, 2012, 08:57:19 PM
 #27

Numbers?

        ▄▄█████████▄▄
     ▄███████ ░████████▄
   ▄█████ ░██  ██  ██████▄
 ░███████  ██  ██ ░████████
░████  ██  ██  █▌ ▐█████████
▐████  ██  ██  █▌ ▐█████████▌
█████░ ▐█  ██  █  ████  █████
█████▌ ▐█████████████  ██████
█████▌ ▐████████████▌ ▐██████
██████  ████████████ ░███████
▐█████  ███████████░ ▐██████▌
 ██████  ████████▀░ ▄███████
  ▀█████░  ▀███▀  ▄███████▀
    ▀█████▄░   ░▄███████▀
       ▀▀███████████▀▀

▄███▄                                                                  ▄███▄
█████░                                                                 █████
█████░                                                                 █████
█████░                                                                 █████
███████████▄▄       ▄███████▄    ▄███▄     ░▄███▄     ░▄██▄░   ▄▄███████████    ▄███████▄░      ▄▄██████▄▄
██████████████▌  ░▄███████████▄░ █████     ▒█████     ▒████▒ ▐██████████████ ░▄███████████▄░  ▄████████████▄
███████████████▌░█████▀  ▀██████ █████     ▒█████     ▒████▒▐█████▀  ▀██████░▐█████▀  ▀█████░▐█████▀  ▀█████▌░
█████▀░   ▀█████▒████      █████ █████     ▒█████     ▒████▒█████░   ░░█████░█████░     ████▒█████      █████
█████░    ░█████▒████      █████ █████░   ░▐█████▌    ▓████▒█████░    ░█████░█████      ████▒█████      █████
█████░    ░█████░█████▄  ▄██████ ██████▄░▄████████▄░▄██████░▐█████▄░░▄█████▌░▐█████▄  ▄█████░▐█████▄  ▄█████▌
█████░    ░█████ ░█████████████░ ░████████████████████████░  ▐████████████▒  ░▀████████████░  ▀████████████▀
▀███▀░    ░▀███▀   ▀▀███████▀▀     ▀▀████████▀ ▀███████▀▀     ░▀████████▀       ▀███████▀▀     ░▀████████▀
  First Decentralized Social Ecosystem
  A SOCIAL NETWORK BUILT FOR THE BLOCKCHAINWORLD






───    APPLY FOR WHITELIST    ───
WEBSITE    WHITEPAPER   TELEGRAM
FACEBOOK  TWITTER         ANN
dooferorg
Full Member
***
Offline Offline

Activity: 163
Merit: 100


View Profile
September 18, 2012, 11:09:35 PM
 #28

The stratum approach is working right now with the current installed based of miners. Just point to the proxy intermediary the 'stratum proxy', simple. Been testing this on a miner I have with BTCGuilds stratum test and it's doing great.

Failover to other pools has been addressed by other proxy software and I use that too in conjunction with the stratum proxy with no problem. Miner can work and upstream proxies (on the local network) just decide what pool and how to communicate. I didn't even have to touch the miner software.

BTC: 1dooferoD3vnwgez3Jo1E4bFfgMf81LR2
ZEC: t1gnToN2HZW4GD52kofEVdijhRijWjCNfYi
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
September 18, 2012, 11:17:49 PM
 #29

The stratum approach is working right now with the current installed based of miners. Just point to the proxy intermediary the 'stratum proxy', simple. Been testing this on a miner I have with BTCGuilds stratum test and it's doing great.
GBT has been working with a proxy since February, and is easily added to existing miners.

Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
September 19, 2012, 06:16:11 PM
 #30

I don't have dialup, but still saving bandwidth is important so the new mining hardware doesn't DDOS your shiny pool by saturating the pools line.
Sure, but GBT does save bandwidth. It just doesn't save as much as StratumMP, because it doesn't keep the miner in the dark.

If the GBT thing has been around since February, why aren't all pools and miners already using it?
Probably the main (only?) incentive for most pools is surviving when ASICs arrive. There's also the problem of Eloipool being the only (publicly available) maintained poolserver (though p2k plans to update ecoinpool when he gets time), and most pools seem to still be using pushpool or PSJ. Technically speaking, pools running older versions of Eloipool should have an older GBT available, but now that BIP 22 is final, it seemed inappropriate to list them until they are running a compatible version.

hahahafr
Hero Member
*****
Offline Offline

Activity: 658
Merit: 500



View Profile
September 20, 2012, 05:15:11 AM
 #31

I don't have dialup, but still saving bandwidth is important so the new mining hardware doesn't DDOS your shiny pool by saturating the pools line.
Sure, but GBT does save bandwidth. It just doesn't save as much as StratumMP, because it doesn't keep the miner in the dark.
Showing the miners the path to enlightenment is nice, but it's not as if we already couldn't do that. Take the Bitminter pool for example: It's clear, transparent, without fees, pool donations are optional, and pays all the transactions fees to the miners. Everyone can check it on the public BTC address that receives the blocks+transactions. After 120 confirmations this same address pays the exact same amount to the miners. More light is not possible.

One should also note that the interest of the pool operator is the same as the miners: Include the best transactions in the next block to maximize his profit (through donations in our case).

        ▄▄█████████▄▄
     ▄███████ ░████████▄
   ▄█████ ░██  ██  ██████▄
 ░███████  ██  ██ ░████████
░████  ██  ██  █▌ ▐█████████
▐████  ██  ██  █▌ ▐█████████▌
█████░ ▐█  ██  █  ████  █████
█████▌ ▐█████████████  ██████
█████▌ ▐████████████▌ ▐██████
██████  ████████████ ░███████
▐█████  ███████████░ ▐██████▌
 ██████  ████████▀░ ▄███████
  ▀█████░  ▀███▀  ▄███████▀
    ▀█████▄░   ░▄███████▀
       ▀▀███████████▀▀

▄███▄                                                                  ▄███▄
█████░                                                                 █████
█████░                                                                 █████
█████░                                                                 █████
███████████▄▄       ▄███████▄    ▄███▄     ░▄███▄     ░▄██▄░   ▄▄███████████    ▄███████▄░      ▄▄██████▄▄
██████████████▌  ░▄███████████▄░ █████     ▒█████     ▒████▒ ▐██████████████ ░▄███████████▄░  ▄████████████▄
███████████████▌░█████▀  ▀██████ █████     ▒█████     ▒████▒▐█████▀  ▀██████░▐█████▀  ▀█████░▐█████▀  ▀█████▌░
█████▀░   ▀█████▒████      █████ █████     ▒█████     ▒████▒█████░   ░░█████░█████░     ████▒█████      █████
█████░    ░█████▒████      █████ █████░   ░▐█████▌    ▓████▒█████░    ░█████░█████      ████▒█████      █████
█████░    ░█████░█████▄  ▄██████ ██████▄░▄████████▄░▄██████░▐█████▄░░▄█████▌░▐█████▄  ▄█████░▐█████▄  ▄█████▌
█████░    ░█████ ░█████████████░ ░████████████████████████░  ▐████████████▒  ░▀████████████░  ▀████████████▀
▀███▀░    ░▀███▀   ▀▀███████▀▀     ▀▀████████▀ ▀███████▀▀     ░▀████████▀       ▀███████▀▀     ░▀████████▀
  First Decentralized Social Ecosystem
  A SOCIAL NETWORK BUILT FOR THE BLOCKCHAINWORLD






───    APPLY FOR WHITELIST    ───
WEBSITE    WHITEPAPER   TELEGRAM
FACEBOOK  TWITTER         ANN
kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
September 21, 2012, 06:42:58 AM
 #32

So ... someone wanna tell me who the fucktard mod was that deleted my post from here?
I guess is was most likely Luke-Jr's pissy little dog who does his bidding - but feel free to correct me if I'm wrong ...

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
ssateneth
Legendary
*
Offline Offline

Activity: 1330
Merit: 1004



View Profile
September 22, 2012, 12:32:57 AM
 #33

So ... someone wanna tell me who the fucktard mod was that deleted my post from here?
I guess is was most likely Luke-Jr's pissy little dog who does his bidding - but feel free to correct me if I'm wrong ...

quoted before deletion

kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
September 22, 2012, 12:50:54 AM
 #34

So ... someone wanna tell me who the fucktard mod was that deleted my post from here?
I guess is was most likely Luke-Jr's pissy little dog who does his bidding - but feel free to correct me if I'm wrong ...

quoted before deletion
Thanks Smiley

It seems that there is a mod (I wonder who Smiley ) who is a slave toy of this stupid thread OP,
who bows down to and deletes posts for him at his whim, to hide the truth about the thread OP.

Not surprising really though, since the thread OP is provably known for lying all over the place to hide his transgressions.

Just reading the first paragraph of the first post and reading exactly what he is saying is rather convincing of the (totally ludicrous) self importance the thread OP places upon himself.
Quote
Since Eleuthria and slush have both recently announced their own developed-behind-closed-doors mining protocols, I felt it appropriate to make a thread here for end-user discussion of the "getblocktemplate" standard which they intend to compete with. As a poolserver author, a pool operator, a miner author, and a contributor to the reference client, my life is not improved by having yet another protocol to support, and what's bad for the authors of the infrastructure tools is also bad for all their users.

Lulz - what the thread OP thinks is bad, is bad for everyone who uses bitcoind Cheesy Cheesy Cheesy

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
September 22, 2012, 02:03:12 PM
 #35

Numbers?
Now that I've had BFGMiner 2.8 running for a number of days, I have some reasonably useful numbers...
getwork session A: 0.83% stale
getwork session B: 1.09% stale
GBT session A: 0.71% stale
GBT session B: 0.66% stale

There's probably a good chance the GBT stales are lower only by coincidence (a lot depends on when blocks are found, after all), but I think it's pretty safe to conclude there's not any notable practical difference just from the protocol.

slush
Legendary
*
Offline Offline

Activity: 1372
Merit: 1019



View Profile WWW
September 22, 2012, 02:20:21 PM
 #36

There's probably a good chance the GBT stales are lower only by coincidence (a lot depends on when blocks are found, after all), but I think it's pretty safe to conclude there's not any notable practical difference just from the protocol.

Comparing both solutions based on HTTP+LP, you obviously don't see any difference.

Stratum session on my machine: 60627 submits, 53 stale, it's 0.08 % stale ratio.

This ^^ also includes short outage of my Internet connectivity overnight ;-).

kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
September 22, 2012, 03:49:44 PM
 #37

There's probably a good chance the GBT stales are lower only by coincidence (a lot depends on when blocks are found, after all), but I think it's pretty safe to conclude there's not any notable practical difference just from the protocol.

Comparing both solutions based on HTTP+LP, you obviously don't see any difference.

Stratum session on my machine: 60627 submits, 53 stale, it's 0.08 % stale ratio.

This ^^ also includes short outage of my Internet connectivity overnight ;-).
... and I'm not sure when the last time I got much above ~0.3% rejects on cgminer using OzCoin or EMC standard pools.
Which is above Stratum but better than this thread protocol.
Stratum looks like it may be OK, but this thread one being worse than the current numbers on a standard pool is ... not good.

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
slush
Legendary
*
Offline Offline

Activity: 1372
Merit: 1019



View Profile WWW
September 22, 2012, 05:04:57 PM
 #38

kano, now you're comparing apples and oranges. Stale ratio is affected by many factors and there's no obvious reason why GBT should be worse than getwork.

kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
September 23, 2012, 12:52:20 AM
 #39

kano, now you're comparing apples and oranges. Stale ratio is affected by many factors and there's no obvious reason why GBT should be worse than getwork.
stale/reject ratio is basically 2 things: 'LP' issues and processing work for too long according to how long the pool will accept the result (and the protocol may allow that all the way up to an LP)

3 different protocols still have that same requirement: accept work or reject it.

There are of course 3 factors in it:
1) The mining software
2) The protocol
3) The connectivity between them

If the miner/protocol/pool is rejecting more work then there is some issue that needs to be resolved.
(or the design is a POS)
As I stated, I get around 0.3% stale with a standard pool protocol
(I can also add that my pool connect is half way around the world, so that should be close to about as bad as it can be)

However, comparing cgminer+getwork/LP+a standard pool
is indeed quite valid.

What other comparison could be made?

Fake up some excuse for bad performance?

His figures are showing his miner+his protocol+his pool are clearly worse than the setup I described ... and I will add that ~0.3% or better is not unexpected from people using cgminer+getwork/LP+'choose random pool'

So I'm not sure where the apples and oranges are ...

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
hahahafr
Hero Member
*****
Offline Offline

Activity: 658
Merit: 500



View Profile
September 23, 2012, 03:20:08 AM
 #40

Numbers?
Now that I've had BFGMiner 2.8 running for a number of days, I have some reasonably useful numbers...
getwork session A: 0.83% stale
getwork session B: 1.09% stale
GBT session A: 0.71% stale
GBT session B: 0.66% stale

There's probably a good chance the GBT stales are lower only by coincidence (a lot depends on when blocks are found, after all), but I think it's pretty safe to conclude there's not any notable practical difference just from the protocol.
If I understand correctly GBT also supports LP, doesn't it? Did you make use of LP with getwork and GBT in your test?

There's probably a good chance the GBT stales are lower only by coincidence (a lot depends on when blocks are found, after all), but I think it's pretty safe to conclude there's not any notable practical difference just from the protocol.

Comparing both solutions based on HTTP+LP, you obviously don't see any difference.

Stratum session on my machine: 60627 submits, 53 stale, it's 0.08 % stale ratio.

This ^^ also includes short outage of my Internet connectivity overnight ;-).
Since stratum is working over a plain TCP connection, one could say that the equivalent of LP is just a simple TCP "message" like any other one, isn't it?

It would really be interesting to see the stale ratio of GBT+LP vs Stratum over one week on the same pool/same connection.

        ▄▄█████████▄▄
     ▄███████ ░████████▄
   ▄█████ ░██  ██  ██████▄
 ░███████  ██  ██ ░████████
░████  ██  ██  █▌ ▐█████████
▐████  ██  ██  █▌ ▐█████████▌
█████░ ▐█  ██  █  ████  █████
█████▌ ▐█████████████  ██████
█████▌ ▐████████████▌ ▐██████
██████  ████████████ ░███████
▐█████  ███████████░ ▐██████▌
 ██████  ████████▀░ ▄███████
  ▀█████░  ▀███▀  ▄███████▀
    ▀█████▄░   ░▄███████▀
       ▀▀███████████▀▀

▄███▄                                                                  ▄███▄
█████░                                                                 █████
█████░                                                                 █████
█████░                                                                 █████
███████████▄▄       ▄███████▄    ▄███▄     ░▄███▄     ░▄██▄░   ▄▄███████████    ▄███████▄░      ▄▄██████▄▄
██████████████▌  ░▄███████████▄░ █████     ▒█████     ▒████▒ ▐██████████████ ░▄███████████▄░  ▄████████████▄
███████████████▌░█████▀  ▀██████ █████     ▒█████     ▒████▒▐█████▀  ▀██████░▐█████▀  ▀█████░▐█████▀  ▀█████▌░
█████▀░   ▀█████▒████      █████ █████     ▒█████     ▒████▒█████░   ░░█████░█████░     ████▒█████      █████
█████░    ░█████▒████      █████ █████░   ░▐█████▌    ▓████▒█████░    ░█████░█████      ████▒█████      █████
█████░    ░█████░█████▄  ▄██████ ██████▄░▄████████▄░▄██████░▐█████▄░░▄█████▌░▐█████▄  ▄█████░▐█████▄  ▄█████▌
█████░    ░█████ ░█████████████░ ░████████████████████████░  ▐████████████▒  ░▀████████████░  ▀████████████▀
▀███▀░    ░▀███▀   ▀▀███████▀▀     ▀▀████████▀ ▀███████▀▀     ░▀████████▀       ▀███████▀▀     ░▀████████▀
  First Decentralized Social Ecosystem
  A SOCIAL NETWORK BUILT FOR THE BLOCKCHAINWORLD






───    APPLY FOR WHITELIST    ───
WEBSITE    WHITEPAPER   TELEGRAM
FACEBOOK  TWITTER         ANN
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
September 23, 2012, 05:00:03 AM
 #41

If I understand correctly GBT also supports LP, doesn't it? Did you make use of LP with getwork and GBT in your test?
Yes, of course.

It would really be interesting to see the stale ratio of GBT+LP vs Stratum over one week on the same pool/same connection.
Unfortunately, I haven't optimized GBT in BFGMiner at all yet, so I doubt it would compare well vs Stratum proxy. It would be interesting for someone to make a quick port of the Stratum proxy to GBT (python-bitcoinrpc + python-blkmaker should do it very easily) to compare with. Of course, you'd also need a biprotocol pool, which AFAIK doesn't exist yet.

jgarzik
Legendary
*
Offline Offline

Activity: 1554
Merit: 1001


View Profile
September 23, 2012, 05:19:03 AM
 #42


Longpolling is a very elegant and useful... hack.  It is a hack because HTTP clients typically initiate requests, and elicit server responses.  This HTTP client behavior does not cover server-initiated messages that the client is not expecting.

Longpolling says "wait, until the next server-initiated message to me"

Most custom designed protocols support server-to-client asynchronous messages, so there is no need for the hack.


Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
kuzetsa
Sr. Member
****
Offline Offline

Activity: 368
Merit: 250


View Profile
September 27, 2012, 04:12:06 AM
 #43

Quote from: #eligius(freenode)
(GMT-0400) North America Eastern Daylight summer time, NY)
[23:39:32] <kuzetsa> is eloipool going to have stratum support?
[23:39:41] <wizkid057> ...
[23:39:50] <@Luke-Jr> kuzetsa: stratum sucks
[23:39:58] <kuzetsa> Luke-Jr: not for bandwidth it doesn't
[23:40:09] <kuzetsa> it's awesome at bandwidth


((...snip...))


[23:59:29] <@Luke-Jr> kuzetsa: you seem to be confused.
[23:59:44] <@Luke-Jr> GBT supports everyone's needs, not just me/my sw/my pool
[23:59:47] <kuzetsa> because I like stratum?
[23:59:54] <kuzetsa> liking stratum = confused?
[23:59:56] <@Luke-Jr> there is no rational reason to prefer stratum
[23:59:59] <@Luke-Jr> so sure

No real explanation given. Just that I'm irrational.

Often as I truely am irrational or even confused, this seems more like  "argumentum ad hominem" to me.




...Edited to add:

Forgot to datestamp.

Local time rolled over from wednesday. Now it is Thursday, september 27th 2012

Quote from: #eligius(freenode)
(GMT-0400) North America Eastern Daylight summer time, NY)
[00:35:41] <@Luke-Jr> kuzetsa: this is a private channel and quoting it is against freenode policy and wiretapping laws, ESPECIALLY out of context
[00:45:52] * You were kicked from #eligius by Luke-Jr (delete the post)



Consider my official response "No"

I'll not be bullied into censoring this.

Feel free to take it up with freenode staff or law enforcement.

I'm confident that I know my local laws:

  • I live in a "one party consent" jurisdiction for purposes of recording a phonecall.
  • Your channel is public (not passworded, nor invite only, etc.) and freenode has no such policy as you described.
DiabloD3
Legendary
*
Offline Offline

Activity: 1162
Merit: 1000


DiabloMiner author


View Profile WWW
September 27, 2012, 05:04:54 AM
 #44

Angry neighborhood bastard mod here.

Luke, what did I tell you about trolling the noobs? Cut it out before one of the admins decide to either scammer/troll tag you or ban you.

jgarzik
Legendary
*
Offline Offline

Activity: 1554
Merit: 1001


View Profile
September 27, 2012, 05:29:38 AM
 #45

The pragmatic solution is to support both GBT and stratum, in mining software.

Both are already deployed, and stratum seems to currently be winning the race for deployments.

GBT is supported in bitcoind and p2pool, and has gone through the BIP process and more review, so a choice seems valid.  GBT is also more easily deployed in existing software, as it uses the more standard HTTP JSON-RPC that miners are familiar with.


Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
September 27, 2012, 06:01:20 AM
 #46

The pragmatic solution is to support both GBT and stratum, in mining software.
Perhaps, but implementing Stratum is a lot of work (at least on the miner end). Better for pools to support both IMO; at least that gives the miners a choice of whether they want to retain control or blindly hand it over to the poolop.

Both are already deployed, and stratum seems to currently be winning the race for deployments.
I see more GBT deployment than Stratum, thankfully.

P.S. Since the mods here don't do their job, I have to point out that kuzetsa is misquoting me.

jgarzik
Legendary
*
Offline Offline

Activity: 1554
Merit: 1001


View Profile
September 27, 2012, 06:22:04 AM
 #47

The pragmatic solution is to support both GBT and stratum, in mining software.
Perhaps, but implementing Stratum is a lot of work (at least on the miner end). Better for pools to support both IMO; at least that gives the miners a choice of whether they want to retain control or blindly hand it over to the poolop.

That is the trade-off.  GBT is far more compatible with existing software.  Stratum is a wholly new protocol requiring wholly new transport implementation, but it gains some efficiency over HTTP.


Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
kuzetsa
Sr. Member
****
Offline Offline

Activity: 368
Merit: 250


View Profile
September 27, 2012, 07:30:57 AM
 #48

((...snip...))
I have to point out that kuzetsa is misquoting me.

I just edited slightly, making the ((...snip...)) in the channel log thingy more obvious by inserting additional vertical whitespace (newline) above and below (3x newline padding instead of
"singlespaced" newline)

Just want to assert: There was no misquote, just an omission for the long duration of "you're wrong, no you are, no you"

--- luke. feel free to fill the ((...snip...)) with what I left out.

My main thought is that I prefer stratum because it gets fewer stale / invalid / orphan or whatever they're called. Stratum was working and had an installed userbase and is ASIC ready, predates GBT, etc.

Longpoll over HTTP-type transport is making GBT less attractive to me than stratum. Any mining protocol using a full TCP transport or even a non-stateless version of udp, etc. (basically, anything with an active connection, therefore by definition not http) will allow for incremental changes to the workload and immediate notification of new block.

There is no reason GBT's features can't be overlaid onto stratum or other non-HTTP mining protocols. (or the reverse for that matter, nothing TOO major that locks GBT's underlying structure to using HTTP)

Before all of this drama started I had planned to be in bed by 30 minutes past midnight. That was 3 hours ago. guess I'll just end with this:

Luke, I've been using IRC since the mid 90s. New to bitcoin and I don't need threats like this when I'm still a n00b because your agenda is threatend or whatever it was that upset you. please be cool.
P_Shep
Legendary
*
Offline Offline

Activity: 1176
Merit: 1000


View Profile
September 27, 2012, 03:41:40 PM
 #49

kuzetsa, meet Luke Smiley
slush
Legendary
*
Offline Offline

Activity: 1372
Merit: 1019



View Profile WWW
October 11, 2012, 08:07:42 PM
 #50

Well, it is not "few kbytes saved versus save the network". I had quite long discussion with gmaxwell yesterday on -dev and I simply don't see any real attack vector which can be performed just because pool send merkle branch instead of transaction hashes (and this is REAL difference between Stratum and GBT). The only real attack vector which we found is that CIA will kill me, Tycho, Eleuthria, Graet, Inaba and some other pool operators, overtake our pools AND users won't notice anything and they'll keep on hashing invalid/broken blocks for CIA.

So from my view, GBT is wasting resources for no real gain.

kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
October 11, 2012, 09:54:57 PM
 #51

I mean - there's no reason to make mining protocol less effective just because decentralization itself.

I wonder what would you say to those who were, are and will be "lured" to Bitcoin with promise of decentralized network once
they realise that same decentralization is being sacrificed for few (k)bytes saved. Your importance list got messed up somehow.

Wait - should I be surprised with promotion of X at the cost of more centralization by pool owner? No, LOL, that's expectable!

As soon as BTCGuild releases me from "0.1 BTC prison", I'll switch to solo mining using BFGMiner with GBT. It won't really bring
a lot to Bitcoin network itself since my hashrate is just 30MH, but I'll sleep better knowing I'm using and supporting right thing.
So why were you already pool mining if this is an issue for you?

If decentralization was in ANY way relevant to your comments then you would have been solo mining or using P2Pool.

... which have been around for a long ....... time ...

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
October 11, 2012, 10:40:29 PM
 #52

The issue is quite simple.

If you personally want 100% decentalization then you must either solo mine or use P2Pool - not GBT to a pool.
GBT to a pool is not 100% 'decentalization'

However, with 30MH/s, you will possibly never get any BTC solo mining.
Your average expected time to find a block at the current difficulty is 5062 days ...

As for pools in general ... luckily, people are allowed to choose hey?
BTC isn't a Luke-jr dictatorship (even though he wants to force that on everyone)

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
mezzomix
Legendary
*
Offline Offline

Activity: 2100
Merit: 1067


View Profile
October 12, 2012, 06:46:50 AM
 #53

So from my view, GBT is wasting resources for no real gain.

That depends on your point of view. As a pool operator or miner with ultimate trust to the pool operator you are right. From the perspective of a miner that cares about what goes into a block it is important to see the complete information.

With Stratum, the pool operator is the one that creates bitcoin blocks and makes the rules. The miner is paid for doing some work for the operator. With GBT the pool operator is a work dispatcher but miners create the bitcoin blocks and make the rules. I don't regard more information and control as a waste of resources.
kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
October 12, 2012, 11:02:30 AM
 #54

So from my view, GBT is wasting resources for no real gain.

That depends on your point of view. As a pool operator or miner with ultimate trust to the pool operator you are right. From the perspective of a miner that cares about what goes into a block it is important to see the complete information.

With Stratum, the pool operator is the one that creates bitcoin blocks and makes the rules. The miner is paid for doing some work for the operator. With GBT the pool operator is a work dispatcher but miners create the bitcoin blocks and make the rules. I don't regard more information and control as a waste of resources.

... actually - no - who actually sees what goes into the blocks?
The software you mine with decides - and if you use the gbt luke-jr implemented - then luke-jr decides for you ...
Did you read the gbt code he wrote when you mined with his BarbieMiner?

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
October 12, 2012, 01:24:40 PM
 #55

So from my view, GBT is wasting resources for no real gain.

That depends on your point of view. As a pool operator or miner with ultimate trust to the pool operator you are right. From the perspective of a miner that cares about what goes into a block it is important to see the complete information.

With Stratum, the pool operator is the one that creates bitcoin blocks and makes the rules. The miner is paid for doing some work for the operator. With GBT the pool operator is a work dispatcher but miners create the bitcoin blocks and make the rules. I don't regard more information and control as a waste of resources.

... actually - no - who actually sees what goes into the blocks?
The software you mine with decides - and if you use the gbt luke-jr implemented - then luke-jr decides for you ...
Did you read the gbt code he wrote when you mined with his BarbieMiner?

You wanna say something is wrong with GBT? Proof?
To correctly state your question:
What are the rules that BarbieMiner uses to select the transactions to put into the GBT blocks?
No doubt you don't already know what they are ... without at least first looking them up now ...
You see the point of my statement?

Edit: and to add to that ... once you finally bother to look up and learn what rules he uses, do you know how to change it to suit what you think it should be? Smiley

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
October 12, 2012, 02:33:40 PM
 #56

I'm using simple logic here:

1. Most of people here which I think suck on so many levels, even fundamental ones, don't like Luke.
2. Luke proved he is concerned with exactly the same issues - and probably even more of them - I'm concerned with.
3. Luke is surely smart enough to know attempting to overtake / manipulate Bitcoin with open source code won't work.

Unless you can prove GBT is flawed, excluding the "it uses more bandwidth / more stales" part, you better rest your case.
Well following your link I've read a lot there of you trying to tell everyone else what they should do.

You wouldn't happen to be Luke-jr would you ...

Anyway, you've made no case either - you've simply stated you don't care what rules Luke-jr has in GBT.

I will point out the following:

1) Luke-jr's pool uses rules to ignore transactions that he will not divulge
2) Luke-jr used his pool's hashing power (without the knowledge of the miners) to effectively kill off an alt-coin
3) Luke-jr had a performance improvement for the BFL device driver for cgminer that he used himself but didn't release

The simple issue is that everyone has their own opinion of what they think is best ... yet you seem to not care what rules are in the software you use ... yet you also seem to care that you don't know the rules that pools are using.

Your argument's main point is in fact also your own ignorance of that point - that you have access to know something that you don't even care to check before you use it.

My initial statement was to simply point out that the comments about GBT being better than Pools due to 'decentralization' were only valid when solo mining and not better than P2Pool (and worse than P2Pool when not solo mining)

That fact that you blindly trust Luke-jr - well good for you.
I have learnt from dealing with him that I certainly should not do that.

Now to top that off, since you say that you can check these things, I will also point out that you can check these things with any pool.
The transactions available on the network at any particular time are visible to everyone.
The transactions placed in every block mined are also visible to everyone as soon as the block gets into the block chain
You can clearly see for every block mined, what transactions were selected and what were ignored ... if you wish to.

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
October 12, 2012, 02:48:25 PM
 #57

I think all pools should be shut-down immidiatelly and code put in place to prevent formation of any sort of groups.
This isn't possible even in theory.

Issues with Luke you posted are nothing but minor incidents.
They also didn't happen, or are being presented by Kano out of (important) context.

For other stuff you posted, I'll wait to see Luke's reply.
What in specific would you like a response to? Kano spouts so much lies and nonsense that I generally ignore his posts.

-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
October 20, 2012, 12:09:05 AM
 #58

An empty (coinbase-only block) StratumMP work payload is 399 bytes.  An empty GBT work payload based on your example is 737 bytes.

StratumMP work payloads increase 67 bytes per power of 2 transactions.  GBT work payloads increase by the size of the transactions being included in a block (roughly 250 bytes for a normal tx with 1 input, 1 payout address, and 1 change address).

For StratumMP, a block containing 257-512 transactions would require an extra 603 bytes, for a total of 1002 bytes.  GBT would require approximately 128,000 bytes.
Using a more common example, lets say each block has 100 transactions on average.  StratumMP would be 868 bytes per work payload.  The GBT equivalent would be approximately 26,000 bytes.

Similarly, work submission payloads are quite different.  GBT would require 501 bytes per share submission in your example.  StratumMP requires only 109.

None of the above factors in the added overhead of wrapping GBT into HTTP Requests.
What a mess. With payloads increasing linearly with transaction count in GBT that makes each getwork equivalent much bigger than plain getwork already is the longer a block takes to solve. As bitcoin becomes more popular and transaction counts increase, gbt becomes less and less efficient. I'd hazard a guess that it may even be less efficient than plain getwork at some stage unless pools start ignoring transactions... there's a disincentive to include them.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
jgarzik
Legendary
*
Offline Offline

Activity: 1554
Merit: 1001


View Profile
October 20, 2012, 12:17:16 AM
 #59

What a mess. With payloads increasing linearly with transaction count in GBT that makes each getwork equivalent much bigger than plain getwork already is the longer a block takes to solve. As bitcoin becomes more popular and transaction counts increase, gbt becomes less and less efficient. I'd hazard a guess that it may even be less efficient than plain getwork at some stage unless pools start ignoring transactions... there's a disincentive to include them.

Unless something has changed, getblocktemplate should include a mode that skips sending the transactions themselves.  See https://en.bitcoin.it/wiki/BIP_0023

I agree it is unlikely that non-p2pool miners will use the full bore GBT + all transactions, even if it is better for network security.

But don't dismiss GBT based on faulty "full mode only" assumptions.


Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
October 20, 2012, 01:03:24 AM
 #60

What a mess. With payloads increasing linearly with transaction count in GBT that makes each getwork equivalent much bigger than plain getwork already is the longer a block takes to solve. As bitcoin becomes more popular and transaction counts increase, gbt becomes less and less efficient. I'd hazard a guess that it may even be less efficient than plain getwork at some stage unless pools start ignoring transactions... there's a disincentive to include them.

Unless something has changed, getblocktemplate should include a mode that skips sending the transactions themselves.  See https://en.bitcoin.it/wiki/BIP_0023

I agree it is unlikely that non-p2pool miners will use the full bore GBT + all transactions, even if it is better for network security.

But don't dismiss GBT based on faulty "full mode only" assumptions.

The problem with not receiving the transactions is that the miner program must have a full bitcoin connection to the network.

Suddenly mining includes bitcoin network transaction and block processing.

... and anyone with half an inkling of understanding of what that entails can see the issues of having to deal with all the complications that entails and then also the importance of the bitcoin network performance of the miner ... which before this point was zero.

The experiment of the issues with this already exists Smiley P2Pool.

The only really acceptable solution to this problem, in my opinion, would be to have bitcoind handle that fully for a GBT miner - not have yet another bitcoin network implementation added onto every miner.

But this still has the issue of the reliability of the local bitcoin network connection and the problems that can cause (that don't exist in the current implementations other than p2pool and solo mining)

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
jgarzik
Legendary
*
Offline Offline

Activity: 1554
Merit: 1001


View Profile
October 20, 2012, 01:18:31 AM
 #61

The problem with not receiving the transactions is that the miner program must have a full bitcoin connection to the network.

No, that is not correct at all.

The pool server may supply sufficient information to modify extranonce in the coinbase txn, and update the merkle root.  Nothing more.

And even that's going over the absolute minimum requirement of simply a block header, with ntime rolling supported.


Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
October 20, 2012, 02:57:20 AM
 #62

What a mess. With payloads increasing linearly with transaction count in GBT that makes each getwork equivalent much bigger than plain getwork already is the longer a block takes to solve. As bitcoin becomes more popular and transaction counts increase, gbt becomes less and less efficient. I'd hazard a guess that it may even be less efficient than plain getwork at some stage unless pools start ignoring transactions... there's a disincentive to include them.

Unless something has changed, getblocktemplate should include a mode that skips sending the transactions themselves.  See https://en.bitcoin.it/wiki/BIP_0023

I agree it is unlikely that non-p2pool miners will use the full bore GBT + all transactions, even if it is better for network security.

But don't dismiss GBT based on faulty "full mode only" assumptions.


So what are you anticipating the pools vs miners will be doing with transactions in pooled mining (ignoring p2pool)?

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
October 20, 2012, 04:00:28 AM
 #63

The problem with not receiving the transactions is that the miner program must have a full bitcoin connection to the network.

No, that is not correct at all.

The pool server may supply sufficient information to modify extranonce in the coinbase txn, and update the merkle root.  Nothing more.

...
Stratum ...

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
jgarzik
Legendary
*
Offline Offline

Activity: 1554
Merit: 1001


View Profile
October 20, 2012, 05:08:54 AM
 #64

So what are you anticipating the pools vs miners will be doing with transactions in pooled mining (ignoring p2pool)?

Transactions don't need to make it to the miners, just the pool server.  The pool server can handle most of the merkle building bits, and verify the returned work from the miner.

The miner runs through nonce.
If miner needs more work, it runs through ntime range.
If miner needs more work, it increments extranonce.

That will keep your ASIC miner mining through the next century ;p


Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
October 20, 2012, 05:20:08 AM
 #65

So what are you anticipating the pools vs miners will be doing with transactions in pooled mining (ignoring p2pool)?

Transactions don't need to make it to the miners, just the pool server.  The pool server can handle most of the merkle building bits, and verify the returned work from the miner.

The miner runs through nonce.
If miner needs more work, it runs through ntime range.
If miner needs more work, it increments extranonce.

That will keep your ASIC miner mining through the next century ;p


The miner can't hash anything without that merkle root though. I didn't see the merkle root being passed regularly to the miners in the sample on the wiki, just how to hash transactions as they come in and generate a merkle root. Did I miss where the merkle root is passed to the miner?

In which case, we're back to my original comment... the miners need all the transactions every so often or they're doing bitcoin a disservice.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
firefop
Sr. Member
****
Offline Offline

Activity: 420
Merit: 250


View Profile
October 20, 2012, 06:08:37 AM
 #66

The problem with not receiving the transactions is that the miner program must have a full bitcoin connection to the network.

Suddenly mining includes bitcoin network transaction and block processing.

... and anyone with half an inkling of understanding of what that entails can see the issues of having to deal with all the complications that entails and then also the importance of the bitcoin network performance of the miner ... which before this point was zero.

Complication like "staying online" is complicated?

This is so much of a non-issue, it's just silly.

Miners SHOULD be running a node even if they aren't mining on it. Depending on the pool for information that can be pulled from bitcoind isn't smart. Having to trust the pool for transaction and block processing isn't smart either. Those add unneeded complexity to an already overcomplicated system.

In fact I think pools in general come at mining form the wrong direction. You don't need to develop a way to hand out work where you send it to the miner and they report back with the completed work - what you ought to be doing is making work assignment system where each miner generates, validates and submits the work for the pool - while transmitting just the nonce (or even better a nonce range completed) back to the pool.

Every miner would include a msg from them to the pool stating they mined the block. And now you've got decentralized and much lower bandwidth. And you aren't trusting the pool for what it may or may not be doing with your hash rate.

And this is just off the top of my head --- start thinking about 'how it should be' instead of 'how it is right now' --- and start developing in that direction also.



-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
October 20, 2012, 06:12:50 AM
 #67

Sorry but a lot of miners mine without their mining rigs being connected to a bitcoind instance. Yes there may be one somewhere in their network, or possibly not at all and they only fire up their wallet when they want to on a separate network. What you think is ideal is asking people to do more than they currently do. They currently use pools and they trust them already. The community is very vigilant with respect to pools misbehaving. The importance of the miner doing it all themselves is grossly exaggerated by proponents of GBT. Pools will continue to do as much as they already do. It's unrealistic to expect otherwise.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
October 20, 2012, 07:05:49 AM
 #68

The problem with not receiving the transactions is that the miner program must have a full bitcoin connection to the network.

Suddenly mining includes bitcoin network transaction and block processing.

... and anyone with half an inkling of understanding of what that entails can see the issues of having to deal with all the complications that entails and then also the importance of the bitcoin network performance of the miner ... which before this point was zero.

Complication like "staying online" is complicated?

This is so much of a non-issue, it's just silly.

Miners SHOULD be running a node even if they aren't mining on it. Depending on the pool for information that can be pulled from bitcoind isn't smart. Having to trust the pool for transaction and block processing isn't smart either. Those add unneeded complexity to an already overcomplicated system.

In fact I think pools in general come at mining form the wrong direction. You don't need to develop a way to hand out work where you send it to the miner and they report back with the completed work - what you ought to be doing is making work assignment system where each miner generates, validates and submits the work for the pool - while transmitting just the nonce (or even better a nonce range completed) back to the pool.

Every miner would include a msg from them to the pool stating they mined the block. And now you've got decentralized and much lower bandwidth. And you aren't trusting the pool for what it may or may not be doing with your hash rate.

And this is just off the top of my head --- start thinking about 'how it should be' instead of 'how it is right now' --- and start developing in that direction also.

A few points here:

The most important is yet again someone telling everyone how they should be mining.
"how it should be"
I'm glad you feel that is how it should be.

"Miners SHOULD be running a node"
Again, glad you feel that's how it should be.

Lucky for everyone else, they have freedom of choice and are not under your control.

Even luckier for you - you can do this already for yourself today - without trying to force everyone else to do it.

--

Meanwhile, your first statement is the only one in this conversation that is "just silly"

Ignoring the blatantly obvious issues and pretending to simplify to a stupid statement of "staying online"

bitcoind cannot be used except for solo mining (or p2pool) - as I mentioned, there needs work done to support a combination of pool GBT mining and bitcoind to supply the transactions.

i.e. miners implementing bitcoind yet again is just a problem waiting to happen.
... and that adds complexity, it doesn't reduce it at all.
Do you actually understand what txn and block processing entails?

Then of course there's the fact that you need to run a high performance bitcoind to ensure maximum work success - yeah everyone has one of those ... you think that may be what most pools spend a lot of effort doing?

As for bandwidth, if you really want to reduce it, how is adding a bitcoind into the equation going to do that?

Go stratum - problem solved Smiley

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
jgarzik
Legendary
*
Offline Offline

Activity: 1554
Merit: 1001


View Profile
October 20, 2012, 05:23:33 PM
 #69


1) In general, everybody should be running a full node, just to support the network.  It's not a requirement, obviously, but we need as many full nodes as possible to keep the network resistant to Sybil attacks and such.

2) p2pool is the preferred solution for the community, with centralized pools being second best.  Centralized pools make it much easier to be a miner, at the cost of centralizing network power in the hands of the pool operator.


Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
October 20, 2012, 09:05:19 PM
 #70

Sigh. In that case, with p2pool being 400 GH of a 20TH network, then GBT is only going to benefit 2% of the miners. It's pretty obvious by now you can't force people to do what you think is best. If anything, GBT with non-p2pool pooled mining will make it more likely that pools will not want to pass on lots of transactions and will be worse than getwork.

How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining?

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
firefop
Sr. Member
****
Offline Offline

Activity: 420
Merit: 250


View Profile
October 20, 2012, 09:24:32 PM
 #71

How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining?

That's an idea... have to consult with Luke on that.

I also wanted to point out that if we (or someone anyway) doesn't build a scalable solution per pooled mining with an integrated node... then nobody will be able to include it in the POS machines that will eventually be developed, and missing the chance for millions of additional nodes seems sad to me.


slush
Legendary
*
Offline Offline

Activity: 1372
Merit: 1019



View Profile WWW
October 20, 2012, 09:27:05 PM
 #72

How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining?

How about adding an extension to Stratum that has some kind of mode that sends "source transactions" for given job?

In that way, paranoid users will have a chance to introspect jobs generated by Stratum pools and common users which don't care will have optimized mining protocol.

And yes, I'm working on it already. Important thing in this extension is that list of transactions can be broadcasted "lazily" (=few miliseconds after job updates), because that list is not important for mining itself. So it won't hurt efficiency of mining, but still give a tool for online block template monitoring.

-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
October 20, 2012, 09:37:43 PM
 #73

slush, not a bad idea but I'm actually not questioning stratum here since I obviously think it's the better protocol for pooled mining as I've already implemented it in cgminer. It's just that this is the GBT thread, I have been asked to include GBT support as well and I'm not really happy with the protocol as is.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
October 20, 2012, 09:39:43 PM
 #74

How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining?
That's an idea... have to consult with Luke on that.
Not really, Con or anyone else is welcome to write a draft BIP to enable centralized mining over GBT, and open the topic to discussion. But I think it's possible to adopt Stratum's benefits without losing the decentralization benefits - though I don't think it is a problem that needs to be addressed immediately, nor do I have time to do it myself right now. BIP 22 and 23 were developed by collaboration of the community, and while I may be the primary organizer, there is no reason someone else couldn't take it forward more on their own initiative.

I also wanted to point out that if we (or someone anyway) doesn't build a scalable solution per pooled mining with an integrated node... then nobody will be able to include it in the POS machines that will eventually be developed, and missing the chance for millions of additional nodes seems sad to me.
GBT is pretty scalable for now.

-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
October 20, 2012, 09:54:51 PM
 #75

No, it's quite alright. I have enough to do myself as well and I'm not in the business of helping pool designs. I'll implement it as is and let the mining community determine which pools and with what protocols they prefer.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
October 20, 2012, 10:10:49 PM
 #76

How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining?
That's an idea... have to consult with Luke on that.
Not really, Con or anyone else is welcome to write a draft BIP to enable centralized mining over GBT, and open the topic to discussion. But I think it's possible to adopt Stratum's benefits without losing the decentralization benefits - though I don't think it is a problem that needs to be addressed immediately, nor do I have time to do it myself right now. BIP 22 and 23 were developed by collaboration of the community, and while I may be the primary organizer, there is no reason someone else couldn't take it forward more on their own initiative.
Um - yes you keep making this 'community' comment - but the reality is the so called 'community' is pretty much you and not much else.

I also wanted to point out that if we (or someone anyway) doesn't build a scalable solution per pooled mining with an integrated node... then nobody will be able to include it in the POS machines that will eventually be developed, and missing the chance for millions of additional nodes seems sad to me.
GBT is pretty scalable for now.
Well, unfortunately 'pretty scalable' seems to be 'worse than getwork' and that should already be a MAJOR flag for anyone to see that work NEEDS to be done, if anyone wants to consider 'choice'
Between using Stratum and GBT at the moment that clearly says to use Stratum

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
October 20, 2012, 11:11:22 PM
 #77

I do intend to improve BFGMiner's GBT implementation such that it transparently combines pooled and solo mining, at least.

Sitarow
Legendary
*
Offline Offline

Activity: 1778
Merit: 1043



View Profile
October 20, 2012, 11:20:59 PM
 #78

I do intend to improve BFGMiner's GBT implementation such that it transparently combines pooled and solo mining, at least.

That is planned in release BFGMiner's 3.0?
kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
October 20, 2012, 11:33:22 PM
 #79

I do intend to improve BFGMiner's GBT implementation such that it transparently combines pooled and solo mining, at least.

That's cool, but solo mining should not be an option - it should be mandatory. Else the whole project is just another house of cards.
See now you know what you should do.
Go create an alt-coin that tells everyone what they must be doing and see how that fares.
If you are right, then that alt-coin will outlast BTC and you'll be famous ...................

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
slush
Legendary
*
Offline Offline

Activity: 1372
Merit: 1019



View Profile WWW
October 20, 2012, 11:35:41 PM
 #80

...I think not... ...I think we should not gamble... ...Bitcoin is extreme fragile... ...demand from users... ...I know it can be done... ...it should be mandatory...

Honestly, as I'm reading your posts, I think that you should create separate blockchain with those rules hardcoded. Feel free to name it NaziCoin.

jgarzik
Legendary
*
Offline Offline

Activity: 1554
Merit: 1001


View Profile
October 20, 2012, 11:37:57 PM
 #81

Sigh. In that case, with p2pool being 400 GH of a 20TH network, then GBT is only going to benefit 2% of the miners. It's pretty obvious by now you can't force people to do what you think is best. If anything, GBT with non-p2pool pooled mining will make it more likely that pools will not want to pass on lots of transactions and will be worse than getwork.

How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining?

Definitely interested in extensions that broaden the userbase as much as possible.

Even for bitcoind (rather than a pool server), extensions like this make sense.  The old "getwork" protocol did as much work as possible on the server side, thereby minimizing the miner's work -- notably the miner did not have to care about anything besides the block header itself.  No transactions or other parsing to worry about.

getblocktemplate is intended to replace getwork.  If there are improvements to be made, I'm quite open to that.

Ping me on IRC, if you want to rapidly iterate some changes to make bitcoind's GBT work better for a wider selection of mining softwares.

I did the final getblocktemplate polish, simplifying and cleaning things up from its original BIP 22/getmemorypool state.  If BIP 22 / BIP 23 are currently missing something you want...  I want to put it in there.


Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
October 20, 2012, 11:41:29 PM
 #82

I do intend to improve BFGMiner's GBT implementation such that it transparently combines pooled and solo mining, at least.
That is planned in release BFGMiner's 3.0?
Probably that will have to wait for 3.1... 3.0 is the ASIC release, and that's coming up too soon.

I do intend to improve BFGMiner's GBT implementation such that it transparently combines pooled and solo mining, at least.
That's cool, but solo mining should not be an option - it should be mandatory. Else the whole project is just another house of cards.
Unfortunately, simply making it mandatory isn't practical. The closest we could come is giving strong incentives for doing so. For example, I intend to set up Eligius in such a way that those who are solo mining in combination can profit more by claiming the fees from transactions they accept over the base set Eligius provides.

-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
October 20, 2012, 11:58:43 PM
 #83

Sigh. In that case, with p2pool being 400 GH of a 20TH network, then GBT is only going to benefit 2% of the miners. It's pretty obvious by now you can't force people to do what you think is best. If anything, GBT with non-p2pool pooled mining will make it more likely that pools will not want to pass on lots of transactions and will be worse than getwork.

How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining?

Definitely interested in extensions that broaden the userbase as much as possible.

Even for bitcoind (rather than a pool server), extensions like this make sense.  The old "getwork" protocol did as much work as possible on the server side, thereby minimizing the miner's work -- notably the miner did not have to care about anything besides the block header itself.  No transactions or other parsing to worry about.

getblocktemplate is intended to replace getwork.  If there are improvements to be made, I'm quite open to that.

Ping me on IRC, if you want to rapidly iterate some changes to make bitcoind's GBT work better for a wider selection of mining softwares.

I did the final getblocktemplate polish, simplifying and cleaning things up from its original BIP 22/getmemorypool state.  If BIP 22 / BIP 23 are currently missing something you want...  I want to put it in there.


Thanks. When I get some extended block of time on IRC in the very near future, and you're online, I will most certainly.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
firefop
Sr. Member
****
Offline Offline

Activity: 420
Merit: 250


View Profile
October 21, 2012, 01:30:01 AM
 #84

I do intend to improve BFGMiner's GBT implementation such that it transparently combines pooled and solo mining, at least.
That's cool, but solo mining should not be an option - it should be mandatory. Else the whole project is just another house of cards.
Unfortunately, simply making it mandatory isn't practical. The closest we could come is giving strong incentives for doing so. For example, I intend to set up Eligius in such a way that those who are solo mining in combination can profit more by claiming the fees from transactions they accept over the base set Eligius provides.

Your efforts are noble, but it might be better for you to start working on alt currency, the one with really strong foundation. Let retards like
those 2 above, who despise you, me and anyone else not fitting their utopian views, finish off Bitcoin. Don't waste your time here, seriously.

I really like your prev-post with the graph substrata - you basically said everything I was thinking but couldn't articulate. Thank you for that!

However, I respectfully disagree. We need to encourage people like Luke to stick around, add even more sanity to the development of the network and slowly marginalize those who don't think in the right direction. If you look at the network as a whole, that's already happening. 2 years ago I wasn't here and you'll see more actual computer scientists become involved in development as bitcoin grows.

@Luke - incentives are nice - but I'd settle for having it be the default option - If you don't have time/inclination to build a custom install script to set that all up (when/if it's supported) I'd be happy to create and submit it to you for approval / publication.

kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
October 21, 2012, 02:52:58 AM
 #85

...
We need to encourage people like Luke to stick around, add even more sanity to the development of the network and slowly marginalize those who don't think in the right direction.
...
Sigh - another nut-case.

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
firefop
Sr. Member
****
Offline Offline

Activity: 420
Merit: 250


View Profile
October 21, 2012, 03:46:12 AM
 #86

Sigh - another nut-case.

Hate all you want, but it doesn't invalidate anything I've said. The fact remains that certain posters are more interested in trash talking rather than contributing, while others are overworking themselves to try and get this whole bitcoin thing working to the point where we can make it grow into it's potential.

kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
October 21, 2012, 04:14:44 AM
 #87

Seriously - what are you looking for?

Decentralisation or control?

Pick one.

Saying that you want decentralisation ... except it has to be how you or Luke-jr or whoever else wants it ... isn't decentralisation.
As soon as you start deciding how things like mining MUST be done, you've lost the plot.

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
Inaba
Legendary
*
Offline Offline

Activity: 1260
Merit: 1000



View Profile WWW
October 21, 2012, 04:36:34 AM
 #88

Requiring a full bitcoind instance on a mining node is ludicrous, at least in the current implementation of bitcoind.  It's far, far, far too resource intensive to put on most serious mining setups.  I don't want to run heavy weight computing back end, just to satisfy the ridiculously complex blockchain processing for a machine that is intended to solely mine, which requires few resources.

Now, if you want to redesign a bitcoind that doesn't require heavy computing resources, then I'm listening, but until then, it's a non-starter.  A mining backend is a netbook, DD-WRT router, Raspberry Pi, Sheeva Plug or Android tablet.  Trying to run a full blockchain on any of those would be an exercise in futility.

If you're searching these lines for a point, you've probably missed it.  There was never anything there in the first place.
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
October 21, 2012, 05:59:06 AM
 #89

Requiring a full bitcoind instance on a mining node is ludicrous, at least in the current implementation of bitcoind.  It's far, far, far too resource intensive to put on most serious mining setups.  I don't want to run heavy weight computing back end, just to satisfy the ridiculously complex blockchain processing for a machine that is intended to solely mine, which requires few resources.

Now, if you want to redesign a bitcoind that doesn't require heavy computing resources, then I'm listening, but until then, it's a non-starter.  A mining backend is a netbook, DD-WRT router, Raspberry Pi, Sheeva Plug or Android tablet.  Trying to run a full blockchain on any of those would be an exercise in futility.

Which means you pretty much unintentionally agree with my position: GBT in its current form is not advantageous to miners. Luckily jgarzik is open to modifying it based on what I have recommended. We will be talking about it in the near future.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
P_Shep
Legendary
*
Offline Offline

Activity: 1176
Merit: 1000


View Profile
October 21, 2012, 06:11:59 AM
 #90

Requiring a full bitcoind instance on a mining node is ludicrous, at least in the current implementation of bitcoind.  It's far, far, far too resource intensive to put on most serious mining setups.  I don't want to run heavy weight computing back end, just to satisfy the ridiculously complex blockchain processing for a machine that is intended to solely mine, which requires few resources.

Now, if you want to redesign a bitcoind that doesn't require heavy computing resources, then I'm listening, but until then, it's a non-starter.  A mining backend is a netbook, DD-WRT router, Raspberry Pi, Sheeva Plug or Android tablet.  Trying to run a full blockchain on any of those would be an exercise in futility.


I've been trying to get bitcoind on the router... not having much luck so far. Not entirely given up yet.
Inaba
Legendary
*
Offline Offline

Activity: 1260
Merit: 1000



View Profile WWW
October 21, 2012, 07:09:20 AM
 #91

Well, I've been mining with GBT for about a month now on a lower power netbook with no problems (and no bitcoind), so I'm not sure what you're referring to exactly?


If you're searching these lines for a point, you've probably missed it.  There was never anything there in the first place.
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
October 21, 2012, 07:13:07 AM
 #92

Well, I've been mining with GBT for about a month now on a lower power netbook with no problems (and no bitcoind), so I'm not sure what you're referring to exactly?
https://bitcointalk.org/index.php?topic=108854.msg1284623#msg1284623

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
Inaba
Legendary
*
Offline Offline

Activity: 1260
Merit: 1000



View Profile WWW
October 21, 2012, 06:50:12 PM
 #93

Interesting... I would like to know more about that at some point.  I've not run into any problems and my server load is dramatically decreased.  Regardless, though, I'm not sure that pools ignoring some transactions in favor of others is necessarily a negative thing.  Right now, I limit the amount of transactions I put into any one block and I don't see any major issue with favoring higher transaction fee transactions over lower ones. 

If there are no higher fee transactions waiting, then the lower fee ones will get processed and that's as designed as far as I know.  I'm not saying it's a bad thing to "fix" GBT, but I've not run into any problems so far and I'd like to know at what point GBT becomes more cumbersome than getwork from a transaction standpoint.


If you're searching these lines for a point, you've probably missed it.  There was never anything there in the first place.
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
October 21, 2012, 08:48:23 PM
 #94

Interesting... I would like to know more about that at some point.  I've not run into any problems and my server load is dramatically decreased.  Regardless, though, I'm not sure that pools ignoring some transactions in favor of others is necessarily a negative thing.  Right now, I limit the amount of transactions I put into any one block and I don't see any major issue with favoring higher transaction fee transactions over lower ones. 

If there are no higher fee transactions waiting, then the lower fee ones will get processed and that's as designed as far as I know.  I'm not saying it's a bad thing to "fix" GBT, but I've not run into any problems so far and I'd like to know at what point GBT becomes more cumbersome than getwork from a transaction standpoint.


The issue is the payload the longer a block takes to solve as the number of transactions rises. It would not be surprising if the pool was sending 100k sized payloads with all the transactions after 10 minutes on the block. While the load on the pool is definitely less, the data being spewed out to miners will eventually rise, and the more transactions and more popular bitcoin becomes, the larger the payload grows in a linear fashion with transactions. Eventually sending out 100kb payloads  say every 30 or 60 seconds from the pool to all your miners will become a burden. Prioritising transactions is your prerogative but with this disincentive to sending transactions we will see a deepbit like situation where pool ops will put a static upper limit on the amount of transactions in total they care about including. This is much worse for bitcoin in general. But with an extension to the gbt protocol that more closely resembles the merkle root base of stratum it can easily be fixed.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
October 21, 2012, 10:43:49 PM
 #95

Real world considerations should be part of GBT - not ideal world ...

The GBT data transfer should currently ALWAYS be high at the moment ...

Checking the bitcoind debug.log for the last 3 days:
The smallest poolsz was:
10/20/12 06:58:09 CTxMemPool::accept() : accepted a661e71a68 (poolsz 579)

Yes the minimum outstanding transactions for the last 3 days (which was after block 204102) was 579 waiting txns
(also note that I restarted my bitcoind 4 days ago - so that figure may be low ...)

So unless pools are ignoring lots of transactions, there should never have been a GBT workload sent from any GBT pool with fewer than a few hundred transactions in it for the past 3 days.

... but of course I've very little doubt that isn't the case ... since we know what pool software supports GBT.


Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
Inaba
Legendary
*
Offline Offline

Activity: 1260
Merit: 1000



View Profile WWW
October 22, 2012, 09:54:16 PM
 #96

You forgot to specify what CPU you are using, also how much RAM total you have.  Also your HD usage.. 4.5GB. That is larger than the entire usage of my 500 GH/s farm.  Combined.  That 0 - 2% CPU usage you quote, I'd wager it's a modern Intel or AMD based CPU.  You just go ahead and try to run that on an ARM based CPU or a C6 or something, see how far you get (you'll never finish loading the block chain, new blocks will come in faster than you can process old ones). 

The blockchain is not suitable for lower power environments.  It simply isn't.  The blockchain would have to be reduced to under a few megabytes and RAM usage would have to be a few hundred kilobytes before it becomes feasible to run a full node on a mining rig. 

If you're searching these lines for a point, you've probably missed it.  There was never anything there in the first place.
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
October 22, 2012, 10:11:29 PM
 #97

The blockchain would have to be reduced to under a few megabytes
Getting off-topic here, but this is already possible.

and RAM usage would have to be a few hundred kilobytes
This is completely unreasonable. Merely being a C++ program requires more than a few hundred KB.

Inaba
Legendary
*
Offline Offline

Activity: 1260
Merit: 1000



View Profile WWW
October 23, 2012, 12:32:40 AM
 #98

You forgot to specify what CPU you are using, also how much RAM total you have.  Also your HD usage.. 4.5GB. That is larger than the entire usage of my 500 GH/s farm.  Combined.  That 0 - 2% CPU usage you quote, I'd wager it's a modern Intel or AMD based CPU.  You just go ahead and try to run that on an ARM based CPU or a C6 or something, see how far you get (you'll never finish loading the block chain, new blocks will come in faster than you can process old ones).

Athlon 64 x2 4800+ and 2GB RAM. Bitcoin client mem usage can be as low as few MB and as high as 100+ MB, depending on something,
no idea what exactly. HDD usage value stands for client installation and stuff in datadir. I have no clue about ARM or C6, but here's the
nasty part = you invested huge money into system that can't help Bitcoin network if all pools go down. That's quite a gamble, you know.

The blockchain is not suitable for lower power environments.  It simply isn't.  The blockchain would have to be reduced to under a few megabytes and RAM usage would have to be a few hundred kilobytes before it becomes feasible to run a full node on a mining rig. 

Nowhere I said my suggestion must be implemented right now. Think long-term. Step by step - in right direction - and it can be done.

Are you using the "royal" "I" in this case?  Because I did no such thing - I can stand up a new pool on my local network in about 30 minutes if all the pools suddenly went down.  But anyway, comparing an A64 X2 to an ARM processor is not exactly a fair comparison.  Go ahead and try to load the blockchain on an old netbook and see how far you get before everything grinds to a halt.


If you're searching these lines for a point, you've probably missed it.  There was never anything there in the first place.
DavinciJ15
Hero Member
*****
Offline Offline

Activity: 750
Merit: 500


Bitcoin - helping to end bankster enslavement.


View Profile WWW
October 26, 2012, 09:18:28 PM
 #99

Go ahead and try to load the blockchain on an old netbook and see how far you get before everything grinds to a halt.

I did that on a Asus EEE and it took 2 weeks!
When I turn it on after a day it's about 1 hour to catch up.

Crazy!

Anyhow: Luke-jr!

Personally I think stratum is better, however, if you want to win create a pool back end that's easy to use.   Better doesn't matter when it comes to wanting to accomplish a task.  I want a pool, and so does many other people!  So create a sold backend that's easy to config and setup and you win.

I know you have created Eloipool but I'm willing to bet it's not so easy to setup. Wink

Another option is to create vm for people to download with everything working only thing that the end user needs to do is add a username and password to the worker table and they can mine and shares can be obtained from the shares table.

Just my 2 Satoshis.
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
October 26, 2012, 09:26:14 PM
 #100

Personally I think stratum is better, however, if you want to win
Win is when Stratum undergoes the BIP process and gets feature parity with GBP. As I understand it, slush is taking it down that road now.

create a pool back end that's easy to use.
Yes, I did that about a year ago...

I know you have created Eloipool but I'm willing to bet it's not so easy to setup. Wink
Easier than the alternatives Smiley

Another option is to create vm for people to download with everything working only thing that the end user needs to do is add a username and password to the worker table and they can mine and shares can be obtained from the shares table.
Eh, why do we need passwords? Tongue

DavinciJ15
Hero Member
*****
Offline Offline

Activity: 750
Merit: 500


Bitcoin - helping to end bankster enslavement.


View Profile WWW
October 27, 2012, 10:54:14 PM
 #101

Easier than the alternatives Smiley

I see your point.

Eh, why do we need passwords? Tongue

It was used for HTTP reuse of code but I guess with a new protocol it's not needed anymore eh?  I can't think of any reason why we should have miner passwords.   LOL
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
October 27, 2012, 10:57:12 PM
 #102

Eh, why do we need passwords? Tongue
It was used for HTTP reuse of code but I guess with a new protocol it's not needed anymore eh?  I can't think of any reason why we should have miner passwords.   LOL
Getting off-topic here, but Eloipool has always ignored passwords entirely, and only used username for share logging (ie, no checking against a db).

kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
October 31, 2012, 12:31:59 PM
 #103

So ... I've had a go at a pool OP about ignoring transactions and found to my surprise they aren't the ones choosing the transactions.

GBT is.

Here's a run of 15 block for the said pool and look at how bad this is for the bitcoin network - shocking in my opinion:

Bitcoind's GBT is deciding the txns, and it's regularly making small blocks when the bitcoin network constantly has a very large number of outstanding transactions (>4000)

This of course means that everyone's bitcoind is sitting there with a list of many thousands of unconfirmed transactions waiting to go into blocks ... and it is bitcoind's own GBT that is part of the cause of it ...

Code:
Block     size txns
205771 209,454 311
205767 188,756 499
205766   9,468  32
205729  30,740  55
205724  37,399  58
205706  11,169  27
205699   9,720  28
205689   8,609  28
205683 102,113 245
205671  27,961  20
205670  59,118  69
205668  46,083 114
205667   4,791  15
205656 105,743 194
205654  71,232 188
205643  10,739  23
205631   4,551  13
205627  16,381  38

6 blocks (1/3) in there you could consider large (>100 txn),
6 blocks (1/3) are very small (~10k or less and few txns)

Is this basically saying that GBT doesn't adhere to the fee structure of bitcoind?

Is GBT's aim to have every bitcoind on the BTC network constantly running large (>4000) txn memorypools and keep them growing?

...

I will also point out that the Eligius pool has for many months restricted all it's blocks to a limit of only 32 transactions and to ignore transactions that pay full fees if they are near the lower limit - as stated by the new pool OP.
The old pool OP is the same proponent of the so called decentralisation holy grail GBT ... that is CLEARLY showing itself to be bad for BTC and the BTC network.

I see the link there as expected, not surprising.

The argument given for the 32 transaction limit was that it reduced the risk of orphans (which of course it does) however, the issue there is that larger pools with better software did not have the orphan problem Eligius had and instead of fixing the performance problems of the pool software, Eligius' solution was instead to detriment the BTC network by reducing the number of transactions being confirmed and thus extend confirmation times.

Even the new pool op has clearly stated that his intererst is in short term return, not considering the bad long term effect of such short sighted decisions.
i.e. a quick buck with risk of long term problems vs a long term strategy that will also pay much better in the long run.

I've seen Luke-Jr make many arguments about what's good for BTC but only to find that he himself is a proponent of doing thing that are quite simply and clearly bad for it - and it seems that GBT follows this same path also.

Edit: this also explain why people mining with GBT aren't having the expected network problems of receiving hundreds of transactions with every request for work - since GBT ignores most transactions.

Easy way to see it: ./bitcoind getblocktemplate | grep sigops | wc
Sometimes you get a lot and sometimes you don't get many at all - even though if you check your debug.log it will show thousands of unconfirmed transactions (poolsz)

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
abracadabra
Hero Member
*****
Offline Offline

Activity: 942
Merit: 1001



View Profile
October 31, 2012, 01:29:17 PM
 #104

If GBT is planning on going anywhere, this has got to get fixed. Ignoring transactions is bad.
If this doesn't get fixed, I doubt pools will support GBT.

Waiting to see how LukeJr responds.
DrHaribo
Legendary
*
Offline Offline

Activity: 2422
Merit: 1019


Bitminter.com Operator


View Profile WWW
October 31, 2012, 05:31:58 PM
 #105

bitcoind has always made a decision about which transactions to include in a block. That has nothing to do with GBT which is just a different API for getting this (and other) data out of bitcoind.

By the way, kano, if you have a suggestion for how bitcoin can be improved, you can set up a pull request on github and I'm sure the bitcoin developers will consider it.

▶▶▶ Bitminter.com - Your trusted mining pool since 2011.
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
October 31, 2012, 06:15:43 PM
 #106

If GBT is planning on going anywhere, this has got to get fixed. Ignoring transactions is bad.
Kano is a troll and basically a liar.

GBT is a protocol, it doesn't and can't "ignore transactions".

Bitcoind, by default design, has always ignored spam transactions when it can detect them.
The Bitcoin protocol, by Satoshi's original paper design, specifically depends on miners choosing to ignore transactions that don't meet miner-defined policies.
But again, one of this has anything to do with GBT; transaction acceptance works exactly the same whether you use getwork, GBT, or Stratum, except that GBT moves the final decision for which transactions to include to the miner (note that no mining software supports changing the pool's decisions yet, but I plan to have future versions of BFGMiner support adding more transactions from bitcoind to collect on fees).

Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
October 31, 2012, 07:24:22 PM
 #107

GBT moves the final decision for which transactions to include to the miner

Transactions are included or excluded from the block based on what criterias? Could criterias be made user-selectable or changeable?
As I said, there is no existing implementation of miner-side transaction control yet.

kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
October 31, 2012, 10:05:21 PM
 #108

GBT moves the final decision for which transactions to include to the miner

Transactions are included or excluded from the block based on what criterias? Could criterias be made user-selectable or changeable?
As I said, there is no existing implementation of miner-side transaction control yet.
So now that yet again Luke-Jr has called me a liar ... which is a common thing for you to do and you have never once proven that true when you have said it ... though I have proven it true in reverse about you ... where is the lie in my post?

The only point you have made is that GBT is a protocol and the implementations are the fault of the implementers ...

So who implemented them?

You in your cloned BarbieMiner, you in your pool, and ... "someone" in bitcoind.

I guess the other place to look will be P2Pool - but I don't know if that just gets the results from bitcoind or if it processes those results.

BitMinter blindly accepts the results from bitcoind (according to DrHaribo's response)

--

Luke-Jr if you are going to use you standard method of distributing FUD by hiding behind definitions of words then I guess you are really saying the GBT is simply an idea and no use in the real world.

The implementations are what use it and if the implementations are bad for BTC then GBT is bad for BTC.

The two first implementations I listed are also by you ...

--

I've explained to you and also mentioned to Gavin a suggestion about the problems with bitcoind and dealing with large blocks.
Specifically that each block transferred from one pool to another sends way too much data that most bitcoinds already know and does way too much work that the bitcoinds have already done.
That's as far as I went - and seems a good enough solution.

(Edit: and GBT doesn't even use this simple and obvious idea)

I don't deal with bitcoin code development directly, there are just way too many issues with doing that and I have (made clear) elements of it that I disagree with and twice when I've raised those issues they have been shot down.
Fine - I disagree with the devs - so I don't think throwing code in that direction would be productive.
Ideas and issues are what I do throw in that direction.

Meanwhile, your solution at your pool was to reduce the number of transactions per block to 32 and ignore full paying transactions (Satoshi Dice) and who knows what else because of that.
I guess the reason you would not divulge that information before was that people would then know that your own pool was ignoring full paying transactions and that would show you as the hypocrite you are.

--

Anyway, I've reset both my bitcoinds and will see how big the memorypool is again in a week or two.

One of my 0.7.0 bitcoind started:
09/22/12 08:30:55 Bitcoin version v0.7.0.3-g0a4e67a-beta ()

It first reached 4000 after less than 4 days:
09/26/12 00:04:23 CTxMemPool::accept() : accepted 8f15d5ec48 (poolsz 4000)

The last time it was under 1000: (one month after it started)
10/22/12 13:35:31 CTxMemPool::accept() : accepted 0a8e298047 (poolsz 999)

The last time it was under 2000: (two days after that)
10/24/12 15:41:28 CTxMemPool::accept() : accepted f14b7e229c (poolsz 1999)

The last time it was under 3000: (almost 8.5 hours after that)
10/25/12 00:02:21 CTxMemPool::accept() : accepted 879a991172 (poolsz 2999)

... and it kept growing ...

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
DavinciJ15
Hero Member
*****
Offline Offline

Activity: 750
Merit: 500


Bitcoin - helping to end bankster enslavement.


View Profile WWW
October 31, 2012, 10:52:53 PM
 #109

If GBT is planning on going anywhere, this has got to get fixed. Ignoring transactions is bad.
Kano is a troll and basically a liar.

GBT is a protocol, it doesn't and can't "ignore transactions".

Calling people names does not prove you are correct.

People called me a troll and liar because I said money is created when people go into debt at a bank.  So does someone calling me a name make it true?  Is the documentation found in Modern Money Mechanics created by the banks a lie?

"Of course, they do not really pay out loans from the money they receive as deposits. If they did this, no additional money would be created. What they do when they make loans is to accept promissory notes in exchange for credits to the borrowers' transaction accounts."
Modern Money Mechanics

I understand the truth of our monetary system and this is why I like bitcoins as a solution to our unfair monetary system.  So lets forget the name calling and find out the truth.

Finally I am not sure if it's a bad thing that miners are ignoring transactions as that's part of the market saying... hey you are not paying me enough to include your transaction!

There will always be miners that work for low to no transaction fees because they believe in the system, thus there is no need to be a "government" and attempt to force good behaviour.

Just my 2 satoshis
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
October 31, 2012, 11:17:22 PM
 #110

Calling people names does not prove you are correct.
Ignoring is the standard appropriate way to deal with trolls. I just put that out there so newbies know why he isn't getting a response.
There's no basis to Kano's accusations in any case, so not much really to say about them other than that they're lies.

DavinciJ15
Hero Member
*****
Offline Offline

Activity: 750
Merit: 500


Bitcoin - helping to end bankster enslavement.


View Profile WWW
October 31, 2012, 11:55:13 PM
 #111

Calling people names does not prove you are correct.
Ignoring is the standard appropriate way to deal with trolls. I just put that out there so newbies know why he isn't getting a response.
There's no basis to Kano's accusations in any case, so not much really to say about them other than that they're lies.

I may not agree that it's an issue but Kano is not a troll for voicing his a opinion.  I have a YouTube channel and I get a lot of trolls and Kano sounds nothing like the trolls I get.  I got so many I did a video for them...

http://www.youtube.com/watch?v=8sQaIeHvuM0

It was funny and even the trolls loved it.
firefop
Sr. Member
****
Offline Offline

Activity: 420
Merit: 250


View Profile
November 01, 2012, 12:58:06 AM
 #112

I may not agree that it's an issue but Kano is not a troll for voicing his a opinion.  I have a YouTube channel and I get a lot of trolls and Kano sounds nothing like the trolls I get.  I got so many I did a video for them...

http://www.youtube.com/watch?v=8sQaIeHvuM0

It was funny and even the trolls loved it.

Hey D, big fan.

That being said, Luke's right. Kano's in this for profit more than love (obvious from his post history here if nothing else). In addition he tends to bend the truth past the breaking point quite often. These two statements are fact I'd encourage you to verify their veracity for yourself.

The following is my personal opinion which I happen to believe is right:

The reason that LukeJr supporters tend to discount poster like Kano is the simply because Luke has a technical mind, understands how things work and also is a proponent if not an advocate of the bitcoin network's need to be transparent. That's basically what this entire post is about... Luke (and myself and anyone who loves liberty) would rather take a slightly less streamlined process that we can see through than trust a pool op to inform us of what's happening. So sure, Stratum is less bandwidth at the cost of giving the pool op the ability do whatever they want behind it and the miners never knowing. It makes me wonder what they'd like to hide.

Also, I'd call it trolling when Kano comes here to decry the 'bad design' of GBT when the real issue is exactly what I've posted above... at the very least he's dishonest for not simply making his case for what it is... the desire to open a wedge into bitcoin that can be exploited later either by himself or others... without those of us actually doing the work knowing about it.



-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
November 01, 2012, 12:59:43 AM
 #113

...And where do I fit in? Despite what it may look like, this is a technical discussion about the disadvantages of GBT.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
DavinciJ15
Hero Member
*****
Offline Offline

Activity: 750
Merit: 500


Bitcoin - helping to end bankster enslavement.


View Profile WWW
November 01, 2012, 01:45:02 AM
 #114

I may not agree that it's an issue but Kano is not a troll for voicing his a opinion.  I have a YouTube channel and I get a lot of trolls and Kano sounds nothing like the trolls I get.  I got so many I did a video for them...

http://www.youtube.com/watch?v=8sQaIeHvuM0

It was funny and even the trolls loved it.

Hey D, big fan.

That being said, Luke's right. Kano's in this for profit more than love (obvious from his post history here if nothing else). In addition he tends to bend the truth past the breaking point quite often. These two statements are fact I'd encourage you to verify their veracity for yourself.

The following is my personal opinion which I happen to believe is right:

The reason that LukeJr supporters tend to discount poster like Kano is the simply because Luke has a technical mind, understands how things work and also is a proponent if not an advocate of the bitcoin network's need to be transparent. That's basically what this entire post is about... Luke (and myself and anyone who loves liberty) would rather take a slightly less streamlined process that we can see through than trust a pool op to inform us of what's happening. So sure, Stratum is less bandwidth at the cost of giving the pool op the ability do whatever they want behind it and the miners never knowing. It makes me wonder what they'd like to hide.

Also, I'd call it trolling when Kano comes here to decry the 'bad design' of GBT when the real issue is exactly what I've posted above... at the very least he's dishonest for not simply making his case for what it is... the desire to open a wedge into bitcoin that can be exploited later either by himself or others... without those of us actually doing the work knowing about it.




That being said, Luke's right. Kano's in this for profit more than love (obvious from his post history here if nothing else). In addition he tends to bend the truth past the breaking point quite often. These two statements are fact I'd encourage you to verify their veracity for yourself.

Thanks I will keep reading and Kano's posts.

The reason that LukeJr supporters tend to discount poster like Kano is the simply because Luke has a technical mind, understands how things work and also is a proponent if not an advocate of the bitcoin network's need to be transparent. That's basically what this entire post is about... Luke (and myself and anyone who loves liberty) would rather take a slightly less streamlined process that we can see through than trust a pool op to inform us of what's happening. So sure, Stratum is less bandwidth at the cost of giving the pool op the ability do whatever they want behind it and the miners never knowing. It makes me wonder what they'd like to hide.

Also, I'd call it trolling when Kano comes here to decry the 'bad design' of GBT when the real issue is exactly what I've posted above... at the very least he's dishonest for not simply making his case for what it is... the desire to open a wedge into bitcoin that can be exploited later either by himself or others... without those of us actually doing the work knowing about it.


Good points, I'm not against GetBlockTemplate (GBT) I just want something that works for everyone not just those with a mensa membership card. :p  lol With that said, I understand that bitcoin is a complex protocol to begin with and we don't want pools to consolidate and use the hashing power for evil once there is nothing anyone can do about it.  However that is not an issue right now so stratum is fine transition from GPU/FPGA mining to ASICs.  In the future we will need GBT.

Bottom line I want GBT but stratum gets the job done faster and with less work for pools, miners and software producers.
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
November 01, 2012, 01:50:53 AM
 #115

Bottom line I want GBT but stratum gets the job done faster and with less work for pools, miners and software producers.
In reality, GBT is much less work for pools/miners/software developers to implement, and does everything Stratum does with a little more overhead (that probably won't be a problem for a while).
I agree GBT could benefit from adopting some of Stratum's design concepts (I wish slush participated in the GBT BIP process so these could have been integrated from the start!), but in practice we have plenty of time for that before it's an issue.
IMO, most likely the next-generation mining protocol to replace GBT (and Stratum) will use a persistent TCP connection, fully transparent, and probably based on the original Bitcoin p2p protocol (it has some convenient overlap, and eliminates the small JSON overhead both GBT and Stratum are subject to) - and maybe even a p2p-friendly design.

DavinciJ15
Hero Member
*****
Offline Offline

Activity: 750
Merit: 500


Bitcoin - helping to end bankster enslavement.


View Profile WWW
November 01, 2012, 02:00:25 AM
 #116

Bottom line I want GBT but stratum gets the job done faster and with less work for pools, miners and software producers.
In reality, GBT is much less work for pools/miners/software developers to implement, and does everything Stratum does with a little more overhead (that probably won't be a problem for a while).
I agree GBT could benefit from adopting some of Stratum's design concepts (I wish slush participated in the GBT BIP process so these could have been integrated from the start!), but in practice we have plenty of time for that before it's an issue.
IMO, most likely the next-generation mining protocol to replace GBT (and Stratum) will use a persistent TCP connection, fully transparent, and probably based on the original Bitcoin p2p protocol (it has some convenient overlap, and eliminates the small JSON overhead both GBT and Stratum are subject to) - and maybe even a p2p-friendly design.

This is my opinion based on what I have read in the threads here, I have no experience developing code for these mining protocols so you take my opinion for what it's worth. (not even a satoshi) :p

With that said it's easy to conclude that since cgminer and others have implemented stratum over GPT kind of gives my opinion a bit of legs to stand on. 
wizkid057
Legendary
*
Offline Offline

Activity: 1220
Merit: 1000


View Profile
November 01, 2012, 02:16:34 AM
 #117

*snip*
Meanwhile, your solution at your pool was to reduce the number of transactions per block to 32 and ignore full paying transactions (Satoshi Dice) and who knows what else because of that.
I guess the reason you would not divulge that information before was that people would then know that your own pool was ignoring full paying transactions and that would show you as the hypocrite you are.
*snip*

I'm going to just chime in to address this particular comment, if for no other reason but to discredit the kano.

In a conversation yesterday evening this particular issue regarding Eligius was discussed on IRC, and kano was given a link to the post on this forum from months prior stating that these actions were being taken on Eligius.  Now, since I personally pointed this out to him, this post of his is clearly after I pointed this out (October 31, 2012, 10:05:21 PM is after both October 30, 2012, 11:57:55 PM and also well after the original post date below), and yet he still publicly says that this information was "not divulged before"? Well sir, you deserve to be called a liar, and it should be obvious that you are simply trying to discredit the work of others in the community, to what end I do not know.  Want the proof?  I'm not going to just make accusations without proof.  Here it is:

Bitcointalk post from June 17th 2012: https://bitcointalk.org/index.php?topic=23768.msg968819;topicseen#msg968819  This post clearly states publicly what the pool operator at the time decided to do with Satoshi Dice transactions and limiting the transaction count.

*snip*
For now, I am blocking transactions to 1dice* addresses and limiting our blocks to 32 transactions until we've caught up on the extra credit or at least have a viable alternative solution.
*snip*

For reference, here is a snippet of IRC log from yesterday evening (timestamps in UTC-0400):
Code:
[23:54:25] <wizkid057> 2012-10-30 23:57:57,913 merkleMaker     WARNING Making merkle tree with 132 transactions (ideal: 132; max: 132)
[23:54:27] <wizkid057> lol
[23:54:31] <wizkid057> 132 unconfirmed txns
[23:54:35] <wizkid057> without satoshidice
[23:54:36] <wizkid057> :P
[23:54:54] <con_> yet you still limit it to 32 ?
[23:55:00] <@kanoi> so every person on the planet who is running the official bitcoind shouldn't be is what you are saying
[23:55:02] <con_> urgh
[23:55:06] <wizkid057> con_: thats actually my personal pool
[23:55:21] <wizkid057> eligius is still 32 atm... was going to change that once I got the new reward system going
[23:55:41] <wizkid057> kanoi: shouldnt be what?
[23:55:45] <@kanoi> you are now gonna get a post in eligius about this ...
[23:55:52] <wizkid057> processing SD txns?
[23:56:00] <wizkid057> ok?
[23:56:03] <wizkid057> i'm confused
[23:56:06] * wizkid057 scratches head
[23:56:14] <con_> no, just all transactions
[23:56:21] <wizkid057> theres already posts in the eligius threads saying the 32 txn limit, and the SD filtering....
[23:56:24] <wizkid057> nothing new
[23:56:27] <con_> sure
[23:56:42] <@kanoi> yeah but pewople don't put 2+2 together ...
[23:57:49] <con_> again my complaint is the protocol as it stands encourages all pool ops to drop transactions
[23:57:55] <wizkid057> https://bitcointalk.org/index.php?topic=23768.msg968819;topicseen#msg968819 <--- satoshidice filtering ......
(anyone is welcome to entirety of this several hour conversation upon request.)

Tired of all the childishness around here, especially now that I've taken over management of Eligius.  Apparently that means that I get to field the BS that kano usually directs towards Luke-Jr, and I'm not going to tolerate that kind of trash.  It gets really old and is very counter productive.  Stop with the FUD because you're just making yourself look stupid.

-wk

Tips: 1LDQrLr6dPVqNJmpZm82eZVKqDFRk7ERW8
Operator of the Eligius Mining Pool - 0% Fee, SAPPLNS, GBT, Stratum, IRC+Phone Support, Share Market (coming soon), Generation payouts, and more.
Don't feed the trolls. Science Confirms: Internet Trolls Really Are Narcissistic, Psychopathic, and Sadistic (1)
wizkid057
Legendary
*
Offline Offline

Activity: 1220
Merit: 1000


View Profile
November 01, 2012, 02:23:37 AM
 #118

Bottom line I want GBT but stratum gets the job done faster and with less work for pools, miners and software producers.
In reality, GBT is much less work for pools/miners/software developers to implement, and does everything Stratum does with a little more overhead (that probably won't be a problem for a while).
I agree GBT could benefit from adopting some of Stratum's design concepts (I wish slush participated in the GBT BIP process so these could have been integrated from the start!), but in practice we have plenty of time for that before it's an issue.
IMO, most likely the next-generation mining protocol to replace GBT (and Stratum) will use a persistent TCP connection, fully transparent, and probably based on the original Bitcoin p2p protocol (it has some convenient overlap, and eliminates the small JSON overhead both GBT and Stratum are subject to) - and maybe even a p2p-friendly design.

This is my opinion based on what I have read in the threads here, I have no experience developing code for these mining protocols so you take my opinion for what it's worth. (not even a satoshi) :p

With that said it's easy to conclude that since cgminer and others have implemented stratum over GPT kind of gives my opinion a bit of legs to stand on. 

By that logic you should say GBT is better because its actually built directly into the official bitcoin client.  I don't see how stratum is better just because cgminer implemented it.  Just because something is implemented somewhere prominent doesn't make it superior.

Tips: 1LDQrLr6dPVqNJmpZm82eZVKqDFRk7ERW8
Operator of the Eligius Mining Pool - 0% Fee, SAPPLNS, GBT, Stratum, IRC+Phone Support, Share Market (coming soon), Generation payouts, and more.
Don't feed the trolls. Science Confirms: Internet Trolls Really Are Narcissistic, Psychopathic, and Sadistic (1)
kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
November 01, 2012, 02:33:03 AM
 #119

I may not agree that it's an issue but Kano is not a troll for voicing his a opinion.  I have a YouTube channel and I get a lot of trolls and Kano sounds nothing like the trolls I get.  I got so many I did a video for them...

http://www.youtube.com/watch?v=8sQaIeHvuM0

It was funny and even the trolls loved it.

Hey D, big fan.

That being said, Luke's right. Kano's in this for profit more than love (obvious from his post history here if nothing else). In addition he tends to bend the truth past the breaking point quite often. These two statements are fact I'd encourage you to verify their veracity for yourself.

The following is my personal opinion which I happen to believe is right:

The reason that LukeJr supporters tend to discount poster like Kano is the simply because Luke has a technical mind, understands how things work and also is a proponent if not an advocate of the bitcoin network's need to be transparent. That's basically what this entire post is about... Luke (and myself and anyone who loves liberty) would rather take a slightly less streamlined process that we can see through than trust a pool op to inform us of what's happening. So sure, Stratum is less bandwidth at the cost of giving the pool op the ability do whatever they want behind it and the miners never knowing. It makes me wonder what they'd like to hide.

Also, I'd call it trolling when Kano comes here to decry the 'bad design' of GBT when the real issue is exactly what I've posted above... at the very least he's dishonest for not simply making his case for what it is... the desire to open a wedge into bitcoin that can be exploited later either by himself or others... without those of us actually doing the work knowing about it.

... another acolyte.
No I wont ignore that OT rubbish.

Quote
That being said, Luke's right. Kano's in this for profit more than love (obvious from his post history here if nothing else). In addition he tends to bend the truth past the breaking point quite often. These two statements are fact I'd encourage you to verify their veracity for yourself.
i.e. rubbish that you seem to think someone else has to prove and you can't since it's not true.

Please - show me these posts of mine you are talking about.
I'd love to have a read.

... and if you can't show them ... then I guess that shows how pointless your post is.

You do realise that every argument Luke-Jr and I have had about implementation issues in cgminer - he has later either done what I said or say he would do it ...

Transparency?

You've blindly decided to believe what Luke-Jr has said and not ever bothered to check it.

I've found this problem.

As for pools - that is simply you being too lazy to check what they do.

All the transactions that go into a block are shown in the block.
All transactions that do NOT go into a block are there for ANYONE to see.
If you don't want to spend ANY effort seeing what they are (as simple as reading the debug.log of bitcoind) that is your ignorance or laziness, nothing more.

--

Hmm you really gonna guess at my financial situation and motive?
And try to use that as an argument against me Cheesy

firepop - subSTRATA - who's next?
I think the problems of higher difficulty and BTC halving are bringing the nut-cases out into the open.

--

Meanwhile ... as I've implied before ... anyone with a linux bitcoind:

Firstly have a look at the last few lines of your debug.log to see how big your memorypool is
Secondly ./bitcoind getblocktemplate

There are 2 issues here:
1) When getblocktemplate does return a lot of txn's per work request - see how large it is ... then consider what you have to send back in reply with each share you find

2) getblocktemplate also returns a small number of transactions when the memory pool isn't small ... say even when the memory pool is in the thousands ...

Arguing that the protocol design is good yet it allows (and all implementations currently use) this sort of thing is purely deceit.

This is what it currently does.
This needs to be fixed before those who don't check such things and believe the rubbish that Luke-Jr spouts start using it.
Oops - too late it's already in use ... oh well.

Actually, the biggest issue with GBT is the lack of GBT transparency itself in the comments the proponents make.
Ignoring or hiding the points about it that will negatively affect it's users and BTC.

Nice to see Luke-Jr actually mention ... some ... of them ... and yet again hide the important points.

Firstly, GBT is less work to implement ONLY because it uses an inferior method of data transfer that is already available and used by GetWork and bitcoind.

It has MUCH more overhead than Stratum.
Consider the data transfer size relationship to txn numbers.
As you mine until a block change, the txn number will continue to increase.
This leads to something that is in direct opposition to the requirements of BTC.
BTC requires transactions to be confirmed - the more the better.
However, with Stratum the relationship is order log2 N with GBT it is order N
Not only that, the size of the actual items sent are much larger in GBT.
GBT sends the whole transaction each time, Stratum just sends the side of the merkle tree and information for the coinbase transaction.
This give direct incentive to reduce the number of txns confirmed by anyone who implements GBT - including Luke-Jr himself on his own pool

Then of course there is
Quote
IMO, most likely the next-generation mining protocol to replace GBT (and Stratum) will use a persistent TCP connection
What Stratum already does - as he already knows.

Meh lots of posts while I was writing this ...

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
November 01, 2012, 02:39:26 AM
 #120

One more point about the GBT protocol. If a miner chooses which transactions they include rather than accept whichever transactions the pool/bitcoind has offered in the template, the miner has to submit all the transactions with every share he submits.

A quick check
./bitcoind getblocktemplate
at this very second produces a output that is 469441 bytes!

If every template request was this size and the miner used the customise coinbase option and chose transactions, they would be receiving 400kb for the block template and each share submitted would be 400kb unless they started withholding transactions.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
November 01, 2012, 02:44:14 AM
 #121

One more point about the GBT protocol. If a miner chooses which transactions they include rather than accept whichever transactions the pool/bitcoind has offered in the template, the miner has to submit all the transactions with every share he submits.
No, that's (part of) what Block Proposal is for.

firefop
Sr. Member
****
Offline Offline

Activity: 420
Merit: 250


View Profile
November 01, 2012, 05:29:08 AM
 #122

...And where do I fit in? Despite what it may look like, this is a technical discussion about the disadvantages of GBT.

imo - you're at least honest in your intent and presentation.

I'm not saying that GBT couldn't or shouldn't be replaced with something that is better that what it is now... I'm just saying given the choice between gbt and stratum --- I'll take transparent and clunkier over obfuscated and streamlined.

Also I'm not saying 'kill stratum with fire' but I won't be using it until the transparency issue is addressed.


-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
November 01, 2012, 05:34:05 AM
 #123

...And where do I fit in? Despite what it may look like, this is a technical discussion about the disadvantages of GBT.

imo - you're at least honest in your intent and presentation.

I'm not saying that GBT couldn't or shouldn't be replaced with something that is better that what it is now... I'm just saying given the choice between gbt and stratum --- I'll take transparent and clunkier over obfuscated and streamlined.

Also I'm not saying 'kill stratum with fire' but I won't be using it until the transparency issue is addressed.
The only apparent transparency issue with stratum is that it does not transmit the transactions included in the merkle branches by default. You do know, however, that it supports a get transaction method as well? Then you can reconstruct the merkle branches for yourself and confirm it does what it says.

The reality is that 99% of miners will mine blindly trusting their pool, and they rely on the 1% of savvy users to keep the pools in check. It is a mechanism which has worked so far and will continue to do so. There is a way to confirm the pool is doing what you hope it's doing with stratum as well, and enforcing the extra information on the other 99% of users is pointless, as they're not even going to look at it.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
wizkid057
Legendary
*
Offline Offline

Activity: 1220
Merit: 1000


View Profile
November 01, 2012, 05:36:39 AM
 #124

...And where do I fit in? Despite what it may look like, this is a technical discussion about the disadvantages of GBT.

imo - you're at least honest in your intent and presentation.

I'm not saying that GBT couldn't or shouldn't be replaced with something that is better that what it is now... I'm just saying given the choice between gbt and stratum --- I'll take transparent and clunkier over obfuscated and streamlined.

Also I'm not saying 'kill stratum with fire' but I won't be using it until the transparency issue is addressed.


I agree with this assessment. con is at least providing constructive criticism and suggestions for improvements, which seem to make sense.   I agree that GBT needs some work to be perfect, but, it's not a terrible implementation even as it sits, despite what people may try to make it look like.

As far as I'm concerned, GBT would be damn near perfect with the following improvements:

  • Reduce redundant data transfer, specifically re-transmitting transaction data when both the pool and the miner know that the other already has this data. (BIP0023 attempts to address this, but could be improved more.)
  • Ditch HTTP.  While using HTTP makes deployment faster, it needs to die for mining. Really. Stratum got it correct in this regard.
  • Miner-side support for all of the features of GBT, such as transaction monitoring, viewing of the coinbase txn, specify miner desired transaction processing policies, etc.

Not that GBT isn't good even as it is, but, if the above is done, then GBT will be indisputably awesome. Smiley

-wk

Tips: 1LDQrLr6dPVqNJmpZm82eZVKqDFRk7ERW8
Operator of the Eligius Mining Pool - 0% Fee, SAPPLNS, GBT, Stratum, IRC+Phone Support, Share Market (coming soon), Generation payouts, and more.
Don't feed the trolls. Science Confirms: Internet Trolls Really Are Narcissistic, Psychopathic, and Sadistic (1)
DrHaribo
Legendary
*
Offline Offline

Activity: 2422
Merit: 1019


Bitminter.com Operator


View Profile WWW
November 01, 2012, 06:57:28 AM
 #125

It's unfortunate that we now have two competing standards. It's more unfortunate that another useless bitcointalk war is breaking out over it.

As usual many of the arguments are just not true, or misunderstood. "You have to send in all the transactions with every share. Think of all those bytes!" No, you don't.

When Luke-Jr made a public proposal and asked for comments, I read through it and sent in my thoughts on how it could be improved. Why didn't more people do that? Instead waiting until there are implementations in production and then finally spitting out nasty words and technical misunderstandings. That's not the way to go about things.

You have to ask yourself: Do I want to be someone with a positive attitude who finds ways to improve things and push Bitcoin forward? Or do I want to be some useless clockwinder who sabotages Bitcoin with FUD, lies and nonsense?

Trolling and spreading FUD may seem like fun. But remember that a lot of newbies who have just discovered Bitcoin come here and read all your nonsense. Also, some of us are trying to build something.

▶▶▶ Bitminter.com - Your trusted mining pool since 2011.
wizkid057
Legendary
*
Offline Offline

Activity: 1220
Merit: 1000


View Profile
November 01, 2012, 06:59:05 AM
 #126

It's unfortunate that we now have two competing standards. It's more unfortunate that another useless bitcointalk war is breaking out over it.

As usual many of the arguments are just not true, or misunderstood. "You have to send in all the transactions with every share. Think of all those bytes!" No, you don't.

When Luke-Jr made a public proposal and asked for comments, I read through it and sent in my thoughts on how it could be improved. Why didn't more people do that? Instead waiting until there are implementations in production and then finally spitting out nasty words and technical misunderstandings. That's not the way to go about things.

You have to ask yourself: Do I want to be someone with a positive attitude who finds ways to improve things and push Bitcoin forward? Or do I want to be some useless clockwinder who sabotages Bitcoin with FUD, lies and nonsense?

Trolling and spreading FUD may seem like fun. But remember that a lot of newbies who have just discovered Bitcoin come here and read all your nonsense. Also, some of us are trying to build something.


+1 * 10^256 Wink

Tips: 1LDQrLr6dPVqNJmpZm82eZVKqDFRk7ERW8
Operator of the Eligius Mining Pool - 0% Fee, SAPPLNS, GBT, Stratum, IRC+Phone Support, Share Market (coming soon), Generation payouts, and more.
Don't feed the trolls. Science Confirms: Internet Trolls Really Are Narcissistic, Psychopathic, and Sadistic (1)
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
November 01, 2012, 07:08:04 AM
 #127

As usual many of the arguments are just not true, or misunderstood. "You have to send in all the transactions with every share. Think of all those bytes!" No, you don't.
Please read my quote. I did NOT say that.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
November 01, 2012, 07:10:42 AM
 #128

And if you blame me for not paying attention at the right time earlier on when this protocol was developed well I'm terribly sorry for not keeping track of every goddamn fucking thing going on in the bitcoin world.

I will implement whatever the fuck this protocol ends up being one way or another.

Sorry for offering suggestions, I will now say no more on this.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
sharky112065
Sr. Member
****
Offline Offline

Activity: 382
Merit: 250



View Profile
November 01, 2012, 07:33:14 AM
 #129

C. Kolivas was going through a devastating family matter at the time of those discussions. We are lucky he has returned at all to continue his work on Cgminer.

People seem to blur the lines that he and Kano are the same person. They do not always agree (Just observe the cgminer IRC channel a bit and you will realize that) on everything.

If I recall, at first he did not like either solution (Stratum or GBT) and he picked the one he had the less issues with to implement first.

I'm not as into the technical end of mining and its underlying code, but from my point of view, all I'm seeing is a lot of bickering and not much getting done to improve and or fix the issues that are causing reservation on implementing GBT.

I hope everyone involved can come together and make both Stratum and GBT work out, because it looks like both will be around for a while.

Donations welcome: 12KaKtrK52iQjPdtsJq7fJ7smC32tXWbWr
DrHaribo
Legendary
*
Offline Offline

Activity: 2422
Merit: 1019


Bitminter.com Operator


View Profile WWW
November 01, 2012, 08:14:06 AM
 #130

My post above was not only about one person or even one side of this. There seem to be two camps fighting rather than improving things.

When you come in late after the discussion has been going on for a long time and things have now even been implemented, you say "I'm sorry I'm so late with this. I've been busy with something else and didn't have time to look through this until now. But it seems to me that X could be done better if you do it more like Y." Notice the polite form and also not just saying that something is bad, but making suggestions on how to improve it.

You don't say "your shit sucks. Fuck this and fuck that." It's bad form and completely unnecessary.

▶▶▶ Bitminter.com - Your trusted mining pool since 2011.
kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
November 01, 2012, 10:36:22 AM
 #131

My post above was not only about one person or even one side of this. There seem to be two camps fighting rather than improving things.

When you come in late after the discussion has been going on for a long time and things have now even been implemented, you say "I'm sorry I'm so late with this. I've been busy with something else and didn't have time to look through this until now. But it seems to me that X could be done better if you do it more like Y." Notice the polite form and also not just saying that something is bad, but making suggestions on how to improve it.

You don't say "your shit sucks. Fuck this and fuck that." It's bad form and completely unnecessary.

Your post further above was rubbish.

The only point you made in it about comments against stratum was actually completely false
(as ckolivas pointed out to you)

What arguments are not true? Or misunderstood? - you gonna go make that statement and not even back it up?

Seriously, you standing there up on high, telling people what they should be doing, is not gonna win you anything.
All it says is that you have no input into the discussion/argument/whatever words you feel the urge to use and should, quite simply, butt out.

If you have arguments to make against the issues being brought up, go ahead make them.

Otherwise be gone - you telling people how they should do this is pointless drivel as is wizkid posting an even more pointless follow up to your post.

You don't like the way I bring up issues and and get a response from those involved? Damn shame about that.
It works and it makes people listen and either it gets resolved or it gets swept under the carpet.

If you think that BTC software should only take noticed of polite happy fairy statements from polite friendly programmers, then you are seriously deluded.

Yeah I'll say it - provide some useful input into the argument/discussion or fuck off.

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
slush
Legendary
*
Offline Offline

Activity: 1372
Merit: 1019



View Profile WWW
November 01, 2012, 11:18:07 AM
 #132

Also I'm not saying 'kill stratum with fire' but I won't be using it until the transparency issue is addressed.

Hm, maybe I didn't say it loudly enough, but this has been solved already: https://bitcointalk.org/index.php?topic=108533.msg1287719#msg1287719

This stratum extension give a full control online to miners about transactions included in the block template. So from side of openess, Stratum and GBT are both equal. Just Stratum is a bit more optimized ;-).

slush
Legendary
*
Offline Offline

Activity: 1372
Merit: 1019



View Profile WWW
November 01, 2012, 11:28:28 AM
 #133

When Luke-Jr made a public proposal and asked for comments, I read through it and sent in my thoughts on how it could be improved. Why didn't more people do that?

I did not so for three reasons:

a) I had no idea how it has been supposed to work. And I still don't understand majority of his proposal, optional parts etc. I understand GBT to point that I can load block template and put valid block back to bitcoind, but the rest seems to be a rubbish for me.
b) If I'd try to help Luke's with new mining protocol, then I'll suggest Stratum. Obviously there were different targets; Luke wanted something which can be implement in bitcoind, but I wanted something optimized for pooled mining. I'm saying it again and again, that Stratum and GBT can live together and only few people have some mental issues with it.
c) It is very hard to discuss with Luke, so creating my own solution which included all my best ideas was a bit easier ;-).

DrHaribo
Legendary
*
Offline Offline

Activity: 2422
Merit: 1019


Bitminter.com Operator


View Profile WWW
November 01, 2012, 12:25:58 PM
 #134

If you think that BTC software should only take noticed of polite happy fairy statements from polite friendly programmers, then you are seriously deluded.

So you think being polite with other people is a fairy land?

What I am saying is that if we work together rather than against each other, then we will get much further.

Yeah I'll say it - provide some useful input into the argument/discussion or fuck off.

I was in this discussion long before you and made contributions before anyone had this in production. With every thing I didn't like I sent in a suggestion for how to improve it. Many, if not all, of my suggestions went in the spec. Now you are telling me to fuck off, because I don't contribute? I think we have different opinions about what a contribution is.

I think your main concern right now is that GBT doesn't provide enough transactions. Correct? Actually that's not an issue with the protocol. If bitcoind or a pool gives you 1 or 1000 transactions is up to bitcoind or the pool, not this interface.

You may have a valid concern that bitcoind is sometimes holding a lot of transactions in the memory pool without wanting to put them in a block with getwork/getmemorypool/getblocktemplate. Maybe we should take that to a different thread?

▶▶▶ Bitminter.com - Your trusted mining pool since 2011.
DavinciJ15
Hero Member
*****
Offline Offline

Activity: 750
Merit: 500


Bitcoin - helping to end bankster enslavement.


View Profile WWW
November 01, 2012, 12:29:55 PM
 #135

With that said it's easy to conclude that since cgminer and others have implemented stratum over GPT kind of gives my opinion a bit of legs to stand on. 

By that logic you should say GBT is better because its actually built directly into the official bitcoin client.  I don't see how stratum is better just because cgminer implemented it.  Just because something is implemented somewhere prominent doesn't make it superior.

Uh, do you know why VHS beat out Betamax?  Simular reason the VHS tapes where created by other businesses at no cost and less issues. Thus Stratum is implemented and used with low cost and less issues that does not make it better it just makes it what the market wants.

A quick check
./bitcoind getblocktemplate
at this very second produces a output that is 469441 bytes!

If every template request was this size and the miner used the customise coinbase option and chose transactions, they would be receiving 400kb for the block template and each share submitted would be 400kb unless they started withholding transactions.

Dam!  If this is true then I would have to ask what nut bar would use this in a pool?  Your network trafic will skyrocket.

One more point about the GBT protocol. If a miner chooses which transactions they include rather than accept whichever transactions the pool/bitcoind has offered in the template, the miner has to submit all the transactions with every share he submits.
No, that's (part of) what Block Proposal is for.

That's not implemented yet is it?

When Luke-Jr made a public proposal and asked for comments, I read through it and sent in my thoughts on how it could be improved. Why didn't more people do that? Instead waiting until there are implementations in production and then finally spitting out nasty words and technical misunderstandings. That's not the way to go about things.

Please share with us all how an open source project where anyone can create anything can be followed by anyone that maybe interested in it?  Note: I only found out that Stratum when I was told on another forum that Bitcoind will not work with my mini rig.  Then I discovered GBT.
 
There should be a bitcoin developer news web site so those that want to see technical announcements so people like me can find planned advancements that effect them.  To bad there isn't so we work with what we have.


Also I'm not saying 'kill stratum with fire' but I won't be using it until the transparency issue is addressed.

Hm, maybe I didn't say it loudly enough, but this has been solved already: https://bitcointalk.org/index.php?topic=108533.msg1287719#msg1287719

This stratum extension give a full control online to miners about transactions included in the block template. So from side of openess, Stratum and GBT are both equal. Just Stratum is a bit more optimized ;-).

Nice! But why do I feel like a I'm watching a boxing fight and you just rang the bell to start another round of the fight? Smiley  lol

but the rest seems to be a rubbish for me.

Oh never mind.  Try to remember Slush that GBT may have some of Luke-jr's best ideas and calling it "rubbish" kind of stings.

RETARDS!!!

We do what we physically can and what the market wants, if that's mentally defective then please do it better on your own and prosper.  Your prosperity will will help us all by providing proper solutions for us to follow and providing better value to the market than these guys can.

If you believe the market will not want your product or service because it's mentally challenged and must be forced to do so, then I must point you to the success of such ideas called U.S.S.R. and Obama-nation. Cheesy
flynn
Hero Member
*****
Offline Offline

Activity: 728
Merit: 540



View Profile
November 01, 2012, 01:03:55 PM
 #136

I have a question.
I must admit I didn't read everything about pools handling asics, but, ...

If solving blocks is roughly returning blocks with the first 53 bits = 0 (well, a bit more), pools request miners to return computed blocks with the first 32 bits = 0.

Why not simply make pools specialized for asics which would request returning blocks with the first (32+8) 40 bits = 0 ? that would divide the numbers of returned blocks by 256 and make everyone happy.

I probably missed something. Can someone explain the n00b I am why this would be bad .. ?

intentionally left blank
DrHaribo
Legendary
*
Offline Offline

Activity: 2422
Merit: 1019


Bitminter.com Operator


View Profile WWW
November 01, 2012, 01:15:52 PM
 #137

Please share with us all how an open source project where anyone can create anything can be followed by anyone that maybe interested in it?  Note: I only found out that Stratum when I was told on another forum that Bitcoind will not work with my mini rig.  Then I discovered GBT.
 
There should be a bitcoin developer news web site so those that want to see technical announcements so people like me can find planned advancements that effect them.  To bad there isn't so we work with what we have.

That's a good point. And you basically answer it yourself. An alternative could be mailing lists. Either way it's probably important to make that a developer-only resource to get a good signal to noise ratio.

▶▶▶ Bitminter.com - Your trusted mining pool since 2011.
kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
November 01, 2012, 01:29:20 PM
 #138

If you think that BTC software should only take noticed of polite happy fairy statements from polite friendly programmers, then you are seriously deluded.

So you think being polite with other people is a fairy land?

What I am saying is that if we work together rather than against each other, then we will get much further.
Lulz - I can post at 2.6 and still see you've posted crap.

Post some grounds for your delusions or ... yeah I've already said it ... oh great you even quoted it below Smiley

Give some grounds for this crap
Quote
It's unfortunate that we now have two competing standards. It's more unfortunate that another useless bitcointalk war is breaking out over it.

As usual many of the arguments are just not true, or misunderstood. "You have to send in all the transactions with every share. Think of all those bytes!" No, you don't.
or be gone.
Quote
Yeah I'll say it - provide some useful input into the argument/discussion or fuck off.

I was in this discussion long before you and made contributions before anyone had this in production. With every thing I didn't like I sent in a suggestion for how to improve it. Many, if not all, of my suggestions went in the spec. Now you are telling me to fuck off, because I don't contribute? I think we have different opinions about what a contribution is.
I don't care if you designed GBT ... ... ... well actually that would be even worse coz that would make you Jesus.

Hell, my ancestor wrote the most important book in Christianity after the damn bible, but so what, who gives a crap about the past.
What matters is that part I just quoted above, not what blame you have in the past for helping creating this abortion called GBT.
Quote
I think your main concern right now is that GBT doesn't provide enough transactions. Correct? Actually that's not an issue with the protocol. If bitcoind or a pool gives you 1 or 1000 transactions is up to bitcoind or the pool, not this interface.

You may have a valid concern that bitcoind is sometimes holding a lot of transactions in the memory pool without wanting to put them in a block with getwork/getmemorypool/getblocktemplate. Maybe we should take that to a different thread?

My concerns have be stated.

Your pool ran 15 blocks that I pointed out, and of them, only 5 were large txn count/size.
5 were tiny. 5 were in the middle.

You claimed that you don't choose the transactions, GBT does ... wow Smiley

Next, GBT currently usually sends more data per minute than a GetWork pool with roll-n-time (probably even if it is ignoring transactions or there are few transactions on the network)
This is due to the fact that it sends all the details of the all transactions you are processing (no matter what, you cannot disable this and request a merkle tree)

Next, GBT uses HTTP not raw sockets, so it is slower and is also expected to have more rejected shares due to this since it isn't a constant connection.
Not using HTTP, Luke-Jr has stated as a negative in Stratum coz it takes more effort to implement a better protocol ... wow gotta love that argument.
Sounds like "Don't implement something better coz it will take more effort"
Or even "Luke-Jr sux at programming and can't do anything that isn't simple"

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
November 01, 2012, 03:13:38 PM
 #139

When Luke-Jr made a public proposal and asked for comments, I read through it and sent in my thoughts on how it could be improved. Why didn't more people do that?

I did not so for three reasons:

a) I had no idea how it has been supposed to work. And I still don't understand majority of his proposal, optional parts etc. I understand GBT to point that I can load block template and put valid block back to bitcoind, but the rest seems to be a rubbish for me.
Why?
b) If I'd try to help Luke's with new mining protocol, then I'll suggest Stratum.
That's fine, Stratum seems like a logical place to start for a new standard. It would have been nice if these things had taken place during the GBT standardization, but that's in the past now. How goes getting StratumMP BIP process off the ground? Are we still waiting on genjix?

Obviously there were different targets; Luke wanted something which can be implement in bitcoind, but I wanted something optimized for pooled mining. I'm saying it again and again, that Stratum and GBT can live together and only few people have some mental issues with it.
You keep saying this, and I keep having to correct you: No, the main focus of GBT was from the start was pooled mining. Citing the Motivation section of BIP 22 from all the way back in the original February draft... "There is reasonable concerns about mining currently being too centralized on pools, and the amount of control these pools hold. By exposing the details of the block proposals to the miners, they are enabled to audit and possibly modify the block before hashing it."

c) It is very hard to discuss with Luke, so creating my own solution which included all my best ideas was a bit easier ;-).
This is just trolling.

Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
November 01, 2012, 03:26:28 PM
 #140

One more point about the GBT protocol. If a miner chooses which transactions they include rather than accept whichever transactions the pool/bitcoind has offered in the template, the miner has to submit all the transactions with every share he submits.
No, that's (part of) what Block Proposal is for.
That's not implemented yet is it?
Allowing miners to modify transactions is not implemented in miners yet at all. Block Proposal only makes sense as part of that. I do have it implemented for the Eloipool<->bitcoind link.

When Luke-Jr made a public proposal and asked for comments, I read through it and sent in my thoughts on how it could be improved. Why didn't more people do that? Instead waiting until there are implementations in production and then finally spitting out nasty words and technical misunderstandings. That's not the way to go about things.
Please share with us all how an open source project where anyone can create anything can be followed by anyone that maybe interested in it?  Note: I only found out that Stratum when I was told on another forum that Bitcoind will not work with my mini rig.  Then I discovered GBT.
There is a Bitcoin development mailing list specifically for discussing all things related to Bitcoin development. GBT had a number of threads of discussion on it since February.

DrHaribo
Legendary
*
Offline Offline

Activity: 2422
Merit: 1019


Bitminter.com Operator


View Profile WWW
November 01, 2012, 04:50:09 PM
 #141

Lulz - I can post at 2.6 and still see you've posted crap.

Being polite and working together towards a common goal is crap? I am surprised that you participate in an open source project.

Give some grounds for this crap
Quote
It's unfortunate that we now have two competing standards. It's more unfortunate that another useless bitcointalk war is breaking out over it.

As usual many of the arguments are just not true, or misunderstood. "You have to send in all the transactions with every share. Think of all those bytes!" No, you don't.
or be gone.

You like to make a lot of demands, don't you?

ckolivas wrote that if the miner selects transactions you have to send in every transaction with every share. This is incorrect and has been discussed enough already, I think. You said that GBT, a protocol, makes decisions on which transactions to include. How does a protocol make a decision like that? And GBT being "an abortion", I don't think that's entirely correct either.

I don't care if you designed GBT ... ... ... well actually that would be even worse coz that would make you Jesus.

I never said I designed GBT. I said I suggested a few improvements for it. And what does religion have to do with this?

You told me to "fuck off" because I don't contribute. I was merely stating that my contributions are already in the spec. Where are yours?

Your pool ran 15 blocks that I pointed out, and of them, only 5 were large txn count/size.
5 were tiny. 5 were in the middle.

You picked 15 random blocks from BitMinter. They were 5 big blocks, 5 medium and 5 small. And so you come to me and demand an explanation? I find that utterly unreasonable.

If you don't like pools that filter out transactions, then talk to them, not me. I don't put any limits on transactions.

You claimed that you don't choose the transactions, GBT does ... wow Smiley

No, you said that, not me. What I said was that I include all the transactions bitcoind gives me, without filtering out the free ones or putting any limit on the number of transactions, like some pools do.

GBT and bitcoind are not the same thing. If a bug in a web server makes it give you half a file, do you blame HTTP?

Next, GBT currently usually sends more data per minute than a GetWork pool with roll-n-time (probably even if it is ignoring transactions or there are few transactions on the network)
This is due to the fact that it sends all the details of the all transactions you are processing (no matter what, you cannot disable this and request a merkle tree)

Maybe that would be a good improvement for GBT. If the miner doesn't care about seeing the transactions or filtering/adding transactions, request a merkle branch instead that you can use to create a merkle root for mining. Not a single transaction goes over the wire, other than the generation tx.

Next, GBT uses HTTP not raw sockets, so it is slower and is also expected to have more rejected shares due to this since it isn't a constant connection.
Not using HTTP, Luke-Jr has stated as a negative in Stratum coz it takes more effort to implement a better protocol ... wow gotta love that argument.
Sounds like "Don't implement something better coz it will take more effort"
Or even "Luke-Jr sux at programming and can't do anything that isn't simple"

Of course HTTP is a constant connection. Use persistent connections, and pipeline requests for speed. My miner uses 1 long poll connection and 1-3 connections to get work and send in proofs of work. For the work connections it will pipeline up to 10 requests at a time. All the connections are persistent. They get closed only if they have been sitting idle in the connection pool for a while.

The advantage of Stratum is that it is designed to be two-way from the start. Long polling in HTTP feels dirty and hackish.

The advantage of HTTP is all the network infrastructure, proxies, libraries that exist, and all the knowledge that developers have about this protocol. When you already know the protocol, you have a library for using it that you already know well, then it is much easier to get started.

▶▶▶ Bitminter.com - Your trusted mining pool since 2011.
slush
Legendary
*
Offline Offline

Activity: 1372
Merit: 1019



View Profile WWW
November 01, 2012, 04:58:48 PM
 #142

How goes getting StratumMP BIP process off the ground? Are we still waiting on genjix?

No, I'm just busy. Writing sane documentation (in english) is pretty hard for me, so I need a lot of time to finalize something.

"There is reasonable concerns about mining currently being too centralized on pools, and the amount of control these pools hold. By exposing the details of the block proposals to the miners, they are enabled to audit and possibly modify the block before hashing it."

You didn't respond to Stratum's "get_transactions" extension explicitly. Can I ask you what's your opinion to this? I think it solves exactly that issue with blind trust for picking transaction by the pool, without unnecessary overhead for common miners.

This is just trolling.

I don't think so. I prefer coding, not discussion. We can spend months on discussions or we can both show our solutions and let others to decide.

Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
November 01, 2012, 05:07:38 PM
 #143

How goes getting StratumMP BIP process off the ground? Are we still waiting on genjix?
No, I'm just busy. Writing sane documentation (in english) is pretty hard for me, so I need a lot of time to finalize something.
Well, we're just talking about a first draft, it doesn't have to be perfect - the rest of the community will probably have things they want added or changed before the final anyway.

"There is reasonable concerns about mining currently being too centralized on pools, and the amount of control these pools hold. By exposing the details of the block proposals to the miners, they are enabled to audit and possibly modify the block before hashing it."
You didn't respond to Stratum's "get_transactions" extension explicitly. Can I ask you what's your opinion to this? I think it solves exactly that issue with blind trust for picking transaction by the pool, without unnecessary overhead for common miners.
It's a step forward, at least; but the design here still encourages miners and pools to avoid (or bill extra for, as you mentioned) decentralization. I don't think "common miners" should be encouraged to centralize.

I prefer coding, not discussion. We can spend months on discussions or we can both show our solutions and let others to decide.
It's not either one or the other. The BIP process generally involves documenting and implementing a first draft, and then discussion to come up with a final standard solution that meets everyone's needs. Eligius had GBT since the first draft in February, but the protocol continued to change significantly as other developers contributed to it before reaching its current form.

Inaba
Legendary
*
Offline Offline

Activity: 1260
Merit: 1000



View Profile WWW
November 01, 2012, 06:33:46 PM
 #144

When Luke was working on GBT and we were working on implementing a working vardiff, I had some concerns from a security standpoint.  The Stratum get_transactions also raises some red flags for me, can you address security/throttling measures in place to prevent get_transactions being used a DoS method to overload a pool?  Anything that leaves arbitrary data requests up to the client is an avenue for exploit and it is one of the reasons I have been so adamantly against user defined difficulties.  If I want to DoS your pool, I crank up a couple of 1.5 TH minirig, set my difficulty to 1 and ask for getworks. 

Same kind of problem with get_transactions:  I'm pissed at you, so I just request all the get_transactions you'll send me over and over and over.


If you're searching these lines for a point, you've probably missed it.  There was never anything there in the first place.
P_Shep
Legendary
*
Offline Offline

Activity: 1176
Merit: 1000


View Profile
November 01, 2012, 06:38:45 PM
 #145

When Luke was working on GBT and we were working on implementing a working vardiff, I had some concerns from a security standpoint.  The Stratum get_transactions also raises some red flags for me, can you address security/throttling measures in place to prevent get_transactions being used a DoS method to overload a pool?  Anything that leaves arbitrary data requests up to the client is an avenue for exploit and it is one of the reasons I have been so adamantly against user defined difficulties.  If I want to DoS your pool, I crank up a couple of 1.5 TH minirig, set my difficulty to 1 and ask for getworks. 

Same kind of problem with get_transactions:  I'm pissed at you, so I just request all the get_transactions you'll send me over and over and over.



Surely you could keep requesting get_works with a script, why would you need any hashing behind it?

As with the get_transactions, surely you'd just place a limit on requests poolside, and ignore those that seem unreasonable?
kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
November 01, 2012, 08:29:38 PM
 #146

When Luke was working on GBT and we were working on implementing a working vardiff, I had some concerns from a security standpoint.  The Stratum get_transactions also raises some red flags for me, can you address security/throttling measures in place to prevent get_transactions being used a DoS method to overload a pool?  Anything that leaves arbitrary data requests up to the client is an avenue for exploit and it is one of the reasons I have been so adamantly against user defined difficulties.  If I want to DoS your pool, I crank up a couple of 1.5 TH minirig, set my difficulty to 1 and ask for getworks.  

Same kind of problem with get_transactions:  I'm pissed at you, so I just request all the get_transactions you'll send me over and over and over.

You've ignored the reverse situation.

With GBT (all current implementations) the pool must always do this.

With Stratum it is optional and yes some users could request it always.

Since it is an available option, then the pool must consider the load of allowing it.
(... and yes I mean they MUST consider the load, not that they should ignore implementing it ... though any that don't would be easy to identify ... due to their failure to respond any miner that requests it)

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
November 01, 2012, 09:01:25 PM
 #147

Must I quote myself?
If every template request was this size and the miner used the customise coinbase option and chose transactions, they would be receiving 400kb for the block template and each share submitted would be 400kb unless they started withholding transactions.
That IS correct.

I was exploring the possible issues with implementation of GBT and got attacked as though I was trying to spread FUD about GBt. Nowhere did I imply this is what all miners would be doing.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
November 01, 2012, 09:07:06 PM
 #148

Must I quote myself?
If every template request was this size and the miner used the customise coinbase option and chose transactions, they would be receiving 400kb for the block template and each share submitted would be 400kb unless they started withholding transactions.
Too bad you're more interested in ignoring me than having a productive discussion, or you'd have noticed I pointed out exactly why that isn't true.

-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
November 01, 2012, 10:10:09 PM
 #149

When Luke was working on GBT and we were working on implementing a working vardiff, I had some concerns from a security standpoint.  The Stratum get_transactions also raises some red flags for me, can you address security/throttling measures in place to prevent get_transactions being used a DoS method to overload a pool?  Anything that leaves arbitrary data requests up to the client is an avenue for exploit and it is one of the reasons I have been so adamantly against user defined difficulties.  If I want to DoS your pool, I crank up a couple of 1.5 TH minirig, set my difficulty to 1 and ask for getworks.  

Same kind of problem with get_transactions:  I'm pissed at you, so I just request all the get_transactions you'll send me over and over and over.



Surely you could keep requesting get_works with a script, why would you need any hashing behind it?

As with the get_transactions, surely you'd just place a limit on requests poolside, and ignore those that seem unreasonable?
Since the point of get_transactions is simply to confirm the merkle branches, you only need to allow it once per stratum template push. All the stratum pools so far only push a new template every 30 seconds. Limiting get transaction requests to 1 for each template would only allow the miner to ask for it once every 30 seconds.

By the same token, GBT pools should limit how often users can ask for a template since that does send out all the transactions.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
November 01, 2012, 10:19:29 PM
 #150

Lulz - I can post at 2.6 and still see you've posted crap.

Being polite and working together towards a common goal is crap? I am surprised that you participate in an open source project.
May your surprise make you happy.

Quote
Give some grounds for this crap
Quote
It's unfortunate that we now have two competing standards. It's more unfortunate that another useless bitcointalk war is breaking out over it.

As usual many of the arguments are just not true, or misunderstood. "You have to send in all the transactions with every share. Think of all those bytes!" No, you don't.
or be gone.

You like to make a lot of demands, don't you?

ckolivas wrote that if the miner selects transactions you have to send in every transaction with every share. This is incorrect and has been discussed enough already, I think. You said that GBT, a protocol, makes decisions on which transactions to include. How does a protocol make a decision like that? And GBT being "an abortion", I don't think that's entirely correct either.
I still don't see any backup to the point you made "As usual many of the arguments are just not true, or misunderstood."

Quote
I don't care if you designed GBT ... ... ... well actually that would be even worse coz that would make you Jesus.

I never said I designed GBT. I said I suggested a few improvements for it. And what does religion have to do with this?
"In" joke Smiley

Quote
You told me to "fuck off" because I don't contribute. I was merely stating that my contributions are already in the spec. Where are yours?
No, because you made a VERY broad statement without any basis and contributed nothing to the current discussion except to tell people how they should be discussing it.

Quote
Your pool ran 15 blocks that I pointed out, and of them, only 5 were large txn count/size.
5 were tiny. 5 were in the middle.

You picked 15 random blocks from BitMinter. They were 5 big blocks, 5 medium and 5 small. And so you come to me and demand an explanation? I find that utterly unreasonable.

If you don't like pools that filter out transactions, then talk to them, not me. I don't put any limits on transactions.
What is your reason for wording it like that?

No I didn't pick 15 random blocks.
I picked 15 blocks in a row that your pool generated.

Your response was that it was out of your control.

The reason I chose those blocks was because one of them orphaned another block at the same time:

205766
BTC Sent  510.69 BTC
Number Of Transactions  32
By BitMinter

Orphaned:
BTC Sent  15,449.927 BTC
Number Of Transactions  619
By EclipseMC

Yes something is badly wrong there ... and your excuse is that you don't have any rules about the transactions, you just call the GBT interface.

[aside]
This is in fact a big issue with Luke-Jr also - him claiming about how what he is doing is better for BTC.
Yet what he was doing with his pool was restricting the number of transactions at his pool to 32 to increase the payments the pool received rather than fixing the problem with his pool software that was getting more orphans than other bigger pools at the same time.
The guy is a hypocrite - even worse: he enables such a thing at his pool ~5 months ago and the pool is still doing it.
I even explained a sizeable solution to this in bitcoind to both him and Gavin back then ...
[/aside]

Quote
You claimed that you don't choose the transactions, GBT does ... wow Smiley

No, you said that, not me. What I said was that I include all the transactions bitcoind gives me, without filtering out the free ones or putting any limit on the number of transactions, like some pools do.

GBT and bitcoind are not the same thing. If a bug in a web server makes it give you half a file, do you blame HTTP?
Tell me how that statement I said is wrong without hiding behind definitions.
There is no difference between what I said and you said except technicalities.
Yes it seems this statement keeps coming up.
This is a text book theory argument that one can make about something that is just that - a theory.
Yes the GBT proponents are out saying to use GBT because it is great.
The implementations are a measure of how great it is and falling back to this constant "It's only an idea you can't say GBT is bad" is pointless also.

Quote
Next, GBT currently usually sends more data per minute than a GetWork pool with roll-n-time (probably even if it is ignoring transactions or there are few transactions on the network)
This is due to the fact that it sends all the details of the all transactions you are processing (no matter what, you cannot disable this and request a merkle tree)

Maybe that would be a good improvement for GBT. If the miner doesn't care about seeing the transactions or filtering/adding transactions, request a merkle branch instead that you can use to create a merkle root for mining. Not a single transaction goes over the wire, other than the generation tx.
Love it - "MAYBE" Tongue
How blatantly obvious is it?
This is a good example of how it becomes just straight out annoying:

Use this great new protocol - it's better than before ........ except it sends more data than getwork ...... that yeah MAYBE that might be worth improving ...... Tongue
But it's better coz the GBT guys say it is - it's transparent ... oh wait Stratum has the option to send the txn list that is 'missing' also.
Um ... what other excuse can they make up ...
Oh yeah it's just the implementations that suck ... GBT is good for everyone ............ except everyone who is using it with any network between the miner and the work generator ...

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
November 01, 2012, 10:33:18 PM
 #151

This is in fact a big issue with Luke-Jr also - him claiming about how what he is doing is better for BTC.
Yet what he was doing with his pool was restricting the number of transactions at his pool to 32 to increase the payments the pool received rather than fixing the problem with his pool software that was getting more orphans than other bigger pools at the same time.
The guy is a hypocrite - even worse: he enables such a thing at his pool ~5 months ago and the pool is still doing it.
I even explained a sizeable solution to this in bitcoind to both him and Gavin back then ...
The problem isn't with the pool software, as you clearly admit to knowing by mentioning fixing bitcoind. Until bitcoind is fixed and the fix is deployed everywhere, this problem remains.
Good job explaining the solution - even though your explanation doesn't actually solve it. Unfortunately, all contributions to bitcoind are done in peoples' spare time and things like a complete rewrite of the p2p code (which is needed to really fix this) takes a lot of time and skill. When this problem is nearly non-existent by simply filtering out a known DDoS attack, there are also higher priorities.

Oh, and stop trolling that GBT uses more bandwidth than getwork. It doesn't.

kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
November 02, 2012, 12:05:41 AM
 #152

This is in fact a big issue with Luke-Jr also - him claiming about how what he is doing is better for BTC.
Yet what he was doing with his pool was restricting the number of transactions at his pool to 32 to increase the payments the pool received rather than fixing the problem with his pool software that was getting more orphans than other bigger pools at the same time.
The guy is a hypocrite - even worse: he enables such a thing at his pool ~5 months ago and the pool is still doing it.
I even explained a sizeable solution to this in bitcoind to both him and Gavin back then ...
The problem isn't with the pool software, as you clearly admit to knowing by mentioning fixing bitcoind. Until bitcoind is fixed and the fix is deployed everywhere, this problem remains.
Good job explaining the solution - even though your explanation doesn't actually solve it. Unfortunately, all contributions to bitcoind are done in peoples' spare time and things like a complete rewrite of the p2p code (which is needed to really fix this) takes a lot of time and skill. When this problem is nearly non-existent by simply filtering out a known DDoS attack, there are also higher priorities.
OK what doesn't it solve?

The solution is based on the fact that
1) Most bitcoinds will already have every transaction that they are sent (all except the coinbase transaction)
So the protocol is sending a large amount of data that the receiving bitcoind already has and knows it has and doesn't need sent
2) The side effect of this is also that the receiving bitcoind again re-validates each transaction that it already has validated in it's memory pool

EDIT: oops left out this comment.
The problem was not with the bigger pools software, since they were not getting as high a % of orphans as you were AT THE SAME TIME.
So yes is was also your software fail that you then used as an excuse to reduce the number of confirmed txns at your pool.

Quote
Oh, and stop trolling that GBT uses more bandwidth than getwork. It doesn't.
Math fail.

I guess the answer to this is that ANY time you quote figures that say something is better it would be foolish for anyone to believe what you say since you will only using some edge case example that is rare or some implementation design that no one uses.

The only possible way for this to be correct is in one real situation:
When a getwork pool doesn't do roll-n-time and the total number of available transaction on the network is small and the hash rate of the device is high.

In the case where the pool does roll-n-time - getwork transfers a LOT less data than GBT in all but edge case rare situations.

To get the size it's VERY easy:

I am saying the work needs to last 2 minutes.
Processing the same work for even 2 minutes has a negative effect on the transaction confirmation times.
But we'll let it be 2 minutes anyway.
This give a single getwork request (allowing a roll of 7000 seconds - slightly under the limit) a limit of 501GH/s

To compare the two I'll use the bitcoind getwork since it approximates to the the size of a pool getwork quite fairly.

./bitcoind getwork

I got: 590 bytes
(Edit: O(1))

./bitcoind getblocktemplate

I got 141 txns and 177,184 bytes
(Edit: O(n))

(AND my bitcoind also says: accepted da0141dc61 (poolsz 1076) ... that sux)

Anyone do it and look at the difference for any getblocktemplate that has even a few transactions in it.
Consider the example in my last post of 619 transactions .... Tongue

Though of course you could double that getwork number to support 1TH/s ... and double it again to support 2TH/s ... etc. until you finally reach the average getblocktemplate sizes.

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
November 02, 2012, 12:30:44 AM
 #153

This is in fact a big issue with Luke-Jr also - him claiming about how what he is doing is better for BTC.
Yet what he was doing with his pool was restricting the number of transactions at his pool to 32 to increase the payments the pool received rather than fixing the problem with his pool software that was getting more orphans than other bigger pools at the same time.
The guy is a hypocrite - even worse: he enables such a thing at his pool ~5 months ago and the pool is still doing it.
I even explained a sizeable solution to this in bitcoind to both him and Gavin back then ...
The problem isn't with the pool software, as you clearly admit to knowing by mentioning fixing bitcoind. Until bitcoind is fixed and the fix is deployed everywhere, this problem remains.
Good job explaining the solution - even though your explanation doesn't actually solve it. Unfortunately, all contributions to bitcoind are done in peoples' spare time and things like a complete rewrite of the p2p code (which is needed to really fix this) takes a lot of time and skill. When this problem is nearly non-existent by simply filtering out a known DDoS attack, there are also higher priorities.
OK what doesn't it solve?

The solution is based on the fact that
1) Most bitcoinds will already have every transaction that they are sent (all except the coinbase transaction)
So the protocol is sending a large amount of data that the receiving bitcoind already has and knows it has and doesn't need sent
2) The side effect of this is also that the receiving bitcoind again re-validates each transaction that it already has validated in it's memory pool
The problem is that Bitcoin clients take much longer to relay blocks with more transactions, while by (paper) design and economic reasoning, including transactions in blocks should be near zero cost. Bitcoin clients in fact do already cache the validation of transactions they know (Gavin added this in 0.7, so it may be a result of your suggestion). However, the process is still absurdly slow.
When a client receives a block, it does this, in order:
  • Request the block from the first peer to mention it
  • Wait for that peer to relay the block in its entirety (ignoring all other peers in the meantime)
  • Validate the block header
  • Validate every transaction in the block (except the ones cached)
  • Tell all our own peers - in no particular order, one by one - that we now have this block (note that if a particular peer's "transfer buffer" is already full, eg if they have a really crappy connection, bitcoind just STOPS AND WAITS FOR IT before moving on to any other peers)
  • Finally, start looking at all/any requests our peers made since we first heard about this block
  • Eventually, one of these peers will have requested the new block. Send it to them, but again do nothing with any other peers until this upload is complete.
  • When that peer is done, another one probably asked for it in the meantime, upload it to them in the same way.
This is absurdly slow, and with large blocks can take a number of minutes to cross the network. The solution involves:
  • Cache signature verifications (done)
  • Rewrite the Bitcoin peer-to-peer protocol handling to work asynchronously: (this is a LOT of work)
    • if one peer can't receive data fast enough, send to other peers at the same time
    • keep responding to other peers while things are going on
  • Begin relaying blocks as soon as the header is downloaded and verified, before even receiving the rest of the block ourselves - this means the entire network is uploading/downloading the new block in parallel; since the difficulty is stupidly high, that makes any DoS attack impractical (I actually implemented this, before I discovered the previous problem which prevents it from working)
  • Don't even upload transactions the other peer knows about already

Also note that it is wrong to assume everyone already knows about every transaction (except coinbase) before the block is found. Peers randomly discard (and don't relay) transactions without fees, so it's likely some of those will be missing on most nodes.

kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
November 02, 2012, 02:21:06 AM
 #154

This is in fact a big issue with Luke-Jr also - him claiming about how what he is doing is better for BTC.
Yet what he was doing with his pool was restricting the number of transactions at his pool to 32 to increase the payments the pool received rather than fixing the problem with his pool software that was getting more orphans than other bigger pools at the same time.
The guy is a hypocrite - even worse: he enables such a thing at his pool ~5 months ago and the pool is still doing it.
I even explained a sizeable solution to this in bitcoind to both him and Gavin back then ...
The problem isn't with the pool software, as you clearly admit to knowing by mentioning fixing bitcoind. Until bitcoind is fixed and the fix is deployed everywhere, this problem remains.
Good job explaining the solution - even though your explanation doesn't actually solve it. Unfortunately, all contributions to bitcoind are done in peoples' spare time and things like a complete rewrite of the p2p code (which is needed to really fix this) takes a lot of time and skill. When this problem is nearly non-existent by simply filtering out a known DDoS attack, there are also higher priorities.
OK what doesn't it solve?

The solution is based on the fact that
1) Most bitcoinds will already have every transaction that they are sent (all except the coinbase transaction)
So the protocol is sending a large amount of data that the receiving bitcoind already has and knows it has and doesn't need sent
2) The side effect of this is also that the receiving bitcoind again re-validates each transaction that it already has validated in it's memory pool
The problem is that Bitcoin clients take much longer to relay blocks with more transactions, while by (paper) design and economic reasoning, including transactions in blocks should be near zero cost. Bitcoin clients in fact do already cache the validation of transactions they know (Gavin added this in 0.7, so it may be a result of your suggestion). However, the process is still absurdly slow.
When a client receives a block, it does this, in order:
  • Request the block from the first peer to mention it
  • Wait for that peer to relay the block in its entirety (ignoring all other peers in the meantime)
  • Validate the block header
  • Validate every transaction in the block (except the ones cached)
  • Tell all our own peers - in no particular order, one by one - that we now have this block (note that if a particular peer's "transfer buffer" is already full, eg if they have a really crappy connection, bitcoind just STOPS AND WAITS FOR IT before moving on to any other peers)
  • Finally, start looking at all/any requests our peers made since we first heard about this block
  • Eventually, one of these peers will have requested the new block. Send it to them, but again do nothing with any other peers until this upload is complete.
  • When that peer is done, another one probably asked for it in the meantime, upload it to them in the same way.
This is absurdly slow, and with large blocks can take a number of minutes to cross the network. The solution involves:
  • Cache signature verifications (done)
  • Rewrite the Bitcoin peer-to-peer protocol handling to work asynchronously: (this is a LOT of work)
    • if one peer can't receive data fast enough, send to other peers at the same time
    • keep responding to other peers while things are going on
  • Begin relaying blocks as soon as the header is downloaded and verified, before even receiving the rest of the block ourselves - this means the entire network is uploading/downloading the new block in parallel; since the difficulty is stupidly high, that makes any DoS attack impractical (I actually implemented this, before I discovered the previous problem which prevents it from working)
  • Don't even upload transactions the other peer knows about already

Also note that it is wrong to assume everyone already knows about every transaction (except coinbase) before the block is found. Peers randomly discard (and don't relay) transactions without fees, so it's likely some of those will be missing on most nodes.
Anyway, instead to implement it well:

1) Create a new transfer mechanism that can be used as well as the old one and ...
2) Prioritise transferring with the new mechanism first (to peers that support it)
3) Remove out the old one some time later (Remember April? Yeah avoid that)

Now I already stated in my solution that not all transactions will be known by the recipient.
The protocol must allow a response to request the txn list it doesn't have.
However, I also specifically wrote code to report the number of previously known transactions each time in my own bitcoind and also mentioned this as well
So regarding that last sentence you wrote:
it was exceedingly rare for me to get new transactions with the block that I didn't know ... other than the coinbase txn which will always be sent

Anyway I will rewrite this with your points:

Quote
Rewrite the Bitcoin peer-to-peer protocol handling to work asynchronously: (this is a LOT of work)
AND
if one peer can't receive data fast enough, send to other peers at the same time
AND
keep responding to other peers while things are going on
Or, as I said in this post, support both. Read above.
We don't want another soft fork debacle like April ...

Quote
1) Begin relaying blocks as soon as the header is downloaded and verified, before even receiving the rest of the block ourselves - this means the entire network is uploading/downloading the new block in parallel; since the difficulty is stupidly high, that makes any DoS attack impractical (I actually implemented this, before I discovered the previous problem which prevents it from working)
AND
2) Don't even upload transactions the other peer knows about already
Yeah that's what I said already ... the 2) effectively resolves 1).


So ... I take this repeat of my solution ... to mean your statement
Quote
Good job explaining the solution - even though your explanation doesn't actually solve it.
is wrong.

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
November 02, 2012, 04:10:36 AM
 #155

Your "explanation" only minorly improves things, it doesn't fix the problem. That's why it was last on the list of things to do to fix it.

None of what I said suggests any kind of fork is needed. Obviously the internals would support both mechanisms - it just isn't relevant to what needs to be done.

DrHaribo
Legendary
*
Offline Offline

Activity: 2422
Merit: 1019


Bitminter.com Operator


View Profile WWW
November 02, 2012, 08:45:06 AM
 #156

Must I quote myself?
ONLY ignoring luke-jr

It appears that you are missing part of the discussion.

Quote
GBT and bitcoind are not the same thing. If a bug in a web server makes it give you half a file, do you blame HTTP?
Tell me how that statement I said is wrong without hiding behind definitions.
There is no difference between what I said and you said except technicalities.

Come on, that's no technicality. Is this really where this discussion is right now?

If you get a virus over HTTP then you blame HTTP?

If you mine a small block over Stratum then you blame Stratum?

Let's be honest now. Does the GBT protocol/API really force you to make small blocks? Does Stratum?


The implementations are a measure of how great it is and falling back to this constant "It's only an idea you can't say GBT is bad" is pointless also.

Come on, how many transactions you decide to include in a block is up to you, not the protocol. Of course you can say that something about GBT is bad, but can you point at the place in the spec where it tells you that you must not include more than X transactions in a block? Which part of GBT are you actually having a problem with?

Do you want it to be a violation of the spec to mine a block with less than X transactions? If so, present your suggestion and make an argument for it. But I doubt there are many that want GBT to have anything to do with which or how many transactions are chosen for a block. Luckily that is not the case now, even though you claim it is.

Quote
Next, GBT currently usually sends more data per minute than a GetWork pool with roll-n-time (probably even if it is ignoring transactions or there are few transactions on the network)
This is due to the fact that it sends all the details of the all transactions you are processing (no matter what, you cannot disable this and request a merkle tree)

Maybe that would be a good improvement for GBT. If the miner doesn't care about seeing the transactions or filtering/adding transactions, request a merkle branch instead that you can use to create a merkle root for mining. Not a single transaction goes over the wire, other than the generation tx.
Love it - "MAYBE" Tongue
How blatantly obvious is it?

Well, again, this is a way to be polite and open for cooperation with other developers. Instead of saying "THIS is the best solution and you better do things MY WAY" you can say "maybe this can help improve things?" You will find this goes over much better with most people. The way you are acting you look like a very angry man who hates the world and everyone in it. There's no need to say that people suck at programming, nor joke about someone's (?) religion.

If you had both a problem and a solution in your head, why did you present only the problem?

ckolivas said "this is a technical discussion about the disadvantages of GBT". Unless the point is to destroy GBT I think a better way to view it is that this is a technical discussion about what GBT is today and how we can make it better tomorrow.

For the problem of using much of the bandwidth on transactions when some miners don't even care about seeing them:

I guess this could be improved by having the client include something in its capabilities list to tell the server it wishes to mine "blindly" without knowing about transactions. If both client and server has this capability then the server sends a merkle branch instead of a list of transactions.

I'm not sure its entirely clean that if the user wishes to see transactions the client would leave out this capability. It makes the capabilities more like options than an actual list of capabilities, but perhaps that's alright.

What do you think, Luke-Jr and others? It could get bandwidth usage down for users who don't care to see the transactions.

▶▶▶ Bitminter.com - Your trusted mining pool since 2011.
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
November 02, 2012, 10:34:10 AM
 #157

ckolivas said "this is a technical discussion about the disadvantages of GBT". Unless the point is to destroy GBT I think a better way to view it is that this is a technical discussion about what GBT is today and how we can make it better tomorrow.

For the problem of using much of the bandwidth on transactions when some miners don't even care about seeing them:

I guess this could be improved by having the client include something in its capabilities list to tell the server it wishes to mine "blindly" without knowing about transactions. If both client and server has this capability then the server sends a merkle branch instead of a list of transactions.

I'm not sure its entirely clean that if the user wishes to see transactions the client would leave out this capability. It makes the capabilities more like options than an actual list of capabilities, but perhaps that's alright.

What do you think, Luke-Jr and others? It could get bandwidth usage down for users who don't care to see the transactions.
My goodness! Go back 3+ pages on this forum thread and you'll see that's what I was discussing with jgarzik. Yes I was trying to be productive and help find some useful pool mining mode for GBT. All it needs is a request in the protocol for branches.

...this may explain why I was so annoyed yesterday.

EDIT: https://bitcointalk.org/index.php?topic=108854.msg1286483#msg1286483

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
DrHaribo
Legendary
*
Offline Offline

Activity: 2422
Merit: 1019


Bitminter.com Operator


View Profile WWW
November 02, 2012, 11:16:03 AM
 #158


I apologize, I missed that in all the noise.

I support your idea. Not sure if "suggested merkle root" is a typo. I'd suggest that on the wire this is a merkle branch that when combined with your generated coinbase tx gives you the merkle root for the block. You request work once, then send several times block header + coinbase. The server will know which txes / merkle tree this belongs to by looking at your work ID.

I believe you are right that the majority will not care to see or change the transactions, so it is a good optimization for the common case. Also, until pools support changing the transactions, it will be a good optimization in every case.

▶▶▶ Bitminter.com - Your trusted mining pool since 2011.
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
November 02, 2012, 11:30:07 AM
 #159

I guess this could be improved by having the client include something in its capabilities list to tell the server it wishes to mine "blindly" without knowing about transactions. If both client and server has this capability then the server sends a merkle branch instead of a list of transactions.

What do you think, Luke-Jr and others? It could get bandwidth usage down for users who don't care to see the transactions.
This specific method is worse than Stratum's new alternative, since the pool would now know which miners it can safely give dirty jobs to.
I also don't think it's a good idea to encourage further pool centralization by making it any easier for miners to be neglegent.

DrHaribo
Legendary
*
Offline Offline

Activity: 2422
Merit: 1019


Bitminter.com Operator


View Profile WWW
November 02, 2012, 11:46:52 AM
 #160

This specific method is worse than Stratum's new alternative, since the pool would now know which miners it can safely give dirty jobs to.

Ok, I didn't think of that. But really, what is the worst that can happen? Mining 5 txes instead of 500? You can see that after-the-fact and I don't think that's a disaster.

Or maybe I am missing something that "dirty" jobs could contain?

▶▶▶ Bitminter.com - Your trusted mining pool since 2011.
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
November 02, 2012, 11:57:33 AM
 #161


I apologize, I missed that in all the noise.

I support your idea. Not sure if "suggested merkle root" is a typo. I'd suggest that on the wire this is a merkle branch that when combined with your generated coinbase tx gives you the merkle root for the block. You request work once, then send several times block header + coinbase. The server will know which txes / merkle tree this belongs to by looking at your work ID.

I believe you are right that the majority will not care to see or change the transactions, so it is a good optimization for the common case. Also, until pools support changing the transactions, it will be a good optimization in every case.


Thank you very much. Yes it was the concept rather than the details, and branch is a better term. I'm thinking the mining software can support both modes and be started in confirm transaction mode only if desired by the user.

EDIT: or alternatively, randomly inject requests for full transactions.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2576
Merit: 1110


Ruu \o/


View Profile WWW
November 05, 2012, 11:36:18 AM
 #162

Almost implemented the first working version of GBT within cgminer.

This sort of behaviour tells the story well though:

Code:
[2012-11-05 22:28:08] Accepted 23c0ba9a Diff 7/1 GPU 2 pool 2
 [2012-11-05 22:28:08] Accepted 163ec800 Diff 11/1 GPU 0 pool 2
 [2012-11-05 22:28:13] Stratum from pool 3 detected new block
 [2012-11-05 22:28:13] Accepted 61bdf4a7 Diff 2/1 GPU 3 pool 2
 [2012-11-05 22:28:13] Stale share detected, submitting as user requested
 [2012-11-05 22:28:14] Accepted 00feebca Diff 257/1 GPU 1 pool 2
 [2012-11-05 22:28:19] Rejected 70b76eb2 Diff 2/1 GPU 1 pool 2 (stale-prevblk)
 [2012-11-05 22:28:19] Rejected 47b723ef Diff 3/1 GPU 3 pool 2 (stale-prevblk)
 [2012-11-05 22:28:20] Rejected 9644eef7 Diff 1/1 GPU 2 pool 2 (stale-prevblk)
 [2012-11-05 22:28:21] Rejected 4cafa99d Diff 3/1 GPU 1 pool 2 (stale-prevblk)
 [2012-11-05 22:28:22] Rejected 093dc4cd Diff 27/1 GPU 0 pool 2 (stale-prevblk)
 [2012-11-05 22:28:23] GBT LONGPOLL from pool 2 requested work restart
 [2012-11-05 22:28:24] Accepted 375a5178 Diff 4/1 GPU 1 pool 2
 [2012-11-05 22:28:25] Accepted 047206f4 Diff 57/1 GPU 3 pool 2
 [2012-11-05 22:28:25] GBT LONGPOLL from pool 2 requested work restart
 [2012-11-05 22:28:30] Accepted 72bee5f8 Diff 2/1 GPU 0 pool 2
 [2012-11-05 22:28:31] Accepted 8e63856d Diff 1/1 GPU 2 pool 2

Now pool 3 here is slush on stratum (ping time 335ms), while pool 2 is EMC on GBT (ping time 225ms).

Note how stratum picks up the block change and notifies me 10 seconds before GBT does. However, this is not the GBT pool simply learning of the block change later, because it starts rejecting my shares as being from the previous block before I even got the longpoll from the GBT pool. Then of course there's a second longpoll with the transactions, which is not an unusual practice.


Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
November 05, 2012, 12:26:57 PM
 #163

Almost implemented the first working version of GBT within cgminer.

This sort of behaviour tells the story well though:

Code:
[2012-11-05 22:28:08] Accepted 23c0ba9a Diff 7/1 GPU 2 pool 2
 [2012-11-05 22:28:08] Accepted 163ec800 Diff 11/1 GPU 0 pool 2
 [2012-11-05 22:28:13] Stratum from pool 3 detected new block
 [2012-11-05 22:28:13] Accepted 61bdf4a7 Diff 2/1 GPU 3 pool 2
 [2012-11-05 22:28:13] Stale share detected, submitting as user requested
 [2012-11-05 22:28:14] Accepted 00feebca Diff 257/1 GPU 1 pool 2
 [2012-11-05 22:28:19] Rejected 70b76eb2 Diff 2/1 GPU 1 pool 2 (stale-prevblk)
 [2012-11-05 22:28:19] Rejected 47b723ef Diff 3/1 GPU 3 pool 2 (stale-prevblk)
 [2012-11-05 22:28:20] Rejected 9644eef7 Diff 1/1 GPU 2 pool 2 (stale-prevblk)
 [2012-11-05 22:28:21] Rejected 4cafa99d Diff 3/1 GPU 1 pool 2 (stale-prevblk)
 [2012-11-05 22:28:22] Rejected 093dc4cd Diff 27/1 GPU 0 pool 2 (stale-prevblk)
 [2012-11-05 22:28:23] GBT LONGPOLL from pool 2 requested work restart
 [2012-11-05 22:28:24] Accepted 375a5178 Diff 4/1 GPU 1 pool 2
 [2012-11-05 22:28:25] Accepted 047206f4 Diff 57/1 GPU 3 pool 2
 [2012-11-05 22:28:25] GBT LONGPOLL from pool 2 requested work restart
 [2012-11-05 22:28:30] Accepted 72bee5f8 Diff 2/1 GPU 0 pool 2
 [2012-11-05 22:28:31] Accepted 8e63856d Diff 1/1 GPU 2 pool 2

Now pool 3 here is slush on stratum (ping time 335ms), while pool 2 is EMC on GBT (ping time 225ms).

Note how stratum picks up the block change and notifies me 10 seconds before GBT does. However, this is not the GBT pool simply learning of the block change later, because it starts rejecting my shares as being from the previous block before I even got the longpoll from the GBT pool. Then of course there's a second longpoll with the transactions, which is not an unusual practice.
Confirming this analysis as accurate... for now. It shouldn't be this much of a difference though, so I suspect there's something going on server-side I need to look into.

kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
November 05, 2012, 12:52:11 PM
 #164

I guess this could be improved by having the client include something in its capabilities list to tell the server it wishes to mine "blindly" without knowing about transactions. If both client and server has this capability then the server sends a merkle branch instead of a list of transactions.

What do you think, Luke-Jr and others? It could get bandwidth usage down for users who don't care to see the transactions.
This specific method is worse than Stratum's new alternative, since the pool would now know which miners it can safely give dirty jobs to.
I also don't think it's a good idea to encourage further pool centralization by making it any easier for miners to be neglegent.
i.e. pools won't want to use GBT since it forces them to always send a TRUCKLOAD of extra data per work item even if the miners don't want it.

So why would any pool use it over Stratum?

... and where are these miners that even show ANYTHING of this TRUCKLOAD ...

Stratum gives the miner the option to allow the dump truck to back up to the house and dump a few hundred K of data that not a single miner reports anything about ... or the miner can not ask for it.
GBT send the dump truck every time, that not a single miner reports anything about the contents - it just uses it to build the merkle tree bigger than what Stratum sends (Stratum only needs to send the side of the tree)

Yes it is a TRUCKLOAD ... and the API stats in cgminer will report this once I get in there and add it after GBT goes live in cgminer ...

You'll be able to load balance 3 pools : Getwork LP, Stratum and GBT and look at how different the numbers are Cheesy Cheesy Cheesy Cheesy

Yeah my post before proving that already no doubt got ignored - but anyone using cgminer will be able to see it soon enough themselves.

This is really just as bad as that idiot subSTRATA.
You must accept a TRUCKLOAD of txns - coz nobody will be safe without them.
Even if no program ever tell you anything about them ...
... and even if anyone can write a program to report exactly what ANY pool is doing ...
... though I guess that may be a bit far beyond your skill set to even understand how to do that.

---

Meanwhile ... there was an accusation thrown at me last week that I ignored but I guess I should clear it up.

Yes my description of Luke-Jr's hiding information was saying he was hiding it for longer than he really did.
Even wizkid mentioned it to me in an IRC discussion that I missed ... somehow when I wrote the post.
(Heh Mobius if you read this ... just keep swimming ... Cheesy)
Yes no one needs to believe me but I did actually completely miss that in the IRC discussion Tongue

But the reality is that he has been hiding information since 1-Jun about how his pool has been "not processing transactions" but I also quoted the information that he did indeed post in his thread (I almost never go near his thread, though once when I posted there then deleted the posts after he had quoted them so I'd not ever have to go back ... one is linked to below)
Also with the dice reject code I'd also be curious if wizkid would be so kind as to show us how it does reject SD transactions ...

Firstly yes Jr did post in his thread about the 32 txn limit he imposed on his pool for over 5 months back when he did it and was still there when wizkid took over:
https://bitcointalk.org/index.php?topic=23768.msg968819#msg968819

However, the actual post I thought about was this one:
https://bitcointalk.org/index.php?topic=23768.msg934320#msg934320
Where Jr says:
Quote
...
And it's not a matter of "ignoring" transactions, it's a matter of not processing them.
...
Eligius continues to employ aggressive anti-spam checks on feeless transactions, to avoid wasting charity on spammers. Because of the nature of spam detection, it is necessary to keep the algorithms used confidential
...
These algorithms have never been disclosed - which is really what I was referring to.
Yes I made a mistake Tongue
The fact that it "doesn't process" transactions based on his definition of the word "feeless" (coz his definitions is not fee=0)

However, for the point of view of giving out a high level, means next to nothing, information ... yes he said
Quote
FWIW, all pools have been experiencing higher stales and orphaned blocks due to the excessive transaction volume lately resulting from SatoshiDice abusing the blockchain (there are much cleaner ways to do the same thing). After our second set of 3 orphans in a row, I'm a bit on the annoyed end. For now, I am blocking transactions to 1dice* addresses and limiting our blocks to 32 transactions until we've caught up on the extra credit or at least have a viable alternative solution. I really hate to do this, as Eligius has traditionally been one of the most accepting mining pools, so any suggestions on other possibilities would be most welcome.
Which is using the fact that Eligius was worse off than every other pool (none of the others changed to 32 txns per block) and none of the others had as many orphans as Eligius - so yeah blaming SD for his crappy software and using it as an excuse to reduce the BTC transaction confirm time for over 5 months really shows something about him ...

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
November 05, 2012, 01:47:54 PM
 #165

Almost implemented the first working version of GBT within cgminer.

This sort of behaviour tells the story well though:

Code:
[2012-11-05 22:28:08] Accepted 23c0ba9a Diff 7/1 GPU 2 pool 2
 [2012-11-05 22:28:08] Accepted 163ec800 Diff 11/1 GPU 0 pool 2
 [2012-11-05 22:28:13] Stratum from pool 3 detected new block
 [2012-11-05 22:28:13] Accepted 61bdf4a7 Diff 2/1 GPU 3 pool 2
 [2012-11-05 22:28:13] Stale share detected, submitting as user requested
 [2012-11-05 22:28:14] Accepted 00feebca Diff 257/1 GPU 1 pool 2
 [2012-11-05 22:28:19] Rejected 70b76eb2 Diff 2/1 GPU 1 pool 2 (stale-prevblk)
 [2012-11-05 22:28:19] Rejected 47b723ef Diff 3/1 GPU 3 pool 2 (stale-prevblk)
 [2012-11-05 22:28:20] Rejected 9644eef7 Diff 1/1 GPU 2 pool 2 (stale-prevblk)
 [2012-11-05 22:28:21] Rejected 4cafa99d Diff 3/1 GPU 1 pool 2 (stale-prevblk)
 [2012-11-05 22:28:22] Rejected 093dc4cd Diff 27/1 GPU 0 pool 2 (stale-prevblk)
 [2012-11-05 22:28:23] GBT LONGPOLL from pool 2 requested work restart
 [2012-11-05 22:28:24] Accepted 375a5178 Diff 4/1 GPU 1 pool 2
 [2012-11-05 22:28:25] Accepted 047206f4 Diff 57/1 GPU 3 pool 2
 [2012-11-05 22:28:25] GBT LONGPOLL from pool 2 requested work restart
 [2012-11-05 22:28:30] Accepted 72bee5f8 Diff 2/1 GPU 0 pool 2
 [2012-11-05 22:28:31] Accepted 8e63856d Diff 1/1 GPU 2 pool 2

Now pool 3 here is slush on stratum (ping time 335ms), while pool 2 is EMC on GBT (ping time 225ms).

Note how stratum picks up the block change and notifies me 10 seconds before GBT does. However, this is not the GBT pool simply learning of the block change later, because it starts rejecting my shares as being from the previous block before I even got the longpoll from the GBT pool. Then of course there's a second longpoll with the transactions, which is not an unusual practice.
Confirming this analysis as accurate... for now. It shouldn't be this much of a difference though, so I suspect there's something going on server-side I need to look into.
Analysis of block 000000000000017eeb9f83b6037b12c26c41abf776c3602b619b0b583ec74b7e:
Code:
1143.46298 Stratum notification of block from slush
1151.00000 Eligius bitcoind finishes processing new block
1152.00000 Eligius eloipool finishes processing new block
1155.90269 Begin receiving GBT longpoll reply from EclipseMC
1156.62722 Begin receiving GBT longpoll reply from Eligius
1162.99977 Finish receiving GBT longpoll reply from Eligius: 822 kB
1167.77308 Finish receiving GBT longpoll reply from EclipseMC: 804 kB
So somehow, slush's pool is getting blocks 8 seconds earlier than EclipseMC and Eligius. That's half the time difference alone.
Now, Eloipool isn't supposed to be including transactions in the first longpoll response (that's what the second one is for). 804/822 kB reflects that in practice it is. That's a bug for me to fix Smiley
There's also a 4 second delay between eloipool processing the block and my beginning to receive the longpoll. The entire process of queuing the LP data to clients is taking 5.5 seconds on Eligius during new blocks. When there isn't a new block, the same thing takes 2.7 seconds. Presumably fixing the size of the longpoll response should reduce this somewhat, but this is an area where Stratum actually does make a difference: the server would only need to encode the JSON response once and queue it at all clients identically.


Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
November 05, 2012, 04:07:07 PM
 #166

Pushed a fix for Eloipool and deployed on Eligius:
Code:
2012-11-05 15:57:27 BFGMiner              NOTICE  Stratum from pool 3 detected new block
2012-11-05 15:57:38 newBlockNotification  INFO    Received new block notification
2012-11-05 15:57:40 merkleMaker           INFO    New block: 0000000000000052ff0915fef8c1144e4a16fe380cf416c991e96a230871512c (height: 206605; bits: 1a0513c5)
2012-11-05 15:57:40 BFGMiner              NOTICE  LONGPOLL from pool 0 caught up to new block
2012-11-05 15:57:45 JSONRPCServer         INFO    Longpoll woke up 7704 clients in 5.137 seconds
Now if only slush would share his secret to getting blocks sooner :p

kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
November 05, 2012, 04:31:53 PM
 #167

Pushed a fix for Eloipool and deployed on Eligius:
Code:
2012-11-05 15:57:27 BFGMiner              NOTICE  Stratum from pool 3 detected new block
2012-11-05 15:57:38 newBlockNotification  INFO    Received new block notification
2012-11-05 15:57:40 merkleMaker           INFO    New block: 0000000000000052ff0915fef8c1144e4a16fe380cf416c991e96a230871512c (height: 206605; bits: 1a0513c5)
2012-11-05 15:57:40 BFGMiner              NOTICE  LONGPOLL from pool 0 caught up to new block
2012-11-05 15:57:45 JSONRPCServer         INFO    Longpoll woke up 7704 clients in 5.137 seconds
Now if only slush would share his secret to getting blocks sooner :p
Ah, so you are saying that Stratum is magically predetermining when blocks are going to arrive to cover for why you see it later.
Well with that suggestion, who would want to use any other protocol Tongue

Also interesting that you have stated that for the sake of the poor performance of LP in GBT you are sending out no transactions with the LP ... yet another poor implementation feature (using LP the same way getwork uses it, is also poor implementation)
I don't know how may times I've thought and said that Tycho must have been drunk the night he thought up LP ...

Also the implemented GBT LP means a % increase in transaction confirmation times when compared to Stratum LP ... and getwork LP ...

So the only supposed performance improvement over Getwork so far is that you can hash a 100TH/s device for 30 seconds but with getwork it will only support a 2TH/s device for 30 seconds ...
(though getwork wins hands down for the amount of data transferred)

Who designed this ... they should be shot.

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
DrHaribo
Legendary
*
Offline Offline

Activity: 2422
Merit: 1019


Bitminter.com Operator


View Profile WWW
November 05, 2012, 09:29:27 PM
 #168

I think GBT would run pretty smoothly if it was allowed to skip the transactions.

What harm can a pool op do when miners don't see the transactions until after a block is created? What exactly would a "dirty job" be?

Filter out transactions? I think miners seeing this a little later is good enough.

Including transactions that should be filtered out? That assumes we should be blacklisting someone. But who/what... stolen coins?

▶▶▶ Bitminter.com - Your trusted mining pool since 2011.
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
November 05, 2012, 09:43:49 PM
 #169

What harm can a pool op do when miners don't see the transactions until after a block is created? What exactly would a "dirty job" be?
Double spending, for example.

DrHaribo
Legendary
*
Offline Offline

Activity: 2422
Merit: 1019


Bitminter.com Operator


View Profile WWW
November 05, 2012, 10:48:55 PM
 #170

What harm can a pool op do when miners don't see the transactions until after a block is created? What exactly would a "dirty job" be?
Double spending, for example.

I always thought about defenses against this in the direction of deciding which fork to build on. So if you have your own bitcoind or public service that you trust, you can refuse to help a pool build more than 2 blocks on a fork parallel to the "main chain" as defined by your trusted source. This should make 6-confirmation double spends practically impossible if a majority of miners followed this scheme, I think?

To implement this you only need a trusted source that you can query for the current height and the block hash at height X. It should work even with getwork as you only need the "prevblockhash" from the block you are mining. A complication is getting the prevblockhash from a block your trusted source doesn't know about (yet).

I guess you could also do this by remembering transactions from before a chain reorganization and refuse to include transactions on the new fork that send those coins somewhere else. You'd probably have to force those transactions into the next few blocks.

You could try to do both. Don't be part of a 51% attack, but if one does happen try to get the orphaned transactions into the new fork as quickly as possible. But chances are that when the chain reorg takes place there is a double spend in the new fork already, if the attacker did his job properly.

You could also use a trusted source to see that a transaction you are currently including in a block you are mining is actuallly a double spend of a transaction from a different fork. But in this case it seems easier to just refuse to deviate much from the main chain.

▶▶▶ Bitminter.com - Your trusted mining pool since 2011.
wizkid057
Legendary
*
Offline Offline

Activity: 1220
Merit: 1000


View Profile
November 06, 2012, 12:27:01 AM
 #171

What harm can a pool op do when miners don't see the transactions until after a block is created? What exactly would a "dirty job" be?
Double spending, for example.

I think it is more for the 0-1 confirmation double spends.  I don't think that there is currently much risk of a 6 confirmation double spend from any single pool.  However, any pool could attack something like satoshidice or other 0 conf service with minor luck.

Tips: 1LDQrLr6dPVqNJmpZm82eZVKqDFRk7ERW8
Operator of the Eligius Mining Pool - 0% Fee, SAPPLNS, GBT, Stratum, IRC+Phone Support, Share Market (coming soon), Generation payouts, and more.
Don't feed the trolls. Science Confirms: Internet Trolls Really Are Narcissistic, Psychopathic, and Sadistic (1)
DrHaribo
Legendary
*
Offline Offline

Activity: 2422
Merit: 1019


Bitminter.com Operator


View Profile WWW
November 06, 2012, 07:23:01 AM
 #172

I think it is more for the 0-1 confirmation double spends.  I don't think that there is currently much risk of a 6 confirmation double spend from any single pool.  However, any pool could attack something like satoshidice or other 0 conf service with minor luck.

Less than 6 confirmations should involve only very small sums. I think in that case it would be good enough to detect after the fact. So a mining pool may be able to help someone pull off a 1 bitcoin "heist". But it would have huge consequences for the pool afterwards.

In the case of 0 conf it may be necessary to have some miners detecting it as it happens. I'm not sure if there is any evidence otherwise. But what if you spread 2 conflicting transactions to different parts of the bitcoin network. When a pool creates a block with one of those transactions it may look like a Finney attack, while the pool is really just including the transaction it sees. Anyone can do this to make a pool look evil, so detecting something like that doesn't actually mean anything.

Is SatoshiDice really 0 conf? So if you have some hashpower you can easily defraud them of large sums using a Finney attack?

▶▶▶ Bitminter.com - Your trusted mining pool since 2011.
kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
November 06, 2012, 07:31:37 AM
 #173

I think it is more for the 0-1 confirmation double spends.  I don't think that there is currently much risk of a 6 confirmation double spend from any single pool.  However, any pool could attack something like satoshidice or other 0 conf service with minor luck.

Less than 6 confirmations should involve only very small sums. I think in that case it would be good enough to detect after the fact. So a mining pool may be able to help someone pull off a 1 bitcoin "heist". But it would have huge consequences for the pool afterwards.

In the case of 0 conf it may be necessary to have some miners detecting it as it happens. I'm not sure if there is any evidence otherwise. But what if you spread 2 conflicting transactions to different parts of the bitcoin network. When a pool creates a block with one of those transactions it may look like a Finney attack, while the pool is really just including the transaction it sees. Anyone can do this to make a pool look evil, so detecting something like that doesn't actually mean anything.

Is SatoshiDice really 0 conf? So if you have some hashpower you can easily defraud them of large sums using a Finney attack?

No, you can make bets and try to undo them and spend the money some other way ... after you learn if they lose.
So yeah - NO one is gonna make much money out of that.

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
wizkid057
Legendary
*
Offline Offline

Activity: 1220
Merit: 1000


View Profile
November 06, 2012, 08:18:47 AM
 #174

I think it is more for the 0-1 confirmation double spends.  I don't think that there is currently much risk of a 6 confirmation double spend from any single pool.  However, any pool could attack something like satoshidice or other 0 conf service with minor luck.

Less than 6 confirmations should involve only very small sums. I think in that case it would be good enough to detect after the fact. So a mining pool may be able to help someone pull off a 1 bitcoin "heist". But it would have huge consequences for the pool afterwards.

In the case of 0 conf it may be necessary to have some miners detecting it as it happens. I'm not sure if there is any evidence otherwise. But what if you spread 2 conflicting transactions to different parts of the bitcoin network. When a pool creates a block with one of those transactions it may look like a Finney attack, while the pool is really just including the transaction it sees. Anyone can do this to make a pool look evil, so detecting something like that doesn't actually mean anything.

Is SatoshiDice really 0 conf? So if you have some hashpower you can easily defraud them of large sums using a Finney attack?

No, you can make bets and try to undo them and spend the money some other way ... after you learn if they lose.
So yeah - NO one is gonna make much money out of that.

Actually, if you look at the SD txns, you'll see that many people pull it off all the time.  I still have zero confirmation payouts in my wallet from back when I was messing with SD months and months ago because somewhere along the lines someone cheated them out of a payout and it broke the whole chain of transactions so that none could confirm.

Basically once you see if you lost or not, you try to get a replacement transaction mined that displaces your losing bet.  With enough hash power or control over txns of enough hash power, it'd be trivial.

Tips: 1LDQrLr6dPVqNJmpZm82eZVKqDFRk7ERW8
Operator of the Eligius Mining Pool - 0% Fee, SAPPLNS, GBT, Stratum, IRC+Phone Support, Share Market (coming soon), Generation payouts, and more.
Don't feed the trolls. Science Confirms: Internet Trolls Really Are Narcissistic, Psychopathic, and Sadistic (1)
DrHaribo
Legendary
*
Offline Offline

Activity: 2422
Merit: 1019


Bitminter.com Operator


View Profile WWW
November 06, 2012, 09:38:21 AM
 #175

Basically once you see if you lost or not, you try to get a replacement transaction mined that displaces your losing bet.  With enough hash power or control over txns of enough hash power, it'd be trivial.

You can mine the block displacing your bet first. After you successfully create the block, hold it back. Now you place some bets and get a winning payout from SD. Finally you release the block to the bitcoin network, getting your bet back as well. (= Finney attack)

Sometimes your block will be orphaned and your bet will be lost. But in most cases you get the winnings (if any) AND your bet. An orphaned block also means losing 50 BTC (soon 25), and holding a block back increases the chance of that happening. But if each of your blocks contain bets for hundreds of bitcoins then the profits should far outweigh those occasional losses.

I don't see any way for GBT to fix this. Services just have to stop accepting payments at zero confirmations.

▶▶▶ Bitminter.com - Your trusted mining pool since 2011.
kano
Legendary
*
Offline Offline

Activity: 2506
Merit: 1043


Linux since 1997 RedHat 4


View Profile
November 06, 2012, 09:41:31 AM
 #176

...
So yeah - NO one is gonna make much money out of that.

Actually, if you look at the SD txns, you'll see that many people pull it off all the time.  I still have zero confirmation payouts in my wallet from back when I was messing with SD months and months ago because somewhere along the lines someone cheated them out of a payout and it broke the whole chain of transactions so that none could confirm.

Basically once you see if you lost or not, you try to get a replacement transaction mined that displaces your losing bet.  With enough hash power or control over txns of enough hash power, it'd be trivial.
List this many ...

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
DrHaribo
Legendary
*
Offline Offline

Activity: 2422
Merit: 1019


Bitminter.com Operator


View Profile WWW
November 06, 2012, 06:05:33 PM
 #177

I missed something about SatoshiDice. If they always make sure that the output of your betting transaction is one of the inputs for the transaction paying out your winnings then you can't both get the winnings and your bet back. But you can still take your coins back if you lose. So you'd win almost every bet, except when the block for your Finney attack is orphaned.

Anyway, my point was that if you can't create an automated way to make Bitcoin/mining safer by use of the transaction data while mining, then it makes no sense for it to be mandatory to push all that redundant data back and forth.

▶▶▶ Bitminter.com - Your trusted mining pool since 2011.
Luke-Jr
Legendary
*
Offline Offline

Activity: 2394
Merit: 1001



View Profile
November 06, 2012, 07:07:28 PM
 #178

Anyway, my point was that if you can't create an automated way to make Bitcoin/mining safer by use of the transaction data while mining, then it makes no sense for it to be mandatory to push all that redundant data back and forth.
But we can. Even without a local bitcoind, miners can setup multiple pools and the miner can avoid pools that have an odd contradicting transaction compared to all the others.

DrHaribo
Legendary
*
Offline Offline

Activity: 2422
Merit: 1019


Bitminter.com Operator


View Profile WWW
November 06, 2012, 07:33:27 PM
 #179

Anyway, my point was that if you can't create an automated way to make Bitcoin/mining safer by use of the transaction data while mining, then it makes no sense for it to be mandatory to push all that redundant data back and forth.
But we can. Even without a local bitcoind, miners can setup multiple pools and the miner can avoid pools that have an odd contradicting transaction compared to all the others.

So if I create two contradicting transactions, send one to pool X and the other to other nodes on the network, then miners would see pool X as the odd man out and stop mining there?

I'm not trying to be annoying, I'd just like to see one example where this works and isn't easily abused.

My thinking so far is that the best we can do is "allow small forks because they happen naturally, but never help create large forks." Perhaps that's good enough.

▶▶▶ Bitminter.com - Your trusted mining pool since 2011.