|
Title: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on September 12, 2012, 12:06:49 AM Since Eleuthria (https://bitcointalk.org/index.php?topic=108441.0) and slush (https://bitcointalk.org/index.php?topic=108533.0) have both recently announced their own developed-behind-closed-doors mining protocols, I felt it appropriate to make a thread here for end-user discussion of the "getblocktemplate" standard which they intend to compete with (http://xkcd.com/927/). As a poolserver author, a pool operator, a miner author, and a contributor to the reference client, my life is not improved by having yet another protocol to support, and what's bad for the authors of the infrastructure tools is also bad for all their users.
Getblocktemplate Features
Documentation See the new Bitcoin Wiki page for getblocktemplate, with lots of useful resources including end user and developer documentation (https://en.bitcoin.it/wiki/Getblocktemplate). Technical specifications:
Supported by... (post or PM me to get yours added) Poolservers (https://en.bitcoin.it/wiki/Poolservers) (providing getblocktemplate to miners):
Mining software:
Libraries for mining software:
Mining pools:
History/Background Getblocktemplate's origins trace back to forrestv's getmemorypool JSON-RPC method for bitcoind. He created it so that his pool (P2Pool) could piggy-back on bitcoind so as to avoid harming the Bitcoin network (up to this point, it mined only empty blocks that never confirmed transactions). Having been fighting with scalability problems in pushpool/bitcoind for months on my pool (Eligius), I set out to implement a fast pool server using getmemorypool to do its own block production (this became Eloipool, this first open source makes-its-own-blocks poolserver). Other poolservers also implemented block creation using getmemorypool over the months following. At about the same time, interest in decentralizing pooled mining became a hot topic. While BitPenny had initially released its own decentralized mining proxy months prior, P2Pool's implementation became rapidly popular. Anyone involved in Bitcoin mining protocols could see the need to move control of block creation back into the hands of the miners. Unfortunately, both BitPenny and P2Pool had used very pool-specific proprietary protocols to implement their decentralization. On the other hand, getmemorypool was almost a perfect fit for the task - it just lacked support for the now-ubiquitous pooled mining extensions that had developed around the getwork protocol over time. In February, I implemented and deployed (https://bitcointalk.org/?topic=23768.msg774497#msg774497) a first draft of getmemorypool mining support in Eloipool (and on Eligius) along with a proof-of-concept getwork proxy (now known as gmp-proxy), adding revisions as needed to function as a general-purpose decentralized mining protocol. After I had confirmed it was working, I documented and proposed it on the Bitcoin development mailing list for review on February 28th, where discussion began on what was missing and what needed to be changed or clarified. During the following few months, a number of others, both developers and testers, provided constructive criticism and suggestions, which were integrated into the standard. I also actively encouraged participation in the development of the standard among pool operators and poolserver authors, especially as it became necessary to move forward into the ASIC "mining generation". Eventually, it was decided it would be best to rename it to the more appropriate "getblocktemplate" name and drop backward compatibility with getmemorypool for simplicity. The standard was split into two pieces and the technical specification can be found in BIP 22 (https://en.bitcoin.it/wiki/BIP_0022) and BIP 23 (https://en.bitcoin.it/wiki/BIP_0023). Support for getblocktemplate mining was announced for BFGMiner (https://bitcointalk.org/index.php?topic=78192.msg1158493#msg1158493) a week ago, and is currently functional (implementing the basics is quite easy) though not entirely optimized yet. A number of other software developers and pools have told me they intend to add getblocktemplate support soon. (side note: all of this occurred before Eleuthria or slush made their protocols public, and neither of them chose to participate in the open development of the getblocktemplate standard) Contribute All that being said, there is always room for improvement. Reception of the proprietary proposals strongly suggests getblocktemplate is in desperate need of good miner-side example documentation. If you have the time to help bring getblocktemplate to everyone sooner in any way, please get in touch :) Thanks to... (in no particular order)
Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: gmaxwell on September 12, 2012, 12:12:14 AM I haven't yet read about about these other protocols, but I'm really disappointed to see more protocol proliferation. Getblocktemplate is well established, peer reviewed, and is good for bitcoin's decentralized security. I hope GBT gains whatever features its missing so it can subsume these things. Having yet another protocol to cope with is a bummer.
Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: Graet on September 12, 2012, 12:35:28 AM I haven't yet read about about these other protocols, but I'm really disappointed to see more protocol proliferation. Getblocktemplate is well established, peer reviewed, and is good for bitcoin's decentralized security. I hope GBT gains whatever features its missing so it can subsume these things. Having yet another protocol to cope with is a bummer. It shows me interest from varied developers is strong - is this not good for Bitcoin?what is wrong with proposing a new way for miners to communicate with pools, it isn't like it is changing a Bitcoin protocol it is a mining change. there is no single pool software everyone uses, there is no single mining software everyone uses - I for one am interested at looking at options. Thanks to all the developers taking interest in this topic and offering some options to consider :) https://bitcointalk.org/index.php?topic=108533.0 Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: eleuthria on September 12, 2012, 12:42:11 AM I started my development privately because I wasn't sure it was ever going to be completed. I did know one thing. The problem that slush and I were both addressing separately (though we wound up doing almost the exact same thing) was that POOLS will not be able to cope with ever increasing speeds, because pooled mining is using the standard getwork protocol that is used in bitcoin.
Our protocols were designed to be specific for pooled mining. The primary Bitcoin project does not need to be polluted with new protocols and RPC commands for mining related purposes. Bitcoin is not about mining, and developer time should be spent on more important features that matter. Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: slush on September 12, 2012, 12:46:49 AM Well, I don't want to start flamewar and personal fights, really. I did the Stratum just because of my requirements, not because I wanted to compete with anybody. Let me explain it:
Actually I see the difference between Stratum and GMP as Electrum client and Satoshi client. You can achieve similar results with both of them, but every of such solution has pros and cons. Two days ago I summarized my reasons for Stratum here: http://mining.bitcoin.cz/stratum-mining I was comparing it mostly with the getwork, because it's the only used protocol by real miners now (voting by the hashrate). Maybe I should write up longer comparsion between Stratum and GMP. Back to the topic. THere's really no need to have one super-protocol which fits all needs. By the way, I'm using getblocktemplate in the Stratum implementation myself and I consider it as a great way how to relocate pool logic outside the bitcoind. GBT is surely very complete protocol and you can do anything what you want, *if* you know how. I see GBT as very complex and very hard to implement full specs without breaking anything. Luke will say the oposite, but GBT has simply unnecesary compexity for the environment of pooled mining. You know, many optional features etc. But pooled miners don't need that. Period. In the oposite we proposed complete stack for pooled mining, which *is* very lightweight and straightforward to implement. Simply said, there's not a *single* optional feature, which is possibly the biggest benefit of whole concept. Miners don't want tons of options, they need simple and clear algorithm how to talk with the server. It includes also exact specification of used transports and mechanisms for real-time broadcasting new jobs to the client. Current BIPs don't care about this anyhow, because they don't need to. Both are just for different use cases. Luke told me yesterday, that GBT is not bounded to HTTP, that it can be implement with any transport. By mistake he expressed the exact reason why it isn't suitable for pooled mining. It gives miners/pool ops degree of freedom which don't help anything, just mess the source code of miners and pools, because interpretation of the current protocol "as is" is ambiguous and cannot be used by some additonal specification about transports etc. By the way, pool servers must be super-optimalized for handle high loads. Current Stratum protocol implementing coinbase templates is definitely the most optimized solution for given purpose and I doubt that any getblocktemplate pool would be more optimized that this. Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: Luke-Jr on September 12, 2012, 01:22:56 AM what is wrong with proposing a new way for miners to communicate with pools, it isn't like it is changing a Bitcoin protocol it is a mining change. The problem is more about when people don't work together with others. Everyone was (and is) welcome to contribute to the development of GBT, so why quietly reinvent an incompatible protocol on the side? GBT vs StratumMP is like W3c HTML/CSS vs Microsoft Internet Explorer "quirks" HTML/CSS.there is no single pool software everyone uses, there is no single mining software everyone uses - I for one am interested at looking at options. The problem that slush and I were both addressing separately (though we wound up doing almost the exact same thing) was that POOLS will not be able to cope with ever increasing speeds, because pooled mining is using the standard getwork protocol that is used in bitcoin. GBT is primarily a pooled mining protocol, and does address the problem you mention perfectly, while also addressing other (arguably) more serious problems (centralized pool control).Our protocols were designed to be specific for pooled mining. The primary Bitcoin project does not need to be polluted with new protocols and RPC commands for mining related purposes. Bitcoin is not about mining, and developer time should be spent on more important features that matter. I did the Stratum just because of my requirements, not because I wanted to compete with anybody. Let me explain it: This analogy doesn't really make sense. Electrum and Satoshi implement things very differently. The only thing StratumMP really does differently from GBT is the protocol/communication syntax.Actually I see the difference between Stratum and GMP as Electrum client and Satoshi client. You can achieve similar results with both of them, but every of such solution has pros and cons. THere's really no need to have one super-protocol which fits all needs. ... I see GBT as very complex and very hard to implement full specs without breaking anything. Luke will say the oposite, but GBT has simply unnecesary compexity for the environment of pooled mining. You know, many optional features etc. But pooled miners don't need that. Period. Pooled miners already need those features enough that they've been hacked on top of getwork for a long time. Probably no miner or pool implements full specs for all the getwork extensions, and that's fine for GBT as well. The important thing is that in cases where those features are needed, there is a consistent protocol for it rather than every pool/miner doing the same thing 5 different ways.Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: slush on September 12, 2012, 01:43:09 AM The problem is more about when people don't work together with others. Everyone was (and is) welcome to contribute to the development of GBT It's the same like designing the universal car. We should have just one model which will fit small city car, family car, a van and the crane. We definitely can design one machine which will fit all these needs just by mounting some special equipment over it. But we just don't want to, because it will lead to overcomplicated monster with many exceptions in the *real* implementation. Look around you at this time. Miner software cannot follow even such dumb protocol as current getwork is. I don't want to show you my pool source, because I'm shy for tons of hacks involved because of crappy miner support. This is real-world lesson and we should learn from it. Please don't repeat same design mistakes with the full featured protocol which is ten-times complicated than getwork. Quote The only thing StratumMP really does differently from GBT is the protocol/communication syntax. Then please show me exact payload done by GBT and providing exactly the same behaviour like Stratum does in the documentation example. And please don't use submitting whole coinbase. It's impossible to do full validation of coinbase for every share. Quote Pooled miners already need those features Which features they *really* need from GBT except? Quote Probably no miner or pool implements full specs for all the getwork extensions, and that's fine for GBT as well. The important thing is that in cases where those features are needed, there is a consistent protocol for it rather than every pool/miner doing the same thing 5 different ways. For the god sake, how you can manage miner implementation if every pool operator will have full freedom in implement any of BIP option? Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: Graet on September 12, 2012, 02:02:43 AM what is wrong with proposing a new way for miners to communicate with pools, it isn't like it is changing a Bitcoin protocol it is a mining change. The problem is more about when people don't work together with others.there is no single pool software everyone uses, there is no single mining software everyone uses - I for one am interested at looking at options. Everyone was (and is) welcome to contribute to the development of GBT but no-one is required to? right?, so why quietly reinvent an incompatible protocol on the side? GBT vs StratumMP is like W3c HTML/CSS vs Microsoft Internet Explorer "quirks" HTML/CSS. Both of these guys announced in their pool threads that they were working on ideas to cope with upcoming changes to mining hardware and most pools have said they are looking at options, either quietly or loudly....I'm probably wrong again but looking around Bitcoin there is no monopoly on anything, there are even alternatives to the satoshi client I think a lot of Bitcoin's growth has come from the different ideas that have been developed by a lot of people both on the main dev team and outside by other interested people, both foss and closed source I welcome choices :) Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: P_Shep on September 12, 2012, 02:05:13 AM I welcome choices :) And may the strongest survive :) Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: zvs on September 12, 2012, 01:33:42 PM For the god sake ahh, reminds me of Adam's Apples subtitles, 'for the sake of fuck', one of my favoritesok, carry on Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: optimator on September 12, 2012, 10:47:27 PM miners decide what they will mine, and make their own blocks - no more blindly giving up mining control to pool operators I thought one of the benefits of a pool was to determine the efficiency of including transactions in a block. It is the pool that could decide whether to include or exclude a transaction based on the transaction fees. Doesn't having the miner take responsibility for the transactions undermine some of the benefit of the pool? If the miner is building the transaction why doesn't the miner submit a block when his hash is less than the difficulty? Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: Luke-Jr on September 13, 2012, 12:45:35 AM miners decide what they will mine, and make their own blocks - no more blindly giving up mining control to pool operators I thought one of the benefits of a pool was to determine the efficiency of including transactions in a block. It is the pool that could decide whether to include or exclude a transaction based on the transaction fees.Doesn't having the miner take responsibility for the transactions undermine some of the benefit of the pool? If the miner is building the transaction why doesn't the miner submit a block when his hash is less than the difficulty? I don't understand this question.Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: optimator on September 13, 2012, 01:57:01 AM If the miner is building the transaction why doesn't the miner submit a block when his hash is less than the difficulty? I don't understand this question.Thanks - I missed the part about flagging transactions. Why can't a miner just submit the block header and the transactions when he finds it? Doesn't the miner have everything he needs to make a block? or am i missing something? Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: Luke-Jr on September 13, 2012, 02:27:31 AM If the miner is building the transaction why doesn't the miner submit a block when his hash is less than the difficulty? I don't understand this question.Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: optimator on September 13, 2012, 02:34:25 AM But then how do the other miners get credit for the completed block? Doesn't this allow dishonest miners to usurp the block from the pool?
Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: Luke-Jr on September 13, 2012, 02:36:50 AM But then how do the other miners get credit for the completed block? Doesn't this allow dishonest miners to usurp the block from the pool? No, the block rewards can't be changed after mining the block.Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on September 15, 2012, 09:55:12 PM BFGMiner 2.8.0 released (https://bitcointalk.org/?topic=78192.msg1193985#msg1193985) with integrated getblocktemplate support.
Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on September 17, 2012, 09:22:20 PM Question off-topic in another thread, pertaining to bitcoind GBT support:
luke-jr: Can you give me a reason why it was felt necessary to remove getmemorypool, instead of having it exist alongside GBT? The GBT patch for bitcoind was originally fully backward compatible, but was changed to replaced GMP instead per the request of the rest of the bitcoind development team.The existing solution breaks many of the pool packages out there. Backwards compatibility is normally a concern for developers. Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: makomk on September 17, 2012, 11:07:56 PM I haven't yet read about about these other protocols, but I'm really disappointed to see more protocol proliferation. Getblocktemplate is well established, peer reviewed, and is good for bitcoin's decentralized security. I hope GBT gains whatever features its missing so it can subsume these things. Having yet another protocol to cope with is a bummer. The problem with GBT is not so much what features it's missing, from what I can tell. Everything that can be done with Stratum can be done with GBT, it'd just far require more bandwidth and every piece of mining software would need code to parse and serialize Bitcoin transactions and blocks and to implement a whole bunch of different cases. Plus, I suspect most mining software would end up requiring a different subset of the GBT protocol than the one bitcoind supports so it's not like using the same protocol for both would even help solo miners. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on September 18, 2012, 04:06:52 AM I've mostly finished a more friendly writeup on getblocktemplate (https://en.bitcoin.it/wiki/Getblocktemplate) on the Bitcoin Wiki, covering (almost?) everything StratumMP's page covered. Take a look and let me know what you think, or how it might be improved! Feel free to edit it directly too, of course :)
Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: Luke-Jr on September 18, 2012, 04:16:24 AM it'd just far require more bandwidth and every piece of mining software would need code to parse and serialize Bitcoin transactions and blocks and to implement a whole bunch of different cases. This isn't correct.Plus, I suspect most mining software would end up requiring a different subset of the GBT protocol than the one bitcoind supports This isn't a problem.Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: eleuthria on September 18, 2012, 05:54:30 AM it'd just far require more bandwidth and every piece of mining software would need code to parse and serialize Bitcoin transactions and blocks and to implement a whole bunch of different cases. This isn't correct.An empty (coinbase-only block) StratumMP work payload is 399 bytes. An empty GBT work payload based on your example is 737 bytes. StratumMP work payloads increase 67 bytes per power of 2 transactions. GBT work payloads increase by the size of the transactions being included in a block (roughly 250 bytes for a normal tx with 1 input, 1 payout address, and 1 change address). For StratumMP, a block containing 257-512 transactions would require an extra 603 bytes, for a total of 1002 bytes. GBT would require approximately 128,000 bytes. Using a more common example, lets say each block has 100 transactions on average. StratumMP would be 868 bytes per work payload. The GBT equivalent would be approximately 26,000 bytes. Similarly, work submission payloads are quite different. GBT would require 501 bytes per share submission in your example. StratumMP requires only 109. None of the above factors in the added overhead of wrapping GBT into HTTP Requests. The biggest hurdle on implementing StratumMP is changing from constant HTTP connections and inefficient longpolling, to a static TCP connection. Beyond that change, StratumMP is a simpler protocol to work with and implement. GBT is more full featured, but your arguments imply that most mining software wouldn't bother implementing the full feature set to begin with. Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: Luke-Jr on September 18, 2012, 06:09:04 AM StratumMP work payloads increase 67 bytes per power of 2 transactions. GBT work payloads increase by the size of the transactions being included in a block (roughly 250 bytes for a normal tx with 1 input, 1 payout address, and 1 change address). Yes, I forgot the exponential factor involved here. However, with a local bitcoind, GBT can shrink far smaller than StratumMP. If bandwidth is a real practical concern for BTCGuild (or anyone else), someone (possibly me, if someone needs and plans to use it) could easily write a "Centralized Mining" extension for GBT providing the same bandwidth savings. Of course, keep in mind that GBT is already going to use far less bandwidth than getwork, so this really shouldn't be a problem. Remember that centralized mining moves the miner's authority blindly to the pool operator, and thus creates a greater risk to the Bitcoin network. Health of the Bitcoin network is IMO far more important than saving a few KB of bandwidth here and there. So... as long as there isn't a practical problem involved, it would seem a shame to keep centralized mining going unnecessarily.The biggest hurdle on implementing StratumMP is changing from constant HTTP connections and inefficient longpolling, to a static TCP connection. Beyond that change, StratumMP is a simpler protocol to work with and implement. Yes, the biggest hurdle is rewriting all the miner code actually relevant to the protocol at all.GBT is more full featured, but your arguments imply that most mining software wouldn't bother implementing the full feature set to begin with. Hopefully mining software will grow closer to full-featured over time. It's pretty important that mining become decentralized, at least.Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: hahahafr on September 18, 2012, 02:01:51 PM The decentralization is important, I agree with you on that Luke. The single-line, TCP connection of slush is still an elegant solution from a miner's point of view. I didn't read much into the technicalities of the GBT protocol, but eventually it's the effective implementations, and the stales rates that come with which ones that is going to determine what protocol is gonna be widely adopted once ASICs are shipped.
Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: slush on September 18, 2012, 02:07:35 PM The decentralization is important I agree on this as well. Currently there are 20+ pools + P2Pools and switching over them is the matter of few seconds, and miners already rotates around backup pools. There's really no reason to think that individual miners know better than poolop what they're doing. And if miner is suspicious about poolop activity, he can just switch the pool to another. I mean - there's no reason to make mining protocol less effective (from any point of view, including CPU resources and used bandwidth, and more difficult implementation in the miners) just because decentralization itself. Market of mining pools is great example of free market and miners can achieve the same level of "decentralization" just by switching to the pool which fits their needs. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on September 18, 2012, 06:23:22 PM ...and the stales rates... FWIW, after 3+ days of use, I can safely say the stale rate is not significantly higher using GBT....And if miner is suspicious about poolop activity, he can just switch the pool to another. Except miners can't viably choose to switch when they are deprived of the information needed to know when to act.I mean - there's no reason to make mining protocol less effective (from any point of view, including CPU resources and used bandwidth, and more difficult implementation in the miners) just because decentralization itself. Market of mining pools is great example of free market and miners can achieve the same level of "decentralization" just by switching to the pool which fits their needs. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: hahahafr on September 18, 2012, 08:57:19 PM Numbers?
Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: dooferorg on September 18, 2012, 11:09:35 PM The stratum approach is working right now with the current installed based of miners. Just point to the proxy intermediary the 'stratum proxy', simple. Been testing this on a miner I have with BTCGuilds stratum test and it's doing great.
Failover to other pools has been addressed by other proxy software and I use that too in conjunction with the stratum proxy with no problem. Miner can work and upstream proxies (on the local network) just decide what pool and how to communicate. I didn't even have to touch the miner software. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on September 18, 2012, 11:17:49 PM The stratum approach is working right now with the current installed based of miners. Just point to the proxy intermediary the 'stratum proxy', simple. Been testing this on a miner I have with BTCGuilds stratum test and it's doing great. GBT has been working with a proxy since February, and is easily added to existing miners.Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on September 19, 2012, 06:16:11 PM I don't have dialup, but still saving bandwidth is important so the new mining hardware doesn't DDOS your shiny pool by saturating the pools line. Sure, but GBT does save bandwidth. It just doesn't save as much as StratumMP, because it doesn't keep the miner in the dark.If the GBT thing has been around since February, why aren't all pools and miners already using it? Probably the main (only?) incentive for most pools is surviving when ASICs arrive. There's also the problem of Eloipool being the only (publicly available) maintained poolserver (though p2k plans to update ecoinpool when he gets time), and most pools seem to still be using pushpool or PSJ. Technically speaking, pools running older versions of Eloipool should have an older GBT available, but now that BIP 22 is final, it seemed inappropriate to list them until they are running a compatible version.Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: hahahafr on September 20, 2012, 05:15:11 AM I don't have dialup, but still saving bandwidth is important so the new mining hardware doesn't DDOS your shiny pool by saturating the pools line. Sure, but GBT does save bandwidth. It just doesn't save as much as StratumMP, because it doesn't keep the miner in the dark.One should also note that the interest of the pool operator is the same as the miners: Include the best transactions in the next block to maximize his profit (through donations in our case). Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on September 21, 2012, 06:42:58 AM So ... someone wanna tell me who the fucktard mod was that deleted my post from here?
I guess is was most likely Luke-Jr's pissy little dog who does his bidding - but feel free to correct me if I'm wrong ... Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: ssateneth on September 22, 2012, 12:32:57 AM So ... someone wanna tell me who the fucktard mod was that deleted my post from here? I guess is was most likely Luke-Jr's pissy little dog who does his bidding - but feel free to correct me if I'm wrong ... quoted before deletion Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on September 22, 2012, 12:50:54 AM So ... someone wanna tell me who the fucktard mod was that deleted my post from here? I guess is was most likely Luke-Jr's pissy little dog who does his bidding - but feel free to correct me if I'm wrong ... quoted before deletion It seems that there is a mod (I wonder who :) ) who is a slave toy of this stupid thread OP, who bows down to and deletes posts for him at his whim, to hide the truth about the thread OP. Not surprising really though, since the thread OP is provably known for lying all over the place to hide his transgressions. Just reading the first paragraph of the first post and reading exactly what he is saying is rather convincing of the (totally ludicrous) self importance the thread OP places upon himself. Quote Since Eleuthria and slush have both recently announced their own developed-behind-closed-doors mining protocols, I felt it appropriate to make a thread here for end-user discussion of the "getblocktemplate" standard which they intend to compete with. As a poolserver author, a pool operator, a miner author, and a contributor to the reference client, my life is not improved by having yet another protocol to support, and what's bad for the authors of the infrastructure tools is also bad for all their users. Lulz - what the thread OP thinks is bad, is bad for everyone who uses bitcoind :D :D :D Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on September 22, 2012, 02:03:12 PM Numbers? Now that I've had BFGMiner 2.8 running for a number of days, I have some reasonably useful numbers...getwork session A: 0.83% stale getwork session B: 1.09% stale GBT session A: 0.71% stale GBT session B: 0.66% stale There's probably a good chance the GBT stales are lower only by coincidence (a lot depends on when blocks are found, after all), but I think it's pretty safe to conclude there's not any notable practical difference just from the protocol. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: slush on September 22, 2012, 02:20:21 PM There's probably a good chance the GBT stales are lower only by coincidence (a lot depends on when blocks are found, after all), but I think it's pretty safe to conclude there's not any notable practical difference just from the protocol. Comparing both solutions based on HTTP+LP, you obviously don't see any difference. Stratum session on my machine: 60627 submits, 53 stale, it's 0.08 % stale ratio. This ^^ also includes short outage of my Internet connectivity overnight ;-). Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on September 22, 2012, 03:49:44 PM There's probably a good chance the GBT stales are lower only by coincidence (a lot depends on when blocks are found, after all), but I think it's pretty safe to conclude there's not any notable practical difference just from the protocol. Comparing both solutions based on HTTP+LP, you obviously don't see any difference. Stratum session on my machine: 60627 submits, 53 stale, it's 0.08 % stale ratio. This ^^ also includes short outage of my Internet connectivity overnight ;-). Which is above Stratum but better than this thread protocol. Stratum looks like it may be OK, but this thread one being worse than the current numbers on a standard pool is ... not good. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: slush on September 22, 2012, 05:04:57 PM kano, now you're comparing apples and oranges. Stale ratio is affected by many factors and there's no obvious reason why GBT should be worse than getwork.
Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on September 23, 2012, 12:52:20 AM kano, now you're comparing apples and oranges. Stale ratio is affected by many factors and there's no obvious reason why GBT should be worse than getwork. stale/reject ratio is basically 2 things: 'LP' issues and processing work for too long according to how long the pool will accept the result (and the protocol may allow that all the way up to an LP)3 different protocols still have that same requirement: accept work or reject it. There are of course 3 factors in it: 1) The mining software 2) The protocol 3) The connectivity between them If the miner/protocol/pool is rejecting more work then there is some issue that needs to be resolved. (or the design is a POS) As I stated, I get around 0.3% stale with a standard pool protocol (I can also add that my pool connect is half way around the world, so that should be close to about as bad as it can be) However, comparing cgminer+getwork/LP+a standard pool is indeed quite valid. What other comparison could be made? Fake up some excuse for bad performance? His figures are showing his miner+his protocol+his pool are clearly worse than the setup I described ... and I will add that ~0.3% or better is not unexpected from people using cgminer+getwork/LP+'choose random pool' So I'm not sure where the apples and oranges are ... Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: hahahafr on September 23, 2012, 03:20:08 AM Numbers? Now that I've had BFGMiner 2.8 running for a number of days, I have some reasonably useful numbers...getwork session A: 0.83% stale getwork session B: 1.09% stale GBT session A: 0.71% stale GBT session B: 0.66% stale There's probably a good chance the GBT stales are lower only by coincidence (a lot depends on when blocks are found, after all), but I think it's pretty safe to conclude there's not any notable practical difference just from the protocol. There's probably a good chance the GBT stales are lower only by coincidence (a lot depends on when blocks are found, after all), but I think it's pretty safe to conclude there's not any notable practical difference just from the protocol. Comparing both solutions based on HTTP+LP, you obviously don't see any difference. Stratum session on my machine: 60627 submits, 53 stale, it's 0.08 % stale ratio. This ^^ also includes short outage of my Internet connectivity overnight ;-). It would really be interesting to see the stale ratio of GBT+LP vs Stratum over one week on the same pool/same connection. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on September 23, 2012, 05:00:03 AM If I understand correctly GBT also supports LP, doesn't it? Did you make use of LP with getwork and GBT in your test? Yes, of course.It would really be interesting to see the stale ratio of GBT+LP vs Stratum over one week on the same pool/same connection. Unfortunately, I haven't optimized GBT in BFGMiner at all yet, so I doubt it would compare well vs Stratum proxy. It would be interesting for someone to make a quick port of the Stratum proxy to GBT (python-bitcoinrpc + python-blkmaker should do it very easily) to compare with. Of course, you'd also need a biprotocol pool, which AFAIK doesn't exist yet.Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: jgarzik on September 23, 2012, 05:19:03 AM Longpolling is a very elegant and useful... hack. It is a hack because HTTP clients typically initiate requests, and elicit server responses. This HTTP client behavior does not cover server-initiated messages that the client is not expecting. Longpolling says "wait, until the next server-initiated message to me" Most custom designed protocols support server-to-client asynchronous messages, so there is no need for the hack. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kuzetsa on September 27, 2012, 04:12:06 AM Quote from: #eligius(freenode) (GMT-0400) North America Eastern Daylight summer time, NY) [23:39:32] <kuzetsa> is eloipool going to have stratum support? [23:39:41] <wizkid057> ... [23:39:50] <@Luke-Jr> kuzetsa: stratum sucks [23:39:58] <kuzetsa> Luke-Jr: not for bandwidth it doesn't [23:40:09] <kuzetsa> it's awesome at bandwidth ((...snip...)) [23:59:29] <@Luke-Jr> kuzetsa: you seem to be confused. [23:59:44] <@Luke-Jr> GBT supports everyone's needs, not just me/my sw/my pool [23:59:47] <kuzetsa> because I like stratum? [23:59:54] <kuzetsa> liking stratum = confused? [23:59:56] <@Luke-Jr> there is no rational reason to prefer stratum [23:59:59] <@Luke-Jr> so sure No real explanation given. Just that I'm irrational. Often as I truely am irrational or even confused, this seems more like "argumentum ad hominem (http://en.wikipedia.org/wiki/Ad_hominem)" to me. ...Edited to add: Forgot to datestamp. Local time rolled over from wednesday. Now it is Thursday, september 27th 2012 Quote from: #eligius(freenode) (GMT-0400) North America Eastern Daylight summer time, NY) [00:35:41] <@Luke-Jr> kuzetsa: this is a private channel and quoting it is against freenode policy and wiretapping laws, ESPECIALLY out of context [00:45:52] * You were kicked from #eligius by Luke-Jr (delete the post) Consider my official response "No" I'll not be bullied into censoring this. Feel free to take it up with freenode staff or law enforcement. I'm confident that I know my local laws:
Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DiabloD3 on September 27, 2012, 05:04:54 AM Angry neighborhood bastard mod here.
Luke, what did I tell you about trolling the noobs? Cut it out before one of the admins decide to either scammer/troll tag you or ban you. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: jgarzik on September 27, 2012, 05:29:38 AM The pragmatic solution is to support both GBT and stratum, in mining software.
Both are already deployed, and stratum seems to currently be winning the race for deployments. GBT is supported in bitcoind and p2pool, and has gone through the BIP process and more review, so a choice seems valid. GBT is also more easily deployed in existing software, as it uses the more standard HTTP JSON-RPC that miners are familiar with. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on September 27, 2012, 06:01:20 AM The pragmatic solution is to support both GBT and stratum, in mining software. Perhaps, but implementing Stratum is a lot of work (at least on the miner end). Better for pools to support both IMO; at least that gives the miners a choice of whether they want to retain control or blindly hand it over to the poolop.Both are already deployed, and stratum seems to currently be winning the race for deployments. I see more GBT deployment than Stratum, thankfully.P.S. Since the mods here don't do their job, I have to point out that kuzetsa is misquoting me. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: jgarzik on September 27, 2012, 06:22:04 AM The pragmatic solution is to support both GBT and stratum, in mining software. Perhaps, but implementing Stratum is a lot of work (at least on the miner end). Better for pools to support both IMO; at least that gives the miners a choice of whether they want to retain control or blindly hand it over to the poolop.That is the trade-off. GBT is far more compatible with existing software. Stratum is a wholly new protocol requiring wholly new transport implementation, but it gains some efficiency over HTTP. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kuzetsa on September 27, 2012, 07:30:57 AM ((...snip...)) I have to point out that kuzetsa is misquoting me. I just edited slightly, making the ((...snip...)) in the channel log thingy more obvious by inserting additional vertical whitespace (newline) above and below (3x newline padding instead of "singlespaced" newline) Just want to assert: There was no misquote, just an omission for the long duration of "you're wrong, no you are, no you" --- luke. feel free to fill the ((...snip...)) with what I left out. My main thought is that I prefer stratum because it gets fewer stale / invalid / orphan or whatever they're called. Stratum was working and had an installed userbase and is ASIC ready, predates GBT, etc. Longpoll over HTTP-type transport is making GBT less attractive to me than stratum. Any mining protocol using a full TCP transport or even a non-stateless version of udp, etc. (basically, anything with an active connection, therefore by definition not http) will allow for incremental changes to the workload and immediate notification of new block. There is no reason GBT's features can't be overlaid onto stratum or other non-HTTP mining protocols. (or the reverse for that matter, nothing TOO major that locks GBT's underlying structure to using HTTP) Before all of this drama started I had planned to be in bed by 30 minutes past midnight. That was 3 hours ago. guess I'll just end with this: Luke, I've been using IRC since the mid 90s. New to bitcoin and I don't need threats like this when I'm still a n00b because your agenda is threatend or whatever it was that upset you. please be cool. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: P_Shep on September 27, 2012, 03:41:40 PM kuzetsa, meet Luke :)
Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: slush on October 11, 2012, 08:07:42 PM Well, it is not "few kbytes saved versus save the network". I had quite long discussion with gmaxwell yesterday on -dev and I simply don't see any real attack vector which can be performed just because pool send merkle branch instead of transaction hashes (and this is REAL difference between Stratum and GBT). The only real attack vector which we found is that CIA will kill me, Tycho, Eleuthria, Graet, Inaba and some other pool operators, overtake our pools AND users won't notice anything and they'll keep on hashing invalid/broken blocks for CIA.
So from my view, GBT is wasting resources for no real gain. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on October 11, 2012, 09:54:57 PM I mean - there's no reason to make mining protocol less effective just because decentralization itself. I wonder what would you say to those who were, are and will be "lured" to Bitcoin with promise of decentralized network once they realise that same decentralization is being sacrificed for few (k)bytes saved. Your importance list got messed up somehow. Wait - should I be surprised with promotion of X at the cost of more centralization by pool owner? No, LOL, that's expectable! As soon as BTCGuild releases me from "0.1 BTC prison", I'll switch to solo mining using BFGMiner with GBT. It won't really bring a lot to Bitcoin network itself since my hashrate is just 30MH, but I'll sleep better knowing I'm using and supporting right thing. If decentralization was in ANY way relevant to your comments then you would have been solo mining or using P2Pool. ... which have been around for a long ....... time ... Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on October 11, 2012, 10:40:29 PM The issue is quite simple.
If you personally want 100% decentalization then you must either solo mine or use P2Pool - not GBT to a pool. GBT to a pool is not 100% 'decentalization' However, with 30MH/s, you will possibly never get any BTC solo mining. Your average expected time to find a block at the current difficulty is 5062 days ... As for pools in general ... luckily, people are allowed to choose hey? BTC isn't a Luke-jr dictatorship (even though he wants to force that on everyone) Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: mezzomix on October 12, 2012, 06:46:50 AM So from my view, GBT is wasting resources for no real gain. That depends on your point of view. As a pool operator or miner with ultimate trust to the pool operator you are right. From the perspective of a miner that cares about what goes into a block it is important to see the complete information. With Stratum, the pool operator is the one that creates bitcoin blocks and makes the rules. The miner is paid for doing some work for the operator. With GBT the pool operator is a work dispatcher but miners create the bitcoin blocks and make the rules. I don't regard more information and control as a waste of resources. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on October 12, 2012, 11:02:30 AM So from my view, GBT is wasting resources for no real gain. That depends on your point of view. As a pool operator or miner with ultimate trust to the pool operator you are right. From the perspective of a miner that cares about what goes into a block it is important to see the complete information. With Stratum, the pool operator is the one that creates bitcoin blocks and makes the rules. The miner is paid for doing some work for the operator. With GBT the pool operator is a work dispatcher but miners create the bitcoin blocks and make the rules. I don't regard more information and control as a waste of resources. The software you mine with decides - and if you use the gbt luke-jr implemented - then luke-jr decides for you ... Did you read the gbt code he wrote when you mined with his BarbieMiner? Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on October 12, 2012, 01:24:40 PM So from my view, GBT is wasting resources for no real gain. That depends on your point of view. As a pool operator or miner with ultimate trust to the pool operator you are right. From the perspective of a miner that cares about what goes into a block it is important to see the complete information. With Stratum, the pool operator is the one that creates bitcoin blocks and makes the rules. The miner is paid for doing some work for the operator. With GBT the pool operator is a work dispatcher but miners create the bitcoin blocks and make the rules. I don't regard more information and control as a waste of resources. The software you mine with decides - and if you use the gbt luke-jr implemented - then luke-jr decides for you ... Did you read the gbt code he wrote when you mined with his BarbieMiner? You wanna say something is wrong with GBT? Proof? What are the rules that BarbieMiner uses to select the transactions to put into the GBT blocks? No doubt you don't already know what they are ... without at least first looking them up now ... You see the point of my statement? Edit: and to add to that ... once you finally bother to look up and learn what rules he uses, do you know how to change it to suit what you think it should be? :) Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on October 12, 2012, 02:33:40 PM I'm using simple logic here: Well following your link I've read a lot there of you trying to tell everyone else what they should do.1. Most of people here which I think suck on so many levels, even fundamental ones, don't like Luke. 2. Luke proved he is concerned with exactly the same issues - and probably even more of them - I'm concerned with (https://bitcointalk.org/index.php?topic=117638.msg1263708#msg1263708). 3. Luke is surely smart enough to know attempting to overtake / manipulate Bitcoin with open source code won't work. Unless you can prove GBT is flawed, excluding the "it uses more bandwidth / more stales" part, you better rest your case. You wouldn't happen to be Luke-jr would you ... Anyway, you've made no case either - you've simply stated you don't care what rules Luke-jr has in GBT. I will point out the following: 1) Luke-jr's pool uses rules to ignore transactions that he will not divulge 2) Luke-jr used his pool's hashing power (without the knowledge of the miners) to effectively kill off an alt-coin 3) Luke-jr had a performance improvement for the BFL device driver for cgminer that he used himself but didn't release The simple issue is that everyone has their own opinion of what they think is best ... yet you seem to not care what rules are in the software you use ... yet you also seem to care that you don't know the rules that pools are using. Your argument's main point is in fact also your own ignorance of that point - that you have access to know something that you don't even care to check before you use it. My initial statement was to simply point out that the comments about GBT being better than Pools due to 'decentralization' were only valid when solo mining and not better than P2Pool (and worse than P2Pool when not solo mining) That fact that you blindly trust Luke-jr - well good for you. I have learnt from dealing with him that I certainly should not do that. Now to top that off, since you say that you can check these things, I will also point out that you can check these things with any pool. The transactions available on the network at any particular time are visible to everyone. The transactions placed in every block mined are also visible to everyone as soon as the block gets into the block chain You can clearly see for every block mined, what transactions were selected and what were ignored ... if you wish to. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on October 12, 2012, 02:48:25 PM I think all pools should be shut-down immidiatelly and code put in place to prevent formation of any sort of groups. This isn't possible even in theory.Issues with Luke you posted are nothing but minor incidents. They also didn't happen, or are being presented by Kano out of (important) context.For other stuff you posted, I'll wait to see Luke's reply. What in specific would you like a response to? Kano spouts so much lies and nonsense that I generally ignore his posts.Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: -ck on October 20, 2012, 12:09:05 AM An empty (coinbase-only block) StratumMP work payload is 399 bytes. An empty GBT work payload based on your example is 737 bytes. What a mess. With payloads increasing linearly with transaction count in GBT that makes each getwork equivalent much bigger than plain getwork already is the longer a block takes to solve. As bitcoin becomes more popular and transaction counts increase, gbt becomes less and less efficient. I'd hazard a guess that it may even be less efficient than plain getwork at some stage unless pools start ignoring transactions... there's a disincentive to include them.StratumMP work payloads increase 67 bytes per power of 2 transactions. GBT work payloads increase by the size of the transactions being included in a block (roughly 250 bytes for a normal tx with 1 input, 1 payout address, and 1 change address). For StratumMP, a block containing 257-512 transactions would require an extra 603 bytes, for a total of 1002 bytes. GBT would require approximately 128,000 bytes. Using a more common example, lets say each block has 100 transactions on average. StratumMP would be 868 bytes per work payload. The GBT equivalent would be approximately 26,000 bytes. Similarly, work submission payloads are quite different. GBT would require 501 bytes per share submission in your example. StratumMP requires only 109. None of the above factors in the added overhead of wrapping GBT into HTTP Requests. Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: jgarzik on October 20, 2012, 12:17:16 AM What a mess. With payloads increasing linearly with transaction count in GBT that makes each getwork equivalent much bigger than plain getwork already is the longer a block takes to solve. As bitcoin becomes more popular and transaction counts increase, gbt becomes less and less efficient. I'd hazard a guess that it may even be less efficient than plain getwork at some stage unless pools start ignoring transactions... there's a disincentive to include them. Unless something has changed, getblocktemplate should include a mode that skips sending the transactions themselves. See https://en.bitcoin.it/wiki/BIP_0023 I agree it is unlikely that non-p2pool miners will use the full bore GBT + all transactions, even if it is better for network security. But don't dismiss GBT based on faulty "full mode only" assumptions. Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: kano on October 20, 2012, 01:03:24 AM What a mess. With payloads increasing linearly with transaction count in GBT that makes each getwork equivalent much bigger than plain getwork already is the longer a block takes to solve. As bitcoin becomes more popular and transaction counts increase, gbt becomes less and less efficient. I'd hazard a guess that it may even be less efficient than plain getwork at some stage unless pools start ignoring transactions... there's a disincentive to include them. Unless something has changed, getblocktemplate should include a mode that skips sending the transactions themselves. See https://en.bitcoin.it/wiki/BIP_0023 I agree it is unlikely that non-p2pool miners will use the full bore GBT + all transactions, even if it is better for network security. But don't dismiss GBT based on faulty "full mode only" assumptions. Suddenly mining includes bitcoin network transaction and block processing. ... and anyone with half an inkling of understanding of what that entails can see the issues of having to deal with all the complications that entails and then also the importance of the bitcoin network performance of the miner ... which before this point was zero. The experiment of the issues with this already exists :) P2Pool. The only really acceptable solution to this problem, in my opinion, would be to have bitcoind handle that fully for a GBT miner - not have yet another bitcoin network implementation added onto every miner. But this still has the issue of the reliability of the local bitcoin network connection and the problems that can cause (that don't exist in the current implementations other than p2pool and solo mining) Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: jgarzik on October 20, 2012, 01:18:31 AM The problem with not receiving the transactions is that the miner program must have a full bitcoin connection to the network. No, that is not correct at all. The pool server may supply sufficient information to modify extranonce in the coinbase txn, and update the merkle root. Nothing more. And even that's going over the absolute minimum requirement of simply a block header, with ntime rolling supported. Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: -ck on October 20, 2012, 02:57:20 AM What a mess. With payloads increasing linearly with transaction count in GBT that makes each getwork equivalent much bigger than plain getwork already is the longer a block takes to solve. As bitcoin becomes more popular and transaction counts increase, gbt becomes less and less efficient. I'd hazard a guess that it may even be less efficient than plain getwork at some stage unless pools start ignoring transactions... there's a disincentive to include them. Unless something has changed, getblocktemplate should include a mode that skips sending the transactions themselves. See https://en.bitcoin.it/wiki/BIP_0023 I agree it is unlikely that non-p2pool miners will use the full bore GBT + all transactions, even if it is better for network security. But don't dismiss GBT based on faulty "full mode only" assumptions. Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: kano on October 20, 2012, 04:00:28 AM The problem with not receiving the transactions is that the miner program must have a full bitcoin connection to the network. No, that is not correct at all. The pool server may supply sufficient information to modify extranonce in the coinbase txn, and update the merkle root. Nothing more. ... Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: jgarzik on October 20, 2012, 05:08:54 AM So what are you anticipating the pools vs miners will be doing with transactions in pooled mining (ignoring p2pool)? Transactions don't need to make it to the miners, just the pool server. The pool server can handle most of the merkle building bits, and verify the returned work from the miner. The miner runs through nonce. If miner needs more work, it runs through ntime range. If miner needs more work, it increments extranonce. That will keep your ASIC miner mining through the next century ;p Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: -ck on October 20, 2012, 05:20:08 AM So what are you anticipating the pools vs miners will be doing with transactions in pooled mining (ignoring p2pool)? Transactions don't need to make it to the miners, just the pool server. The pool server can handle most of the merkle building bits, and verify the returned work from the miner. The miner runs through nonce. If miner needs more work, it runs through ntime range. If miner needs more work, it increments extranonce. That will keep your ASIC miner mining through the next century ;p In which case, we're back to my original comment... the miners need all the transactions every so often or they're doing bitcoin a disservice. Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: firefop on October 20, 2012, 06:08:37 AM The problem with not receiving the transactions is that the miner program must have a full bitcoin connection to the network. Suddenly mining includes bitcoin network transaction and block processing. ... and anyone with half an inkling of understanding of what that entails can see the issues of having to deal with all the complications that entails and then also the importance of the bitcoin network performance of the miner ... which before this point was zero. Complication like "staying online" is complicated? This is so much of a non-issue, it's just silly. Miners SHOULD be running a node even if they aren't mining on it. Depending on the pool for information that can be pulled from bitcoind isn't smart. Having to trust the pool for transaction and block processing isn't smart either. Those add unneeded complexity to an already overcomplicated system. In fact I think pools in general come at mining form the wrong direction. You don't need to develop a way to hand out work where you send it to the miner and they report back with the completed work - what you ought to be doing is making work assignment system where each miner generates, validates and submits the work for the pool - while transmitting just the nonce (or even better a nonce range completed) back to the pool. Every miner would include a msg from them to the pool stating they mined the block. And now you've got decentralized and much lower bandwidth. And you aren't trusting the pool for what it may or may not be doing with your hash rate. And this is just off the top of my head --- start thinking about 'how it should be' instead of 'how it is right now' --- and start developing in that direction also. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: -ck on October 20, 2012, 06:12:50 AM Sorry but a lot of miners mine without their mining rigs being connected to a bitcoind instance. Yes there may be one somewhere in their network, or possibly not at all and they only fire up their wallet when they want to on a separate network. What you think is ideal is asking people to do more than they currently do. They currently use pools and they trust them already. The community is very vigilant with respect to pools misbehaving. The importance of the miner doing it all themselves is grossly exaggerated by proponents of GBT. Pools will continue to do as much as they already do. It's unrealistic to expect otherwise.
Title: Re: Decentralized mining protocol standard: getblocktemplate Post by: kano on October 20, 2012, 07:05:49 AM The problem with not receiving the transactions is that the miner program must have a full bitcoin connection to the network. Suddenly mining includes bitcoin network transaction and block processing. ... and anyone with half an inkling of understanding of what that entails can see the issues of having to deal with all the complications that entails and then also the importance of the bitcoin network performance of the miner ... which before this point was zero. Complication like "staying online" is complicated? This is so much of a non-issue, it's just silly. Miners SHOULD be running a node even if they aren't mining on it. Depending on the pool for information that can be pulled from bitcoind isn't smart. Having to trust the pool for transaction and block processing isn't smart either. Those add unneeded complexity to an already overcomplicated system. In fact I think pools in general come at mining form the wrong direction. You don't need to develop a way to hand out work where you send it to the miner and they report back with the completed work - what you ought to be doing is making work assignment system where each miner generates, validates and submits the work for the pool - while transmitting just the nonce (or even better a nonce range completed) back to the pool. Every miner would include a msg from them to the pool stating they mined the block. And now you've got decentralized and much lower bandwidth. And you aren't trusting the pool for what it may or may not be doing with your hash rate. And this is just off the top of my head --- start thinking about 'how it should be' instead of 'how it is right now' --- and start developing in that direction also. The most important is yet again someone telling everyone how they should be mining. "how it should be" I'm glad you feel that is how it should be. "Miners SHOULD be running a node" Again, glad you feel that's how it should be. Lucky for everyone else, they have freedom of choice and are not under your control. Even luckier for you - you can do this already for yourself today - without trying to force everyone else to do it. -- Meanwhile, your first statement is the only one in this conversation that is "just silly" Ignoring the blatantly obvious issues and pretending to simplify to a stupid statement of "staying online" bitcoind cannot be used except for solo mining (or p2pool) - as I mentioned, there needs work done to support a combination of pool GBT mining and bitcoind to supply the transactions. i.e. miners implementing bitcoind yet again is just a problem waiting to happen. ... and that adds complexity, it doesn't reduce it at all. Do you actually understand what txn and block processing entails? Then of course there's the fact that you need to run a high performance bitcoind to ensure maximum work success - yeah everyone has one of those ... you think that may be what most pools spend a lot of effort doing? As for bandwidth, if you really want to reduce it, how is adding a bitcoind into the equation going to do that? Go stratum - problem solved :) Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: jgarzik on October 20, 2012, 05:23:33 PM 1) In general, everybody should be running a full node, just to support the network. It's not a requirement, obviously, but we need as many full nodes as possible to keep the network resistant to Sybil attacks and such. 2) p2pool is the preferred solution for the community, with centralized pools being second best. Centralized pools make it much easier to be a miner, at the cost of centralizing network power in the hands of the pool operator. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: -ck on October 20, 2012, 09:05:19 PM Sigh. In that case, with p2pool being 400 GH of a 20TH network, then GBT is only going to benefit 2% of the miners. It's pretty obvious by now you can't force people to do what you think is best. If anything, GBT with non-p2pool pooled mining will make it more likely that pools will not want to pass on lots of transactions and will be worse than getwork.
How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining? Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: firefop on October 20, 2012, 09:24:32 PM How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining? That's an idea... have to consult with Luke on that. I also wanted to point out that if we (or someone anyway) doesn't build a scalable solution per pooled mining with an integrated node... then nobody will be able to include it in the POS machines that will eventually be developed, and missing the chance for millions of additional nodes seems sad to me. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: slush on October 20, 2012, 09:27:05 PM How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining? How about adding an extension to Stratum that has some kind of mode that sends "source transactions" for given job? In that way, paranoid users will have a chance to introspect jobs generated by Stratum pools and common users which don't care will have optimized mining protocol. And yes, I'm working on it already. Important thing in this extension is that list of transactions can be broadcasted "lazily" (=few miliseconds after job updates), because that list is not important for mining itself. So it won't hurt efficiency of mining, but still give a tool for online block template monitoring. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: -ck on October 20, 2012, 09:37:43 PM slush, not a bad idea but I'm actually not questioning stratum here since I obviously think it's the better protocol for pooled mining as I've already implemented it in cgminer. It's just that this is the GBT thread, I have been asked to include GBT support as well and I'm not really happy with the protocol as is.
Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on October 20, 2012, 09:39:43 PM How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining? That's an idea... have to consult with Luke on that.I also wanted to point out that if we (or someone anyway) doesn't build a scalable solution per pooled mining with an integrated node... then nobody will be able to include it in the POS machines that will eventually be developed, and missing the chance for millions of additional nodes seems sad to me. GBT is pretty scalable for now.Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: -ck on October 20, 2012, 09:54:51 PM No, it's quite alright. I have enough to do myself as well and I'm not in the business of helping pool designs. I'll implement it as is and let the mining community determine which pools and with what protocols they prefer.
Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on October 20, 2012, 10:10:49 PM How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining? That's an idea... have to consult with Luke on that.I also wanted to point out that if we (or someone anyway) doesn't build a scalable solution per pooled mining with an integrated node... then nobody will be able to include it in the POS machines that will eventually be developed, and missing the chance for millions of additional nodes seems sad to me. GBT is pretty scalable for now.Between using Stratum and GBT at the moment that clearly says to use Stratum Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on October 20, 2012, 11:11:22 PM I do intend to improve BFGMiner's GBT implementation such that it transparently combines pooled and solo mining, at least.
Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Sitarow on October 20, 2012, 11:20:59 PM I do intend to improve BFGMiner's GBT implementation such that it transparently combines pooled and solo mining, at least. That is planned in release BFGMiner's 3.0? Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on October 20, 2012, 11:33:22 PM I do intend to improve BFGMiner's GBT implementation such that it transparently combines pooled and solo mining, at least. That's cool, but solo mining should not be an option - it should be mandatory. Else the whole project is just another house of cards. Go create an alt-coin that tells everyone what they must be doing and see how that fares. If you are right, then that alt-coin will outlast BTC and you'll be famous ................... Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: slush on October 20, 2012, 11:35:41 PM ...I think not... ...I think we should not gamble... ...Bitcoin is extreme fragile... ...demand from users... ...I know it can be done... ...it should be mandatory... Honestly, as I'm reading your posts, I think that you should create separate blockchain with those rules hardcoded. Feel free to name it NaziCoin. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: jgarzik on October 20, 2012, 11:37:57 PM Sigh. In that case, with p2pool being 400 GH of a 20TH network, then GBT is only going to benefit 2% of the miners. It's pretty obvious by now you can't force people to do what you think is best. If anything, GBT with non-p2pool pooled mining will make it more likely that pools will not want to pass on lots of transactions and will be worse than getwork. How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining? Definitely interested in extensions that broaden the userbase as much as possible. Even for bitcoind (rather than a pool server), extensions like this make sense. The old "getwork" protocol did as much work as possible on the server side, thereby minimizing the miner's work -- notably the miner did not have to care about anything besides the block header itself. No transactions or other parsing to worry about. getblocktemplate is intended to replace getwork. If there are improvements to be made, I'm quite open to that. Ping me on IRC, if you want to rapidly iterate some changes to make bitcoind's GBT work better for a wider selection of mining softwares. I did the final getblocktemplate polish, simplifying and cleaning things up from its original BIP 22/getmemorypool state. If BIP 22 / BIP 23 are currently missing something you want... I want to put it in there. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on October 20, 2012, 11:41:29 PM I do intend to improve BFGMiner's GBT implementation such that it transparently combines pooled and solo mining, at least. That is planned in release BFGMiner's 3.0?I do intend to improve BFGMiner's GBT implementation such that it transparently combines pooled and solo mining, at least. That's cool, but solo mining should not be an option - it should be mandatory. Else the whole project is just another house of cards.Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: -ck on October 20, 2012, 11:58:43 PM Sigh. In that case, with p2pool being 400 GH of a 20TH network, then GBT is only going to benefit 2% of the miners. It's pretty obvious by now you can't force people to do what you think is best. If anything, GBT with non-p2pool pooled mining will make it more likely that pools will not want to pass on lots of transactions and will be worse than getwork. How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining? Definitely interested in extensions that broaden the userbase as much as possible. Even for bitcoind (rather than a pool server), extensions like this make sense. The old "getwork" protocol did as much work as possible on the server side, thereby minimizing the miner's work -- notably the miner did not have to care about anything besides the block header itself. No transactions or other parsing to worry about. getblocktemplate is intended to replace getwork. If there are improvements to be made, I'm quite open to that. Ping me on IRC, if you want to rapidly iterate some changes to make bitcoind's GBT work better for a wider selection of mining softwares. I did the final getblocktemplate polish, simplifying and cleaning things up from its original BIP 22/getmemorypool state. If BIP 22 / BIP 23 are currently missing something you want... I want to put it in there. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: firefop on October 21, 2012, 01:30:01 AM I do intend to improve BFGMiner's GBT implementation such that it transparently combines pooled and solo mining, at least. That's cool, but solo mining should not be an option - it should be mandatory. Else the whole project is just another house of cards.Your efforts are noble, but it might be better for you to start working on alt currency, the one with really strong foundation. Let retards like those 2 above, who despise you, me and anyone else not fitting their utopian views, finish off Bitcoin. Don't waste your time here, seriously. I really like your prev-post with the graph substrata - you basically said everything I was thinking but couldn't articulate. Thank you for that! However, I respectfully disagree. We need to encourage people like Luke to stick around, add even more sanity to the development of the network and slowly marginalize those who don't think in the right direction. If you look at the network as a whole, that's already happening. 2 years ago I wasn't here and you'll see more actual computer scientists become involved in development as bitcoin grows. @Luke - incentives are nice - but I'd settle for having it be the default option - If you don't have time/inclination to build a custom install script to set that all up (when/if it's supported) I'd be happy to create and submit it to you for approval / publication. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on October 21, 2012, 02:52:58 AM ... Sigh - another nut-case.We need to encourage people like Luke to stick around, add even more sanity to the development of the network and slowly marginalize those who don't think in the right direction. ... Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: firefop on October 21, 2012, 03:46:12 AM Sigh - another nut-case. Hate all you want, but it doesn't invalidate anything I've said. The fact remains that certain posters are more interested in trash talking rather than contributing, while others are overworking themselves to try and get this whole bitcoin thing working to the point where we can make it grow into it's potential. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on October 21, 2012, 04:14:44 AM Seriously - what are you looking for?
Decentralisation or control? Pick one. Saying that you want decentralisation ... except it has to be how you or Luke-jr or whoever else wants it ... isn't decentralisation. As soon as you start deciding how things like mining MUST be done, you've lost the plot. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Inaba on October 21, 2012, 04:36:34 AM Requiring a full bitcoind instance on a mining node is ludicrous, at least in the current implementation of bitcoind. It's far, far, far too resource intensive to put on most serious mining setups. I don't want to run heavy weight computing back end, just to satisfy the ridiculously complex blockchain processing for a machine that is intended to solely mine, which requires few resources.
Now, if you want to redesign a bitcoind that doesn't require heavy computing resources, then I'm listening, but until then, it's a non-starter. A mining backend is a netbook, DD-WRT router, Raspberry Pi, Sheeva Plug or Android tablet. Trying to run a full blockchain on any of those would be an exercise in futility. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: -ck on October 21, 2012, 05:59:06 AM Requiring a full bitcoind instance on a mining node is ludicrous, at least in the current implementation of bitcoind. It's far, far, far too resource intensive to put on most serious mining setups. I don't want to run heavy weight computing back end, just to satisfy the ridiculously complex blockchain processing for a machine that is intended to solely mine, which requires few resources. Which means you pretty much unintentionally agree with my position: GBT in its current form is not advantageous to miners. Luckily jgarzik is open to modifying it based on what I have recommended. We will be talking about it in the near future.Now, if you want to redesign a bitcoind that doesn't require heavy computing resources, then I'm listening, but until then, it's a non-starter. A mining backend is a netbook, DD-WRT router, Raspberry Pi, Sheeva Plug or Android tablet. Trying to run a full blockchain on any of those would be an exercise in futility. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: P_Shep on October 21, 2012, 06:11:59 AM Requiring a full bitcoind instance on a mining node is ludicrous, at least in the current implementation of bitcoind. It's far, far, far too resource intensive to put on most serious mining setups. I don't want to run heavy weight computing back end, just to satisfy the ridiculously complex blockchain processing for a machine that is intended to solely mine, which requires few resources. Now, if you want to redesign a bitcoind that doesn't require heavy computing resources, then I'm listening, but until then, it's a non-starter. A mining backend is a netbook, DD-WRT router, Raspberry Pi, Sheeva Plug or Android tablet. Trying to run a full blockchain on any of those would be an exercise in futility. I've been trying to get bitcoind on the router... not having much luck so far. Not entirely given up yet. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Inaba on October 21, 2012, 07:09:20 AM Well, I've been mining with GBT for about a month now on a lower power netbook with no problems (and no bitcoind), so I'm not sure what you're referring to exactly?
Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: -ck on October 21, 2012, 07:13:07 AM Well, I've been mining with GBT for about a month now on a lower power netbook with no problems (and no bitcoind), so I'm not sure what you're referring to exactly? https://bitcointalk.org/index.php?topic=108854.msg1284623#msg1284623Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Inaba on October 21, 2012, 06:50:12 PM Interesting... I would like to know more about that at some point. I've not run into any problems and my server load is dramatically decreased. Regardless, though, I'm not sure that pools ignoring some transactions in favor of others is necessarily a negative thing. Right now, I limit the amount of transactions I put into any one block and I don't see any major issue with favoring higher transaction fee transactions over lower ones.
If there are no higher fee transactions waiting, then the lower fee ones will get processed and that's as designed as far as I know. I'm not saying it's a bad thing to "fix" GBT, but I've not run into any problems so far and I'd like to know at what point GBT becomes more cumbersome than getwork from a transaction standpoint. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: -ck on October 21, 2012, 08:48:23 PM Interesting... I would like to know more about that at some point. I've not run into any problems and my server load is dramatically decreased. Regardless, though, I'm not sure that pools ignoring some transactions in favor of others is necessarily a negative thing. Right now, I limit the amount of transactions I put into any one block and I don't see any major issue with favoring higher transaction fee transactions over lower ones. The issue is the payload the longer a block takes to solve as the number of transactions rises. It would not be surprising if the pool was sending 100k sized payloads with all the transactions after 10 minutes on the block. While the load on the pool is definitely less, the data being spewed out to miners will eventually rise, and the more transactions and more popular bitcoin becomes, the larger the payload grows in a linear fashion with transactions. Eventually sending out 100kb payloads say every 30 or 60 seconds from the pool to all your miners will become a burden. Prioritising transactions is your prerogative but with this disincentive to sending transactions we will see a deepbit like situation where pool ops will put a static upper limit on the amount of transactions in total they care about including. This is much worse for bitcoin in general. But with an extension to the gbt protocol that more closely resembles the merkle root base of stratum it can easily be fixed.If there are no higher fee transactions waiting, then the lower fee ones will get processed and that's as designed as far as I know. I'm not saying it's a bad thing to "fix" GBT, but I've not run into any problems so far and I'd like to know at what point GBT becomes more cumbersome than getwork from a transaction standpoint. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on October 21, 2012, 10:43:49 PM Real world considerations should be part of GBT - not ideal world ...
The GBT data transfer should currently ALWAYS be high at the moment ... Checking the bitcoind debug.log for the last 3 days: The smallest poolsz was: 10/20/12 06:58:09 CTxMemPool::accept() : accepted a661e71a68 (poolsz 579) Yes the minimum outstanding transactions for the last 3 days (which was after block 204102) was 579 waiting txns (also note that I restarted my bitcoind 4 days ago - so that figure may be low ...) So unless pools are ignoring lots of transactions, there should never have been a GBT workload sent from any GBT pool with fewer than a few hundred transactions in it for the past 3 days. ... but of course I've very little doubt that isn't the case ... since we know what pool software supports GBT. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Inaba on October 22, 2012, 09:54:16 PM You forgot to specify what CPU you are using, also how much RAM total you have. Also your HD usage.. 4.5GB. That is larger than the entire usage of my 500 GH/s farm. Combined. That 0 - 2% CPU usage you quote, I'd wager it's a modern Intel or AMD based CPU. You just go ahead and try to run that on an ARM based CPU or a C6 or something, see how far you get (you'll never finish loading the block chain, new blocks will come in faster than you can process old ones).
The blockchain is not suitable for lower power environments. It simply isn't. The blockchain would have to be reduced to under a few megabytes and RAM usage would have to be a few hundred kilobytes before it becomes feasible to run a full node on a mining rig. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on October 22, 2012, 10:11:29 PM The blockchain would have to be reduced to under a few megabytes Getting off-topic here, but this is already possible.and RAM usage would have to be a few hundred kilobytes This is completely unreasonable. Merely being a C++ program requires more than a few hundred KB.Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Inaba on October 23, 2012, 12:32:40 AM You forgot to specify what CPU you are using, also how much RAM total you have. Also your HD usage.. 4.5GB. That is larger than the entire usage of my 500 GH/s farm. Combined. That 0 - 2% CPU usage you quote, I'd wager it's a modern Intel or AMD based CPU. You just go ahead and try to run that on an ARM based CPU or a C6 or something, see how far you get (you'll never finish loading the block chain, new blocks will come in faster than you can process old ones). Athlon 64 x2 4800+ and 2GB RAM. Bitcoin client mem usage can be as low as few MB and as high as 100+ MB, depending on something, no idea what exactly. HDD usage value stands for client installation and stuff in datadir. I have no clue about ARM or C6, but here's the nasty part = you invested huge money into system that can't help Bitcoin network if all pools go down. That's quite a gamble, you know. The blockchain is not suitable for lower power environments. It simply isn't. The blockchain would have to be reduced to under a few megabytes and RAM usage would have to be a few hundred kilobytes before it becomes feasible to run a full node on a mining rig. Nowhere I said my suggestion must be implemented right now. Think long-term. Step by step - in right direction - and it can be done. Are you using the "royal" "I" in this case? Because I did no such thing - I can stand up a new pool on my local network in about 30 minutes if all the pools suddenly went down. But anyway, comparing an A64 X2 to an ARM processor is not exactly a fair comparison. Go ahead and try to load the blockchain on an old netbook and see how far you get before everything grinds to a halt. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DavinciJ15 on October 26, 2012, 09:18:28 PM Go ahead and try to load the blockchain on an old netbook and see how far you get before everything grinds to a halt. I did that on a Asus EEE and it took 2 weeks! When I turn it on after a day it's about 1 hour to catch up. Crazy! Anyhow: Luke-jr! Personally I think stratum is better, however, if you want to win create a pool back end that's easy to use. Better doesn't matter when it comes to wanting to accomplish a task. I want a pool, and so does many other people! So create a sold backend that's easy to config and setup and you win. I know you have created Eloipool but I'm willing to bet it's not so easy to setup. ;) Another option is to create vm for people to download with everything working only thing that the end user needs to do is add a username and password to the worker table and they can mine and shares can be obtained from the shares table. Just my 2 Satoshis. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on October 26, 2012, 09:26:14 PM Personally I think stratum is better, however, if you want to win Win is when Stratum undergoes the BIP process and gets feature parity with GBP. As I understand it, slush is taking it down that road now.create a pool back end that's easy to use. Yes, I did that about a year ago...I know you have created Eloipool but I'm willing to bet it's not so easy to setup. ;) Easier than the alternatives :)Another option is to create vm for people to download with everything working only thing that the end user needs to do is add a username and password to the worker table and they can mine and shares can be obtained from the shares table. Eh, why do we need passwords? :PTitle: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DavinciJ15 on October 27, 2012, 10:54:14 PM Easier than the alternatives :) I see your point. Eh, why do we need passwords? :P It was used for HTTP reuse of code but I guess with a new protocol it's not needed anymore eh? I can't think of any reason why we should have miner passwords. LOL Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on October 27, 2012, 10:57:12 PM Eh, why do we need passwords? :P It was used for HTTP reuse of code but I guess with a new protocol it's not needed anymore eh? I can't think of any reason why we should have miner passwords. LOLTitle: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on October 31, 2012, 12:31:59 PM So ... I've had a go at a pool OP about ignoring transactions and found to my surprise they aren't the ones choosing the transactions.
GBT is. Here's a run of 15 block for the said pool and look at how bad this is for the bitcoin network - shocking in my opinion: Bitcoind's GBT is deciding the txns, and it's regularly making small blocks when the bitcoin network constantly has a very large number of outstanding transactions (>4000) This of course means that everyone's bitcoind is sitting there with a list of many thousands of unconfirmed transactions waiting to go into blocks ... and it is bitcoind's own GBT that is part of the cause of it ... Code: Block size txns 6 blocks (1/3) in there you could consider large (>100 txn), 6 blocks (1/3) are very small (~10k or less and few txns) Is this basically saying that GBT doesn't adhere to the fee structure of bitcoind? Is GBT's aim to have every bitcoind on the BTC network constantly running large (>4000) txn memorypools and keep them growing? ... I will also point out that the Eligius pool has for many months restricted all it's blocks to a limit of only 32 transactions and to ignore transactions that pay full fees if they are near the lower limit - as stated by the new pool OP. The old pool OP is the same proponent of the so called decentralisation holy grail GBT ... that is CLEARLY showing itself to be bad for BTC and the BTC network. I see the link there as expected, not surprising. The argument given for the 32 transaction limit was that it reduced the risk of orphans (which of course it does) however, the issue there is that larger pools with better software did not have the orphan problem Eligius had and instead of fixing the performance problems of the pool software, Eligius' solution was instead to detriment the BTC network by reducing the number of transactions being confirmed and thus extend confirmation times. Even the new pool op has clearly stated that his intererst is in short term return, not considering the bad long term effect of such short sighted decisions. i.e. a quick buck with risk of long term problems vs a long term strategy that will also pay much better in the long run. I've seen Luke-Jr make many arguments about what's good for BTC but only to find that he himself is a proponent of doing thing that are quite simply and clearly bad for it - and it seems that GBT follows this same path also. Edit: this also explain why people mining with GBT aren't having the expected network problems of receiving hundreds of transactions with every request for work - since GBT ignores most transactions. Easy way to see it: ./bitcoind getblocktemplate | grep sigops | wc Sometimes you get a lot and sometimes you don't get many at all - even though if you check your debug.log it will show thousands of unconfirmed transactions (poolsz) Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: abracadabra on October 31, 2012, 01:29:17 PM If GBT is planning on going anywhere, this has got to get fixed. Ignoring transactions is bad.
If this doesn't get fixed, I doubt pools will support GBT. Waiting to see how LukeJr responds. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DrHaribo on October 31, 2012, 05:31:58 PM bitcoind has always made a decision about which transactions to include in a block. That has nothing to do with GBT which is just a different API for getting this (and other) data out of bitcoind.
By the way, kano, if you have a suggestion for how bitcoin can be improved, you can set up a pull request on github and I'm sure the bitcoin developers will consider it. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on October 31, 2012, 06:15:43 PM If GBT is planning on going anywhere, this has got to get fixed. Ignoring transactions is bad. Kano is a troll and basically a liar.GBT is a protocol, it doesn't and can't "ignore transactions". Bitcoind, by default design, has always ignored spam transactions when it can detect them. The Bitcoin protocol, by Satoshi's original paper design, specifically depends on miners choosing to ignore transactions that don't meet miner-defined policies. But again, one of this has anything to do with GBT; transaction acceptance works exactly the same whether you use getwork, GBT, or Stratum, except that GBT moves the final decision for which transactions to include to the miner (note that no mining software supports changing the pool's decisions yet, but I plan to have future versions of BFGMiner support adding more transactions from bitcoind to collect on fees). Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on October 31, 2012, 07:24:22 PM GBT moves the final decision for which transactions to include to the miner Transactions are included or excluded from the block based on what criterias? Could criterias be made user-selectable or changeable? Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on October 31, 2012, 10:05:21 PM GBT moves the final decision for which transactions to include to the miner Transactions are included or excluded from the block based on what criterias? Could criterias be made user-selectable or changeable? The only point you have made is that GBT is a protocol and the implementations are the fault of the implementers ... So who implemented them? You in your cloned BarbieMiner, you in your pool, and ... "someone" in bitcoind. I guess the other place to look will be P2Pool - but I don't know if that just gets the results from bitcoind or if it processes those results. BitMinter blindly accepts the results from bitcoind (according to DrHaribo's response) -- Luke-Jr if you are going to use you standard method of distributing FUD by hiding behind definitions of words then I guess you are really saying the GBT is simply an idea and no use in the real world. The implementations are what use it and if the implementations are bad for BTC then GBT is bad for BTC. The two first implementations I listed are also by you ... -- I've explained to you and also mentioned to Gavin a suggestion about the problems with bitcoind and dealing with large blocks. Specifically that each block transferred from one pool to another sends way too much data that most bitcoinds already know and does way too much work that the bitcoinds have already done. That's as far as I went - and seems a good enough solution. (Edit: and GBT doesn't even use this simple and obvious idea) I don't deal with bitcoin code development directly, there are just way too many issues with doing that and I have (made clear) elements of it that I disagree with and twice when I've raised those issues they have been shot down. Fine - I disagree with the devs - so I don't think throwing code in that direction would be productive. Ideas and issues are what I do throw in that direction. Meanwhile, your solution at your pool was to reduce the number of transactions per block to 32 and ignore full paying transactions (Satoshi Dice) and who knows what else because of that. I guess the reason you would not divulge that information before was that people would then know that your own pool was ignoring full paying transactions and that would show you as the hypocrite you are. -- Anyway, I've reset both my bitcoinds and will see how big the memorypool is again in a week or two. One of my 0.7.0 bitcoind started: 09/22/12 08:30:55 Bitcoin version v0.7.0.3-g0a4e67a-beta () It first reached 4000 after less than 4 days: 09/26/12 00:04:23 CTxMemPool::accept() : accepted 8f15d5ec48 (poolsz 4000) The last time it was under 1000: (one month after it started) 10/22/12 13:35:31 CTxMemPool::accept() : accepted 0a8e298047 (poolsz 999) The last time it was under 2000: (two days after that) 10/24/12 15:41:28 CTxMemPool::accept() : accepted f14b7e229c (poolsz 1999) The last time it was under 3000: (almost 8.5 hours after that) 10/25/12 00:02:21 CTxMemPool::accept() : accepted 879a991172 (poolsz 2999) ... and it kept growing ... Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DavinciJ15 on October 31, 2012, 10:52:53 PM If GBT is planning on going anywhere, this has got to get fixed. Ignoring transactions is bad. Kano is a troll and basically a liar.GBT is a protocol, it doesn't and can't "ignore transactions". Calling people names does not prove you are correct. People called me a troll and liar because I said money is created when people go into debt at a bank. So does someone calling me a name make it true? Is the documentation found in Modern Money Mechanics created by the banks a lie? "Of course, they do not really pay out loans from the money they receive as deposits. If they did this, no additional money would be created. What they do when they make loans is to accept promissory notes in exchange for credits to the borrowers' transaction accounts." Modern Money Mechanics I understand the truth of our monetary system and this is why I like bitcoins as a solution to our unfair monetary system. So lets forget the name calling and find out the truth. Finally I am not sure if it's a bad thing that miners are ignoring transactions as that's part of the market saying... hey you are not paying me enough to include your transaction! There will always be miners that work for low to no transaction fees because they believe in the system, thus there is no need to be a "government" and attempt to force good behaviour. Just my 2 satoshis Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on October 31, 2012, 11:17:22 PM Calling people names does not prove you are correct. Ignoring is the standard appropriate way to deal with trolls. I just put that out there so newbies know why he isn't getting a response.There's no basis to Kano's accusations in any case, so not much really to say about them other than that they're lies. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DavinciJ15 on October 31, 2012, 11:55:13 PM Calling people names does not prove you are correct. Ignoring is the standard appropriate way to deal with trolls. I just put that out there so newbies know why he isn't getting a response.There's no basis to Kano's accusations in any case, so not much really to say about them other than that they're lies. I may not agree that it's an issue but Kano is not a troll for voicing his a opinion. I have a YouTube channel and I get a lot of trolls and Kano sounds nothing like the trolls I get. I got so many I did a video for them... http://www.youtube.com/watch?v=8sQaIeHvuM0 It was funny and even the trolls loved it. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: firefop on November 01, 2012, 12:58:06 AM I may not agree that it's an issue but Kano is not a troll for voicing his a opinion. I have a YouTube channel and I get a lot of trolls and Kano sounds nothing like the trolls I get. I got so many I did a video for them... http://www.youtube.com/watch?v=8sQaIeHvuM0 It was funny and even the trolls loved it. Hey D, big fan. That being said, Luke's right. Kano's in this for profit more than love (obvious from his post history here if nothing else). In addition he tends to bend the truth past the breaking point quite often. These two statements are fact I'd encourage you to verify their veracity for yourself. The following is my personal opinion which I happen to believe is right: The reason that LukeJr supporters tend to discount poster like Kano is the simply because Luke has a technical mind, understands how things work and also is a proponent if not an advocate of the bitcoin network's need to be transparent. That's basically what this entire post is about... Luke (and myself and anyone who loves liberty) would rather take a slightly less streamlined process that we can see through than trust a pool op to inform us of what's happening. So sure, Stratum is less bandwidth at the cost of giving the pool op the ability do whatever they want behind it and the miners never knowing. It makes me wonder what they'd like to hide. Also, I'd call it trolling when Kano comes here to decry the 'bad design' of GBT when the real issue is exactly what I've posted above... at the very least he's dishonest for not simply making his case for what it is... the desire to open a wedge into bitcoin that can be exploited later either by himself or others... without those of us actually doing the work knowing about it. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: -ck on November 01, 2012, 12:59:43 AM ...And where do I fit in? Despite what it may look like, this is a technical discussion about the disadvantages of GBT.
Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DavinciJ15 on November 01, 2012, 01:45:02 AM I may not agree that it's an issue but Kano is not a troll for voicing his a opinion. I have a YouTube channel and I get a lot of trolls and Kano sounds nothing like the trolls I get. I got so many I did a video for them... http://www.youtube.com/watch?v=8sQaIeHvuM0 It was funny and even the trolls loved it. Hey D, big fan. That being said, Luke's right. Kano's in this for profit more than love (obvious from his post history here if nothing else). In addition he tends to bend the truth past the breaking point quite often. These two statements are fact I'd encourage you to verify their veracity for yourself. The following is my personal opinion which I happen to believe is right: The reason that LukeJr supporters tend to discount poster like Kano is the simply because Luke has a technical mind, understands how things work and also is a proponent if not an advocate of the bitcoin network's need to be transparent. That's basically what this entire post is about... Luke (and myself and anyone who loves liberty) would rather take a slightly less streamlined process that we can see through than trust a pool op to inform us of what's happening. So sure, Stratum is less bandwidth at the cost of giving the pool op the ability do whatever they want behind it and the miners never knowing. It makes me wonder what they'd like to hide. Also, I'd call it trolling when Kano comes here to decry the 'bad design' of GBT when the real issue is exactly what I've posted above... at the very least he's dishonest for not simply making his case for what it is... the desire to open a wedge into bitcoin that can be exploited later either by himself or others... without those of us actually doing the work knowing about it. That being said, Luke's right. Kano's in this for profit more than love (obvious from his post history here if nothing else). In addition he tends to bend the truth past the breaking point quite often. These two statements are fact I'd encourage you to verify their veracity for yourself. Thanks I will keep reading and Kano's posts. The reason that LukeJr supporters tend to discount poster like Kano is the simply because Luke has a technical mind, understands how things work and also is a proponent if not an advocate of the bitcoin network's need to be transparent. That's basically what this entire post is about... Luke (and myself and anyone who loves liberty) would rather take a slightly less streamlined process that we can see through than trust a pool op to inform us of what's happening. So sure, Stratum is less bandwidth at the cost of giving the pool op the ability do whatever they want behind it and the miners never knowing. It makes me wonder what they'd like to hide. Also, I'd call it trolling when Kano comes here to decry the 'bad design' of GBT when the real issue is exactly what I've posted above... at the very least he's dishonest for not simply making his case for what it is... the desire to open a wedge into bitcoin that can be exploited later either by himself or others... without those of us actually doing the work knowing about it. Good points, I'm not against GetBlockTemplate (GBT) I just want something that works for everyone not just those with a mensa membership card. :p lol With that said, I understand that bitcoin is a complex protocol to begin with and we don't want pools to consolidate and use the hashing power for evil once there is nothing anyone can do about it. However that is not an issue right now so stratum is fine transition from GPU/FPGA mining to ASICs. In the future we will need GBT. Bottom line I want GBT but stratum gets the job done faster and with less work for pools, miners and software producers. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on November 01, 2012, 01:50:53 AM Bottom line I want GBT but stratum gets the job done faster and with less work for pools, miners and software producers. In reality, GBT is much less work for pools/miners/software developers to implement, and does everything Stratum does with a little more overhead (that probably won't be a problem for a while).I agree GBT could benefit from adopting some of Stratum's design concepts (I wish slush participated in the GBT BIP process so these could have been integrated from the start!), but in practice we have plenty of time for that before it's an issue. IMO, most likely the next-generation mining protocol to replace GBT (and Stratum) will use a persistent TCP connection, fully transparent, and probably based on the original Bitcoin p2p protocol (it has some convenient overlap, and eliminates the small JSON overhead both GBT and Stratum are subject to) - and maybe even a p2p-friendly design. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DavinciJ15 on November 01, 2012, 02:00:25 AM Bottom line I want GBT but stratum gets the job done faster and with less work for pools, miners and software producers. In reality, GBT is much less work for pools/miners/software developers to implement, and does everything Stratum does with a little more overhead (that probably won't be a problem for a while).I agree GBT could benefit from adopting some of Stratum's design concepts (I wish slush participated in the GBT BIP process so these could have been integrated from the start!), but in practice we have plenty of time for that before it's an issue. IMO, most likely the next-generation mining protocol to replace GBT (and Stratum) will use a persistent TCP connection, fully transparent, and probably based on the original Bitcoin p2p protocol (it has some convenient overlap, and eliminates the small JSON overhead both GBT and Stratum are subject to) - and maybe even a p2p-friendly design. This is my opinion based on what I have read in the threads here, I have no experience developing code for these mining protocols so you take my opinion for what it's worth. (not even a satoshi) :p With that said it's easy to conclude that since cgminer and others have implemented stratum over GPT kind of gives my opinion a bit of legs to stand on. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: wizkid057 on November 01, 2012, 02:16:34 AM *snip* Meanwhile, your solution at your pool was to reduce the number of transactions per block to 32 and ignore full paying transactions (Satoshi Dice) and who knows what else because of that. I guess the reason you would not divulge that information before was that people would then know that your own pool was ignoring full paying transactions and that would show you as the hypocrite you are. *snip* I'm going to just chime in to address this particular comment, if for no other reason but to discredit the kano. In a conversation yesterday evening this particular issue regarding Eligius was discussed on IRC, and kano was given a link to the post on this forum from months prior stating that these actions were being taken on Eligius. Now, since I personally pointed this out to him, this post of his is clearly after I pointed this out (October 31, 2012, 10:05:21 PM is after both October 30, 2012, 11:57:55 PM and also well after the original post date below), and yet he still publicly says that this information was "not divulged before"? Well sir, you deserve to be called a liar, and it should be obvious that you are simply trying to discredit the work of others in the community, to what end I do not know. Want the proof? I'm not going to just make accusations without proof. Here it is: Bitcointalk post from June 17th 2012: https://bitcointalk.org/index.php?topic=23768.msg968819;topicseen#msg968819 (https://bitcointalk.org/index.php?topic=23768.msg968819;topicseen#msg968819) This post clearly states publicly what the pool operator at the time decided to do with Satoshi Dice transactions and limiting the transaction count. *snip* For now, I am blocking transactions to 1dice* addresses and limiting our blocks to 32 transactions until we've caught up on the extra credit or at least have a viable alternative solution. *snip* For reference, here is a snippet of IRC log from yesterday evening (timestamps in UTC-0400): Code: [23:54:25] <wizkid057> 2012-10-30 23:57:57,913 merkleMaker WARNING Making merkle tree with 132 transactions (ideal: 132; max: 132) Tired of all the childishness around here, especially now that I've taken over management of Eligius. Apparently that means that I get to field the BS that kano usually directs towards Luke-Jr, and I'm not going to tolerate that kind of trash. It gets really old and is very counter productive. Stop with the FUD because you're just making yourself look stupid. -wk Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: wizkid057 on November 01, 2012, 02:23:37 AM Bottom line I want GBT but stratum gets the job done faster and with less work for pools, miners and software producers. In reality, GBT is much less work for pools/miners/software developers to implement, and does everything Stratum does with a little more overhead (that probably won't be a problem for a while).I agree GBT could benefit from adopting some of Stratum's design concepts (I wish slush participated in the GBT BIP process so these could have been integrated from the start!), but in practice we have plenty of time for that before it's an issue. IMO, most likely the next-generation mining protocol to replace GBT (and Stratum) will use a persistent TCP connection, fully transparent, and probably based on the original Bitcoin p2p protocol (it has some convenient overlap, and eliminates the small JSON overhead both GBT and Stratum are subject to) - and maybe even a p2p-friendly design. This is my opinion based on what I have read in the threads here, I have no experience developing code for these mining protocols so you take my opinion for what it's worth. (not even a satoshi) :p With that said it's easy to conclude that since cgminer and others have implemented stratum over GPT kind of gives my opinion a bit of legs to stand on. By that logic you should say GBT is better because its actually built directly into the official bitcoin client. I don't see how stratum is better just because cgminer implemented it. Just because something is implemented somewhere prominent doesn't make it superior. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on November 01, 2012, 02:33:03 AM I may not agree that it's an issue but Kano is not a troll for voicing his a opinion. I have a YouTube channel and I get a lot of trolls and Kano sounds nothing like the trolls I get. I got so many I did a video for them... http://www.youtube.com/watch?v=8sQaIeHvuM0 It was funny and even the trolls loved it. Hey D, big fan. That being said, Luke's right. Kano's in this for profit more than love (obvious from his post history here if nothing else). In addition he tends to bend the truth past the breaking point quite often. These two statements are fact I'd encourage you to verify their veracity for yourself. The following is my personal opinion which I happen to believe is right: The reason that LukeJr supporters tend to discount poster like Kano is the simply because Luke has a technical mind, understands how things work and also is a proponent if not an advocate of the bitcoin network's need to be transparent. That's basically what this entire post is about... Luke (and myself and anyone who loves liberty) would rather take a slightly less streamlined process that we can see through than trust a pool op to inform us of what's happening. So sure, Stratum is less bandwidth at the cost of giving the pool op the ability do whatever they want behind it and the miners never knowing. It makes me wonder what they'd like to hide. Also, I'd call it trolling when Kano comes here to decry the 'bad design' of GBT when the real issue is exactly what I've posted above... at the very least he's dishonest for not simply making his case for what it is... the desire to open a wedge into bitcoin that can be exploited later either by himself or others... without those of us actually doing the work knowing about it. No I wont ignore that OT rubbish. Quote That being said, Luke's right. Kano's in this for profit more than love (obvious from his post history here if nothing else). In addition he tends to bend the truth past the breaking point quite often. These two statements are fact I'd encourage you to verify their veracity for yourself. i.e. rubbish that you seem to think someone else has to prove and you can't since it's not true.Please - show me these posts of mine you are talking about. I'd love to have a read. ... and if you can't show them ... then I guess that shows how pointless your post is. You do realise that every argument Luke-Jr and I have had about implementation issues in cgminer - he has later either done what I said or say he would do it ... Transparency? You've blindly decided to believe what Luke-Jr has said and not ever bothered to check it. I've found this problem. As for pools - that is simply you being too lazy to check what they do. All the transactions that go into a block are shown in the block. All transactions that do NOT go into a block are there for ANYONE to see. If you don't want to spend ANY effort seeing what they are (as simple as reading the debug.log of bitcoind) that is your ignorance or laziness, nothing more. -- Hmm you really gonna guess at my financial situation and motive? And try to use that as an argument against me :D firepop - subSTRATA - who's next? I think the problems of higher difficulty and BTC halving are bringing the nut-cases out into the open. -- Meanwhile ... as I've implied before ... anyone with a linux bitcoind: Firstly have a look at the last few lines of your debug.log to see how big your memorypool is Secondly ./bitcoind getblocktemplate There are 2 issues here: 1) When getblocktemplate does return a lot of txn's per work request - see how large it is ... then consider what you have to send back in reply with each share you find 2) getblocktemplate also returns a small number of transactions when the memory pool isn't small ... say even when the memory pool is in the thousands ... Arguing that the protocol design is good yet it allows (and all implementations currently use) this sort of thing is purely deceit. This is what it currently does. This needs to be fixed before those who don't check such things and believe the rubbish that Luke-Jr spouts start using it. Oops - too late it's already in use ... oh well. Actually, the biggest issue with GBT is the lack of GBT transparency itself in the comments the proponents make. Ignoring or hiding the points about it that will negatively affect it's users and BTC. Nice to see Luke-Jr actually mention ... some ... of them ... and yet again hide the important points. Firstly, GBT is less work to implement ONLY because it uses an inferior method of data transfer that is already available and used by GetWork and bitcoind. It has MUCH more overhead than Stratum. Consider the data transfer size relationship to txn numbers. As you mine until a block change, the txn number will continue to increase. This leads to something that is in direct opposition to the requirements of BTC. BTC requires transactions to be confirmed - the more the better. However, with Stratum the relationship is order log2 N with GBT it is order N Not only that, the size of the actual items sent are much larger in GBT. GBT sends the whole transaction each time, Stratum just sends the side of the merkle tree and information for the coinbase transaction. This give direct incentive to reduce the number of txns confirmed by anyone who implements GBT - including Luke-Jr himself on his own pool Then of course there is Quote IMO, most likely the next-generation mining protocol to replace GBT (and Stratum) will use a persistent TCP connection What Stratum already does - as he already knows.Meh lots of posts while I was writing this ... Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: -ck on November 01, 2012, 02:39:26 AM One more point about the GBT protocol. If a miner chooses which transactions they include rather than accept whichever transactions the pool/bitcoind has offered in the template, the miner has to submit all the transactions with every share he submits.
A quick check ./bitcoind getblocktemplate at this very second produces a output that is 469441 bytes! If every template request was this size and the miner used the customise coinbase option and chose transactions, they would be receiving 400kb for the block template and each share submitted would be 400kb unless they started withholding transactions. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on November 01, 2012, 02:44:14 AM One more point about the GBT protocol. If a miner chooses which transactions they include rather than accept whichever transactions the pool/bitcoind has offered in the template, the miner has to submit all the transactions with every share he submits. No, that's (part of) what Block Proposal is for.Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: firefop on November 01, 2012, 05:29:08 AM ...And where do I fit in? Despite what it may look like, this is a technical discussion about the disadvantages of GBT. imo - you're at least honest in your intent and presentation. I'm not saying that GBT couldn't or shouldn't be replaced with something that is better that what it is now... I'm just saying given the choice between gbt and stratum --- I'll take transparent and clunkier over obfuscated and streamlined. Also I'm not saying 'kill stratum with fire' but I won't be using it until the transparency issue is addressed. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: -ck on November 01, 2012, 05:34:05 AM ...And where do I fit in? Despite what it may look like, this is a technical discussion about the disadvantages of GBT. imo - you're at least honest in your intent and presentation. I'm not saying that GBT couldn't or shouldn't be replaced with something that is better that what it is now... I'm just saying given the choice between gbt and stratum --- I'll take transparent and clunkier over obfuscated and streamlined. Also I'm not saying 'kill stratum with fire' but I won't be using it until the transparency issue is addressed. The reality is that 99% of miners will mine blindly trusting their pool, and they rely on the 1% of savvy users to keep the pools in check. It is a mechanism which has worked so far and will continue to do so. There is a way to confirm the pool is doing what you hope it's doing with stratum as well, and enforcing the extra information on the other 99% of users is pointless, as they're not even going to look at it. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: wizkid057 on November 01, 2012, 05:36:39 AM ...And where do I fit in? Despite what it may look like, this is a technical discussion about the disadvantages of GBT. imo - you're at least honest in your intent and presentation. I'm not saying that GBT couldn't or shouldn't be replaced with something that is better that what it is now... I'm just saying given the choice between gbt and stratum --- I'll take transparent and clunkier over obfuscated and streamlined. Also I'm not saying 'kill stratum with fire' but I won't be using it until the transparency issue is addressed. I agree with this assessment. con is at least providing constructive criticism and suggestions for improvements, which seem to make sense. I agree that GBT needs some work to be perfect, but, it's not a terrible implementation even as it sits, despite what people may try to make it look like. As far as I'm concerned, GBT would be damn near perfect with the following improvements:
Not that GBT isn't good even as it is, but, if the above is done, then GBT will be indisputably awesome. :) -wk Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DrHaribo on November 01, 2012, 06:57:28 AM It's unfortunate that we now have two competing standards. It's more unfortunate that another useless bitcointalk war is breaking out over it.
As usual many of the arguments are just not true, or misunderstood. "You have to send in all the transactions with every share. Think of all those bytes!" No, you don't. When Luke-Jr made a public proposal and asked for comments, I read through it and sent in my thoughts on how it could be improved. Why didn't more people do that? Instead waiting until there are implementations in production and then finally spitting out nasty words and technical misunderstandings. That's not the way to go about things. You have to ask yourself: Do I want to be someone with a positive attitude who finds ways to improve things and push Bitcoin forward? Or do I want to be some useless clockwinder who sabotages Bitcoin with FUD, lies and nonsense? Trolling and spreading FUD may seem like fun. But remember that a lot of newbies who have just discovered Bitcoin come here and read all your nonsense. Also, some of us are trying to build something. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: wizkid057 on November 01, 2012, 06:59:05 AM It's unfortunate that we now have two competing standards. It's more unfortunate that another useless bitcointalk war is breaking out over it. As usual many of the arguments are just not true, or misunderstood. "You have to send in all the transactions with every share. Think of all those bytes!" No, you don't. When Luke-Jr made a public proposal and asked for comments, I read through it and sent in my thoughts on how it could be improved. Why didn't more people do that? Instead waiting until there are implementations in production and then finally spitting out nasty words and technical misunderstandings. That's not the way to go about things. You have to ask yourself: Do I want to be someone with a positive attitude who finds ways to improve things and push Bitcoin forward? Or do I want to be some useless clockwinder who sabotages Bitcoin with FUD, lies and nonsense? Trolling and spreading FUD may seem like fun. But remember that a lot of newbies who have just discovered Bitcoin come here and read all your nonsense. Also, some of us are trying to build something. +1 * 10^256 ;) Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: -ck on November 01, 2012, 07:08:04 AM As usual many of the arguments are just not true, or misunderstood. "You have to send in all the transactions with every share. Think of all those bytes!" No, you don't. Please read my quote. I did NOT say that.Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: -ck on November 01, 2012, 07:10:42 AM And if you blame me for not paying attention at the right time earlier on when this protocol was developed well I'm terribly sorry for not keeping track of every goddamn fucking thing going on in the bitcoin world.
I will implement whatever the fuck this protocol ends up being one way or another. Sorry for offering suggestions, I will now say no more on this. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: sharky112065 on November 01, 2012, 07:33:14 AM C. Kolivas was going through a devastating family matter at the time of those discussions. We are lucky he has returned at all to continue his work on Cgminer.
People seem to blur the lines that he and Kano are the same person. They do not always agree (Just observe the cgminer IRC channel a bit and you will realize that) on everything. If I recall, at first he did not like either solution (Stratum or GBT) and he picked the one he had the less issues with to implement first. I'm not as into the technical end of mining and its underlying code, but from my point of view, all I'm seeing is a lot of bickering and not much getting done to improve and or fix the issues that are causing reservation on implementing GBT. I hope everyone involved can come together and make both Stratum and GBT work out, because it looks like both will be around for a while. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DrHaribo on November 01, 2012, 08:14:06 AM My post above was not only about one person or even one side of this. There seem to be two camps fighting rather than improving things.
When you come in late after the discussion has been going on for a long time and things have now even been implemented, you say "I'm sorry I'm so late with this. I've been busy with something else and didn't have time to look through this until now. But it seems to me that X could be done better if you do it more like Y." Notice the polite form and also not just saying that something is bad, but making suggestions on how to improve it. You don't say "your shit sucks. Fuck this and fuck that." It's bad form and completely unnecessary. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on November 01, 2012, 10:36:22 AM My post above was not only about one person or even one side of this. There seem to be two camps fighting rather than improving things. Your post further above was rubbish.When you come in late after the discussion has been going on for a long time and things have now even been implemented, you say "I'm sorry I'm so late with this. I've been busy with something else and didn't have time to look through this until now. But it seems to me that X could be done better if you do it more like Y." Notice the polite form and also not just saying that something is bad, but making suggestions on how to improve it. You don't say "your shit sucks. Fuck this and fuck that." It's bad form and completely unnecessary. The only point you made in it about comments against stratum was actually completely false (as ckolivas pointed out to you) What arguments are not true? Or misunderstood? - you gonna go make that statement and not even back it up? Seriously, you standing there up on high, telling people what they should be doing, is not gonna win you anything. All it says is that you have no input into the discussion/argument/whatever words you feel the urge to use and should, quite simply, butt out. If you have arguments to make against the issues being brought up, go ahead make them. Otherwise be gone - you telling people how they should do this is pointless drivel as is wizkid posting an even more pointless follow up to your post. You don't like the way I bring up issues and and get a response from those involved? Damn shame about that. It works and it makes people listen and either it gets resolved or it gets swept under the carpet. If you think that BTC software should only take noticed of polite happy fairy statements from polite friendly programmers, then you are seriously deluded. Yeah I'll say it - provide some useful input into the argument/discussion or fuck off. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: slush on November 01, 2012, 11:18:07 AM Also I'm not saying 'kill stratum with fire' but I won't be using it until the transparency issue is addressed. Hm, maybe I didn't say it loudly enough, but this has been solved already: https://bitcointalk.org/index.php?topic=108533.msg1287719#msg1287719 This stratum extension give a full control online to miners about transactions included in the block template. So from side of openess, Stratum and GBT are both equal. Just Stratum is a bit more optimized ;-). Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: slush on November 01, 2012, 11:28:28 AM When Luke-Jr made a public proposal and asked for comments, I read through it and sent in my thoughts on how it could be improved. Why didn't more people do that? I did not so for three reasons: a) I had no idea how it has been supposed to work. And I still don't understand majority of his proposal, optional parts etc. I understand GBT to point that I can load block template and put valid block back to bitcoind, but the rest seems to be a rubbish for me. b) If I'd try to help Luke's with new mining protocol, then I'll suggest Stratum. Obviously there were different targets; Luke wanted something which can be implement in bitcoind, but I wanted something optimized for pooled mining. I'm saying it again and again, that Stratum and GBT can live together and only few people have some mental issues with it. c) It is very hard to discuss with Luke, so creating my own solution which included all my best ideas was a bit easier ;-). Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DrHaribo on November 01, 2012, 12:25:58 PM If you think that BTC software should only take noticed of polite happy fairy statements from polite friendly programmers, then you are seriously deluded. So you think being polite with other people is a fairy land? What I am saying is that if we work together rather than against each other, then we will get much further. Yeah I'll say it - provide some useful input into the argument/discussion or fuck off. I was in this discussion long before you and made contributions before anyone had this in production. With every thing I didn't like I sent in a suggestion for how to improve it. Many, if not all, of my suggestions went in the spec. Now you are telling me to fuck off, because I don't contribute? I think we have different opinions about what a contribution is. I think your main concern right now is that GBT doesn't provide enough transactions. Correct? Actually that's not an issue with the protocol. If bitcoind or a pool gives you 1 or 1000 transactions is up to bitcoind or the pool, not this interface. You may have a valid concern that bitcoind is sometimes holding a lot of transactions in the memory pool without wanting to put them in a block with getwork/getmemorypool/getblocktemplate. Maybe we should take that to a different thread? Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DavinciJ15 on November 01, 2012, 12:29:55 PM With that said it's easy to conclude that since cgminer and others have implemented stratum over GPT kind of gives my opinion a bit of legs to stand on. By that logic you should say GBT is better because its actually built directly into the official bitcoin client. I don't see how stratum is better just because cgminer implemented it. Just because something is implemented somewhere prominent doesn't make it superior. Uh, do you know why VHS beat out Betamax? Simular reason the VHS tapes where created by other businesses at no cost and less issues. Thus Stratum is implemented and used with low cost and less issues that does not make it better it just makes it what the market wants. A quick check ./bitcoind getblocktemplate at this very second produces a output that is 469441 bytes! If every template request was this size and the miner used the customise coinbase option and chose transactions, they would be receiving 400kb for the block template and each share submitted would be 400kb unless they started withholding transactions. Dam! If this is true then I would have to ask what nut bar would use this in a pool? Your network trafic will skyrocket. One more point about the GBT protocol. If a miner chooses which transactions they include rather than accept whichever transactions the pool/bitcoind has offered in the template, the miner has to submit all the transactions with every share he submits. No, that's (part of) what Block Proposal is for.That's not implemented yet is it? When Luke-Jr made a public proposal and asked for comments, I read through it and sent in my thoughts on how it could be improved. Why didn't more people do that? Instead waiting until there are implementations in production and then finally spitting out nasty words and technical misunderstandings. That's not the way to go about things. Please share with us all how an open source project where anyone can create anything can be followed by anyone that maybe interested in it? Note: I only found out that Stratum when I was told on another forum that Bitcoind will not work with my mini rig. Then I discovered GBT. There should be a bitcoin developer news web site so those that want to see technical announcements so people like me can find planned advancements that effect them. To bad there isn't so we work with what we have. Also I'm not saying 'kill stratum with fire' but I won't be using it until the transparency issue is addressed. Hm, maybe I didn't say it loudly enough, but this has been solved already: https://bitcointalk.org/index.php?topic=108533.msg1287719#msg1287719 This stratum extension give a full control online to miners about transactions included in the block template. So from side of openess, Stratum and GBT are both equal. Just Stratum is a bit more optimized ;-). Nice! But why do I feel like a I'm watching a boxing fight and you just rang the bell to start another round of the fight? :) lol but the rest seems to be a rubbish for me. Oh never mind. Try to remember Slush that GBT may have some of Luke-jr's best ideas and calling it "rubbish" kind of stings. RETARDS!!! We do what we physically can and what the market wants, if that's mentally defective then please do it better on your own and prosper. Your prosperity will will help us all by providing proper solutions for us to follow and providing better value to the market than these guys can. If you believe the market will not want your product or service because it's mentally challenged and must be forced to do so, then I must point you to the success of such ideas called U.S.S.R. and Obama-nation. :D Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: flynn on November 01, 2012, 01:03:55 PM I have a question.
I must admit I didn't read everything about pools handling asics, but, ... If solving blocks is roughly returning blocks with the first 53 bits = 0 (well, a bit more), pools request miners to return computed blocks with the first 32 bits = 0. Why not simply make pools specialized for asics which would request returning blocks with the first (32+8) 40 bits = 0 ? that would divide the numbers of returned blocks by 256 and make everyone happy. I probably missed something. Can someone explain the n00b I am why this would be bad .. ? Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DrHaribo on November 01, 2012, 01:15:52 PM Please share with us all how an open source project where anyone can create anything can be followed by anyone that maybe interested in it? Note: I only found out that Stratum when I was told on another forum that Bitcoind will not work with my mini rig. Then I discovered GBT. There should be a bitcoin developer news web site so those that want to see technical announcements so people like me can find planned advancements that effect them. To bad there isn't so we work with what we have. That's a good point. And you basically answer it yourself. An alternative could be mailing lists. Either way it's probably important to make that a developer-only resource to get a good signal to noise ratio. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on November 01, 2012, 01:29:20 PM If you think that BTC software should only take noticed of polite happy fairy statements from polite friendly programmers, then you are seriously deluded. So you think being polite with other people is a fairy land? What I am saying is that if we work together rather than against each other, then we will get much further. Post some grounds for your delusions or ... yeah I've already said it ... oh great you even quoted it below :) Give some grounds for this crap Quote It's unfortunate that we now have two competing standards. It's more unfortunate that another useless bitcointalk war is breaking out over it. or be gone.As usual many of the arguments are just not true, or misunderstood. "You have to send in all the transactions with every share. Think of all those bytes!" No, you don't. Quote Yeah I'll say it - provide some useful input into the argument/discussion or fuck off. I was in this discussion long before you and made contributions before anyone had this in production. With every thing I didn't like I sent in a suggestion for how to improve it. Many, if not all, of my suggestions went in the spec. Now you are telling me to fuck off, because I don't contribute? I think we have different opinions about what a contribution is. Hell, my ancestor wrote the most important book in Christianity after the damn bible, but so what, who gives a crap about the past. What matters is that part I just quoted above, not what blame you have in the past for helping creating this abortion called GBT. Quote I think your main concern right now is that GBT doesn't provide enough transactions. Correct? Actually that's not an issue with the protocol. If bitcoind or a pool gives you 1 or 1000 transactions is up to bitcoind or the pool, not this interface. My concerns have be stated.You may have a valid concern that bitcoind is sometimes holding a lot of transactions in the memory pool without wanting to put them in a block with getwork/getmemorypool/getblocktemplate. Maybe we should take that to a different thread? Your pool ran 15 blocks that I pointed out, and of them, only 5 were large txn count/size. 5 were tiny. 5 were in the middle. You claimed that you don't choose the transactions, GBT does ... wow :) Next, GBT currently usually sends more data per minute than a GetWork pool with roll-n-time (probably even if it is ignoring transactions or there are few transactions on the network) This is due to the fact that it sends all the details of the all transactions you are processing (no matter what, you cannot disable this and request a merkle tree) Next, GBT uses HTTP not raw sockets, so it is slower and is also expected to have more rejected shares due to this since it isn't a constant connection. Not using HTTP, Luke-Jr has stated as a negative in Stratum coz it takes more effort to implement a better protocol ... wow gotta love that argument. Sounds like "Don't implement something better coz it will take more effort" Or even "Luke-Jr sux at programming and can't do anything that isn't simple" Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on November 01, 2012, 03:13:38 PM When Luke-Jr made a public proposal and asked for comments, I read through it and sent in my thoughts on how it could be improved. Why didn't more people do that? I did not so for three reasons: a) I had no idea how it has been supposed to work. And I still don't understand majority of his proposal, optional parts etc. I understand GBT to point that I can load block template and put valid block back to bitcoind, but the rest seems to be a rubbish for me. b) If I'd try to help Luke's with new mining protocol, then I'll suggest Stratum. That's fine, Stratum seems like a logical place to start for a new standard. It would have been nice if these things had taken place during the GBT standardization, but that's in the past now. How goes getting StratumMP BIP process off the ground? Are we still waiting on genjix?Obviously there were different targets; Luke wanted something which can be implement in bitcoind, but I wanted something optimized for pooled mining. I'm saying it again and again, that Stratum and GBT can live together and only few people have some mental issues with it. You keep saying this, and I keep having to correct you: No, the main focus of GBT was from the start was pooled mining. Citing the Motivation section of BIP 22 from all the way back in the original February draft... "There is reasonable concerns about mining currently being too centralized on pools, and the amount of control these pools hold. By exposing the details of the block proposals to the miners, they are enabled to audit and possibly modify the block before hashing it."c) It is very hard to discuss with Luke, so creating my own solution which included all my best ideas was a bit easier ;-). This is just trolling.Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on November 01, 2012, 03:26:28 PM One more point about the GBT protocol. If a miner chooses which transactions they include rather than accept whichever transactions the pool/bitcoind has offered in the template, the miner has to submit all the transactions with every share he submits. No, that's (part of) what Block Proposal is for.When Luke-Jr made a public proposal and asked for comments, I read through it and sent in my thoughts on how it could be improved. Why didn't more people do that? Instead waiting until there are implementations in production and then finally spitting out nasty words and technical misunderstandings. That's not the way to go about things. Please share with us all how an open source project where anyone can create anything can be followed by anyone that maybe interested in it? Note: I only found out that Stratum when I was told on another forum that Bitcoind will not work with my mini rig. Then I discovered GBT.Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DrHaribo on November 01, 2012, 04:50:09 PM Lulz - I can post at 2.6 and still see you've posted crap. Being polite and working together towards a common goal is crap? I am surprised that you participate in an open source project. Give some grounds for this crap Quote It's unfortunate that we now have two competing standards. It's more unfortunate that another useless bitcointalk war is breaking out over it. or be gone.As usual many of the arguments are just not true, or misunderstood. "You have to send in all the transactions with every share. Think of all those bytes!" No, you don't. You like to make a lot of demands, don't you? ckolivas wrote that if the miner selects transactions you have to send in every transaction with every share. This is incorrect and has been discussed enough already, I think. You said that GBT, a protocol, makes decisions on which transactions to include. How does a protocol make a decision like that? And GBT being "an abortion", I don't think that's entirely correct either. I don't care if you designed GBT ... ... ... well actually that would be even worse coz that would make you Jesus. I never said I designed GBT. I said I suggested a few improvements for it. And what does religion have to do with this? You told me to "fuck off" because I don't contribute. I was merely stating that my contributions are already in the spec. Where are yours? Your pool ran 15 blocks that I pointed out, and of them, only 5 were large txn count/size. 5 were tiny. 5 were in the middle. You picked 15 random blocks from BitMinter. They were 5 big blocks, 5 medium and 5 small. And so you come to me and demand an explanation? I find that utterly unreasonable. If you don't like pools that filter out transactions, then talk to them, not me. I don't put any limits on transactions. You claimed that you don't choose the transactions, GBT does ... wow :) No, you said that, not me. What I said was that I include all the transactions bitcoind gives me, without filtering out the free ones or putting any limit on the number of transactions, like some pools do. GBT and bitcoind are not the same thing. If a bug in a web server makes it give you half a file, do you blame HTTP? Next, GBT currently usually sends more data per minute than a GetWork pool with roll-n-time (probably even if it is ignoring transactions or there are few transactions on the network) This is due to the fact that it sends all the details of the all transactions you are processing (no matter what, you cannot disable this and request a merkle tree) Maybe that would be a good improvement for GBT. If the miner doesn't care about seeing the transactions or filtering/adding transactions, request a merkle branch instead that you can use to create a merkle root for mining. Not a single transaction goes over the wire, other than the generation tx. Next, GBT uses HTTP not raw sockets, so it is slower and is also expected to have more rejected shares due to this since it isn't a constant connection. Not using HTTP, Luke-Jr has stated as a negative in Stratum coz it takes more effort to implement a better protocol ... wow gotta love that argument. Sounds like "Don't implement something better coz it will take more effort" Or even "Luke-Jr sux at programming and can't do anything that isn't simple" Of course HTTP is a constant connection. Use persistent connections, and pipeline requests for speed. My miner uses 1 long poll connection and 1-3 connections to get work and send in proofs of work. For the work connections it will pipeline up to 10 requests at a time. All the connections are persistent. They get closed only if they have been sitting idle in the connection pool for a while. The advantage of Stratum is that it is designed to be two-way from the start. Long polling in HTTP feels dirty and hackish. The advantage of HTTP is all the network infrastructure, proxies, libraries that exist, and all the knowledge that developers have about this protocol. When you already know the protocol, you have a library for using it that you already know well, then it is much easier to get started. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: slush on November 01, 2012, 04:58:48 PM How goes getting StratumMP BIP process off the ground? Are we still waiting on genjix? No, I'm just busy. Writing sane documentation (in english) is pretty hard for me, so I need a lot of time to finalize something. "There is reasonable concerns about mining currently being too centralized on pools, and the amount of control these pools hold. By exposing the details of the block proposals to the miners, they are enabled to audit and possibly modify the block before hashing it." You didn't respond to Stratum's "get_transactions" extension explicitly. Can I ask you what's your opinion to this? I think it solves exactly that issue with blind trust for picking transaction by the pool, without unnecessary overhead for common miners. This is just trolling. I don't think so. I prefer coding, not discussion. We can spend months on discussions or we can both show our solutions and let others to decide. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on November 01, 2012, 05:07:38 PM How goes getting StratumMP BIP process off the ground? Are we still waiting on genjix? No, I'm just busy. Writing sane documentation (in english) is pretty hard for me, so I need a lot of time to finalize something."There is reasonable concerns about mining currently being too centralized on pools, and the amount of control these pools hold. By exposing the details of the block proposals to the miners, they are enabled to audit and possibly modify the block before hashing it." You didn't respond to Stratum's "get_transactions" extension explicitly. Can I ask you what's your opinion to this? I think it solves exactly that issue with blind trust for picking transaction by the pool, without unnecessary overhead for common miners.I prefer coding, not discussion. We can spend months on discussions or we can both show our solutions and let others to decide. It's not either one or the other. The BIP process generally involves documenting and implementing a first draft, and then discussion to come up with a final standard solution that meets everyone's needs. Eligius had GBT since the first draft in February, but the protocol continued to change significantly as other developers contributed to it before reaching its current form.Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Inaba on November 01, 2012, 06:33:46 PM When Luke was working on GBT and we were working on implementing a working vardiff, I had some concerns from a security standpoint. The Stratum get_transactions also raises some red flags for me, can you address security/throttling measures in place to prevent get_transactions being used a DoS method to overload a pool? Anything that leaves arbitrary data requests up to the client is an avenue for exploit and it is one of the reasons I have been so adamantly against user defined difficulties. If I want to DoS your pool, I crank up a couple of 1.5 TH minirig, set my difficulty to 1 and ask for getworks.
Same kind of problem with get_transactions: I'm pissed at you, so I just request all the get_transactions you'll send me over and over and over. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: P_Shep on November 01, 2012, 06:38:45 PM When Luke was working on GBT and we were working on implementing a working vardiff, I had some concerns from a security standpoint. The Stratum get_transactions also raises some red flags for me, can you address security/throttling measures in place to prevent get_transactions being used a DoS method to overload a pool? Anything that leaves arbitrary data requests up to the client is an avenue for exploit and it is one of the reasons I have been so adamantly against user defined difficulties. If I want to DoS your pool, I crank up a couple of 1.5 TH minirig, set my difficulty to 1 and ask for getworks. Same kind of problem with get_transactions: I'm pissed at you, so I just request all the get_transactions you'll send me over and over and over. Surely you could keep requesting get_works with a script, why would you need any hashing behind it? As with the get_transactions, surely you'd just place a limit on requests poolside, and ignore those that seem unreasonable? Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on November 01, 2012, 08:29:38 PM When Luke was working on GBT and we were working on implementing a working vardiff, I had some concerns from a security standpoint. The Stratum get_transactions also raises some red flags for me, can you address security/throttling measures in place to prevent get_transactions being used a DoS method to overload a pool? Anything that leaves arbitrary data requests up to the client is an avenue for exploit and it is one of the reasons I have been so adamantly against user defined difficulties. If I want to DoS your pool, I crank up a couple of 1.5 TH minirig, set my difficulty to 1 and ask for getworks. You've ignored the reverse situation.Same kind of problem with get_transactions: I'm pissed at you, so I just request all the get_transactions you'll send me over and over and over. With GBT (all current implementations) the pool must always do this. With Stratum it is optional and yes some users could request it always. Since it is an available option, then the pool must consider the load of allowing it. (... and yes I mean they MUST consider the load, not that they should ignore implementing it ... though any that don't would be easy to identify ... due to their failure to respond any miner that requests it) Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: -ck on November 01, 2012, 09:01:25 PM Must I quote myself?
If every template request was this size and the miner used the customise coinbase option and chose transactions, they would be receiving 400kb for the block template and each share submitted would be 400kb unless they started withholding transactions. That IS correct.I was exploring the possible issues with implementation of GBT and got attacked as though I was trying to spread FUD about GBt. Nowhere did I imply this is what all miners would be doing. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on November 01, 2012, 09:07:06 PM Must I quote myself? Too bad you're more interested in ignoring me than having a productive discussion, or you'd have noticed I pointed out exactly why that isn't true.If every template request was this size and the miner used the customise coinbase option and chose transactions, they would be receiving 400kb for the block template and each share submitted would be 400kb unless they started withholding transactions. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: -ck on November 01, 2012, 10:10:09 PM When Luke was working on GBT and we were working on implementing a working vardiff, I had some concerns from a security standpoint. The Stratum get_transactions also raises some red flags for me, can you address security/throttling measures in place to prevent get_transactions being used a DoS method to overload a pool? Anything that leaves arbitrary data requests up to the client is an avenue for exploit and it is one of the reasons I have been so adamantly against user defined difficulties. If I want to DoS your pool, I crank up a couple of 1.5 TH minirig, set my difficulty to 1 and ask for getworks. Same kind of problem with get_transactions: I'm pissed at you, so I just request all the get_transactions you'll send me over and over and over. Surely you could keep requesting get_works with a script, why would you need any hashing behind it? As with the get_transactions, surely you'd just place a limit on requests poolside, and ignore those that seem unreasonable? By the same token, GBT pools should limit how often users can ask for a template since that does send out all the transactions. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on November 01, 2012, 10:19:29 PM Lulz - I can post at 2.6 and still see you've posted crap. Being polite and working together towards a common goal is crap? I am surprised that you participate in an open source project. Quote Give some grounds for this crap Quote It's unfortunate that we now have two competing standards. It's more unfortunate that another useless bitcointalk war is breaking out over it. or be gone.As usual many of the arguments are just not true, or misunderstood. "You have to send in all the transactions with every share. Think of all those bytes!" No, you don't. You like to make a lot of demands, don't you? ckolivas wrote that if the miner selects transactions you have to send in every transaction with every share. This is incorrect and has been discussed enough already, I think. You said that GBT, a protocol, makes decisions on which transactions to include. How does a protocol make a decision like that? And GBT being "an abortion", I don't think that's entirely correct either. Quote I don't care if you designed GBT ... ... ... well actually that would be even worse coz that would make you Jesus. I never said I designed GBT. I said I suggested a few improvements for it. And what does religion have to do with this? Quote You told me to "fuck off" because I don't contribute. I was merely stating that my contributions are already in the spec. Where are yours? No, because you made a VERY broad statement without any basis and contributed nothing to the current discussion except to tell people how they should be discussing it.Quote Your pool ran 15 blocks that I pointed out, and of them, only 5 were large txn count/size. 5 were tiny. 5 were in the middle. You picked 15 random blocks from BitMinter. They were 5 big blocks, 5 medium and 5 small. And so you come to me and demand an explanation? I find that utterly unreasonable. If you don't like pools that filter out transactions, then talk to them, not me. I don't put any limits on transactions. No I didn't pick 15 random blocks. I picked 15 blocks in a row that your pool generated. Your response was that it was out of your control. The reason I chose those blocks was because one of them orphaned another block at the same time: 205766 BTC Sent 510.69 BTC Number Of Transactions 32 By BitMinter Orphaned: BTC Sent 15,449.927 BTC Number Of Transactions 619 By EclipseMC Yes something is badly wrong there ... and your excuse is that you don't have any rules about the transactions, you just call the GBT interface. [aside] This is in fact a big issue with Luke-Jr also - him claiming about how what he is doing is better for BTC. Yet what he was doing with his pool was restricting the number of transactions at his pool to 32 to increase the payments the pool received rather than fixing the problem with his pool software that was getting more orphans than other bigger pools at the same time. The guy is a hypocrite - even worse: he enables such a thing at his pool ~5 months ago and the pool is still doing it. I even explained a sizeable solution to this in bitcoind to both him and Gavin back then ... [/aside] Quote You claimed that you don't choose the transactions, GBT does ... wow :) No, you said that, not me. What I said was that I include all the transactions bitcoind gives me, without filtering out the free ones or putting any limit on the number of transactions, like some pools do. GBT and bitcoind are not the same thing. If a bug in a web server makes it give you half a file, do you blame HTTP? There is no difference between what I said and you said except technicalities. Yes it seems this statement keeps coming up. This is a text book theory argument that one can make about something that is just that - a theory. Yes the GBT proponents are out saying to use GBT because it is great. The implementations are a measure of how great it is and falling back to this constant "It's only an idea you can't say GBT is bad" is pointless also. Quote Next, GBT currently usually sends more data per minute than a GetWork pool with roll-n-time (probably even if it is ignoring transactions or there are few transactions on the network) This is due to the fact that it sends all the details of the all transactions you are processing (no matter what, you cannot disable this and request a merkle tree) Maybe that would be a good improvement for GBT. If the miner doesn't care about seeing the transactions or filtering/adding transactions, request a merkle branch instead that you can use to create a merkle root for mining. Not a single transaction goes over the wire, other than the generation tx. How blatantly obvious is it? This is a good example of how it becomes just straight out annoying: Use this great new protocol - it's better than before ........ except it sends more data than getwork ...... that yeah MAYBE that might be worth improving ...... :P But it's better coz the GBT guys say it is - it's transparent ... oh wait Stratum has the option to send the txn list that is 'missing' also. Um ... what other excuse can they make up ... Oh yeah it's just the implementations that suck ... GBT is good for everyone ............ except everyone who is using it with any network between the miner and the work generator ... Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on November 01, 2012, 10:33:18 PM This is in fact a big issue with Luke-Jr also - him claiming about how what he is doing is better for BTC. The problem isn't with the pool software, as you clearly admit to knowing by mentioning fixing bitcoind. Until bitcoind is fixed and the fix is deployed everywhere, this problem remains.Yet what he was doing with his pool was restricting the number of transactions at his pool to 32 to increase the payments the pool received rather than fixing the problem with his pool software that was getting more orphans than other bigger pools at the same time. The guy is a hypocrite - even worse: he enables such a thing at his pool ~5 months ago and the pool is still doing it. I even explained a sizeable solution to this in bitcoind to both him and Gavin back then ... Good job explaining the solution - even though your explanation doesn't actually solve it. Unfortunately, all contributions to bitcoind are done in peoples' spare time and things like a complete rewrite of the p2p code (which is needed to really fix this) takes a lot of time and skill. When this problem is nearly non-existent by simply filtering out a known DDoS attack, there are also higher priorities. Oh, and stop trolling that GBT uses more bandwidth than getwork. It doesn't. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on November 02, 2012, 12:05:41 AM This is in fact a big issue with Luke-Jr also - him claiming about how what he is doing is better for BTC. The problem isn't with the pool software, as you clearly admit to knowing by mentioning fixing bitcoind. Until bitcoind is fixed and the fix is deployed everywhere, this problem remains.Yet what he was doing with his pool was restricting the number of transactions at his pool to 32 to increase the payments the pool received rather than fixing the problem with his pool software that was getting more orphans than other bigger pools at the same time. The guy is a hypocrite - even worse: he enables such a thing at his pool ~5 months ago and the pool is still doing it. I even explained a sizeable solution to this in bitcoind to both him and Gavin back then ... Good job explaining the solution - even though your explanation doesn't actually solve it. Unfortunately, all contributions to bitcoind are done in peoples' spare time and things like a complete rewrite of the p2p code (which is needed to really fix this) takes a lot of time and skill. When this problem is nearly non-existent by simply filtering out a known DDoS attack, there are also higher priorities. The solution is based on the fact that 1) Most bitcoinds will already have every transaction that they are sent (all except the coinbase transaction) So the protocol is sending a large amount of data that the receiving bitcoind already has and knows it has and doesn't need sent 2) The side effect of this is also that the receiving bitcoind again re-validates each transaction that it already has validated in it's memory pool EDIT: oops left out this comment. The problem was not with the bigger pools software, since they were not getting as high a % of orphans as you were AT THE SAME TIME. So yes is was also your software fail that you then used as an excuse to reduce the number of confirmed txns at your pool. Quote Oh, and stop trolling that GBT uses more bandwidth than getwork. It doesn't. Math fail.I guess the answer to this is that ANY time you quote figures that say something is better it would be foolish for anyone to believe what you say since you will only using some edge case example that is rare or some implementation design that no one uses. The only possible way for this to be correct is in one real situation: When a getwork pool doesn't do roll-n-time and the total number of available transaction on the network is small and the hash rate of the device is high. In the case where the pool does roll-n-time - getwork transfers a LOT less data than GBT in all but edge case rare situations. To get the size it's VERY easy: I am saying the work needs to last 2 minutes. Processing the same work for even 2 minutes has a negative effect on the transaction confirmation times. But we'll let it be 2 minutes anyway. This give a single getwork request (allowing a roll of 7000 seconds - slightly under the limit) a limit of 501GH/s To compare the two I'll use the bitcoind getwork since it approximates to the the size of a pool getwork quite fairly. ./bitcoind getwork I got: 590 bytes (Edit: O(1)) ./bitcoind getblocktemplate I got 141 txns and 177,184 bytes (Edit: O(n)) (AND my bitcoind also says: accepted da0141dc61 (poolsz 1076) ... that sux) Anyone do it and look at the difference for any getblocktemplate that has even a few transactions in it. Consider the example in my last post of 619 transactions .... :P Though of course you could double that getwork number to support 1TH/s ... and double it again to support 2TH/s ... etc. until you finally reach the average getblocktemplate sizes. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on November 02, 2012, 12:30:44 AM This is in fact a big issue with Luke-Jr also - him claiming about how what he is doing is better for BTC. The problem isn't with the pool software, as you clearly admit to knowing by mentioning fixing bitcoind. Until bitcoind is fixed and the fix is deployed everywhere, this problem remains.Yet what he was doing with his pool was restricting the number of transactions at his pool to 32 to increase the payments the pool received rather than fixing the problem with his pool software that was getting more orphans than other bigger pools at the same time. The guy is a hypocrite - even worse: he enables such a thing at his pool ~5 months ago and the pool is still doing it. I even explained a sizeable solution to this in bitcoind to both him and Gavin back then ... Good job explaining the solution - even though your explanation doesn't actually solve it. Unfortunately, all contributions to bitcoind are done in peoples' spare time and things like a complete rewrite of the p2p code (which is needed to really fix this) takes a lot of time and skill. When this problem is nearly non-existent by simply filtering out a known DDoS attack, there are also higher priorities. The solution is based on the fact that 1) Most bitcoinds will already have every transaction that they are sent (all except the coinbase transaction) So the protocol is sending a large amount of data that the receiving bitcoind already has and knows it has and doesn't need sent 2) The side effect of this is also that the receiving bitcoind again re-validates each transaction that it already has validated in it's memory pool When a client receives a block, it does this, in order:
Also note that it is wrong to assume everyone already knows about every transaction (except coinbase) before the block is found. Peers randomly discard (and don't relay) transactions without fees, so it's likely some of those will be missing on most nodes. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on November 02, 2012, 02:21:06 AM This is in fact a big issue with Luke-Jr also - him claiming about how what he is doing is better for BTC. The problem isn't with the pool software, as you clearly admit to knowing by mentioning fixing bitcoind. Until bitcoind is fixed and the fix is deployed everywhere, this problem remains.Yet what he was doing with his pool was restricting the number of transactions at his pool to 32 to increase the payments the pool received rather than fixing the problem with his pool software that was getting more orphans than other bigger pools at the same time. The guy is a hypocrite - even worse: he enables such a thing at his pool ~5 months ago and the pool is still doing it. I even explained a sizeable solution to this in bitcoind to both him and Gavin back then ... Good job explaining the solution - even though your explanation doesn't actually solve it. Unfortunately, all contributions to bitcoind are done in peoples' spare time and things like a complete rewrite of the p2p code (which is needed to really fix this) takes a lot of time and skill. When this problem is nearly non-existent by simply filtering out a known DDoS attack, there are also higher priorities. The solution is based on the fact that 1) Most bitcoinds will already have every transaction that they are sent (all except the coinbase transaction) So the protocol is sending a large amount of data that the receiving bitcoind already has and knows it has and doesn't need sent 2) The side effect of this is also that the receiving bitcoind again re-validates each transaction that it already has validated in it's memory pool When a client receives a block, it does this, in order:
Also note that it is wrong to assume everyone already knows about every transaction (except coinbase) before the block is found. Peers randomly discard (and don't relay) transactions without fees, so it's likely some of those will be missing on most nodes. 1) Create a new transfer mechanism that can be used as well as the old one and ... 2) Prioritise transferring with the new mechanism first (to peers that support it) 3) Remove out the old one some time later (Remember April? Yeah avoid that) Now I already stated in my solution that not all transactions will be known by the recipient. The protocol must allow a response to request the txn list it doesn't have. However, I also specifically wrote code to report the number of previously known transactions each time in my own bitcoind and also mentioned this as well So regarding that last sentence you wrote: it was exceedingly rare for me to get new transactions with the block that I didn't know ... other than the coinbase txn which will always be sent Anyway I will rewrite this with your points: Quote Rewrite the Bitcoin peer-to-peer protocol handling to work asynchronously: (this is a LOT of work) Or, as I said in this post, support both. Read above.AND if one peer can't receive data fast enough, send to other peers at the same time AND keep responding to other peers while things are going on We don't want another soft fork debacle like April ... Quote 1) Begin relaying blocks as soon as the header is downloaded and verified, before even receiving the rest of the block ourselves - this means the entire network is uploading/downloading the new block in parallel; since the difficulty is stupidly high, that makes any DoS attack impractical (I actually implemented this, before I discovered the previous problem which prevents it from working) Yeah that's what I said already ... the 2) effectively resolves 1).AND 2) Don't even upload transactions the other peer knows about already So ... I take this repeat of my solution ... to mean your statement Quote Good job explaining the solution - even though your explanation doesn't actually solve it. is wrong.Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on November 02, 2012, 04:10:36 AM Your "explanation" only minorly improves things, it doesn't fix the problem. That's why it was last on the list of things to do to fix it.
None of what I said suggests any kind of fork is needed. Obviously the internals would support both mechanisms - it just isn't relevant to what needs to be done. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DrHaribo on November 02, 2012, 08:45:06 AM Must I quote myself? ONLY ignoring luke-jr It appears that you are missing part of the discussion. Quote GBT and bitcoind are not the same thing. If a bug in a web server makes it give you half a file, do you blame HTTP? Tell me how that statement I said is wrong without hiding behind definitions.There is no difference between what I said and you said except technicalities. Come on, that's no technicality. Is this really where this discussion is right now? If you get a virus over HTTP then you blame HTTP? If you mine a small block over Stratum then you blame Stratum? Let's be honest now. Does the GBT protocol/API really force you to make small blocks? Does Stratum? The implementations are a measure of how great it is and falling back to this constant "It's only an idea you can't say GBT is bad" is pointless also. Come on, how many transactions you decide to include in a block is up to you, not the protocol. Of course you can say that something about GBT is bad, but can you point at the place in the spec where it tells you that you must not include more than X transactions in a block? Which part of GBT are you actually having a problem with? Do you want it to be a violation of the spec to mine a block with less than X transactions? If so, present your suggestion and make an argument for it. But I doubt there are many that want GBT to have anything to do with which or how many transactions are chosen for a block. Luckily that is not the case now, even though you claim it is. Quote Next, GBT currently usually sends more data per minute than a GetWork pool with roll-n-time (probably even if it is ignoring transactions or there are few transactions on the network) This is due to the fact that it sends all the details of the all transactions you are processing (no matter what, you cannot disable this and request a merkle tree) Maybe that would be a good improvement for GBT. If the miner doesn't care about seeing the transactions or filtering/adding transactions, request a merkle branch instead that you can use to create a merkle root for mining. Not a single transaction goes over the wire, other than the generation tx. How blatantly obvious is it? Well, again, this is a way to be polite and open for cooperation with other developers. Instead of saying "THIS is the best solution and you better do things MY WAY" you can say "maybe this can help improve things?" You will find this goes over much better with most people. The way you are acting you look like a very angry man who hates the world and everyone in it. There's no need to say that people suck at programming, nor joke about someone's (?) religion. If you had both a problem and a solution in your head, why did you present only the problem? ckolivas said "this is a technical discussion about the disadvantages of GBT". Unless the point is to destroy GBT I think a better way to view it is that this is a technical discussion about what GBT is today and how we can make it better tomorrow. For the problem of using much of the bandwidth on transactions when some miners don't even care about seeing them: I guess this could be improved by having the client include something in its capabilities list to tell the server it wishes to mine "blindly" without knowing about transactions. If both client and server has this capability then the server sends a merkle branch instead of a list of transactions. I'm not sure its entirely clean that if the user wishes to see transactions the client would leave out this capability. It makes the capabilities more like options than an actual list of capabilities, but perhaps that's alright. What do you think, Luke-Jr and others? It could get bandwidth usage down for users who don't care to see the transactions. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: -ck on November 02, 2012, 10:34:10 AM ckolivas said "this is a technical discussion about the disadvantages of GBT". Unless the point is to destroy GBT I think a better way to view it is that this is a technical discussion about what GBT is today and how we can make it better tomorrow. My goodness! Go back 3+ pages on this forum thread and you'll see that's what I was discussing with jgarzik. Yes I was trying to be productive and help find some useful pool mining mode for GBT. All it needs is a request in the protocol for branches.For the problem of using much of the bandwidth on transactions when some miners don't even care about seeing them: I guess this could be improved by having the client include something in its capabilities list to tell the server it wishes to mine "blindly" without knowing about transactions. If both client and server has this capability then the server sends a merkle branch instead of a list of transactions. I'm not sure its entirely clean that if the user wishes to see transactions the client would leave out this capability. It makes the capabilities more like options than an actual list of capabilities, but perhaps that's alright. What do you think, Luke-Jr and others? It could get bandwidth usage down for users who don't care to see the transactions. ...this may explain why I was so annoyed yesterday. EDIT: https://bitcointalk.org/index.php?topic=108854.msg1286483#msg1286483 Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DrHaribo on November 02, 2012, 11:16:03 AM EDIT: https://bitcointalk.org/index.php?topic=108854.msg1286483#msg1286483 I apologize, I missed that in all the noise. I support your idea. Not sure if "suggested merkle root" is a typo. I'd suggest that on the wire this is a merkle branch that when combined with your generated coinbase tx gives you the merkle root for the block. You request work once, then send several times block header + coinbase. The server will know which txes / merkle tree this belongs to by looking at your work ID. I believe you are right that the majority will not care to see or change the transactions, so it is a good optimization for the common case. Also, until pools support changing the transactions, it will be a good optimization in every case. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on November 02, 2012, 11:30:07 AM I guess this could be improved by having the client include something in its capabilities list to tell the server it wishes to mine "blindly" without knowing about transactions. If both client and server has this capability then the server sends a merkle branch instead of a list of transactions. This specific method is worse than Stratum's new alternative, since the pool would now know which miners it can safely give dirty jobs to.What do you think, Luke-Jr and others? It could get bandwidth usage down for users who don't care to see the transactions. I also don't think it's a good idea to encourage further pool centralization by making it any easier for miners to be neglegent. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DrHaribo on November 02, 2012, 11:46:52 AM This specific method is worse than Stratum's new alternative, since the pool would now know which miners it can safely give dirty jobs to. Ok, I didn't think of that. But really, what is the worst that can happen? Mining 5 txes instead of 500? You can see that after-the-fact and I don't think that's a disaster. Or maybe I am missing something that "dirty" jobs could contain? Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: -ck on November 02, 2012, 11:57:33 AM EDIT: https://bitcointalk.org/index.php?topic=108854.msg1286483#msg1286483 I apologize, I missed that in all the noise. I support your idea. Not sure if "suggested merkle root" is a typo. I'd suggest that on the wire this is a merkle branch that when combined with your generated coinbase tx gives you the merkle root for the block. You request work once, then send several times block header + coinbase. The server will know which txes / merkle tree this belongs to by looking at your work ID. I believe you are right that the majority will not care to see or change the transactions, so it is a good optimization for the common case. Also, until pools support changing the transactions, it will be a good optimization in every case. EDIT: or alternatively, randomly inject requests for full transactions. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: -ck on November 05, 2012, 11:36:18 AM Almost implemented the first working version of GBT within cgminer.
This sort of behaviour tells the story well though: Code: [2012-11-05 22:28:08] Accepted 23c0ba9a Diff 7/1 GPU 2 pool 2 Now pool 3 here is slush on stratum (ping time 335ms), while pool 2 is EMC on GBT (ping time 225ms). Note how stratum picks up the block change and notifies me 10 seconds before GBT does. However, this is not the GBT pool simply learning of the block change later, because it starts rejecting my shares as being from the previous block before I even got the longpoll from the GBT pool. Then of course there's a second longpoll with the transactions, which is not an unusual practice. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on November 05, 2012, 12:26:57 PM Almost implemented the first working version of GBT within cgminer. Confirming this analysis as accurate... for now. It shouldn't be this much of a difference though, so I suspect there's something going on server-side I need to look into.This sort of behaviour tells the story well though: Code: [2012-11-05 22:28:08] Accepted 23c0ba9a Diff 7/1 GPU 2 pool 2 Now pool 3 here is slush on stratum (ping time 335ms), while pool 2 is EMC on GBT (ping time 225ms). Note how stratum picks up the block change and notifies me 10 seconds before GBT does. However, this is not the GBT pool simply learning of the block change later, because it starts rejecting my shares as being from the previous block before I even got the longpoll from the GBT pool. Then of course there's a second longpoll with the transactions, which is not an unusual practice. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on November 05, 2012, 12:52:11 PM I guess this could be improved by having the client include something in its capabilities list to tell the server it wishes to mine "blindly" without knowing about transactions. If both client and server has this capability then the server sends a merkle branch instead of a list of transactions. This specific method is worse than Stratum's new alternative, since the pool would now know which miners it can safely give dirty jobs to.What do you think, Luke-Jr and others? It could get bandwidth usage down for users who don't care to see the transactions. I also don't think it's a good idea to encourage further pool centralization by making it any easier for miners to be neglegent. So why would any pool use it over Stratum? ... and where are these miners that even show ANYTHING of this TRUCKLOAD ... Stratum gives the miner the option to allow the dump truck to back up to the house and dump a few hundred K of data that not a single miner reports anything about ... or the miner can not ask for it. GBT send the dump truck every time, that not a single miner reports anything about the contents - it just uses it to build the merkle tree bigger than what Stratum sends (Stratum only needs to send the side of the tree) Yes it is a TRUCKLOAD ... and the API stats in cgminer will report this once I get in there and add it after GBT goes live in cgminer ... You'll be able to load balance 3 pools : Getwork LP, Stratum and GBT and look at how different the numbers are :D :D :D :D Yeah my post before proving that already no doubt got ignored - but anyone using cgminer will be able to see it soon enough themselves. This is really just as bad as that idiot subSTRATA. You must accept a TRUCKLOAD of txns - coz nobody will be safe without them. Even if no program ever tell you anything about them ... ... and even if anyone can write a program to report exactly what ANY pool is doing ... ... though I guess that may be a bit far beyond your skill set to even understand how to do that. --- Meanwhile ... there was an accusation thrown at me last week that I ignored but I guess I should clear it up. Yes my description of Luke-Jr's hiding information was saying he was hiding it for longer than he really did. Even wizkid mentioned it to me in an IRC discussion that I missed ... somehow when I wrote the post. (Heh Mobius if you read this ... just keep swimming ... :D) Yes no one needs to believe me but I did actually completely miss that in the IRC discussion :P But the reality is that he has been hiding information since 1-Jun about how his pool has been "not processing transactions" but I also quoted the information that he did indeed post in his thread (I almost never go near his thread, though once when I posted there then deleted the posts after he had quoted them so I'd not ever have to go back ... one is linked to below) Also with the dice reject code I'd also be curious if wizkid would be so kind as to show us how it does reject SD transactions ... Firstly yes Jr did post in his thread about the 32 txn limit he imposed on his pool for over 5 months back when he did it and was still there when wizkid took over: https://bitcointalk.org/index.php?topic=23768.msg968819#msg968819 However, the actual post I thought about was this one: https://bitcointalk.org/index.php?topic=23768.msg934320#msg934320 Where Jr says: Quote ... These algorithms have never been disclosed - which is really what I was referring to.And it's not a matter of "ignoring" transactions, it's a matter of not processing them. ... Eligius continues to employ aggressive anti-spam checks on feeless transactions, to avoid wasting charity on spammers. Because of the nature of spam detection, it is necessary to keep the algorithms used confidential ... Yes I made a mistake :P The fact that it "doesn't process" transactions based on his definition of the word "feeless" (coz his definitions is not fee=0) However, for the point of view of giving out a high level, means next to nothing, information ... yes he said Quote FWIW, all pools have been experiencing higher stales and orphaned blocks due to the excessive transaction volume lately resulting from SatoshiDice abusing the blockchain (there are much cleaner ways to do the same thing). After our second set of 3 orphans in a row, I'm a bit on the annoyed end. For now, I am blocking transactions to 1dice* addresses and limiting our blocks to 32 transactions until we've caught up on the extra credit or at least have a viable alternative solution. I really hate to do this, as Eligius has traditionally been one of the most accepting mining pools, so any suggestions on other possibilities would be most welcome. Which is using the fact that Eligius was worse off than every other pool (none of the others changed to 32 txns per block) and none of the others had as many orphans as Eligius - so yeah blaming SD for his crappy software and using it as an excuse to reduce the BTC transaction confirm time for over 5 months really shows something about him ...Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on November 05, 2012, 01:47:54 PM Almost implemented the first working version of GBT within cgminer. Confirming this analysis as accurate... for now. It shouldn't be this much of a difference though, so I suspect there's something going on server-side I need to look into.This sort of behaviour tells the story well though: Code: [2012-11-05 22:28:08] Accepted 23c0ba9a Diff 7/1 GPU 2 pool 2 Now pool 3 here is slush on stratum (ping time 335ms), while pool 2 is EMC on GBT (ping time 225ms). Note how stratum picks up the block change and notifies me 10 seconds before GBT does. However, this is not the GBT pool simply learning of the block change later, because it starts rejecting my shares as being from the previous block before I even got the longpoll from the GBT pool. Then of course there's a second longpoll with the transactions, which is not an unusual practice. Code: 1143.46298 Stratum notification of block from slush Now, Eloipool isn't supposed to be including transactions in the first longpoll response (that's what the second one is for). 804/822 kB reflects that in practice it is. That's a bug for me to fix :) There's also a 4 second delay between eloipool processing the block and my beginning to receive the longpoll. The entire process of queuing the LP data to clients is taking 5.5 seconds on Eligius during new blocks. When there isn't a new block, the same thing takes 2.7 seconds. Presumably fixing the size of the longpoll response should reduce this somewhat, but this is an area where Stratum actually does make a difference: the server would only need to encode the JSON response once and queue it at all clients identically. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on November 05, 2012, 04:07:07 PM Pushed a fix for Eloipool and deployed on Eligius:
Code: 2012-11-05 15:57:27 BFGMiner NOTICE Stratum from pool 3 detected new block Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on November 05, 2012, 04:31:53 PM Pushed a fix for Eloipool and deployed on Eligius: Ah, so you are saying that Stratum is magically predetermining when blocks are going to arrive to cover for why you see it later.Code: 2012-11-05 15:57:27 BFGMiner NOTICE Stratum from pool 3 detected new block Well with that suggestion, who would want to use any other protocol :P Also interesting that you have stated that for the sake of the poor performance of LP in GBT you are sending out no transactions with the LP ... yet another poor implementation feature (using LP the same way getwork uses it, is also poor implementation) I don't know how may times I've thought and said that Tycho must have been drunk the night he thought up LP ... Also the implemented GBT LP means a % increase in transaction confirmation times when compared to Stratum LP ... and getwork LP ... So the only supposed performance improvement over Getwork so far is that you can hash a 100TH/s device for 30 seconds but with getwork it will only support a 2TH/s device for 30 seconds ... (though getwork wins hands down for the amount of data transferred) Who designed this ... they should be shot. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DrHaribo on November 05, 2012, 09:29:27 PM I think GBT would run pretty smoothly if it was allowed to skip the transactions.
What harm can a pool op do when miners don't see the transactions until after a block is created? What exactly would a "dirty job" be? Filter out transactions? I think miners seeing this a little later is good enough. Including transactions that should be filtered out? That assumes we should be blacklisting someone. But who/what... stolen coins? Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on November 05, 2012, 09:43:49 PM What harm can a pool op do when miners don't see the transactions until after a block is created? What exactly would a "dirty job" be? Double spending, for example.Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DrHaribo on November 05, 2012, 10:48:55 PM What harm can a pool op do when miners don't see the transactions until after a block is created? What exactly would a "dirty job" be? Double spending, for example.I always thought about defenses against this in the direction of deciding which fork to build on. So if you have your own bitcoind or public service that you trust, you can refuse to help a pool build more than 2 blocks on a fork parallel to the "main chain" as defined by your trusted source. This should make 6-confirmation double spends practically impossible if a majority of miners followed this scheme, I think? To implement this you only need a trusted source that you can query for the current height and the block hash at height X. It should work even with getwork as you only need the "prevblockhash" from the block you are mining. A complication is getting the prevblockhash from a block your trusted source doesn't know about (yet). I guess you could also do this by remembering transactions from before a chain reorganization and refuse to include transactions on the new fork that send those coins somewhere else. You'd probably have to force those transactions into the next few blocks. You could try to do both. Don't be part of a 51% attack, but if one does happen try to get the orphaned transactions into the new fork as quickly as possible. But chances are that when the chain reorg takes place there is a double spend in the new fork already, if the attacker did his job properly. You could also use a trusted source to see that a transaction you are currently including in a block you are mining is actuallly a double spend of a transaction from a different fork. But in this case it seems easier to just refuse to deviate much from the main chain. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: wizkid057 on November 06, 2012, 12:27:01 AM What harm can a pool op do when miners don't see the transactions until after a block is created? What exactly would a "dirty job" be? Double spending, for example.I think it is more for the 0-1 confirmation double spends. I don't think that there is currently much risk of a 6 confirmation double spend from any single pool. However, any pool could attack something like satoshidice or other 0 conf service with minor luck. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DrHaribo on November 06, 2012, 07:23:01 AM I think it is more for the 0-1 confirmation double spends. I don't think that there is currently much risk of a 6 confirmation double spend from any single pool. However, any pool could attack something like satoshidice or other 0 conf service with minor luck. Less than 6 confirmations should involve only very small sums. I think in that case it would be good enough to detect after the fact. So a mining pool may be able to help someone pull off a 1 bitcoin "heist". But it would have huge consequences for the pool afterwards. In the case of 0 conf it may be necessary to have some miners detecting it as it happens. I'm not sure if there is any evidence otherwise. But what if you spread 2 conflicting transactions to different parts of the bitcoin network. When a pool creates a block with one of those transactions it may look like a Finney attack, while the pool is really just including the transaction it sees. Anyone can do this to make a pool look evil, so detecting something like that doesn't actually mean anything. Is SatoshiDice really 0 conf? So if you have some hashpower you can easily defraud them of large sums using a Finney attack? Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on November 06, 2012, 07:31:37 AM I think it is more for the 0-1 confirmation double spends. I don't think that there is currently much risk of a 6 confirmation double spend from any single pool. However, any pool could attack something like satoshidice or other 0 conf service with minor luck. Less than 6 confirmations should involve only very small sums. I think in that case it would be good enough to detect after the fact. So a mining pool may be able to help someone pull off a 1 bitcoin "heist". But it would have huge consequences for the pool afterwards. In the case of 0 conf it may be necessary to have some miners detecting it as it happens. I'm not sure if there is any evidence otherwise. But what if you spread 2 conflicting transactions to different parts of the bitcoin network. When a pool creates a block with one of those transactions it may look like a Finney attack, while the pool is really just including the transaction it sees. Anyone can do this to make a pool look evil, so detecting something like that doesn't actually mean anything. Is SatoshiDice really 0 conf? So if you have some hashpower you can easily defraud them of large sums using a Finney attack? So yeah - NO one is gonna make much money out of that. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: wizkid057 on November 06, 2012, 08:18:47 AM I think it is more for the 0-1 confirmation double spends. I don't think that there is currently much risk of a 6 confirmation double spend from any single pool. However, any pool could attack something like satoshidice or other 0 conf service with minor luck. Less than 6 confirmations should involve only very small sums. I think in that case it would be good enough to detect after the fact. So a mining pool may be able to help someone pull off a 1 bitcoin "heist". But it would have huge consequences for the pool afterwards. In the case of 0 conf it may be necessary to have some miners detecting it as it happens. I'm not sure if there is any evidence otherwise. But what if you spread 2 conflicting transactions to different parts of the bitcoin network. When a pool creates a block with one of those transactions it may look like a Finney attack, while the pool is really just including the transaction it sees. Anyone can do this to make a pool look evil, so detecting something like that doesn't actually mean anything. Is SatoshiDice really 0 conf? So if you have some hashpower you can easily defraud them of large sums using a Finney attack? So yeah - NO one is gonna make much money out of that. Actually, if you look at the SD txns, you'll see that many people pull it off all the time. I still have zero confirmation payouts in my wallet from back when I was messing with SD months and months ago because somewhere along the lines someone cheated them out of a payout and it broke the whole chain of transactions so that none could confirm. Basically once you see if you lost or not, you try to get a replacement transaction mined that displaces your losing bet. With enough hash power or control over txns of enough hash power, it'd be trivial. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DrHaribo on November 06, 2012, 09:38:21 AM Basically once you see if you lost or not, you try to get a replacement transaction mined that displaces your losing bet. With enough hash power or control over txns of enough hash power, it'd be trivial. You can mine the block displacing your bet first. After you successfully create the block, hold it back. Now you place some bets and get a winning payout from SD. Finally you release the block to the bitcoin network, getting your bet back as well. (= Finney attack) Sometimes your block will be orphaned and your bet will be lost. But in most cases you get the winnings (if any) AND your bet. An orphaned block also means losing 50 BTC (soon 25), and holding a block back increases the chance of that happening. But if each of your blocks contain bets for hundreds of bitcoins then the profits should far outweigh those occasional losses. I don't see any way for GBT to fix this. Services just have to stop accepting payments at zero confirmations. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on November 06, 2012, 09:41:31 AM ... So yeah - NO one is gonna make much money out of that. Actually, if you look at the SD txns, you'll see that many people pull it off all the time. I still have zero confirmation payouts in my wallet from back when I was messing with SD months and months ago because somewhere along the lines someone cheated them out of a payout and it broke the whole chain of transactions so that none could confirm. Basically once you see if you lost or not, you try to get a replacement transaction mined that displaces your losing bet. With enough hash power or control over txns of enough hash power, it'd be trivial. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DrHaribo on November 06, 2012, 06:05:33 PM I missed something about SatoshiDice. If they always make sure that the output of your betting transaction is one of the inputs for the transaction paying out your winnings then you can't both get the winnings and your bet back. But you can still take your coins back if you lose. So you'd win almost every bet, except when the block for your Finney attack is orphaned.
Anyway, my point was that if you can't create an automated way to make Bitcoin/mining safer by use of the transaction data while mining, then it makes no sense for it to be mandatory to push all that redundant data back and forth. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on November 06, 2012, 07:07:28 PM Anyway, my point was that if you can't create an automated way to make Bitcoin/mining safer by use of the transaction data while mining, then it makes no sense for it to be mandatory to push all that redundant data back and forth. But we can. Even without a local bitcoind, miners can setup multiple pools and the miner can avoid pools that have an odd contradicting transaction compared to all the others.Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DrHaribo on November 06, 2012, 07:33:27 PM Anyway, my point was that if you can't create an automated way to make Bitcoin/mining safer by use of the transaction data while mining, then it makes no sense for it to be mandatory to push all that redundant data back and forth. But we can. Even without a local bitcoind, miners can setup multiple pools and the miner can avoid pools that have an odd contradicting transaction compared to all the others.So if I create two contradicting transactions, send one to pool X and the other to other nodes on the network, then miners would see pool X as the odd man out and stop mining there? I'm not trying to be annoying, I'd just like to see one example where this works and isn't easily abused. My thinking so far is that the best we can do is "allow small forks because they happen naturally, but never help create large forks." Perhaps that's good enough. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: wizkid057 on November 06, 2012, 08:44:06 PM Anyway, my point was that if you can't create an automated way to make Bitcoin/mining safer by use of the transaction data while mining, then it makes no sense for it to be mandatory to push all that redundant data back and forth. But we can. Even without a local bitcoind, miners can setup multiple pools and the miner can avoid pools that have an odd contradicting transaction compared to all the others.So if I create two contradicting transactions, send one to pool X and the other to other nodes on the network, then miners would see pool X as the odd man out and stop mining there? I'm not trying to be annoying, I'd just like to see one example where this works and isn't easily abused. My thinking so far is that the best we can do is "allow small forks because they happen naturally, but never help create large forks." Perhaps that's good enough. Perhaps avoid pools is too strong of a response. Perhaps ignore the odd transaction (assuming all GBT pools :) ) would be a better response? Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on November 06, 2012, 11:59:04 PM ... So yeah - NO one is gonna make much money out of that. Actually, if you look at the SD txns, you'll see that many people pull it off all the time. I still have zero confirmation payouts in my wallet from back when I was messing with SD months and months ago because somewhere along the lines someone cheated them out of a payout and it broke the whole chain of transactions so that none could confirm. Basically once you see if you lost or not, you try to get a replacement transaction mined that displaces your losing bet. With enough hash power or control over txns of enough hash power, it'd be trivial. Listing a billion transactions that failed means nothing. Most of the failed transactions most likely are simply a chain based on a few failed transactions. The failed transactions that are the cause would have to be transactions that were sent out BEFORE the later matching transaction that won by going into a block So basically to do ANY useful analysis to see if finney attacks can be related to the FUD you are bandying about: "MANY", you would have to find the cause of the transactions - the transactions that all parent transactions are committed into a block, but that transaction itself failed BUT was valid otherwise. Then you'd have to find the transaction that was sent out AFTER this transaction and committed into a block that caused that other transaction to never be committed. Then you would have found ONE successful finney attack (though it could also be an SD correction to fix a mess up in SD) Saying 'many' people pull off finney attacks is FUD ... and your a pool OP ... Jesus found a good replacement. The risk of attempting a successful finney attack is simple - you need to put a transaction into a block and the txn must replace another valid txn already out there (0 confirmed) Firstly to do this either you would have to rely on a VERY LARGE amount of luck to expect to find a valid block in the near future and use GBT (since the other protocols wont allow you to supply transactions) ... and yeah the luck option is not gonna happen ... OR you would have to withhold the block that has the new transaction replacing the old transaction e.g. then that old transaction SD bet would have to lose (and you wait to find out) and then if the bet fails, allow the block out. Now looking at this without psychosis involved: If your bet wins you would have to withhold the block permanently or your bet would be undone ... so you have to make big bets. You would have to do it regularly to get the chance of winning a big bet You would need a lot of hashing power. Now who has a lot of hashing power? A pool. ... and as soon as a pool starts doing this sort of crap they will be found out ... so again it's pointless. Though there is one pool that is known in the recent past to have been using merged mining to do things behind the backs of the community mining on that pool: Eligius. ... and ever since has been trying to hide this fact ... Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Graet on December 04, 2012, 10:52:56 AM When, someday, the ASIC may see the light, what will happen if a lot of this new computational power goes to the same pool? Which is quite feasible Hope the pool supports GBT so the control remains with the miners.Since you chimed in with this...does Eligius or EclipseMC actually support miners being able to build their own custom blocks yet? Last I heard GBT pools were still sending miners the entire block to build, rather than expecting the miner to submit one of their own. Just because GBT makes it possible doesn't mean it's being done. Answer the Question! Quite a few people are interested in the answer it seems An honest answer would be appreciated Thanks Graet Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on December 04, 2012, 01:21:12 PM I think luke-jr must have stopped looking in that thread. No, I just continued ignoring known trolls such as yourself.The question is also irrelevant. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: os2sam on December 04, 2012, 01:41:24 PM I think luke-jr must have stopped looking in that thread. No, I just continued ignoring known trolls such as yourself.The question is also irrelevant. Is that an example of going the extra mile? I think that comment was a bit unnecessary. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Graet on December 04, 2012, 01:50:37 PM I think luke-jr must have stopped looking in that thread. No, I just continued ignoring known trolls such as yourself.The question is also irrelevant. anyone that is interested in a question is a troll unless it conforms to your views thanks for not answering - it is what I expected and does your credibility even more harm. this is a shame as it is harming Bitcoin - something you so quickly accuse others of. Please answer questions re: GBT honestly, there is much interest in the community. EDIT: the question is very relevant considering the FUD you spread trying to discredit other peoples work Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: abracadabra on December 04, 2012, 01:57:54 PM I think luke-jr must have stopped looking in that thread. No, I just continued ignoring known trolls such as yourself.The question is also irrelevant. Uhm, a "little" over the top. Don't you think? I'd say Graet is pretty well-respected. Calling him a troll brings your credibility into question. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on December 04, 2012, 02:00:55 PM I think luke-jr must have stopped looking in that thread. No, I just continued ignoring known trolls such as yourself.The question is also irrelevant. Uhm, a "little" over the top. Don't you think? I'd say Graet is pretty well-respected. Calling him a troll brings your credibility into question. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: SomeoneWeird on December 04, 2012, 02:20:41 PM I think luke-jr must have stopped looking in that thread. No, I just continued ignoring known trolls such as yourself.The question is also irrelevant. Uhm, a "little" over the top. Don't you think? I'd say Graet is pretty well-respected. Calling him a troll brings your credibility into question. And your evidence of this...? Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Pontius on December 04, 2012, 02:21:20 PM The question is also irrelevant. [...] constantly spreading FUD and other lies, with utter disregard for the truth. So, how are we stupid beings supposed to know the "truth" if you don't tell it to us? Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on December 04, 2012, 02:35:33 PM The question is also irrelevant. [...] constantly spreading FUD and other lies, with utter disregard for the truth. So, how are we stupid beings supposed to know the "truth" if you don't tell it to us?Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: slush on December 04, 2012, 02:37:12 PM Quote from: eleuthria Does Eligius or EclipseMC actually support miners being able to build their own custom blocks yet? Last I heard GBT pools were still sending miners the entire block to build, rather than expecting the miner to submit one of their own. Just because GBT makes it possible doesn't mean it's being done. The question is also irrelevant. This is perfectly valid and on-topic question. Please step away from personal acusations and stay on-topic as well. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DiabloD3 on December 04, 2012, 05:00:36 PM I think luke-jr must have stopped looking in that thread. No, I just continued ignoring known trolls such as yourself.The question is also irrelevant. Luke, what did I tell you about trolling on the forum? Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on December 04, 2012, 05:25:14 PM Luke, what did I tell you about trolling on the forum? Did you confess to doing it too much?Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on December 05, 2012, 07:22:08 AM I think luke-jr must have stopped looking in that thread. No, I just continued ignoring known trolls such as yourself.The question is also irrelevant. Uhm, a "little" over the top. Don't you think? I'd say Graet is pretty well-respected. Calling him a troll brings your credibility into question. And your evidence of this...? Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: crazyates on December 06, 2012, 03:45:49 AM Guys, this makes perfect sense. When his protocol/JesusHolyMiner fails when the ASICs hit and he just can't remain competitive, he gets to blame his failure on the Graet, Kano, and everyone else ganging up on and trolling him to death. It's not that he wrote an inferior product, or that his product even has any issues whatsoever, it's that he is nothing more than a victim of cyber bullies who are jealous and threatened by his competition. ::) ::) ::) ::)
Just you watch. I give it less than 60 days. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: firefop on December 07, 2012, 03:28:21 AM Guys, this makes perfect sense. When his protocol/JesusHolyMiner fails when the ASICs hit and he just can't remain competitive, he gets to blame his failure on the Graet, Kano, and everyone else ganging up on and trolling him to death. It's not that he wrote an inferior product, or that his product even has any issues whatsoever, it's that he is nothing more than a victim of cyber bullies who are jealous and threatened by his competition. ::) ::) ::) ::) Just you watch. I give it less than 60 days. Should be interesting to see how it goes. My prediction: nothing happens at all. But to be rational about it, it comes down to a fight over wanting to do it transparently or wanting to do it more obscurely. Personally, I'll take transparent over obscure any day... Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on December 07, 2012, 04:15:10 AM Guys, this makes perfect sense. When his protocol/JesusHolyMiner fails when the ASICs hit and he just can't remain competitive, he gets to blame his failure on the Graet, Kano, and everyone else ganging up on and trolling him to death. It's not that he wrote an inferior product, or that his product even has any issues whatsoever, it's that he is nothing more than a victim of cyber bullies who are jealous and threatened by his competition. ::) ::) ::) ::) Just you watch. I give it less than 60 days. Should be interesting to see how it goes. My prediction: nothing happens at all. But to be rational about it, it comes down to a fight over wanting to do it transparently or wanting to do it more obscurely. Personally, I'll take transparent over obscure any day... The comment above, which you clearly didn't read, was about miners. cgminer supports both GBT and Stratum. Little Luke-Jr's complaint about GBT in cgminer was that it gets too much work - when it gets work as often as Stratum does. However, that directly equates to the statement that his miner increase transaction confirm times. Which, I guess I also need to explain to you, is bad for BTC. His excuse is actually that GBT sends too much data. Since that is also exactly what that statement equates to. If Stratum and GBT get/receive the same amount of work, then according to Little Luke-Jr, that is GBT getting too much work. So for the good of BTC it is actually better to use cgminer Stratum than his GBT implementation since the biggest proponent of GBT says that transaction confirm times should be longer and has programmed his miner to make sure they are. Looking at the opposite issue: Little Luke-Jr set his miner to get the Stratum transaction list MORE OFTEN than it gets work. Which is totally STUPID since the transaction list is static for each item of work. Yes he really does do totally MIND NUMBINGLY STUPID things. He also put this up as a pull request in cgminer - which of course was ignored. Bottom line is: If you want MANY gigabytes of transactions sent to your miner every day, then yes use GBT on cgminer. If you don't want MANY gigabytes of transactions send to your miner every day, then yes us Stratum on cgminer. Think about that when you consider things like 3G/4G ... -- So, I'd actually like to know which pools have been putting bad transactions into their blocks for the last 4 years? Did anyone actually do ANY analysis of this? Of course there is the possibility of pools putting transactions into their blocks that could be bad, or even using their pools to attack alt-chains like Like-Jr did and GBT doesn't stop that from happening - gee I wonder why that wasn't an option in GBT. However, any pool stupid enough to put bad transactions in their blocks, would be easy to prove that it did it and unless they LIE as well as Luke-Jr and try and cover everything up as he does, they would get caught out. You can already do a full block analysis on any block as it is generated using an active bitcoind and a bit of code to watch the transactions that come and go and that end up in each block. There is no trick to doing this at all. Just requires a small amount of coding. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on December 07, 2012, 04:17:44 AM Bottom line is still: Kano is a clueless liar.
Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on December 07, 2012, 04:36:09 AM Bottom line is still: Kano is a clueless liar. I've proven you've lied before.Go ahead and show me where I've lied? No - as usual, you simply fall down to the level of name calling and show no intelligence at all. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DiabloD3 on December 07, 2012, 08:06:14 AM Bottom line is still: Kano is a clueless liar. Luke, continued trolling can earn you a ban on the forums. Are you sure you really want that? Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DrHaribo on December 07, 2012, 09:16:46 AM Little Luke-Jr This is how representatives of cgminer address other developers? No - as usual, you simply fall down to the level of name calling and show no intelligence at all. I LOLed. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: kano on December 07, 2012, 09:26:36 AM Little Luke-Jr This is how representatives of cgminer address other developers? No - as usual, you simply fall down to the level of name calling and show no intelligence at all. I LOLed. My point is that all he ever does is calling me a liar and never once has backed that statement up. Most of my replies include the details of what I am talking about and the reasons why. He even said of Graet: "constantly spreading FUD and other lies" Which is pure bull shit - though I guess you swallowed it whole right? DrHaribo? Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: -ck on December 07, 2012, 09:47:12 AM Little Luke-Jr This is how representatives of cgminer address other developers?Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Nachtwind on December 10, 2012, 01:09:43 AM ...and i really thought i could get more information on GBT here.. found myself in a flamewar though. Honestly guys.. noone here is giving a good example.
Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: DrHaribo on December 12, 2012, 11:37:06 PM ...and i really thought i could get more information on GBT here.. found myself in a flamewar though. Honestly guys.. noone here is giving a good example. Sorry that I added to the spam. But there is a reason these forums are so unfriendly. kano has a habit of trolling, name calling, religious harassment, etc. etc. I don't know why he hasn't been banned yet. But if noone says anything, things won't get any better. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Thistled on January 20, 2013, 05:07:30 PM How do you guys propose to compare dick size?
That could be interesting. However, I am sure there are forum rules against that. lol. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: crazyates on January 20, 2013, 05:18:16 PM How do you guys propose to compare dick size? I don't want to play tit for tat and name calling and dick size comparisons Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Thistled on January 20, 2013, 05:24:13 PM lol.
Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on January 20, 2013, 06:02:46 PM How do you guys propose to compare dick size? http://en.wikipedia.org/wiki/Yao's_Millionaires'_ProblemTitle: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: W-M on April 04, 2013, 01:56:16 PM Hey guys, sorry to interrupt your discussion about penis sizes, but I have a question about the workings of getblocktemplate :P.
Recently I have been working on a small getblocktemplate miner implementation in JavaScript/PHP, mainly to understand how the bitcoin works internally a lot better. The Wiki page (https://en.bitcoin.it/wiki/Getblocktemplate) about getblocktemplate has helped me a lot, but there is one crucial thing I cannot figure out: How to send a proper block template request to a pool. (This is the request a mining program makes to the pool to indicate it wants to mine something) Everything I've tried thus far makes the pool return Code: {"result":null,"id":null,"error":{"name":"JSONRPCError","code":100,"message":"Parse error"}}The Wiki page tells me the following syntax: Code: {"id": 0, "method": "getblocktemplate", "params": [{"capabilities": ["coinbasetxn", "workid", "coinbase/append"]}]}Problems I stumble across: 1. I'm unsure if the "coinbasetxn", "workid", "coinbase/append" and even the whole "capabilities" are optional, or that the brackets in the example are actually part of the syntax to forward the different arguments as an array (as they are usually used in JSON). 2. the BIP 22 (https://en.bitcoin.it/wiki/BIP_0022) does not list this request at all, only telling me about two non-optional arguments. 3. Analyzing the python-blkmaker source code, I find another argument called "maxversion". This is not mentioned on the wiki page and not on the BIP 22 page or anywhere else either. So, what do I do? How do I make a proper block template request? Thanks, ~W-M Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on April 04, 2013, 03:35:37 PM Code: {"id": 0, "method": "getblocktemplate", "params": [{"capabilities": ["coinbasetxn", "workid", "coinbase/append"]}]}Problems I stumble across: It's all optional, but necessary for some servers which make decisions on what features to use in your template.1. I'm unsure if the "coinbasetxn", "workid", "coinbase/append" and even the whole "capabilities" are optional, or that the brackets in the example are actually part of the syntax to forward the different arguments as an array (as they are usually used in JSON). 2. the BIP 22 (https://en.bitcoin.it/wiki/BIP_0022) does not list this request at all, only telling me about two non-optional arguments. These are part of BIP 23 mainly.3. Analyzing the python-blkmaker source code, I find another argument called "maxversion". This is not mentioned on the wiki page and not on the BIP 22 page or anywhere else either. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: W-M on April 04, 2013, 07:16:37 PM Thank you for your reply, Luke.
Code: {"id": 0, "method": "getblocktemplate", "params": [{"capabilities": ["coinbasetxn", "workid", "coinbase/append"]}]}Problems I stumble across: It's all optional, but necessary for some servers which make decisions on what features to use in your template.1. I'm unsure if the "coinbasetxn", "workid", "coinbase/append" and even the whole "capabilities" are optional, or that the brackets in the example are actually part of the syntax to forward the different arguments as an array (as they are usually used in JSON). 2. the BIP 22 (https://en.bitcoin.it/wiki/BIP_0022) does not list this request at all, only telling me about two non-optional arguments. These are part of BIP 23 mainly.3. Analyzing the python-blkmaker source code, I find another argument called "maxversion". This is not mentioned on the wiki page and not on the BIP 22 page or anywhere else either. [/quote] Hmm... Yes, in BIP 23 more optional parts are noted. However, the structure of the JSON-RPC request isn't outlined anywhere. Unless someone had worked with JSON-RPC before they wouldn't know about the exact way to invoke this. Maybe a link to the JSON-RPC Wikipedia page would be nice :). Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: W-M on April 06, 2013, 07:56:38 PM In the meantime, I got it working. The problem was that I parsed all my JSON objects to post parameters. (i.e. vara=valuea&varb=valueb). This is the way AJAX requests are normally done from Javascript or by any browser submitting a form. It never occured to me that the solution would be to just put the JSON-RPC objects as a string in the postdata of the HTTP request, and voilà, the server was able to parse it and returned a block list.
So I got it working! My Javascript/PHP miner is mining, albeit extremely slow(and with some stops where my host tells me I've done too much requests from one IP in a short time), but it works. I have yet to submit my first real share to the server, and probably some other bugs will pop up later down the line, but for now it is hashing. Hooray! :-) Thank you for your help, Luke-jr ~W-M Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: KNK on May 06, 2013, 11:21:11 AM Hello,
i am looking at making my own poll app based on GWT, but got confused at some pint and still can't figure it out. I've read BIP 22 and 23, but some real world example would be welcome. bitcoind returns: Code: {Is the first transaction listed the coinbase one (guess not) or i should create my own? If the later - how do i make it in order to contain the flags (and what do they mean)? Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: Luke-Jr on May 06, 2013, 11:23:34 AM Hello, If there is no "coinbasetxn" key, you need to make it.i am looking at making my own poll app based on GWT, but got confused at some pint and still can't figure it out. I've read BIP 22 and 23, but some real world example would be welcome. bitcoind returns: Code: {Is the first transaction listed the coinbase one (guess not) or i should create my own? If the later - how do i make it in order to contain the flags (and what do they mean)? https://en.bitcoin.it/wiki/Protocol_specification#tx Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: KNK on May 06, 2013, 12:13:44 PM Yes, that was my understanding too (glad it's correct), but where may i find information about the coinbaseaux flags?
BIP 22 states 'data that SHOULD be included in the coinbase's scriptSig content', but no mention of flags in https://en.bitcoin.it/wiki/Transactions P.S. In short my idea was to have each frontend worker have it's own address for coinbase (to avoid share duplication) and have the blocks valid/accepted even if they submit it to another bitcoind instance as backend. Title: Re: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) Post by: zeffirsky on September 15, 2017, 12:13:20 PM Hello, sorry for maybe stupid question. Let's suppose that I run two instances of bitcoind on my server. Can I accept block template from first instance but submit share to second? If I understand it correct: getblocktemplate is stateless in terms of blockchain, and pool sever can choose which transactions to include in candidate block. And can submit blocks to any bitcoind instance.
Is it possible to load balance between two bitcoind instances using getblocktemplate? |