Algorithmically placed FPGA miner: 255MH/s/chip, supports all known boards

[BR0KK]

I'm only on 5 Ztex 1.15x Boards ATM but i'd love to see a faster Bitsream for them. But since i'm only a small scale Miner i'm not shure how much i could invest into a new Bitstream...... Sending all my boards to the US or locking them to a single Bitstream is no option.

Maybe if there were a Community funded Project so that he can get some money for his efforts i could invest at least something into it

Question:

How safe is it to use that Bitstream? i really do not want my boards go up in flames while mining .....

[eldentyrell]

Hence I will soon have over 25 LX150 boards and it'd be trivial to use my business-case spreadsheet to work out how much I'd be prepared to pay for a bitstream that converts my 208 MH/s units into 300-odd MH/s units.

Indeed, a hypothetical 300MH/s bitstream would make your equipment 50% more valuable for bitcoin mining purposes (let's not kid ourselves here: the non-mining market for these stripped-down boards is nearly zero).  So a rational miner would be willing to pay 50% of the cost of their boards in order to get these results.  Rest assured that I have no intention of charging nearly this much, although it is an important data point in assessing the value of what I'm providing.

However, unless there are miners out there with tens of thousands of LX150-based boards already running,

I think you are greatly underestimating the number of FPGAs out there that are mining.  The number of people involved in FPGA mining is small, but there are already several very large FPGA farms.  Also, I have pretty good info on Xilinx' pricing curve, which lets me put a lower bound on how many units the major vendors have sold.  Lastly, I've been watching AVNet's inventory data, and they are moving pretty huge quantities of LX150's compared to LX75's -- and I don't know of any other non-scientific-computing product that uses them.

I can't see any single miner paying enough to 'compensate' for the number of hours development involved,

Neither can I, which is why kickstartr is currently the most likely option.

But... all this is academic.  Until there is a larger performance boost and I have done a demo in front of independent witnesses there is neither enough data nor a real need for this discussion.

If a unit fails, and needs re-programming ... If the only way to prevent 'theft' of the bitstream would be to lock the FPGA so it can't be used for other purposes

I'm not enthusiastic about the bitstream encryption route, but I do want to point out that this is just flat-out false.  ADDING a decryption key to a Spartan in no way prevents it from being used for other purposes -- you don't have to use it (in fact, most bitstreams don't!).  The encryption key is stored by writing to eFuses; Xilinx has multi-million dollar customers relying on those fuses.  They are no more likely to fail than the rest of the device (in which case you're screwed anyways).

it's probable that the open-source effort will, eventually, squeeze enough hints from the thread to build their own version.

I doubt it.  You really do have to start over from scratch to do an algorithmically mapped and placed design.  Sure, somebody might do that, but it most certainly won't arise as a derivative of an xst-and-xilinx-map design.  And even if they're twice as fast as me it will take them four months -- so if you haven't heard about somebody starting an open-source project like this today you can be sure there won't be any ready to use until at least 4Q2012.

[Energizer]

So when are you planning to start offering the bitstream for Icarus boards? how much it will cost? and what are the procedures? We really have to send the boards to you? This actually would be imposible for me as am not in USA!

[shackleford]

Whatever happened to that guy that figured out how to implement BF_INT (or something) on the GPU miners before the rest of the coders did shortly after?

I looked for a few min on to forum and did not find it as if people have completly forgotten.

[mrb]

Whatever happened to that guy that figured out how to implement BF_INT (or something) on the GPU miners before the rest of the coders did shortly after?

I looked for a few min on to forum and did not find it as if people have completly forgotten.

I am the guy.

What happened to me? Well for a while I was selling my BFI_INT-enabled GPU miner. My strategy to prevent it from being leaked and pirated (I think the only strategy that can work) was to price it quite high (400 BTC which was about $200 at the time), so that the buyers would feel they had a valuable, exclusive product that they would not want to leak. AFAIK it worked and was never leaked. At the time I was 10-15% faster than the other open source miners.

Then the open source ones progressively caught up, so I discounted the price down to 300, then 250 BTC. And eventually I stopped selling it altogether.

[shackleford]

Whatever happened to that guy that figured out how to implement BF_INT (or something) on the GPU miners before the rest of the coders did shortly after?

I looked for a few min on to forum and did not find it as if people have completly forgotten.

I am the guy.

What happened to me? Well for a while I was selling my BFI_INT-enabled GPU miner. My strategy to prevent it from being leaked and pirated (I think the only strategy that can work) was to price it quite high (400 BTC which was about $200 at the time), so that the buyers would feel they had a valuable, exclusive product that they would not want to leak. AFAIK it worked and was never leaked. At the time I was 10-15% faster than the other open source miners.

Then the open source ones progressively caught up, so I discounted the price down to 300, then 250 BTC. And eventually I stopped selling it altogether.

I am glad you saw this. I know I would have used  your miner if I could have (I was and still am pretty small time).  I don’t know if other people do but every piece of open source software like miners and desktop gadgets or free pools and bounties I have used for this hobby I have contributed with BTC. Not windfall amounts but respectable at the time. Would you have done it the same way again? Maybe what you did was the best way to go about it. Do you feel you were properly compensated for the time you put into it? (perhaps after the rise in btc value)

I don’t really care what is decided with this new FPGA approach.  I know eventually people are going to find every way possible to squeeze every bit of performance out of these chips as possible.  I only mentioned the BF_INT thing because the situation sounded familiar.

One thing is certain, this community has people that are very passionate about the related technologies and you can’t really compete long term with people who work for free.

[mrb]

I certainly feel I was properly compensated.

Today, I would either do it the same way, or do it using the Kickstarter model like suggested on this thread. For those not familiar with it: http://en.wikipedia.org/wiki/Kickstarter
In 2 words: funds are pledged by potential customers. If a funding target is not reached by a certain date, money is returned to those who funded. Else the funds go to the seller who can finish producing and releasing the product.

[Wandering Albatross]

The value of your improvements, if they are for real, might be diminished as time goes by. It seems if this is legit you should have a kickstartr going real soon.
Also may be that a project has been announced, due in May (see icarus thread) that will be as fast/efficient/inexpensive as your target.

So you toil away, achieve your goal, then find out you were beaten to market. Ouch.

[eldentyrell]

Just a ping here since I haven't updated the thread in a while; now that the semester is winding down (I teach CS) I should have enough time to take this across the finish line.

I've come to a much better understanding of how the Spartan6 routing fabric works.  Originally I just focused on packing all the logic into the device and keeping things that communicate nearby.  That's not quite enough: in order to make sure Xilinx's router doesn't do stupid things you have to make sure that things that communicate are aligned either vertically or horizontally.  All the retiming and reorganization needed for this took a long time, but it's paying off: the latest iteration has a design clock rate of 180mhz and meets timing for all of the "ordinary" stages -- i.e. all of them except the funny ones at the very beginning, the very end, and the corner turn (stages 30-31).  I also factored in a lot of changes I'd been putting off that reduce the register count by about 10% (mostly to save routing resources).

Lastly, my "non-homebrew" Spartan6 board just arrived so I will be able to post useful power numbers soon.

[Garr255]

This looks awesome. What school do you teach at? It seems like it'd be a good place to learn CS

[TheSeven]

Dr. Tyrell, can you please answer an important question for us board designers?

How much current on VCCINT does your design currently use, at which clock frequency?

It's probably in your own interest to ensure that future boards meet the power requirements of your design.

[c_k]

I really look forward to your finished product, it sounds very exciting

[Inspector 2211]

the latest iteration has a design clock rate of 180mhz and meets timing for all of the "ordinary" stages

Drool.

-- i.e. all of them except the funny ones at the very beginning, the very end, and the corner turn (stages 30-31).

Could one, two, or all three of these corner cases be solved by adding a "dummy" stage?

[eldentyrell]

How much current on VCCINT does your design currently use, at which clock frequency?

Short answer: I don't know right now.  Measuring that is actually why I shelled out for a non-DIY Spartan6 board, which *just* arrived.  My own homebrew boards are crap and probably leak power all over the place.  However, getting the 180mhz design running is a higher priority right now.

I have not yet put any effort at all into minimizing power consumption, and don't plan on doing so until I feel that further performance improvements are tapped out.  FWIW I am still using SRL16s, which are power hogs, instead of RAM32Ms with LFSR address generators.

It's probably in your own interest to ensure that future boards meet the power requirements of your design.

The best strategy is to have the Spartan and power supply on separate boards so that you can replace or underpopulate the power supply boards if needed.  This is what I and at least one other person do.  If I find out that I underdesigned the power supply I can just leave a slot empty.

If you can't put them on separate boards, you're going to have to overdesign by a wide margin to be sure you don't get left behind due to running out of power.  Artforz' boards can deliver a whopping 15A of current to each chip, which is so much current that you'll hit insurmountable cooling problems long before you run out of power -- plenty of margin.  Ztex is a far better board designer than I am, but I do feel that he skimped on the power supply by providing only 8A and I was disappointed to see that he hasn't upgraded this on his 4-chip board.

[TheSeven]

The best strategy is to have the Spartan and power supply on separate boards so that you can replace or underpopulate the power supply boards if needed.  This is what I and at least one other person do.  If I find out that I underdesigned the power supply I can just leave a slot empty.

If you can't put them on separate boards, you're going to have to overdesign by a wide margin to be sure you don't get left behind due to running out of power.  Artforz' boards can deliver a whopping 15A of current to each chip, which is so much current that you'll hit insurmountable cooling problems long before you run out of power -- plenty of margin.  Ztex is a far better board designer than I am, but I do feel that he skimped on the power supply by providing only 8A and I was disappointed to see that he hasn't upgraded this on his 4-chip board.

My personal bet was on ~10A per FPGA until now. (And you can probably actually pull 11-12A from most 10A supplies if you need to.)
PSU on a separate board is not an option, trace/connector voltage drop quickly becomes unmanageable with that. For these 50A that you ask for (for a quad-FPGA board) you just need solid power supply layers.

[eldentyrell]

PSU on a separate board is not an option, trace/connector voltage drop quickly becomes unmanageable with that.

Hrm, look, I'm an amateur when it comes to PCB design, but you might want to reconsider that.  What you say is certainly true for the good old IDT headers (the kind you connect ribbon cables to) -- I've seen that myself and got burned by it.  But the Molex Mini-Fit connectors can carry 13A per pin -- they're the same connectors that deliver power to your motherboard and your graphics card.  I measure no more than 15mV drop across the connector on my boards when running at full power.  That's not to say I don't have power distribution headaches -- just that the IR drop across the connector isn't one of them.

For these 50A that you ask for (for a quad-FPGA board) you just need solid power supply layers.

Actually I think I've been extremely careful to avoid giving a specific power number.  I'm stalling (sorry) until I can measure the new 180mhz design -- which has much shorter wires and 10% fewer registers -- on a high-quality board.

In other news, I've got some used Virtex5-155 boards showing up next week.  Porting the design to them was stupidly easy.  Virtex5 looks like Spartan6 with faster routing, more carry chains, and (most important) without all the idiotic "potholes".  So much more pleasant to have a perfectly regular fabric.  Porting to Artix7 is the same story except that it has "potholes" so I'm not terribly motivated to go through the hassle until I know when the boards will be available.

[TheSeven]

PSU on a separate board is not an option, trace/connector voltage drop quickly becomes unmanageable with that.

Hrm, look, I'm an amateur when it comes to PCB design, but you might want to reconsider that.  What you say is certainly true for the good old IDT headers (the kind you connect ribbon cables to) -- I've seen that myself and got burned by it.  But the Molex Mini-Fit connectors can carry 13A per pin -- they're the same connectors that deliver power to your motherboard and your graphics card.  I measure no more than 15mV drop across the connector on my boards when running at full power.  That's not to say I don't have power distribution headaches -- just that the IR drop across the connector isn't one of them.

The connector itself is only part of the problem here. An you certainly don't have 13A on that pin you measured...
If you have to route power through connectors, this also extends the trace length/decreases the trace width towards the connector. In the end we're talking more like maybe 50mV drop here, which can already cause trouble in applications like this (and generally lowers efficiency). Having the regulator on the FPGA board itself just makes more sense IMHO, for a variety of reasons. Ask the guy who made the design you quoted above, he also got burned by that

For these 50A that you ask for (for a quad-FPGA board) you just need solid power supply layers.

Actually I think I've been extremely careful to avoid giving a specific power number.  I'm stalling (sorry) until I can measure the new 180mhz design -- which has much shorter wires and 10% fewer registers -- on a high-quality board.

In other news, I've got some used Virtex5-155 boards showing up next week.  Porting the design to them was stupidly easy.  Virtex5 looks like Spartan6 with faster routing, more carry chains, and (most important) without all the idiotic "potholes".  So much more pleasant to have a perfectly regular fabric.  Porting to Artix7 is the same story except that it has "potholes" so I'm not terribly motivated to go through the hassle until I know when the boards will be available.

How about Kintex btw? What kind of fabric is that one using?

I think it would already be very valuable to us if you could measure the power consumption of whatever you have currently at 100MHz. We can compare to the existing designs running at the same frequency and will get at least a ball park estimate of what the requirements for your design are going to be. Sure, this is likely to be optimized over time, but an upper bound would be very helpful

[eldentyrell]

An you certainly don't have 13A on that pin you measured...

Of course not, because I'm using four of them! (and another five for ground)

In the end we're talking more like maybe 50mV drop here, which can already cause trouble in applications like this (and generally lowers efficiency).

You have to ask yourself here, which is worse: losing a bit of efficiency or being stuck at (say) 220MH/s while everybody else is getting 270MH/s?

Just trying to be helpful.

BTW, you might want to consider software-controlled voltage.  It isn't hard; my boards have it.  Most high-quality DC-DC converters determine the output voltage based on a resistor.  Stick a digipot in there instead.  If you do the math right you can arrange things so that even if the digipot fails completely (0 ohms resistance) the voltage doesn't cause damage to the FPGAs.  There's a fair bit of room between the maximum operating voltage and the voltage Xilinx says they can handle without being damaged.

Another alternative is to put a jumper on your board that disables the power supply (they all have "enable" pins these days) and include space for an 8-pin mini-fit connector that isn't soldered down by default.  This gives you an emergency escape, albeit an ugly one, in the event that you underbudgeted for power: add on the connector and use an off-board power supply.

I think it would already be very valuable to us if you could measure the power consumption of whatever you have currently at 100MHz.

I don't.  The numbers would not be representative of the final design.  And I don't want to start rumors about "eldentyrell's design won't work on board XYZ because it can't supply enough power" unless they're actually true (which is highly unlikely).  Having to shift my focus from performance to power prematurely in order to fight these rumors would be an inefficient use of my time.

[antirack]

I will offer a 10BTC bounty to anybody who posts the JTAG IDCODE readout from the BFL single -- merely to satisfy my curiosity.  There was a JTAG header on the last PCB I saw them post.

Here you go:
https://bitcointalk.org/index.php?topic=66244.msg870733#msg870733

Send your bitcoins to: 139uZdmLamPy2uifmijbGJBJAfYg4HqUZp

 

[eldentyrell]

I will offer a 10BTC bounty to anybody who posts the JTAG IDCODE readout from the BFL single -- merely to satisfy my curiosity.  There was a JTAG header on the last PCB I saw them post.

Here you go:
https://bitcointalk.org/index.php?topic=66244.msg870733#msg870733

You've only posted half of the IDCODE readout (there are two chains).

Send your bitcoins to: 139uZdmLamPy2uifmijbGJBJAfYg4HqUZp

I will be happy to do that once you post the other half of the IDCODE readout!