Ahh ok - I completely understand what you’re saying now. I misread as the individual algorithm took the chip.

The way i’d attack Lyra2Rev2 in hardware is a literal pipeline of chips, sized according to paralyzed throughput. It looks like the whole chain is 256 bit hashes, so 400gbps interconnect could handle your 1Gh+. For chip to hip interconnect on the same board 3 quads of 32 Gbps should be sufficient.  The Blake/keccak skein is probably all on the same chip or a much smaller chip.

Your 9P is probably $3000, you could buy 2-3x the luts for pipelines on smaller chips for that...

All that said it looks like the 1GH is worth about $1000/mo right now, so still quite a long payout if you’re using $12k in hardware.

Are you saying the algorithm  for an individual cubehash pipeline hardly fits on a 9P???... while I haven’t studied cubehash specifically, it’s claimed to take 200 cycle on a basic CPU, and I can implement a lot of basic CPU cores on a 9P...

That’s exactly what I’ve been working on, for reasonable definitions of small and fast.

I have two active projects in the first spin batch phase. One is nVME with basically the biggest thing you can fit on there, and it augments GPUs more than works standalone.

The second is 4 chips on one PCIe card, with a switch, but the most reasonable chip that can be used in that configuration is still not what you would call cheap. Frankly even the nVME chip is as much as some graphics cards to get 4x 3.0 PCIe lanes.

The one advantage is the 4-chip board uses modules, so you could buy one with 1 module populated in the 3 figure range. When it is ready, which is likely August at this point for mass production.

You can’t fit a Stratix 10 on a nVME stick... I’ve tried. Kintex or Arrria is about as big as you can get. Damn 22x80 form factor.

I’ll back up senseless here - the board he is proposing is useful and valuable for all the FPGA mining being done. There will be many FPGA mining options soon, but if you’re interested in getting into this space it is a good option.  


As to the picture,
VCU118 - aka the ol’ Ultrascale+ test/dev bed. http://imgur.com/FKVHPNs

This is not the same space as GPU mining. Expect Bitstreams, miner development, etc. to be more ASIC like.

Edit: I would be interested in 8 of these for some testing.

This isn’t hard. I do it all the time for a few algorithms. Here’s a contrived example - fill the scratchpad for CN7 on one FPGA dedicated to that, spitting out 2MB scratch pads all day long, and taking them back in and compressing / finalizing them. Total bandwidth for (example) 22kH is 343 Gbps. That’s achievable on lots of current hardware.

This makes it a lot easier build two sets of pipelines on two FPGAs for two related but very different set of operations. Doing all the things on two separate FPGAs couldn’t achieve the same performance.

This was a back of the envelope example, but the Xxx algorithms that just chain more on to the process definitely lean them selves to this kind of operation. This (and memory bandiwdth, and easier cooling) is why my accelerator cards have 4x75W
Ultrascale + FPGAs and not one big Virtex. Interconnect on those is 256 Gbps.


Edit: Let me try to phrase this in a few words. Don’t waste the extremely high bandwidth interconnect and resources inside the FPGA for something you can use the slow external interfaces to accomplish.

Partial reconfiguration - you don’t need every combination, just every building block.

Really? This is standard operating procedure for Xilinx. Dev boards are usually sold cheaper than even just the chip cost for normal quantities. They want to make it easy for people to build things around their expensive chips, but still make all the margins on chips. I too have been talking to Xilinx and trying to get them to see there is a big market here, big enough to make up for their margins on some of these chips.

There is more FPGA hardware coming pretty soon (over the next 2-3 months)  from a few sources (disclosure - myself included). Generally speaking most are going to be in the same ballpark cost/performance, but there may be outliers or easier entry level costs.

The problem with FPGAs is it is very hard to protect the bitstream if you aren’t selling the hardware. You can sell hardware that will work with your encrypted bitstreams and also support unencrypted bitstreams (anyone can make new miners). You can also lock hardware to only your encrypted bitstreams. But if you don’t sell the hardware (and burn in the encryption keys) anyone can copy it. So that leaves virtuous open source, or people making profit on hardware alone. The tools and time to develop these things are very expensive, so you likely won’t find a lot of people giving it away for free. Development time is also much longer. If I make a change that actually needs tested on real hardware of VCU1525 size, expect hours for a very fast computer  to synthesize and route it before I can test it. A lot can be done in simulation, but the last mile takes a lot of time.

It is possible to use normal methods to try and protect the “command and control” normal software that drives the FPGA, but someone could reverse engineer that - just like OpenCL kernels.

If you think the FPGA is magic and the OpenCL compiler will give you 10-50x performance over your GPU, you will be sorely disappointed. Those of us seeing these gains are (afaik) writing very low level RTL to do so. There’s a big difference between OpenCL and RTL for FPGAs vs OpenCL vs ISA Assembky for GPUs.

What is hoped to be accomplished with HBM/HBC? What algorithms ? These typically offer in the order of 500GB/s. That means for ETH they support a max of 62MH, and for CN7 they alone support “only” 16KH, but they add huge cost premiums to FPGAs, and offer no advantage over GPUs.

The FPGA advantage is in calculation bound code, or algorithms whose work happens in memory spaces < 40MB or so, where the onboard 11+ TB/s of ultraram/BRAM can be used.

External QDR is going to top out at 80-160 GB/s too because of pin count. Sure it is low latency but that is only part of the equation.

The absolute most bandwidth you can get off chip is if you used 128 32 GBit/s transceivers in a very expensive chip, and that will give you... 512 GByte/second again.

There’s little to be gained with off chip memory of the very expensive variety, in most cases.

Aren’t we all cussing out whitefire990 for telling the world?

Poor smartcash, it’s not gonna know what to do with all the hashpower it’s about to get

I second this. What chip, or even brand are you using?

To sell for this price in Xilinx brand, you’d be Artix with some external QDR maybe? Still the hashrate is... tricky at that power consumption. I’d buy a bunch of them if I knew what FPGA so I could program them myself...

I too have a 7W board in development/ initial prototypes along with the big one. I don’t think I could come anywhere close to even the AES for 7KH on that one. @200Mhz artix chips would be capable of that’s around 5-10 M/AES full passes a second, 10 pipelines maybe on chip by the time other logic needed.. That only supports even scratch pad fill/inner/compress of 1 kh.  So I would say again what chip?

Did I miss the detailed performances spec, price, and power consumption figures on the 1170 getting published?

There isn’t really much magic to the RTL for these things. Most of the performance gains are more about working around the nuances of the FPGA design and finding hardware where someone has laid out all the needed components (memory bandwidth and/or low latency memory, multiple chips with high speed interconnect, etc. I do wonder how the the OP intends to secure the bitstream if he distributed it. It would be pretty easy to write your own miner, and you can only encrypt the bitstream if it is your own hardware and you burn eFuses or similar.

Also with FPGAs and dev boards you are usually given very detailed manuals and all the schematics.

Anyone equipped to do ASICs could implement the algorithms themselves, it’s all about the capital - and accepting that all that cost is wasted if the algorithm is changed. For a lot of modern algorithms it is more about assembling the memory components than even the ASIC cores.

The reason some of us are long on FPGAs is they have the potential to be lower cost, much lower power, and higher performance than GPUs for algorithms that choose to resist ASIC development (aka change regularly). There is very little a GPU can do to gain advantage on modern (upcoming) FPGAs in that world. The transition to HBM as standard and Intel ending the artificial high price racket are the only barriers. It doesn’t take that long to stand up working RTL for an algorithm change. Personally I’m thrilled if Crypto pressure ushers in a whole new era of high performance computing.

I stand corrected, I was not aware of the existence of that flag! I knew you could read the XADC, but did not realize there was a bitstream flag. Good to know, although I don’t know if I would rely on that as your only protection.

I appreciate the wealth of knowledge and skill coming out on this forum regarding the FPGA work.

Regarding workstation GPUs, I said they could be useful for mining, and can work alongside FPGAs given the direct access to their memory, not that they were better than normal GPUs for mining or that they were like FPGAs. You can however very often find large batches of the older generation passive cards for cheap as most people don’t have the ability to use them. I last paid $130/ea about a month ago for a bunch of older FirePro workstation cards that are now hashing at 24MH ea.

“Turn on overtemp shutdown in the bitstream”

AFAIK there’s no such magic button. If you want temp monitoring and thermal limits with an FPGA you have to include them in your design logic. Same as an ASIC. Otherwise you very much can exceed junction temps and damage the hardware if you have enough power going into the board to begin with.

This is also possible on some GPUs with poor drivers. A few of my old Titan Blacks from HPC work had power stages for memory that would overheat if the memory was trashed too much for too long, and the mosfet would slip down on the boards till they caused a short. Actually caught a entire server on fire that way once...

We operate custom hardware. We’ve done the 100 device first spin, and are currently working on a full batch order > 1000 FPGAs for the Ultrascale+ end.

I haven’t posted here much, so I seem to have a lot of message reply and rate limits.

Regarding AWS - once the cat is out of the bag it won’t be profitable.

Im not sure you actually read my posts on the topic, as you’re repeating a few things I already stated - such as I don’t do secondary hashes on the FPGA. 30k LUTs for AES? That’s a huge amount more than my cores...

You mean the FirePro type GPUs? Definitely. I have many many older FirePro cards that had plenty of memory bandwidth but underpowered (or over powered depending how you look at it) cores compared to the current gen. This cards got large page table support and some other niceties because of who the customers were. The also support an FPGA on the PCIe bus directly writing into / readying from their GPU memory (albeit at PCIe speeds). This works great with something like ETH, because you can do all the compute heavy Keccak off GPU but still use all the memory bandwidth.

We are low level RTL here, so I haven’t messed with much of the HLS side. The big issue with FPGAs is people used to buying graphics cards that are all commoditized close to a low MSRP don’t understand that the FPGA market is nothing like this. The traditional market for these things was Avionics, Defense, Government, Nation States, etc. the price on digikey is no where near the actual price for someone building their own hardware, and the dev kits are expensive because they come with every bell and whistle you might want to use with a particular chip and they are low low volume. This is changing a bit but these VCU1525 dev accelerators are no different than Nvidia’s P100/V100 PCIe card’s or AMD’s old FirePro, now Radeon Pro multi-thousand dollar cards. They don’t cost 10x more to make, the target market is just far less cost sensitive.

The gap between ASIC and FPGA is also much narrower now than it was a few years ago.

If a market emerges to consume 100,000 FPGA accelerators for mining, someone building their own hardware with a good relationship with Xilinx or Intel can and will drive the cost down at least an order of magnitude.