Bitcoin Forum

Hi

I'm looking into creating a rig using FGPAs.
To control them I would like to use a Beagleboard-xM or Pandaboard ES.

The Beagleboard-xM has a 1GHz ARM Cortex-A8.
The PandaBoard ES has a Dual-core ARM Cortex-A9 MPCore with Symmetric Multiprocessing (SMP) at 1.2 GHz each.

Has somebody used these boards?
How many FGPAs do you think it will able to manage?

Thanks.

Estimated hashes? ;D

As a start 900 - 1000 MHash

You really could use almost anything to control the FPGAs. For example, the Icarus board has simply a UART-USB chip. It takes USB data and feeds it serial into the FPGA. So the interface/control of the FPGA is trivial computing wise. The task is more about how you get work for the miner and UI/RPC.

I'm building a single chip PIC with Ethernet board as a miner control to feed a cluster of FPGAs. It's not built yet but the PCB design is done. I'm waiting for the Chinese PCB fab to re-open after their New Year so I can make some boards. Good they took a holiday as I'm having extra time to fix mistakes and improve the design.

I would rather go with a RaspberryPi than a beagleboard.

http://www.raspberrypi.org/

Your DD-WRT'd USB router :)

I'm eager to get my hands on one when they come out. Credit card size coolness.

that would be really awesome if someone can get that to work.

Well, got cgminer to compile for the mips (broadcom) processor my ASUS RT-N16 has.
Absolutely no idea if it does ANYTHING, but it compiled!

All in need to do now is:
Actually install DD-WRT on my router.
Acquire a BFL unit.
See if it does anything :)

WAT? how? on what board. Let's break this down.

ps. The Raspberry Pi seems like it wouldn't hash much... But $25, I guess you can just buy LOTS of them.

I think you missed it completely there n0ne. LMAO!

Hes wanting to use a Raspberry Pi to control a farm of FPGA's.  The BFL's are supposed to get around 854 mHash.

haha yea maybe it went over my head. I'm a bit new to this FPGA stuff. The way I under stand it i RaspberryPi board with it's on-board chip would be way to weak. So I'm guessing the idea is to control something else with it?

ps. I'm drunk, be nice :D

NA. I think it would be OK.  Its a pretty powerful liittle arm processor.
All you really need is just something to distribute work to the BFL units, server as a "hub" and connect to your pool.  All the hasing is done on the BFL (FPGA) in much the same context as a GPU does the work in a "normal" rig.

Oh ok I see. The idea is to use one of these to be the controller for all the BFL boxes. Neat, would make things more stream line.

Unfortunately they are not for sale yet.
But as soon as they are I will get myself one of these anyways.


Exactly.
This should manage several independant FPGA boards.
BFL..., well let's see when they are out.

I use a mini-itx atom D510MO as desktop and run my fpga from that, economical, silent and better for the environment and with proper linux!

<3 Pico-ITX formfactor with a Via processor.

The RaspberryPi is supposed to be around 5 watts.  
Most Mini-ITX boards will be in the 50 to 75 watt range.

Oh, the RaspberryPi also runs a proper linux distro....Debian :)

50-70W? For a atom or via board? In what universe?

Im talking total power draw measured using a kill-a-watt.  You figure in the ineffeciency of the PSU, HDD, and the mobo, you will see around 50 watts.

PSU? What else are you going to run your FPGAs off of? A few dozen <75% efficient chinese wall warts?
HDD? USB stick or CF card in PATA adapter. Even a 7.2k 2.5" is <1W spinning.
Mobo? I give you that, 3.5W vs. 20W peak ...
So we're talking about 430 vs 450W for 10Gh with S6s, 23.3 vs. 22.2 Mh/J ... not exactly a massive deal.
For something with efficiency similar to BFLs vaporboards, it's even less of a difference, 2-3% max.

Now, what something like a pi does have going for it is power (see above, not *that* big of an issue), cost and size. = A lot cheaper, and way easier to tuck into a corner somewhere.

Benefits of a generic x86 ... can install any random OS, way easier to set up a full dev toolchain without having to fiiddle with crosscompiling, > 256MB RAM. Also potentially a lot faster and has GigE (somewhat pointless here...).

So yeah, if I wanted to build a few dozen 10Gh rigs something like the pi as a controller would make sense... for a one-off... too much work, just slap a x86 in.

My GA-970-D3 with Athlon draws < 25W. My Jetway mini-ITX about 20W.

Using a Raspberry-Pi is more for the novelty of it and just finding a use for one when they finally are available.

To really save power... here's my current project: a PIC18F86J65 (cost $3), a RJ45 with MAGS ($1), a 25MHz crystal ($.50) and a few passive parts. The only real power draw here is driving the ethernet signals. And a simple PIC miner program that relays work to the FPGA cluster of up to 128 devices.

PCBs done, just waiting to send to fab. Overhead cost per FPGA < $21 qty 1, <$16 qty 25.

@ArtForz - I'd be interested in comments on PCB/Design if you were willing to look. I can post PNGs here.

I would love to see your design on this - have you considered making it PoE powered?

I've heard reports of the Pandaboard being unstable. I've heard better things of the Efika MX Smarttop (http://www.genesi-usa.com/products/efika). One user, I remember, reported an uptime of 30 days, which would have been more if he didn't have to reboot the machine for a kernel upgrade. This seems to me to be of chief importance in a mining rig configuration. The Smarttop only sports an 800 MHz Cortex-A8 though. For pooled mining that should be more than sufficient to drive several FPGAs, as far as I know. If you're talking p2pool, where you have to do transaction verification, initial block chain download might be slow (though we can just move the block chian over from another computer), but at the current transaction rate I think it should be able to keep up with incoming transactions. It has an onboard cryptographic HW processor too, but I'm not sure if this does ECC-DSA.
It comes with Ubuntu 10.10 installed by default, but they have a Debian hardfloat image up on their support/community site (http://www.powerdeveloper.org/).

If you are going this in-depth and installing a whole OS, I prefer VIA's Pico-ITX formfactor, in particular the ARTiGO A1150 (http://www.e-itx.com/artigo-a1150.html) (Dual core VIA Eden X2, x86_64 compatible, max 4 GB RAM). I've got the single core version (The A1100), and it is damn sweet - just be sure to install the video drivers, otherwise the thing is slower than molasses due to video being processed on the CPU instead of the chipset. ::)

It supports expansion boards, and includes I/O headers for tons of things including serial ports, plenty of USB ports, standalone LCD displays, and tons more. Kind of expensive, but compact and reliable.

^ Installing the OS is as simple as downloading an image and booting it. The Smarttop is $129 (€129 in Europe with free shipping). But I admit to it being somewhat of an experiment. It's possible that you will hit some sort of bottleneck in the system if you're attaching 10 FPGAs to it doing 5 GH/s in total. I know memory bandwidth in particular is pretty poor. But if I were to build a mining rig consisting of only a single or a few FPGAs, I'd definitely go for this. It uses 5-10W, is cheap as hell and stable. Also, I think ARM is cool :).

Well, BkkCoins' post up at the top of this page makes x86 (and ARM) look like power hogs by comparison. I would like to know how well his design can scale, and how much power it uses.

^ I must admit that I'm satisfied with the 5W that the Smarttop consumes. Personally, I'd be worried about the upgradability of custom hardware. What if I want to do p2pool mining on BkkCoins' hardware, where I have to verify transactions and store the block chain?
Also, Smarttops are ready to ship now, with working software. It seems BkkCoins hasn't even had hardware built yet. But if OP is not in a hurry, and he wants to save a couple of watts, it might be worth waiting for, cause we won't get lower power than a custom design like that.

I did consider POE but it was too much hassle. And since there is power from the FPGA board I chose the economy approach. The first FPGA board provides 3.3V power to the Ethernet PIC via it's I2C connector. Simple.

I did the design in Eagle on Linux. I don't mind sharing the design files but since they use a custom library I'm not sure it's the easiest for a quick view. So I've exported a rather large PNG of the schematic, and then converted a couple Gerber views to PNG for easy viewing.

Some Notes:

This is based on an Icarus compatible FPGA design. I tried pretty hard to synthesize and place my own version of the Verilog so I could optimize everything into the FPGA (no on board MPU) but kept on having failures (maybe due to some Ubuntu issues or whatnot). Finally I decided to just adapt to the serial input on the Icarus design. The Ztex uses the CSBGA and I can't see myself working with 0.65 pitch layout on a 2 sided board. That's even more crazy than just attempting a 2 sided board in the first place.

Power is from a good quality PSU. It uses 12V, 5V, and 3.3V input. 5V is used as bias for FAN2108 DC converter, 12V -> 1.2V, and 3.3V is used directly for 1 FPGA bank and dropped to 2.5V for VCCAUX and other banks (as required by Icarus design).

Each (5cm x 5cm) FPGA board can act as master for up to 3 slaves. They tile together like square dominoes, up to 127 boards (limited by I2C address, seems like enough to me). There is a PIC16LF1503 on board that queues work/results and acts as multiplexer for CFG init and work data. It co-ordinates data from the Ethernet PIC and passes it off to slaves. It talks via bitbanged serial to FPGA. This chip was chosen as it has a CLC (configurable logic block) inside allowing routing high speed CFG data via itself without external parts. It talks to master and slaves using I2C.

There is fan control, temperature monitoring on each board. It uses the PWM of the PIC and internal temp sensor. Not the most accurate but likely good enough for this.

The Ethernet PIC is master of the whole cluster. It gets work, feeds it to the first FPGA slave, and requests work back. It has a 64Mb Flash chip on board to store FPGA CFG. I'm building these parts as modular units so that I can use them for other purposes.

So, here is PNG version of schematic and PNG version of PCB layouts with screenprint overlay.
This is just the FPGA board now. I can post the EthernetPIC/Flash board soon.

Miner Schematic (http://public.optimalseller.com/miner4.png)
Top Layer (http://public.optimalseller.com/miner4top.png)
Bottom Layer (http://public.optimalseller.com/miner4btm.png)

I'll draw up a quick overview diagram so it makes more sense how these connect together in a cluster. I'm going to send the Gerbers off for making 10 boards once I'm more confident this is a final layout. As you can see this is my 4th revision (miner4)!

BkkCoins, epically awesome. So does the program that gets flashed to the PIC also contain stuff like what pool to mine on, and IP address, etc? Or am I misunderstanding how it will be used?

The program for the Ethernet PIC (MasterPIC below) would have a web interface like your router. You can login and edit the pool values (no reason it can't have a few for failover). Also, later should be able to view some stats/graphics, though haven't thought much about that yet. It would have an upload button for updating firmware for both itself and each FPGA, and PIC too. Both these model PICs support self-writing to their EEPROM allowing online updates. Only the initial programming needs to be done via programmer to install bootloader. That's why little space is given to ISP connectors.

Each FPGA-PIC has simple code to inspect I2C data, queue work data for FPGA, or relay on to slaves. During power up it does a process of device enumeration to set addresses for whatever devices are connected. Then it disconnects itself and passes thru CFG data until "Done" on each FPGA tells it to reconnect as I2C relay. This code has to fit in 2K so it's assembler and minimal. Since Spartan 6s are happy with hotplugging I don't see a reason why devices couldn't be added/removed while running, with minimal disruption, as long as my addressing code adapts. It would have to rescan every few seconds maybe.

Here is a PNG of the overall idea of how you would connect a cluster of these units. Note that each FPGA board has a Molex MicroFit connector (small!) for power. The I2C connectors don't provide power to slaves (each has it's own cable to PSU). But the master connector (top one, red) has power on it to provide power to the EthernetPIC.


http://public.optimalseller.com/MinerGrid.png

watching

Here's a few more images to check out.

Schematic of MasterPIC.  (http://public.optimalseller.com/Net2Blox.png)
Schematic of Flash support for MasterPIC. (http://public.optimalseller.com/Flash2Blox.png)
PCB Top Layer MasterPIC. (http://public.optimalseller.com/Net2BloxTop.png)
PCB Bottom Layer MasterPIC. (http://public.optimalseller.com/Net2BloxBtm.png)
(I may still replace Xtal with Oscillator pkg on Net2Blox. I like the small size but it costs about 50 cents more.)

I've created them as little I2C/SPI modules allowing them to be used for other things. I tentatively call these modules "2Blox" as I may market them later.

Net2Blox and Flash2Blox. The Flash2Blox allows using either W25Q64 64Mbit flash devices or alternately a microSD card. I haven't decided my choice yet but designed board to work with either. If the microSD can't output at 16Mbps then I'd choose the chip instead. I don't want it to take too long to load FPGA CFG. At 16Mbps it's approx 2 seconds to do the whole cluster.

The Flash2Blox has a gated oscillator so the Net2Blox (MasterPIC) can set it up, enable the clock, and let the CFG fly out to the slaves from the Flash - very fast.

These little 2Blox boards stack on top of each other. So the Flash sits just on top of the Net one. And the Net one plugs into the first FPGA slave.

This is very much a work-in-progress.

No, the processor draws 13W for the one I have and 10W for the next generation. If you compare that to the ztex FPGA 8W it's really ok.

The peripherals will draw on any other card too.

And the linux you get on any non x86 architecture has NO software support.

The choice is easy!

That's wrong. If you buy a Smarttop from Genesi there is a software support community at http://www.powerdeveloper.org/ (http://www.powerdeveloper.org/). Of course, if you're not interested in playing around with a new architecture, which will have issues that are not present on firmly established archs like x86, it's probably not for you.

And come on dude, do you really know all vendors of Linux ARM hardware well enough to say that none of them offer software support? It's a bold statement to make at least, and pretty much impossible to verify.

I'm going to have to get an FPGA and see if I can plug it into my chumby.

How many FPGAs do you run?
How much is theCPU load?

Just try to use your ARM linux as a workstation with java, eclipse, chrome WITH flash support and spotify, skype, wine etc. If you wan't to do anything else in your life you won't be able to.

Now I run 5 via USB, but its only 1 java process that controls the cluster. CPU is 0% for that process... It would be close to 0% for the 128 I could run with the limitations of USB.

Exactly. All these fools thinking they will go for ARM or MIPS or another BS architecture like that will hit a brick wall.

Fact of the matter is that x86 is KING and most supported. No x86, no dice.

Stick to i686 or amd64 if you don't want to be dreaming about cross compiling in your sleep :P

I don't think anyone plans on using the beagleboard as a workstation.  If this is a dedicated FPGA mining setup, it won't need to have any of that bloat on it.  I don't see a problem.

I'm not sure you two have any idea what you're talking about.
I've already cross compiled cgminer for my router.

Thanks for your input though, I'll now not use my router as a workstation.

::)

Why?

Still, don't try and tell me that it is easier / faster to run cgminer on ARM compared to x86 because that simply is not true.

Most people are running it on x86 and that is where the support is at. ARM is just a hack and rarely done except by the complicated few who can't spend a few bucks to get a proper desktop for FPGA only mining.

It is a mining rig and so should be considered a server, not a desktop.

The whole point of FPGAs is the lower wattage. ARM beats the crap out of x86 when it comes to power usage.

So I can attach my BFLs to it! Why else!?


Who the crap said it would be easier? It's not, but it certainly ain't no 'Brick wall'!
Cheaper components (USB router $60, ATOM et al. = $200), less energy (USB router 20W, ATOM et al. 80W).
We're here to make money, right?

Oh, and in-built wireless. Can put it anywhere in the house.

Ah yes. Forgot what this thread was about lol

P_Shep, sorry if this has been covered already, but what model of router are you using that has USB port(s) on it? Most that I have seen with that feature either couldn't be used for custom fimrware, or were rather expensive.

Newegg currently has Linksys E3000 (MIPSEL 480MHz, 64Mb RAM, 8Mb FLASH, 1xUSB 2.0) for $56
http://www.newegg.com/Product/Product.aspx?Item=N82E16833124419
Or there's the ASUS RT-N16 (MIPSEL 480, 128Mb RAM, 32Mb Flash, 2xUSB 2.0) for $90
http://www.newegg.com/Product/Product.aspx?Item=N82E16833320038

You can install DD-WRT on both of these, and add extra tools/utilities with optware on an external drive.

I currently have the asus running as a media server, my Anubis server, VPN server, Torrent manager, and various other bits and pieces.

How much would you charge to buy one, flash it with DD-WRT, install Anubis, and make it BFL/Icarus/Ztex/x6500-ready? That would be really cool.

WAY more than it's worth :D

I'll be compiling cgminer for myself when new versions come out, so I can release that (it'll work on both the above routers).
I could probably write some sort of guide on how to set it up.

I don't think cgminer has api support for FPGAs (yet), so I don't think Anubis will work.

EDIT: Here's the bounty https://bitcointalk.org/index.php?topic=66699.0

Just ran cgminer on my router with CPU mining enabled for the hell of it (because I can).
Over the 20 mins I ran it, it averaged a whopping 0.1MH/s, and didn't submit a single share... it was taking so long, the long-poll came around and reset the work.

Incidently, memory usage was ~1900bytes or maybe Kbytes (I'm not sure what HTOP lists mem in), either way, nice small footprint.
Also, the shares counters for the CPU made there way across the screen every update, must be some problem with ncurses there.

I am pretty sure you are killing your router with that.  The CPUs usually have pretty poor cooling.

Indeed. Most of them get very hot.

I would discourage trying to mine on any router simply because the heat mining produces was not meant to be dissipated into that tiny router case.

Maybe try adding a heatsink and fan and air paths if you want to do that.

FTFY :)

I was just experimenting :D
There's no way the router could mine, even with long-poll disabled, it's so slow that by the time a single share has been calculated, it'll be stale. Just ran it for 20 mins to see how it works. Without a BFL (currently on my 8th of 4-6 weeks) cgminer don't do much :)

Jesus, why would it cross anyone's mind that P_Shep wanted to MINE on a router? Daaaamn... Wouldn't you guys think that if a guy managed to figure out how to cross-compile cgminer for MIPS (or what's in there) would be stupid enough to do mining on a router? Almost everyone these days knows cgminer has preliminary support for FPGA's...

Jeez...

In case anyone runs into this like I did, it is perfectly possible to do this. I just got it up and running with cgminer 2.11.4 on a BeagleBoard revision B5 board. I just used the Ubuntu instructions here:

http://elinux.org/BeagleBoardUbuntu (http://elinux.org/BeagleBoardUbuntu)

Then built with the cgminer instructions. My BFL Single is happily hashing along with no problems.

I'm mining using a RaspberryPi with MinePeon.
Most FPGA are supported by bfgminer.