Dear BitCoiners, BitCoin talk users!
First of all I would like to thank to other developers of Spartan-6 based works, especially those who contacted me soon - shalab.si, fpgamining.com team, Greg and all those who understand how hard that work is. Also I would like to thank all those who supported me during development of these mining racks and invested money, so I haven't went into "preorder"
fall like offering more Mh/s power than hardware actually could deliver.
I am actually in love with bitcoins, because this is exactly the thing I lacked in 2006 year, and tried hard to invent... Unfortunately I were not smart enough to find out proof-of-work based blockchain. And this solution actually useful even in wider range of applications than money transfer themselves, as it can limit or remove human factor in distributed database consistency. It is interesting how this will evolve.
Second, I was surprised by reaction about comparision with ButterFly Labs with 'Estimated price'. People, that's $90k is with 20% VAT, which is not paid in US for example and with having in mind that it would have single hot air outlet and that's all about it... but to get mini-rig we would pay +20% on customs VAT, so it would be not $15'295 but $18'354 + shipment costs, so it is $0,728 per Mh/s. We've spent on boards production about $0,50 per Mh/s and on installation about $0,12 per Mh/s, totalling $0,62 per Mh/s. Count bitstream offer (it will be later in this topic) - it would be $0,69 per Mh/s. SO IT IS $0,03 LESS THAN THAN BFL Mini-Rig and it really works for almost 2 month, so we are sure now that it does not fry chips etc. We have not even managed to fry chip, when fed it with 1.5 V VCCINT core voltage. But, price would be definately higher (near estimated production cost), if counted other issues, like climate control, getting specialized area for them, getting power (as it consumes 20 kW together with chiller). All of these costs were already paid, because it is installed in basement of household with already existing heat pump.
Third, 300 Mh/s is not limit of this bitstream, it can give even more rounds, if you count that almost all DSPs are not used in left part of design, and some free space in topmost part. So it could definitely give 8% at least better performance, it would then cost to us $0,57 per Mh/s, which is even less than BFL Mini-Rig.
I've been asked by email and skype about smaller editions. And I would say that in my opinion best solution would be standard 4-U chassis, 0.5 meter long, with 14-15 boards with 6 chips on board installed (that is 84 - 90 chips), like with shalab.si original ideas, but a bit different layout to prevent overheating chips. Withing this chassis single Intel Atom D525 motherboard is installed. Boards with 6 spartans can be even without microcontrollers and flash, everything could be programmed right via LPT-port. Bandwith required to communicate with every chip is quite low - about 300 bps. So with all chips it would be about 27 kbps. Bitstream loading over LPT port however will be slow. For smaller scale RS-485 is overkill. Why to bother about it and not implement using USB - simply because flashing chip or flashing controller
adds up cost of controller and also costs of programming and testing them, also when something should be updated, and you have to reprogram every controller - that rises service cost. I would like to say, that current design of BitFury rack, where controller only translates RS-485 to SPI bus with Spartan and back requires almost zero maintenance.
Cost of such chassis with power supply and Intel ATOM motherboard could vary in $400 - $600 range. Cost of Spartan6 chips when purchased in bulk quantities (WITHOUT VAT) would vary in $70 - $95 range, depending on shipment location and quantity of chips ordered. Cost of other components (using numbers from our current design):
motherboard 4-layer $111 ($24,5 PCB and $8,53 soldering), daugthercard 6-layer $18.39 ($2,2 PCB and $5,5 soldering). these are costs with components with VAT and connectors. If connectors will be removed and VAT will be removed, then cost would be like $77,7 for motherboard for 6 chips and daugthercard $12,83 . Totalling $154,68 per board .
So server with 14-15 boards could cost in range $8'445 to $11'470. More likely that actual manufacturing cost plus work will be somewhere about $10'000 to $10'500. Then, say it would have 90 chips. I would point out how important Mh/s are:
BFL Mining rig quite comparative product promised to be sold at $15'295 for 25,2 Gh/s - that is $0.607 per Mh/s
200 Mh/s bitstream - would produce 18 Gh/s - it would be compared to 71% of Mini-Rig and so product price could be $10'859.
250 Mh/s bitstream - would produce 22.5 Gh/s - it would be compared to 89,2% of Mini-Rig and so product price could be $13'643.
300 MH/s bitstream - would produce 27 Gh/s - it would be compared to 107% of Mini-Rig and so product price could be $16'365.
325 MH/s bitstream - would produce 29.2 Gh/s - it would be compared to 116% of Mini-Rig and so product price could be $17'722.
So at 200 Mh/s - there's almost no difference between bulk order and product price. At 250 Mh/s there's $3'143 income.
At 300 Mh/s there's $5'865 income. And at 325 Mh/s there's $7'222 income. Additional 50 Mh/s per chip gives 86% income increase if prices are set at BFL levels of $0.607 per Mh/s. Additional 75 Mh/s per chips gives +129% income.
Calculating these costs and also costs handling sales, manufacturing etc. lead me to following licensing targets for about 1000 chips per month installed:
$20 - $25 per chip (depending on chip price and costs incurred by AES-key programming) for current bitstream and
$5 - $7.5 for future upgrade (separately), which can be opted when such update actually done.
From our side most important point is bitstream protection. This incurs costs of AES-key programming by moving trusted person to assembly plant from time-to time, powering on boards and fusing chips with AES-key. For simplicity it would be great to have 6 spartans on single board tied on JTAG line via buffers. Then encrypted bitstream could be available without any additional protection to it. Of course if quantities will be small and location distant, it would be difficult to execute programming. We are already planning to program chips with AES key in Hong-Kong, and we have good access to EU, because we are located in Ukraine. Existing boards could be upgraded as well.
Also - why I am insisting that 4U design one of the best sizes:
1) it would consume 1.3 - 1.5 kW per 4U, and that fits into envelope 350W per 1U heat production, which is not difficult
to implement in either datacenter or homebrew setup;
2) Chassis itself could be sold with minimal margin, so people could build mining power within nice chassis step by step;
3) When local price to electricity would be unaffordable, people can send what they built to Iceland for $0.04 per kw*h
special setup, so compatibility with datacenters is nice feature;
4) It gives $1'800-$1'900 "entry-level" price and board-by-board upgrade possibility;
5) "special single board" can be available as well - with USB, and it would be nice option that it can be put into server,
basically I expect to lower costs to have boards with pads for USB-related components, just they will not be soldered for
items that will be put into servers;
6) also it would be nice to have some DRAM on board for other purposes, but again - this would be not soldered at all, but later, if boards will be re-used for other tasks such DRAM capability and at least 1G ethernet external connectivity would make difference for long-term product life. At stage of PCB design it adds only NRE costs of designing PCB itself, and no cost at manufacturing.
Thinking where equipment will be placed - in small quantities - probably in homes, but when it will get larger - I have already found location and discussed this issue with Andreas Fink (Skype: andreasfink ) from datacell.com
[5/22/2012 10:24:36 AM] Andreas Fink: Our datacenter is designed for 20kW cooling per rack
[5/22/2012 10:24:37 AM] bitfury.org: but if air inlet drops 10 degrees Celsius, performance can be increased. and optimal temperature is about 0 degrees Celsius if we use industrial chips. that gives 20% performance increase.
[5/22/2012 10:24:44 AM] Andreas Fink: we use hot aisle concept
[5/22/2012 10:25:17 AM] bitfury.org: buyers of these devices actually concentrate on pure performance and cost of electricity, cooling etc. many of them installing such installations right at home, because it is cheaper.
[5/22/2012 10:25:27 AM] bitfury.org: you can google for "bitcoin mining"
[5/22/2012 10:25:40 AM] Andreas Fink: Well we operate our datacenter in iceland. energy is cheap there and coolign is easy.
[5/22/2012 10:27:05 AM] Andreas Fink: We could build a custom datacenter just for that if there's enough demand.
[5/22/2012 10:28:05 AM] Andreas Fink: what is the potential of your installation if it produces bitcoins?
[5/22/2012 10:29:38 AM] bitfury.org: very difficult to say, because we have just started to announce our solution.
[5/22/2012 10:29:54 AM] Andreas Fink: a rough estimate to get an idea.
[5/22/2012 10:29:54 AM] bitfury.org: current bitcoin network hashpower is 10'000 Gh/s, single bitfury produces 110 Gh/s
[5/22/2012 10:30:03 AM] bitfury.org: so estimation is about 30-40 racks
[5/22/2012 10:30:28 AM] bitfury.org: that's potential for FPGA bitcoin mining.
...
[5/22/2012 11:37:42 AM] Andreas Fink: the electricity company charges like 3-4 eurocents per kWh
...
It depends - if we manage to start with 4U working with them and without using diesel/ups-backed power which is expensive and climate control equipment - it would be cheapest collocation for mining available in world.
[5/22/2012 11:02:04 AM] Andreas Fink:
http://www.sgi.com/products/data_center/ice_cube_air/[5/22/2012 11:02:17 AM] Andreas Fink: put them in such a shelter, we provide power and internet and put it into the green field
I suppose that shelter should be designed as well as SGI could be expensive, but generally you've got the point. And that is why I am sticking with rackmount design. It will be big pain to move devices that are non-compliant with datacenters, putting Mini-Rig there for example. I suppose this is the moment, when buyers of BFL should start thinking twice, what they would do with their rigs, when income will be about the same as consumed power. For myself that problem is solved - mining equipment is integral part of house heating now. For cold countries this can be as well solution - integrating house / DHW heating with mining. Today you will probably say that it is absolutely crazy, but 5 years in the future things would change, and those who make useful use of excessive heat will save on electricity bills, while those who will pay additionally for air conditioning would loose, as they could not compete.
So finally, I would welcome everyone interested to join efforts. First - design chassis and PCB with more-or-less interexchangable parts. Getting for example with-USB and without-USB version, with-flash and without-flash version, etc. so actual board can be customized by needs of our partner who provide product, but in general they all are near the same, and could be fitted inside 4U chassis. This way we can save on designing this thing, build product in which our customers will be confident, that even if one supplier stops selling boards, he can possibly with higher costs order elsewhere, but still finish his 4U box. More important is different metal pieces like holders of boards, etc. Which would depend on supplier of these chassis solution.
I am going to provide soon draft of 4U box, as I see it in 3D with airflow calculations. For those who asked about current interface, I am attaching right into this message .vhd source code of bitstream side, current .ucf file (but please, it can change - we used 1 mm FGG484C, and it could be possibly better to use CSG 0.8 mm steps), only important point that communication enters BOTTOM of chip. And communication part in dsPIC33F firmware. As you probably would see - same communication can be performed over LPT port without bothering with production of additional conversion boards.
Here are links to download:
http://www.bitfury.org/bfdetails/sha_top.ucfhttp://www.bitfury.org/bfdetails/sha_top.vhdhttp://www.bitfury.org/bfdetails/shaspi.vhdhttp://www.bitfury.org/bfdetails/jobs.cMeanwhile - we used FGG484 because we though to put more bypass capacitors below chip for overclocking purpose, as with CSG there
will be difficult to place such numbers of capacitors. However we've aimed for 320 Mhz that makes too much errors, with 240 Mhz
it would be fine to place less capacitance, and also get industrial CSG chip, as this would remove any possible overheating problems.
CSG chips have about 1.5 times (!) lower junction-case thermal resistance according to datasheet. However this would be great to be
confirmed by someone who used CSG chips - I saw ZTEX used them.
So - how we shall proceed ? I suppose best way will be to:
1) Write to this topic, if you are in (possibly after getting in touch in skype: bitfury.org );
2) Writing estimated quantities per month, assembly plant location (place where chips can be turned on);
3) Writing your final destination where you are going to ship these boards for further packaging;
4) Writing number of chips you already use - because if there's many already - then possibly re-work on IR station could cost
significantly less than buying new chips;
5) Writing your requirements to board - specifying way of Spartan programming, currently used interfaces (i.e. USB with USB chip name),
so we can get common specification for board interfaces and chips that shall be on board. Probably for overclockers it would be nice
to have ability of voltage setting.
Then next iteration will be to approve specification and doing PCB schematic design, and then doing layout, and
doing similar way with chassis. Hopefully there's people who like working 24 hours + night
Kind regards,
Mr. V // Skype: BitFury.org
PS. Special note for those who know me personally. I dislike indexing of personal information via google, etc, so please, do not write here - "oh I know this guy"... Write it in skype in personal communication (it will be visible at least for
special services and not to general johny hacker who mines information and would like to get some cash), not on public forums or public web pages. And I discourage everyone from doing so with their information. Imagine that it is possible to enter your name to google not only to your friends and partners, but also to your foes, those who you even do not know...
PPS. I've also uploaded stats file
http://www.bitfury.org/bfdetails/bitfury_status.txt with current number of working cores, you may see that some cores are not working, error rates.... There were no special handling with them - we've just put cores "as is" into motherboards, and there are no visible defects. We've not bothered replacing these cores. Just to those who say that it is not real thing
EDITED - also forgot part of conversation with Andreas Fink about Iceland VAT there is 25.5% but for collocated servers it is returned. So it is like having 0% VAT compared to 18-20% VAT in EU countries.
