So I updated the controller FPGA on board serial n. 8 and it went withouth problems. Then I started flashing twin_test.bit into my boards and I've found that outside of virtual box it takes a lot less to flash a FPGA. user@t5570:~$ time sudo ./xc3sprog -c cm1 -p 3 -Ixc6lx150.bit twin_test.bit
XC3SPROG (c) 2004-2011 xc3sprog project $Rev: 674 $ OS: Linux
Free software: If you contribute nothing, expect nothing!
Feedback on success/failure/enhancement requests:
http://sourceforge.net/mail/?group_id=170565
Check Sourceforge for updates:
http://sourceforge.net/projects/xc3sprog/develop
Using Libftdi,
DNA is 0x390dc1e841c73bf1
JEDEC: 20 20 0x18 0x00
Found Numonyx Device, Device ID 0x2018
256 bytes/page, 65536 pages = 16777216 bytes total
Verify: Success!
real 2m50.569s
user 0m4.340s
sys 0m10.517s
Less than 3 minutes per FPGA.  spiccioli. Doing the direct programming of the FPGA is fast but won't stay when you power cycle. Loading ointo the SPI Flash takes longer but is there every time you power up the board. |
|
|
|
|
Looks like you guys have found at least one silly thing for me too look for today. When I get a chance I will ask about the -I thing and if we can in any way make it more consistant with the rest of the switches. If nothing else we can at least highlight that fact in the instructions.
|
|
|
|
@yohan: any news or piece of software we can play with?
Ok nothing too exciting today. The Power Distribution Boards PCBs arrived today and we part built one of them and tested the ATX turn on function and that all worked as we expected. We are going to fit one of the switch positions with a header so a semi-remote switch can be used to turn on the stack. There will be a local switch as well. We will do a full board build up on Monday when the Pheonix connectors arrive and we should have some available to ship Tuesday or Wednesday next week. Bitstream moved another notch closer but nothing out of the ordinary there. The first release isn't far away. I believe we are talking days now but as this is very hard to predict I won't totally promise that. On the programming today was busy and I have not managed to go through that process yet. However I have a board with me to test in the relative peace of my home office. I will attempt the programming of a board using Win7/64 machine as the host to start with following the instruction you all have to work with. From that I will see how I do and hopefully identify things that are not totally clear and what we might do better. We do have a couple of further development activities planned to improve the programming but that will be after we get the first bitstream going. |
|
|
|
Anything we can do in Cairnsmore1 can be done better in Cairnsmore2 where we have more technology to play with.
Any hints on what's up for specs?  There are many variables still be tied down and I also don't want to give too much away to the competition but it might be interesting. There is more that one unique idea floating round the development team and I'm not going to promise anything we have not got fully working yet. You are going to have to faith with us to deliver the interesting bit. There are many barely documented features of FPGAs that even most people designing daily with FPGAs simply don't know about and that's about all I will say for the moment. |
|
|
|
Yohan, could you please clarify what you mean with 'you should be able to do that now (i.e. writing to SPI)'? As said above, right now no one successfully wrote to the SPI flash and it is really a PITA to reprogram all devices non persistently again when one of them crashed and need to be powered fully down (you do not know which one is the problematic one and need to turn off all of them).
So either it is possible with the current controller programming, then could you please provide a detailed step-by-step instruction. Or it is not doable over USB but needs to be done via JTAG.
Either way it would be very helpful to flash the Icarus bitstream into SPI until your dedicated one is ready. Any clarification is highly appreciated.
Thanks.
Yes I think all of this is currently possible and I will double check that this morning that I am not talking crap in any shape or form. We use these processes on our lines already. The load of the 4 array SPI Flash does take over 30 minutes so it does need patience to do. We do a 5-10 boards together here to stop it becoming a line blocker. I will also see if we can somehow streamline the instructions. I too would prefer working instructions for flashing to the SPI for the twin_test bitstream. Personally I would prefer this now as opposed to getting the working bitstream a day earlier so then at least I can run the twin_test bitstream without going through reflashing it every day. I would be happy to give enterpoint an interest free loan if they could beat BFl to it.
Who's with me?
I'm not giving anyone an interest free loan. Let's not worry about the name that shall not be mentioned. Let's see firstly how the technology gong into Cairnsmore1 works. If it achieves even some of what think it can, and that is still an if, but and maybe, we have something that that is potentially a major step forward. Anything we can do in Cairnsmore1 can be done better in Cairnsmore2 where we have more technology to play with. |
|
|
|
Programming wise it's our intention that everything becomes resident in the SPI flash and doen't need to be live loaded every time. You should be able to do that now but obviously that's not straightforward for all of you and we are going to come back and look at that in more detail and try and make it better. We still have the ultimate fallback of using stand alone cables if that is necessary. It should not be necessary to go this far though.
[...]
Yohan, could you please clarify what you mean with 'you should be able to do that now (i.e. writing to SPI)'? As said above, right now no one successfully wrote to the SPI flash and it is really a PITA to reprogram all devices non persistently again when one of them crashed and need to be powered fully down (you do not know which one is the problematic one and need to turn off all of them). So either it is possible with the current controller programming, then could you please provide a detailed step-by-step instruction. Or it is not doable over USB but needs to be done via JTAG. Either way it would be very helpful to flash the Icarus bitstream into SPI until your dedicated one is ready. Any clarification is highly appreciated. Thanks. Yes I think all of this is currently possible and I will double check that this morning that I am not talking crap in any shape or form. We use these processes on our lines already. The load of the 4 array SPI Flash does take over 30 minutes so it does need patience to do. We do a 5-10 boards together here to stop it becoming a line blocker. I will also see if we can somehow streamline the instructions. |
|
|
|
I can confirm that the twin configuration works reliably (at least for 30hours straight on my 2 boards).
No luck storing it in flash though so if the power to the boards is lost I'll have to go to their location to reset them manually as it involves moving dip switches.
@yohan : is bitstream flashing supposed to need a manual intervention forever or will it be possible to do so remotely (without touching the dip switches) in the future ?
Programming wise it's our intention that everything becomes resident in the SPI flash and doen't need to be live loaded every time. You should be able to do that now but obviously that's not straightforward for all of you and we are going to come back and look at that in more detail and try and make it better. We still have the ultimate fallback of using stand alone cables if that is necessary. It should not be necessary to go this far though. We are looking to make things nicer and that you all don't have to fiddle with dip switches very much if at all. On the array FPGAs it's our intention that the dip switches will be totally unused. So once we have that SW2, SW3, SW4, and SW5 won't even be a consideration. The dip switches SW1 and SW6 for the controller will at least be simplified. We are still debating if we need a "safety" dip switch to act as a block to accidentially programming a board but pretty much everything else can become software controlled and I think that is the way it will go. At the moment only 7 of the 8 switch bits of SW1/6 are used but that's still 127 possibilities to get it wrong. The drive for all of this will be the delivery of our original bitstream design and with that we will have the next controller build. With the first version of that our aim will be stability but achieving at least the circa 800MH/s everyone is looking for. It's a very radical design but I think everyone will be pleased with the results assuming it works the way we think it should. Either that we have totally lost our touch at doing these sorts of things. Hopefully that's not the case. It's now our big focus to deliver this now we are more or less sorted on the hardward bits of the design. |
|
|
|
Ok some running dip switch settings. These are now on support page as well.   |
|
|
|
|
I think the Icarus design is built off the Ztex one but I might be wrong on that. If anyone happens to have the NCD files for either these and wants to supply them I will try a couple of modifications to see if they will work seperately of the main team on our original bitstream. It's a process not guaranteed to work but sometimes it does but most importantly these sorts of hack can be done in 30 mins not 2 days of a rebuild. I know you all are keen to see the performance side of this and I don't think we will have any major problems getting to about the 800MH/s mark it's more a case of how long to get there.
|
|
|
|
Are you planning on contacting TMC (eldentyrell) and providing him with stuff he needs to make his bitstream compatible with Cairnsmore1?
We have had a conversation but it does need some work from our side that would take from our bitstream progression. So it's not likely to get us there much faster that our own approach. so for the moment we continue as we are. That's the wrong sex for what we are trying to find for the PDB. We have gone with the 3x2 that we already have on the Cairnsmore1 already so we know where to get it and we have stock anyway. A small update on the PDB is that we hope to have the first bare boards a bit earlier now on Friday and we will build one up quickly to test and show the real thing. If that's all ok we will have production quantities a couple of days later. |
|
|
|
It doesn't surprise me that PCIsig would try and keep a secret of this. It's a very greedy organisation. The amount they charge just for a PCIe spec is enormous. However I would not be surprised if there isn't a Chinese clone available somewhere out there. It's the identifying and getting hold of the parts that is the biggest issue. On a side note we are looking at the emails send to the new support email and we will try and make a proper response on these before too long. Please bear with us on this we aren't doing *** style response here and deliberately ignoring you all but at the moment our main focus is the new design bitstream which is now getting more time and resource to bring that into shape. Once we have the first version of that the up/down function won't be far behind and the Cairnsmore1 design will start to show what we think it is really capable of doing. |
|
|
|
Yohan, got my first batch of Cairnsmores delivered and would like to ask you for clarification on: 1) Patches for xc3sprog I can't access the boards from VirtualBox. Since I am using Ubuntu as host system anyhow and xc3sprog is open source, I built the tools for my host system. With all prerequisites met claims that it can not find devices using libftdi. Could you please check whether you had to patch the source code for the binaries you delivered with your VirtualBox image - and if so please publish the patches (since it is GPLv2 anyway)? 2) Up/Down cable support I have a 2x5 IDC ribbon cable at hand and connected two boards over the up/down interface with one board attached via USB to the host. It does not seem to work as I expected, i.e. neither I see 8 ttyUSB ports nor I get a higher hashing rate with the second board attached via up/down cable. Could you please confirm whether the up/down functionality is currently supported at all? 3) FTDI disconnects One of the four boards I am testing has obvious problems with the FTDI chip. It causes the PC to become non responsive or freezes it completely until I plug the USB cable to the related unit. Tried with several USB hubs and swapped the cables, but it is always the one unit that causes problems. This is the syslog when things go bad: Jun 25 19:43:08 d620 kernel: [15947.382768] ftdi_sio 1-4.1:1.2: FTDI USB Serial Device converter detected
Jun 25 19:43:08 d620 kernel: [15947.382833] usb 1-4.1: Detected FT4232H
Jun 25 19:43:08 d620 kernel: [15947.382838] usb 1-4.1: Number of endpoints 2
Jun 25 19:43:08 d620 kernel: [15947.382843] usb 1-4.1: Endpoint 1 MaxPacketSize 512
Jun 25 19:43:08 d620 kernel: [15947.382849] usb 1-4.1: Endpoint 2 MaxPacketSize 512
Jun 25 19:43:08 d620 kernel: [15947.382854] usb 1-4.1: Setting MaxPacketSize 512
Jun 25 19:43:08 d620 kernel: [15947.383210] usb 1-4.1: FTDI USB Serial Device converter now attached to ttyUSB2
Jun 25 19:43:08 d620 kernel: [15947.384871] ftdi_sio 1-4.1:1.3: FTDI USB Serial Device converter detected
Jun 25 19:43:08 d620 kernel: [15947.384942] usb 1-4.1: Detected FT4232H
Jun 25 19:43:08 d620 kernel: [15947.384947] usb 1-4.1: Number of endpoints 2
Jun 25 19:43:08 d620 kernel: [15947.384952] usb 1-4.1: Endpoint 1 MaxPacketSize 512
Jun 25 19:43:08 d620 kernel: [15947.384958] usb 1-4.1: Endpoint 2 MaxPacketSize 512
Jun 25 19:43:08 d620 kernel: [15947.384963] usb 1-4.1: Setting MaxPacketSize 512
Jun 25 19:43:08 d620 kernel: [15947.385366] usb 1-4.1: FTDI USB Serial Device converter now attached to ttyUSB3
Jun 25 19:43:08 d620 kernel: [15947.460411] usb 1-4.2: new high speed USB device using ehci_hcd and address 101
Jun 25 19:43:08 d620 kernel: [15947.544437] usb 1-4.2: device descriptor read/64, error -71
Jun 25 19:43:08 d620 kernel: [15947.732438] usb 1-4.2: device descriptor read/64, error -71
Jun 25 19:43:09 d620 kernel: [15947.908443] usb 1-4.2: new high speed USB device using ehci_hcd and address 102
Jun 25 19:43:09 d620 kernel: [15947.992441] usb 1-4.2: device descriptor read/64, error -71
Jun 25 19:43:09 d620 kernel: [15948.180403] usb 1-4.2: device descriptor read/64, error -71
Jun 25 19:43:09 d620 kernel: [15948.356446] usb 1-4.2: new high speed USB device using ehci_hcd and address 103
Jun 25 19:43:10 d620 kernel: [15948.772066] usb 1-4.2: device not accepting address 103, error -71
Jun 25 19:43:10 d620 kernel: [15948.844451] usb 1-4.2: new high speed USB device using ehci_hcd and address 104
Jun 25 19:43:10 d620 kernel: [15949.260132] usb 1-4.2: device not accepting address 104, error -71
Jun 25 19:43:10 d620 kernel: [15949.260486] hub 1-4:1.0: unable to enumerate USB device on port 2
Are you aware of such problems and is there anything I can do SW wise to bypass it (aside of the udev rules I am already using)? I considered sending this support request to the support email address you gave, but think that it might be interesting for other users and/or it could already be solved by someone. Thanks. Ok I am told we didn't do any patches for the xc3sprog so it is as the original. Up/Down is not working yet. That will come after our first original bitstream. That will need another controller update. Are the USB hubs you are using powered with a brick supply or bus powered? You might get this issue on un-powered hubs and it just one board is marginally more power hungary. Do send this stuff to the emial support.bitcoin at enterpoint.co.uk. Response won't be terribly fast as we are balancing getting on with the new stuff with supporting the temporary things. On a different point on CGminer we will probably have put a second build for Windows up on the website shortly. It's specifically for the twin build. Basically the one that is currently was up there is for the 50MHz build and we should have made that a little clearer. The twin build version is more or less the same as Icarus. I think there are a couple of minor switch settings changed but that's all. Both windows versions are now on support page in a single zip download. When we bring in our own original build bitstream I expect to have quite a lot of changes including a new step of CGminer and new revision of the Controller. The up/down function may, or not, make the first of these versions but if it doesn't it should not be far behind. When we have that it will take a lot of the historical problems of large numbers of USB trying to run on one machine and will start to look much more like our planned long term structure of network nodes. That's a key in to what is coming in Cairnsmore2 where we will extend that idea somewhat more. |
|
|
|
I'm playing at the moment with different bitstreams. I wonder why the icarus bitstream "190M_V4.bit" is the same as the "twin_test.bit" ?? I did a binary compare and it's the same 99,9%. Only the date and the directory from where it is compiled is different (comare the first bits in HEX). @yohan: Is the "twin_test.bit" bitstream the same as the "190M_V4.bit" bitstream from icarus? I thought your team have make some changes to it to get it work? If it is the same and i think so after the binary compare then read this: 190M for test.
in this bitsteam, the FPGA will continue working until nonce_to, even found a valid nonce. Found at https://github.com/ngzhang/Icarus/blob/master/Downloads/bitsteam/V4/Then it makes sense that the board is stop working after some time... Never the less, i'm testing the 190M_V3.bit, it looks more stable at the moment (0% invalids with both fpga's so far), but know for sure in some hours. The twin test is more or less a standard Icarus build. I think the ID and maybe the DCM settings were changed but that was about it. |
|
|
|
|
We think we know what the problem is where performance drops after some time usually on one FPGA. It's related to the comms and basically messages coming back with results are being lost. It won't be anything more than a short term issue and will be sorted out with our own bitstream.
|
|
|
|
12V is a good choice for Cairnsmore1 although it can actually operate with a bit higher distribution voltage if needed.
I come prepared for all contingencies! How does 36kw of 48vdc sound? Keeping the estimate of 60 watts per board, that gives us 36,000 / 60 = 600 boards. 600 boards x $640 = $384,000.00.  That's some serious power and bad if you short it. Although I do of an incident of a spanner droppin over a battery out of a submarine and that was bad apparently. |
|
|
|
I believe it is correct that the EPS12V connectors have the positive and the negative swapped relative to a 6/8-pin PCIe connector, so be careful. Even if it fits, you might end up with reverse polarity and that would be bad. So the 6-pin goes like this (with the connector latch on top): And I believe the ESP12V goes like this (also with the connector latch on top): But I don't have one next to me and so I can't check. Yes they are opposite and that is one of the most stupid things they did in the PSU spec. They could have been the same. The 2x4 connectors are differently polarised but here's the worst bit the 2x3 PCIE can plug into the EPS socket and is totally the wrong way round so lots of smoke and probably fire if you do that. This was a side reason for not putting on the EPS on the PDB. They also didn't do a good job going to the 2x4 PCIE either. Adding sense and not capacity over the 2x3. However that is what we have to work with. It's common in general industry to have Point Of Load structure where power is distributed at higher voltage and lower current within a rack or rig. This makes for smaller copper wires and less distribution loss. Against that your local regulations stages are not 100% efficient so it's a balance between copper loss and POL regulator loss. The distribution voltage varies with system and usually depends on amount of power. In our extreme board Merrick1 we use 48V but that needs special PSUs at the front end and each board can be using up to 1000W. 12V is a good choice for Cairnsmore1 although it can actually operate with a bit higher distribution voltage if needed. |
|
|
|
|
[/quote]
Most powersupplies can be maxed out on PCIe and +12v EPS pins. I don't know why you're throwing a hissy fit over obsolete and bulky 4-pin molex connectors. The better question is why aren't the 8-pin EPS connectors being utilized.
[/quote]
The EPS certain would have ben nice and we could have got them in on a bigger board. It is a design compromise on the current size and we can always do another one if it looks the right thing to do. We could have shoehorned in more connectors but that might have meant more copper loss because we would have had to thin down the conduction paths. So as i say it is a compromise.
We did look at putting in the 4x2 PCIE as well but could not find the part number for the special keying it has and the Molex website doesn't help very much. Ironically the parts we got samples of could take take the EPS and had the right keying for that. If anyone has the 4x2 board side connector part number let me know and we can look at it for next time. I don't think it makes much difference on power handling they way they do it but might be easier for plugging in the 3x2+2x1 connectors currently common on ATX PSUs.
How this board can be used will be very dependent on how the host ATX PSU is configured and there are lots of variations there to deal with.
|
|
|
|
Currently your board only limits 24pin and PCI-e connectors, i want to know why? Every PSU still has 2-3 molex 4-pin wires (i'm counting only wires because other connectors share the same wire are useless). Why not maximizing the available wires of a PSU? to save few bucks cents on the connector?
Yes we didn't use every last wire but there can be difficulties here as some ATX power supplies have seperate regulation stages and combining them together can mean that they fight and one regulator stage can be very stressed. So for the loss of 2 or 3 wires we left out the molex. That's only equivalent of one PCIE and we do have six of them. What would have been more useful to use was the 12V connectors for motherboards. In the end we went for a practical layout. I would have liked to have 8 PCIE but space didn't allow. However 2 PDBs can be used with one ATX in that case. We didn't connect the PCIE together because of the fight issue so each of these has 3 Pheonix connectors on it and there are more or less independent. Of course the remaining Molex can be used directly if you really want every last bit of power out of the PSU. Making simple adaptors for the motherboard 12V is also possible. The PDB gives you the easy way to use use the ATX PSU. There are a bunch of things we could have done and this was just the balance we choose. If you do use the extra strings for any of this you do need to check individual output, or group, limits from the ATX PSU. That does go for the PCIE as well but much less of a chance of an issue here. We will try and build up a list of configurations we have checked or tested with a given PSU. This product isn't just designed just for this week. It is designed for the limit of the Cairnsmore1 limit which is about 60W or 5A at 12V. |
|
|
|
|
Actuallly I was looking at the post before. Yes that's very much the idea that you can do more or less what you want with the wiring. It's always difficult to get PCIE leads to wire up nicely in a rig always the wrong length and it's hard to split them nicely. Hopefully this board will be of use here. I imagine it will get used for other non-Cairnsmore uses as well. ATX PSU are hard to beat in efficiency terms and high power at a reasonable cost.
The plan is that this is an initial version and we will follow up with one supporting Ethernet as well. That enhanced one won't be done for 2-3 months yet. Depends on how busy we are,
|
|
|
|
|
The green connectors are not Molex. They are the Pheonix type that we already use a 6 way of on the Cairnsmore1 itself. The contacts on these are rated 12A roughly double that a Molex DD contact can carry. We also have no 5V. It's all 12V and 0V/GND so each PCIe string has 3 connectors connected and therefore 6 contacts each for both 12V and 0V/GND. So the Pheonix side is good for 72A if fully fitted up and that's way more that a PCIE string is rated for.
The Pheonix connector has a range of plug in mating halves including screw terminals. We also have an option to go directly to screw terminals on the boards so there are a range of options. It allows low tech wiring that suits your rig with no special criming tools needed.
I would expect 3-4 boards to be powered from each PCIE string this way but 5 or 6 might be possible. A lot depends on the host ATX power supply and it's wiring. We will look at those and experiment a bit. More on that in the coming weeks. We will be looking at wire grades and what currents can be carried to come up with the best usage in the stage. Whatever way you want to do it this board gives you easy usage of low cost ATX PSUs and the ability to make nice neat wiring rather than fighting the unbendable ATX power strings.
Of course if you want to stack these as well that is possible.
|
|
|
|
|
Show Posts
