[ANN][BLC] Blakecoin Blake-256 for GPU/FPGA With Merged Mined Pools Stable Net

[kramble]

I changed line 431 of /mpbm/modules/fpgamining/x6500/x6500worker.py from:
self.checksuccess = False
to:
self.checksuccess = True

With just these modifications the x6500 successfully submits shares to a pool. The hashrate is reported incorrectly. Just a quick and dirty copy pasta to get mpbm going in windows with blakecoin firmware from kramble and an x6500. Thanks for all your efforts kramble.

Thanks. I've updated the github with this change (ideally the validation job would be changed instead but this will do for now I guess). I've also changed the default branch from testing to master in the repo (the testing branch does not include my blake mods).

I need to update the README with some guidelines for driver installation and miner instructions, so let me know what you think I should add.

Should I recommend using MPBM or the X6500 python miner?

Also with the python miner, what overclock setting did you use to achieve 400MH/s? Looking at the verilog code, it defaults to 50MHz which seems a bit slow compared with the 200MHz we're getting on the ztex port, but there may be a scaling factor to confuse matters, which is why I'd like to know the actual hash rate achieved and clock setting. If its just 400MH/s for the entire board (dual fpgas) it should be possible to roughly double this with some tuning of the bitstream (not that I enjoy this task, Xilinx ISE is a right PITA).

[1713308999]

1713308999

[1713308999]

1713308999

[1713308999]

1713308999

[Ignatius]

Thanks. I've updated the github with this change (ideally the validation job would be changed instead but this will do for now I guess). I've also changed the default branch from testing to master in the repo (the testing branch does not include my blake mods).

I need to update the README with some guidelines for driver installation and miner instructions, so let me know what you think I should add.

Should I recommend using MPBM or the X6500 python miner?

Also with the python miner, what overclock setting did you use to achieve 400MH/s? Looking at the verilog code, it defaults to 50MHz which seems a bit slow compared with the 200MHz we're getting on the ztex port, but there may be a scaling factor to confuse matters, which is why I'd like to know the actual hash rate achieved and clock setting. If its just 400MH/s for the entire board (dual fpgas) it should be possible to roughly double this with some tuning of the bitstream (not that I enjoy this task, Xilinx ISE is a right PITA).

Ever since I uncommented line 104 of mine.py the hashrate is reported as 0 but things still work and shares are submitted. I have not tried modifying the clock settings, I have just left things at the default so far. IIRC mine.py reported ~400MHs. Currently MPBM reports ~600MHs effective against the blake getwork source and 402MHs effective for the x6500 device itself.  I am trying to mess with the clock speeds now in MPBM but it seems once I set them to go above 80 mpbm starts reporting drastically incorrect hashrates.

I will try to think of things to add to the readme. I am not sure which mining software should be recommended, mine.py or mpbm. It seems like personal preference to me, I like mine.py. I left MPBM running and attempted to work in TheSevens fixes for the stratum divide by zero issue with eloi pool, but have not succeeded yet. Still going fine via getwork though.


EDIT: When the clocks step above ~62 strange things start to happen.

Code:

2014-02-20 13:05:35.016 [400] X6500 board AH01A6C3 FPGA0: Mining Blake:0000000230**
2014-02-20 13:05:35.042 [500] X6500 board AH01A6C3 FPGA1: Warmup: Setting clock speed to 64.00 MHz...
2014-02-20 13:05:35.179 [500] X6500 board AH01A6C3 FPGA0: Warmup: Setting clock speed to 64.00 MHz...
2014-02-20 13:05:35.246 [400] X6500 board AH01A6C3 FPGA1: Job interval: 47.000000 seconds
2014-02-20 13:05:35.292 [400] X6500 board AH01A6C3 FPGA0: Job interval: 0.500000 seconds
2014-02-20 13:05:35.292 [100] X6500 board AH01A6C3 FPGA0: Setting clock speed failed!

Then the reported hashrate is ~4.2 THs, this causes the job interval to be decreased drastically, so I changed the hardcoded "max" value of 0.5 to 2.5. It still manages to submit valid shares.

[kramble]

OK, it looks like the overclock is broken (I vaguely remember being suspicious of it).

I've just checked the build logs for X6500-Robust-v04-2core-fmax-100MHz.bit and what it ought to be doing is initializing the DCM to 100MHz clock then immediately reprogramming to 50MHz (BOOTUP_FREQUENCY). What I suspect is happening is that the reprogramming fails and the FPGA runs at 100MHz which corresponds to 400MHash/sec (2 FPGA with 2 cores each).

The overclock parameter ought to be reprogramming the clock speed each time mine.py is run, but you should be able to get it significantly above that initial 100MHz. It is possible there is a factor of 2 in there somewhere, and 64MHz corresponds to an actual clock speed of 128MHz (which would be reasonable given fmax=100MHz), but I can't see it in the code.

To confirm this, it would be useful to see if overclock is having any effect at all, both on the reported hash rate and that at the pool. If it is actually changing the clock speed then we can make some use of it (even if I don't understand where the factor of 2 is coming in), otherwise I could just get rid of it entirely and do a fixed clock speed build. I've found a 125MHz version I did earlier that you may want to try https://www.dropbox.com/s/g7iqzfl4yzmsnk2/X6500-StaticClock-v01-2core-125MHz.bit

EDIT: I see that "Setting clock speed failed!" comes from MPBM rather than mine.py (no matter, I assume it works pretty much the same way, its just that MPBM is a lot more complex to understand what's happening). If you were using mine.py you would probably need to manually increase the getwork rate as it defaults to 20sec which is OK for bitcoin but not for the blake bitstreams since only half of the nonce space is scanned (this was to support up to 4 cores, though I could drop this requirement now). I run my lancelot at 2 seconds for 800MH/s.

Ah, (crosspost), thanks for that, so 62 is effectively giving 500MHash/sec? If so then that's useful to know as I can just try rebuilding the original bitstream for a faster FMAX.

[Ignatius]

OK, it looks like the overclock is broken (I vaguely remember being suspicious of it).

I've just checked the build logs for X6500-Robust-v04-2core-fmax-100MHz.bit and what it ought to be doing is initializing the DCM to 100MHz clock then immediately reprogramming to 50MHz (BOOTUP_FREQUENCY). What I suspect is happening is that the reprogramming fails and the FPGA runs at 100MHz which corresponds to 400MHash/sec (2 FPGA with 2 cores each).

The overclock parameter ought to be reprogramming the clock speed each time mine.py is run, but you should be able to get it significantly above that initial 100MHz. It is possible there is a factor of 2 in there somewhere, and 64MHz corresponds to an actual clock speed of 128MHz (which would be reasonable given fmax=100MHz), but I can't see it in the code.

To confirm this, it would be useful to see if overclock is having any effect at all, both on the reported hash rate and that at the pool. If it is actually changing the clock speed then we can make some use of it (even if I don't understand where the factor of 2 is coming in), otherwise I could just get rid of it entirely and do a fixed clock speed build. I've found a 125MHz version I did earlier that you may want to try https://www.dropbox.com/s/g7iqzfl4yzmsnk2/X6500-StaticClock-v01-2core-125MHz.bit

You would know better than I, but I do not believe the clock problem is an issue with *2 somewhere. Despite the hashrate being reported incorrectly in the gui the device seems to operate correctly, not overheating, and at a hashrate pool side closer to that which is reported by mpbm as "effective hashrate". When using the x6500s with mpbm for BTC mining the clock values are given as actual values, not /2. When set at a clock speed which does not result in incorrect reporting of hashrate (~62) the device hashes at a correspondingly low rate.

I am trying the 125MHz.bit, I just had the 100MHz.bit up over 500MHs.

[kramble]

Ah, (crosspost, see my edit above), thanks for that, so 62 is effectively giving 500MHash/sec? If so then that's useful to know as I can just try rebuilding the original bitstream for a faster FMAX.

[Ignatius]

Ah, (crosspost, see my edit above), thanks for that, so 62 is effectively giving 500MHash/sec? If so then that's useful to know as I can just try rebuilding the original bitstream for a faster FMAX.

No, I'm sorry, mis-communication. Setting the clock to ~62 in MPBM results in the correct hashrate and clock speeds being shown in MPBM, but the hashrate is very low. It is around the speed one would expect if the chips were actually clocked at ~62MHz.

I am currently running X6500-StaticClock-v01-2core-125MHz.bit in MPBM, min clock set to 115, max clock set to 135. Effective hashrate 474MHs. I would need to leave things running for far longer to get more accurate numbers of avg hashrate.

EDIT: Effective hashrate settling above 580MHs. I will let it run on these settings for a while and try to grab a more accurate average MHs value.

[kramble]

No, I'm sorry, mis-communication. Setting the clock to ~62 in MPBM results in the correct hashrate and clock speeds being shown in MPBM, but the hashrate is very low. It is around the speed one would expect if the chips were actually clocked at ~62MHz.

I am currently running X6500-StaticClock-v01-2core-125MHz.bit in MPBM, min clock set to 115, max clock set to 135. Effective hashrate 474MHs. I would need to leave things running for far longer to get more accurate numbers of avg hashrate.

EDIT: Effective hashrate settling above 580MHs. I will let it run on these settings for a while and try to grab a more accurate average MHs value.

Good, so it is working then. What was confusing me was the log (code block) from a couple of posts up where it failed to set it above 64. I'm still a little confused by this as you're now setting it to 115 and it is working, but 64 was not? Is there some window of values where it is broken? EDIT, no, I'm definitely not thinking straight, the StaticClock version has a fixed 125MHz clock speed, it won't overclock or underclock, it just runs at that speed, which should be exactly 500MHash/sec.

What's even more confusing is that the "Setting clock speed failed!" message is the result of just reading back the configuration register from the FPGA, which is just dependent on the jtag clock and should not break even if the DCM failed to program.

I'll get on with trying for a faster build, but this can take a while (like days sometimes). EDIT. Well I just kicked it off looking for FMAX=130, but I'm not so sure that this is worthwhile now. It'll probably fail PAR anyway, so the next try will be for a 150MHz StaticClock I think.

[Ignatius]

No, I'm sorry, mis-communication. Setting the clock to ~62 in MPBM results in the correct hashrate and clock speeds being shown in MPBM, but the hashrate is very low. It is around the speed one would expect if the chips were actually clocked at ~62MHz.

I am currently running X6500-StaticClock-v01-2core-125MHz.bit in MPBM, min clock set to 115, max clock set to 135. Effective hashrate 474MHs. I would need to leave things running for far longer to get more accurate numbers of avg hashrate.

EDIT: Effective hashrate settling above 580MHs. I will let it run on these settings for a while and try to grab a more accurate average MHs value.

Good, so it is working then. What was confusing me was the log (code block) from a couple of posts up where it failed to set it above 64. I'm still a little confused by this as you're now setting it to 115 and it is working, but 64 was not? Is there some window of values where it is broken?

What's even more confusing is that the "Setting clock speed failed!" message is the result of just reading back the configuration register from the FPGA, which is just dependent on the jtag clock and should not break even if the DCM failed to program.

I'll get on with trying for a faster build, but this can take a while (like days sometimes).

When setting a clock value in MPBM above roughly ~62MHz MPBM reports "Setting clock speed failed!", and from that point on reports a wildy inaccurate hashrate under "Current Hashrate" and "Average Hashrate". More sane values are shown under "Effective Hashrate" though. However, the device continues to operate and submit valid shares. The more I increase the clock past 62, the higher the effective hashrate becomes.

Thank you for your work on these devices kramble. I am in no rush for faster bitstreams, I was happy at ~400MHs.

I added BlueDragons ny1.blakecoin.com pool to MPBM and it reports my current hashrate at ~550MHs. I will update later.

[kramble]

When setting a clock value in MPBM above roughly ~62MHz MPBM reports "Setting clock speed failed!", and from that point on reports a wildy inaccurate hashrate under "Current Hashrate" and "Average Hashrate". More sane values are shown under "Effective Hashrate" though. However, the device continues to operate and submit valid shares. The more I increase the clock past 62, the higher the effective hashrate becomes.

OK, the MPBM code is doing this

Code:

def _set_speed(self, speed):
    speed = min(max(speed, 4), self.parent.settings.maximumspeed)
    if self.stats.speed == speed: return
    if speed == self.parent.settings.maximumspeed: self.initialramp = False
    self.core.log(self, "%s: Setting clock speed to %.2f MHz...\n" % ("Warmup" if self.initialramp else "Tracking", speed), 500, "B")
    self.parent.set_speed(self.fpga, speed)
    self.stats.speed = self.parent.get_speed(self.fpga)
    self.stats.mhps = self.stats.speed
    self._update_job_interval()
    if self.stats.speed != speed:
      self.core.log(self, "Setting clock speed failed!\n", 100, "rB")

So after a failure to set the speed, we have self.stats.speed != speed, so self.stats.speed is probably garbage which may be the reason for the bad stats. It would be useful to know what the actual value of self.stats.speed is as I still don't understand how this can fail (I'm gazing at the verilog, and it just seems impossible to not read back the same value as it wrote).

Anyway, its the actual pool speed that matters (for valid shares). At some frequency the device is going to start to giving invalid shares. I expect its going to be around the 500MHash/sec range (for the variable clock speed bitstream).

Thank you for your work on these devices kramble. I am in no rush for faster bitstreams, I was happy at ~400MHs.

No problem. This is quite a difficult one to get working blind, so it will be good to make some progress. It was rather left in limbo back at the end of October as I moved on to do the ztex and CM1 ports, which is why the hash rate is so slow compared to those ports (it should be able to get 800MHash/sec from each board). If you're happy to test, then I'm happy to supply bitstreams. It will be quite a slow process though (probably be a couple of days). I'll PM you when I've got something.

[Ignatius]

It would be useful to know what the actual value of self.stats.speed is as I still don't understand how this can fail (I'm gazing at the verilog, and it just seems impossible to not read back the same value as it wrote).

I believe this:
2014-02-20 15:27:15.972   [100]   X6500 board AH01A6C3 FPGA1:    Setting clock speed failed!
2014-02-20 15:27:16.182   [300]   X6500 board AH01A6C3 FPGA0:    Running at 4294967295.000000 MH/s

means that self.stats.speed == 4294967295.000000

?

[kramble]

means that self.stats.speed == 4294967295.000000

Which is exactly 0xFFFFFFFF, which makes some sense. The JTAG code is quite horrible for me to understand (both the python, and to a lesser extent the verilog). There may be something strange going on (there is a checksum bit which looks suspicious as the change from 63 to 64 is just at a binary turnover). Perhaps I broke something in the original blake port from the bitcoin code? I'll have a look. Thanks.

Got it (I think), the python code and verilog are almost, but not quite, talking the same protocol. It's the checksum that's the issue (but only affected by the higher bits of the clock speed), so it makes sense now. The python has been written to the spec (described in the comments at the start of the jtag_comm.v verilog), but the verilog does not actually implement this correctly (the read address checksum should be differently calculated to the write address+data checksum). So it's fixable (if a bit tricky to simulate).

[ByronP]

SPREAD THE WORD: Fri 2/21/14 starting at 6pm EST 0% BUY FEE and 0.15% SELL FEE

[Ignatius]

1.5GH/s on a Enterpoint Cairnsmore 1 Quad Spartan-6 LX150 Development Board

What settings are needed to achieve these stated hashrates?

With --cainsmore-clock 200 and CM1-hv-v04a-175MHz-ucf-150-fmax-161.bit I see ~750MHs pool side.

At first I just thought cgminer was reporting the hashrate /2. But ny1.blakecoin.com shows the same numbers.

Any help is appreciated, thanks in advance.

[BlueDragon747]

1.5GH/s on a Enterpoint Cairnsmore 1 Quad Spartan-6 LX150 Development Board

What settings are needed to achieve these stated hashrates?

With --cainsmore-clock 200 and CM1-hv-v04a-175MHz-ucf-150-fmax-161.bit I see ~750MHs pool side.

At first I just thought cgminer was reporting the hashrate /2. But ny1.blakecoin.com shows the same numbers.

Any help is appreciated, thanks in advance.

are you sure its all programmed and you are using all 4 chips seems lower than default 

here is my command for cm1 at ~1.5Gh/s on pool

cgminer --disable-gpu --icarus-timing 2.45 -S \\.\COM28 -S \\.\COM29 -S \\.\COM30 -S \\.\COM31 --url stratum+tcp://ny1.blakecoin.com:3334 --userpass Username.Worker:WorkerPass --cainsmore-clock 200,200,190,180

[Ignatius]

are you sure its all programmed and you are using all 4 chips seems lower than default 

here is my command for cm1 at ~1.5Gh/s on pool

cgminer --disable-gpu --icarus-timing 2.45 -S \\.\COM28 -S \\.\COM29 -S \\.\COM30 -S \\.\COM31 --url stratum+tcp://ny1.blakecoin.com:3334 --userpass Username.Worker:WorkerPass --cainsmore-clock 200,200,190,180

Yes I am using all 4 chips. Which bitstream do you use at 1.5GHs? Clock rates show roughly 200MHz per chip. I take it the bitstream you are using gives 2 MHs per 1 MHz? I wonder why mine doesn't

[BlueDragon747]

same bitstream CM1-hv-v04a-175MHz-ucf-150-fmax-161.bit

did you use the hashvoodoo on the controller?

[kramble]

I left MPBM running and attempted to work in TheSevens fixes for the stratum divide by zero issue with eloi pool, but have not

You will also have to change the merkle hash as blake uses a single sha256 rather than the double sha256 used in bitcoin. The stratum code is missing in my fork (I only modified the master branch for blake, while stratum is in testing), but it's at around lines 131-132 in TheSeven's original repo. Though it's not quite that simple as the double hash is still used in some places! See cgminer diff. The gen_hash() function was changed to a single hash, but the double hash is still used at line 5487. I'm not quite sure how this will need to be implemented in stratumworksource.py, but I thought you'd want the heads-up.

Further to my last post about the clock speed issues. As understand the problem, it only affects the readback of the clock speed from the FPGA (the clock is always successfully changed). For clock speeds between 64-191MHz the value read is 0xFFFFFFFF, while it should be correct for <64MHz and >191MHz. What is happening is that only 6 bits are shifted into the JTAG user register when setting the address for read, this shifts out the lower 6 bits of the previous clock value, and the remaining bits are all-zero for speeds <64MHz, but at 64MHz a 1 remains in the LSB, which toggles the checksum value (really just a parity bit) and the command is ignored. At 128MHz we get 10 which is also gives an invalid checksum, and only at 192MHz do we get 11 which is valid. I don't know if this is the behavior for the official bitcoin bitstream, but this behavior is in the source code on fpgaminer's github (it may be that the bug is fixed in the official bitstream but did not get published on github).

Anyway I'll do the fix and get a new bitstream built. The fix is trivial, but testing it in simulation is going to be a pain so I'll do the bitstream build in parallel to the simulation and it may well be ready before I've even got the simulation working 

[BlueDragon747]

I left MPBM running and attempted to work in TheSevens fixes for the stratum divide by zero issue with eloi pool, but have not

You will also have to change the merkle hash as blake uses a single sha256 rather than the double sha256 used in bitcoin. The stratum code is missing in my fork (I only modified the master branch for blake, while stratum is in testing), but it's at around lines 131-132 in TheSeven's original repo. Though it's not quite that simple as the double hash is still used in some places! See cgminer diff. The gen_hash() function was changed to a single hash, but the double hash is still used at line 5487. I'm not quite sure how this will need to be implemented in stratumworksource.py, but I thought you'd want the heads-up.

Further to my last post about the clock speed issues. As understand the problem, it only affects the readback of the clock speed from the FPGA (the clock is always successfully changed). For clock speeds between 64-191MHz the value read is 0xFFFFFFFF, while it should be correct for <64MHz and >191MHz. What is happening is that only 6 bits are shifted into the JTAG user register when setting the address for read, this shifts out the lower 6 bits of the previous clock value, and the remaining bits are all-zero for speeds <64MHz, but at 64MHz a 1 remains in the LSB, which toggles the checksum value (really just a parity bit) and the command is ignored. At 128MHz we get 10 which is also gives an invalid checksum, and only at 192MHz do we get 11 which is valid. I don't know if this is the behavior for the official bitcoin bitstream, but this behavior is in the source code on fpgaminer's github (it may be that the bug is fixed in the official bitstream but did not get published on github).

Anyway I'll do the fix and get a new bitstream built. The fix is trivial, but testing it in simulation is going to be a pain so I'll do the bitstream build in parallel to the simulation and it may well be ready before I've even got the simulation working 

Thanks for your continued help and support kramble you are awesome

[Ignatius]

same bitstream CM1-hv-v04a-175MHz-ucf-150-fmax-161.bit

did you use the hashvoodoo on the controller?

Yes, hashvoodoo_controller_25.bit. I have no idea why we would get such vastly different results

After an overnight run my x6500 with X6500-Robust-v04-2core-fmax-100MHz.bit, running on MPBM, initial clock 100 max clock 135, MPBM shows an effective hashrate of 272MHs. I left this bitstream going since it seemed to provide a higher hashrate than the 125MHz static clock bitstream. I am now leaving X6500-StaticClock-v01-2core-125MHz.bit for an extended period of time to see where it's hashrate ends up.

[kramble]

After an overnight run my x6500 with X6500-Robust-v04-2core-fmax-100MHz.bit, running on MPBM, initial clock 100 max clock 135, MPBM shows an effective hashrate of 272MHs. I left this bitstream going since it seemed to provide a higher hashrate than the 125MHz static clock bitstream. I am now leaving X6500-StaticClock-v01-2core-125MHz.bit for an extended period of time to see where it's hashrate end sup.

Strange. I'd expect around 500MHs for that clock speed. Maybe it's just ramped the clock up too far and you're getting bad hashes (HW errors in cgminer terminology). I'll see if I can work out what MPBM is doing with the clock, but I need to finish my current work on simulations first (almost done, then I'll put on a quick single core build for functional verification).