This Titan did it again. Weird:

DC/DC   Voltage (V)   Current (A)   Power (W)   Temperature (°C)
0   0.7900   41.5625   32.834   59.900
1   0.7897   41.2500   32.575   73.400
2   0.7867   41.0625   32.304   57.400
3   0.7886   41.6250   32.825   52.700
5   0.0035   0   0.000   17.300
6   0.7883   40.3750   31.828   44.900
7   0.7928   39.8125   31.563   45.100

Thank you for doing this. Next, get Titan to hash with Beaglebone. Working on it.

And I'm sending out my first repaired Neptune/Titan controller boards.

Sweet. Apparently there is a soft reset (my guess is "soft reset all power supplies explicitly using waas or direct SPI pokes) followed by 8 power ups. That will do it.

And Tarkin knows about this and is writing the solution into the next gen of his code. So this can be solved by software which is a lot better than sending it to me to swap power supplies.

Noted, moving on. The right way to turn off a Titan remotely of course is to just remotely turn off and on a small relay connected to the power supply enable line (the one we always jumper). But software is simpler.

Heh. I've been pulling power supplies from Neptunes and now a Titan. Sometimes it's because I am troubleshooting the supply reading 0.0 volts (dead short, usually under chip, bitch to reflow) sometimes because the power supply turned into a little cute inferno. In these cases advanced screen will show the skipped supplies like 0,1,4,5,6,7

This is a bit different. What I see is a Titan where it hashes away with supplies 0,1,2,3,4,5,6,7 all purring, then when I come back the next day it shows only supplies 0,1,3,4,5,6,7. 2 literally is not there from the advanced view. If I reset the second die, 3 will try to supply *all* the power. Which will probably cause a quick and fun fireball.

Power cycle the cube and 2 reappears and it hashes happily. But sometimes it's a different supply that "vanishes". Really odd.
If theres a true hardware problem w/ ur titan tho then of course my firmware cant do much bout that =P

That's it. Your firmware rocks, and while I think I can see how to fix this with appropriate calls, it's a billion times better to just say "Pay Tarkin $50, it's the best $50 you spend" than pay me substantially more to pull and swap power supplies. I will flag this as "logical sense solution exists" and focus on the fireballs.

Next up: Why a shorted Titan shorts down the whole FPGA by shorting pins 4,6 on the 10 pin header. What's that all about :-)

Tarkin, question: Does your source code fix the issue where a power supply literally disappears? I've had this happen twice on a Titan in, power cycling clears it and it is different supplies at different times. If your code fixes that then it is exceptionally "worth it" as trying to power a die with a single supply will result in a blown up component quickly...

France is not too far away from the US, I can PM you a price but posting pictures is the best way to start.

Yep, there are "islands of stability" in Neptunes. Strange, but something to note. Maybe the slower clock speed interferes with the power draw of the chip, causing the power supplies to restart. Hm. Tried running the junker at 100mhz all chips and the power supplies are at .8 volts, but all currents match and it's hashing at a very cool 100gh.

Anyway, I'm a bit curious as to why Titans are still such a good deal. With one running at 60mh @200 watts one should make 1.15ltc a day or 4 bucks a day. Difficulty doesn't seem to be doing squat in the LTC world. Power usage of 200w is 4.8kw a day or 50 cents of power at .10c/kw.

So each unit makes a profit of $3.50 a day. Hundred a month profit, 1.2k in a year per cube if not much changes in the ltc world.

That's interesting. Somewhere in there is a math error or something, because price always seems to match power required to hash the coin.....

Oh well. Other things to do in the next day or two, will pick this back up mid-week.

That's a bit odd. I tuned down one of the test Neptunes here (the water block equipped junker) to a clock speed of 300mhz and this morning noticed it was hashing slower. Power supplies 0,1 were .8 volts, no current. (note: They should be .78v). Reboots did not change anything.

Put the clock speed on that engine back to 350mhz and the power supplies came right back. Tried a similar thing on the other Neptune (600gh, 350/450/450/450 solid as a rock) and sure enough the supplies I moved to 300 shut down. Did through a reboot. Reset, no problem.

Weird. I am running 1.02 of the firmware for testing (was the first I got, consistency is key) but weird.

No, and since I am just starting on the dead die thing I am wondering if the dc-dc power supplies are what is causing dead die (not the same as board shorts out whole Titan thing). So don't replace everything yet :-)


I can ask the guy that sent me this Rpi but when I am fixing controller boards (Titan or Neptune) I always use a BeagleBone with Neptune code as the test CPU and there is no problem testing/running them. I would find it spectacularly odd that a Neppie board will not work with a Titan at this point.

But there are some oddball differences in the Titan. This power supply vanishing act is not like the Neptunes, and these boards that shut down a controller (and quite possibly would burn up the FPGA) are a different animal as well. I'll need to spend a week probing gently to see what these are doing.

Hm. Well, this is interesting.

So I fixed Titan #1 here and brought it online for testing. All 8 power supplies come up, all cores, hashied at 60mh with temps on the chip in the low 30's, power supplied averaging around 50c. Left it alone last night, checked in this morning.

Hash speed in the 40's. But under advanced tab I see that one of the power supplies is just *missing*. And the other had a current of zero, rhat's odd.... So I tried changing the voltage, applied changes, and quickly noticed that power supply 0 (the partner to the now missing supply 1) was trying to put out 50a to meet the chip need. Turned that to OFF *fast*, then turned it to a low 50mhz. Which it will supply:

DC/DC   Voltage (V)   Current (A)   Power (W)   Temperature (°C)
0   0.7394   12.1562   8.988   44.100
2   0.7963   27.8125   22.147   55.400
3   0.7972   27.8750   22.222   54.700
4   0.7933   33.9375   26.923   52.100
5   0.7938   34.0625   27.039   47.800
6   0.7930   34.3750   27.259   49.400
7   0.7955   33.5625   26.699   47.200

(note I also cut the voltage back on core zero).

That's weird. So I went over, power cycled the cube and controller (need to put those on separate supplies) and:

DC/DC   Voltage (V)   Current (A)   Power (W)   Temperature (°C)
0   0.7397   5.8984   4.363   38.100
1   0.7413   5.2500   3.892   41.500
2   0.7981   27.1250   21.648   49.400
3   0.7987   27.1562   21.690   48.800
4   0.7952   32.8125   26.093   47.900
5   0.7938   33.4375   26.543   45.600
6   0.7948   32.8125   26.079   47.100
7   0.7960   32.8125   26.119   43.500

Back and purring. Bringing it to full speed gets me:

DC/DC   Voltage (V)   Current (A)   Power (W)   Temperature (°C)
0   0.7988   28.6250   22.866   56.000
1   0.7992   28.2812   22.602   65.500
2   0.7950   28.8438   22.931   57.200
3   0.7958   29.0000   23.078   54.800
4   0.7930   33.3125   26.417   52.000
5   0.7938   33.8125   26.840   49.600
6   0.7924   34.0000   26.942   50.900
7   0.7955   33.5000   26.649   47.800

Right back to normal.

This is an odd thing, and if it's something that happens to others I wonder if it is the source of the "bad die" issue. Is this what Tarkin's software fixes?

What concerns me on the side is that I was able to try and get that one remaining power supply to supply all the hashing power, that would blow up a supply quickly.

Thoughts? If this is known and Tarkin fixes it for $50 by turning off then on the errant supply pair then this is not an issue I will fiddle with.  (Running Titan 2.0 software on this particular system, no special hacks)

Honest mode: With a soldering iron and some luck? No.

In order to fix boards properly you really need a good pre-heater (not a toaster oven. No. Just. No.) and a ok air rework tool (Aoyue isn't bad) and experience in using both. I've got another thread where I have been documenting all of my Neptune and Titan repairs here:

https://bitcointalk.org/index.php?topic=1283859.new#new

If you would like me to work on it, send me a PM.

Well with the power supply pulled it comes up. That's one Titan miner running. Got another Titan going here as well, testing it with a very low burn rate of 150mhz, stock voltage, purring at 42mh, 31c chip temp, 42c dc-dc temp. Will let it purr for awhile.

Edit: That Titan ran the night but soft-dropped a power supply. Odd.

Does the power supply shut down as soon as you turn it on? Did fire shoot out the back of the unit?

If the answer to either of these is "yes" then one of the power modules has shorted. I have one on the bench here that was being run in a barn (sigh). The dirt, straw, and whatnot built up on the caps until they overheated resulting in blown FETs and a small pretty plasma ball when I tested the supply off-board.

I can oddly enough fix this.

If it doesn't see the miner and Tarkin's code doesn't get it back online, then the chip overheated and fused the solder underneath. Mess. Can be fixed sometimes, but bit of a bitch on wheels to do.

Note to self: Charge a fee to clean a filth-covered board. Yuck.


On the other hand the back of the board shows no heat damage, just destroyed power supplies. My guess is that the caps and fets were covered in filth and the usual happened....

Oh and the power Molex connector is on the way out. Need to replace that too.

On the other hand, first Titan in. Next batch coming. Busy weekend ahead.

Meantime second and third boards have the FPGAs off. TPS chips now come up properly, will put on new FPGAs as soon as they come in. And it looks like USPS has finally gotten off their asses and sent the one day express package across the country in FOUR DAYS! Big round of applause....

Yep, works. There is no physical difference between a T and N controller baseboard. In fact I'm working to get a Beaglebone to run the 2.00 Titan code. It's a little different but not much and they included all sorts of nice source code....

I see that Antminers and now exploding in flames too. Though that's probably because people are running 8,000 watt power supplies into them. When there is a capacitor short those don't just go pfft and trip the power supply, they explode the capacitor and the manufacturer can't do a whole lot about that... (the technical term is AIR or Amp Interrupt Rating. It's why you can't plug a 15 amp appliance into a 20 amp outlet....)

Status: Had two more Neptune/Titan boards come in, both were solid dead. One is back up and running with the TPS power chip replaced, the second one still does not come up and it looks like the FPGA is shorted. Contacted user to see if they want me to replace the FPGA chip (costs .15btc by itself), will see.

It's looking to be about 50/50 in terms of replacing the power chip vs. the FPGA to fix these Titan/Neptune controller boards.

Now I just need to wait for the postal service to deliver more packages. Then we can start on reflows/rebuilds of the power supplies (bit more fun)

Some people get all the fun.

Looks like the fire started in the hashing chip area, then moved on up the board. Interesting, was this person using a normal power supply or one of those monster rail concepts that power a lot of miners off a common bus?

If so one should ensure that they have an appropriately rated fuse on each tap that goes to a miner, because those power planes are probably only designed for a max input current of 100a or so (1,200 watts). A single power supply would crowbar. Or run the risk of burning your mine down, things happen.

How does the other side of the board look?

I'm beginning to think they are all the same thing, at least since October Jupiters. Check if you can to see if the bottoms of the boards are smoked.

Like this:



That's a Titan board, crashed. Will be working on it.

View it as a smaller image and you can see the discoloration all around the chip. Especially in the corner where the die burned the board, that's where the reflow failure is.

Interesting.

Easy is relative. Was KNC shipping dead units?

Anyway, pull the board and tell me if there is a big black burn spot on the back. I just got some pictures of a Titan coming over and sure enough it has the same burn pattern as the neptunes.

Interesting.