Bitcoin Forum
December 07, 2016, 06:43:23 PM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: EMI and PCIe extenders. Problem or just internet forum noise?  (Read 1980 times)
catfish
Sr. Member
****
Offline Offline

Activity: 270


teh giant catfesh


View Profile
November 02, 2011, 10:32:13 PM
 #1

I've had some odd cases when trying to populate ALL the available PCIe slots on any given logic board with GPUs. In the vast majority of the times, the problem is either bad logic board design or bad power distribution strategies. (Note that I'm talking about getting the board to boot up and a basic OS install to recognise the PCIe slots being filled. Actually getting ATI's Catalyst driver to work with different OEM graphics cards is a whole different problem, and not what I'm talking about here).

Due to problem 1, I've given up on certain fancy logic boards and standardised on one that, so far, has worked very well with all 4 slots populated (I have four of these boards now, and one rig uses three boards and 12 GPUs with no problem). I've mentioned it before, it's a Gigabyte H61M-D2-B3.

That leaves power distribution issues being the main cause of cards failing to be recognised, and I've learned the hard way that you need to spend as much on PSUs as you do on GPUs - cutting costs on PSUs or using 'just about powerful enough' models is a bad idea. Wasted time is the biggest issue, since systems not getting *quite* enough clean power don't just 'stop', but have intermittent problems.


However, on these forums and elsewhere, when I've been trying to debug a problem board with the help of you all in one thread or another, and also in other peoples' threads, the issue of EMI between the PCIe extender cables has recently started coming up.

In other words, electromagnetic interference - current flowing through the extenders causing an EM field to flicker around the conductors, which *theoretically* could induce changes in the current flowing in nearby extender cables. Since we're dealing with digital electronics, a single change in a bitstream from 1 to 0 could make the data invalid, or if it's control data, may crash the card, etc. etc.


Can any proper electronic engineer put this issue straight for me? Can there theoretically be enough EMI from an average PCIe extender cable running Bitcoin software (i.e. not using all the bandwidth, shouldn't be taking LOTS of power) to cause problems with adjacent cards? Assume my worst-case builds, where two cards end up with their extender cables lying ON TOP of each other - i.e. separated by the thickness of their insulation only.

My gut feeling is that this is bullshit, and engineers have already thought of this and made sure it's not a problem. After all, all slots end up running current through lines in the logic board, which are VERY close to each other and not surrounded by loads of EM shielding. And if it *was* a real known issue, then Cablesaurus et al would surely sell 'shielded' extenders. Or all the extenders would be expected to be 'shielded' as a matter of course.

But I'd like someone who REALLY knows the answer to nail it once and for all. If EM interference is potentially a problem for multi-card miners using PCIe extenders, then we need to know because these types of failures can be random intermittent problems that are an absolute BITCH to debug. Making different OEM cards play nice with ATI's Catalyst drivers is hard enough, and this 'EMI in your extenders' is another variable I want to remove.

If there *is* a potential EMI problem, how do I shield the typical x1 -> x16 extender? They need to remain flexible, and the last thing I want is some 'solution' to a rare 'problem' that introduces new BIG risks. So wrapping the extenders in aluminium foil isn't really what I want to hear, I can just imagine all my boards shorted out and my wooden rigs up in flames.


Is this a *real* issue, or is it yet another example of how a random mistaken / drunk / high / irrelevant comment on a specialised topic forum can explode by repetition to become a 'genuine issue' in that field?

...so I give in to the rhythm, the click click clack
I'm too wasted to fight back...


BTC: 1A7HvdGGDie3P5nDpiskG8JxXT33Yu6Gct
1481136203
Hero Member
*
Offline Offline

Posts: 1481136203

View Profile Personal Message (Offline)

Ignore
1481136203
Reply with quote  #2

1481136203
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218


Gerald Davis


View Profile
November 02, 2011, 11:34:58 PM
 #2

Server grade ribbon extenders are normally shielded.  They also tend to run $30 to $70 a piece. 

If you want a homemade version use aluminum tape.  It is more than flexible enough.

http://www.amazon.com/JVCC-AF20-Aluminum-Foil-Linered/dp/B000QDEQQU



If you are worried about shorting (which is highly unlikely) then wrap the aluminum tape in a layer of clear package tape.
bulanula
Hero Member
*****
Offline Offline

Activity: 518



View Profile
November 02, 2011, 11:46:55 PM
 #3

I think this is a valid concern but don't quote me on this.

What do you mean by getting different OEM cards working with the ATI drivers ? Please enlighten me mate ! Maybe this is the problem I have been having all along and not the extenders ? Can you combine XFX, Sapphire and Dell 5870s all running on one mobo without special instructions ? If not, what are those special instructions.

Thank you !
sveetsnelda
Hero Member
*****
Offline Offline

Activity: 644


View Profile
November 03, 2011, 05:11:11 AM
 #4

Check out my post here, my friend.  I ended up having to wrap almost all of my PCI-E extenders.  Curiously enough, it seems that I only have the issue with a particular motherboard (which I happen to have 12 of now).

https://bitcointalk.org/index.php?topic=38331.20

14u2rp4AqFtN5jkwK944nn741FnfF714m7
sveetsnelda
Hero Member
*****
Offline Offline

Activity: 644


View Profile
November 03, 2011, 05:13:24 AM
 #5

If you are worried about shorting (which is highly unlikely) then wrap the aluminum tape in a layer of clear package tape.
I actually already shorted an extender against the back of one card already (bumped a card while passing by).  Fortunately the card lived and is still mining away.  Most of my extenders aren't wrapped after shielding, but I have a few with electrician's tape.

14u2rp4AqFtN5jkwK944nn741FnfF714m7
Transisto
Donator
Legendary
*
Offline Offline

Activity: 1624



View Profile WWW
November 03, 2011, 07:59:58 AM
 #6

I cut lengths of Al. foil and wrap em once from inside out, then use one stripe of shipping tape to finish the job. All done in 1min.

But I don't like the result, the extenders become harder to bend and that stress the base soldering.
It may also be just another layer of FUD bringing in new troubles.

Please let's bring an high frequency electronic engineer to clarify this once and for all !

You say there are already traces very close to each other on the board, but could it be that those trace are being shielded by intermediary trace ?

paraipan
Legendary
*
Offline Offline

Activity: 924


Firstbits: 1pirata


View Profile WWW
November 03, 2011, 12:32:52 PM
 #7

I'm not a real engineer here, but just like your "guts" tell you the issues you have aren't from the extenders, my experience in electronics tell the same.

You have a 5v low amp passing on those extenders just like the whole motherboard circuitry, too low to create a real em interference. You just have to look at the printed circuits mb for the answer, they are not shielded in any way.

The real issues i had was with the PSU's having a common rail for the 12v feeding more than one GPU. The interference created on the rail by every card makes the ones with lower quality filtering stage, capacitors and such, have an extreme varying supply in their actual circuits. All components connected to a common rail induce interference back to it (GPU, HDD, CD-ROM, you name it).

The best solution i could think of was have an independent PSU for every card and the MB too. Using cheap 450w PSU (20 $) for every card solved the internal interference in the setup i have. Off course a little hacking is involved to trick every PSU it has a load and supply the power. All PSU must share the same ground connection [edit->] on the high voltage side, same 110-220v extender [/edit], take extreme care with that if you get into it some day.

A nice guide on the "solution" i posted found on the inter-webs.

BTCitcoin: An Idea Worth Saving - Q&A with bitcoins on rugatu.com - Check my rep
catfish
Sr. Member
****
Offline Offline

Activity: 270


teh giant catfesh


View Profile
November 07, 2011, 02:28:56 PM
 #8

Cheers everyone for the feedback.

Sveetsnelda - looks like it's a miracle the Catfish Mining Shelf works at all mate!!!! I will come up with a typically catfish-like answer to this - sounds like the shielding is necessary for maximum output and stability, but I'm NOT having my extenders covered with conductive material unless it's insulated. But I've seen the power of your rig and I'm not going to question your experience Wink

Point in question - does the 'shielding' work on faraday-cage principles or does the 'shielding' conductive cover have to be connected to ground? Can I simply wrap the extenders in alu foil and then insulate the whole lot - or does the alu foil wrap need to be connected electrically to a ground line?

Independent PSUs isn't an answer for a high-density 12-GPU rig and it also adds a bunch of additional electrical losses that cost a fortune in the UK (electricity is pricey).

...so I give in to the rhythm, the click click clack
I'm too wasted to fight back...


BTC: 1A7HvdGGDie3P5nDpiskG8JxXT33Yu6Gct
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218


Gerald Davis


View Profile
November 07, 2011, 02:46:35 PM
 #9

For optimal performance the shield would need to be continuous so that means connecting to both the motherboard ground and the video card ground.

This is the principal that both RG-6 ("coax cable") and shielded networking cables use.  The connectors at both ends bit into the shield to provide a continual conductive link.  The equipment at both ends grounds the connectors to the common ground ensuring a "perfect" shield.

However these cables are going hundreds of feet (sometimes miles) and thus can be exposed to significant interference.  A electrically isolated shield is less effective but likely more than sufficient for the small distances you are dealing with.  If you wanted you could possibly attach a lead from the shield to an unpainted screw in the computer chassis but that is likely overkill. 

As far as conductive danger, wrapping the aluminum tape in a layer of packaging tape is sufficient to reduce those dangers.
Transisto
Donator
Legendary
*
Offline Offline

Activity: 1624



View Profile WWW
November 07, 2011, 07:17:27 PM
 #10

I would guess shielded extender can be more useful when chaining 2 or 3.

I am more concerned about having the pc not in a case.
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!