Onda B250-D12P-D1 cannot boot more then two Tesla P100 -16GB Cards - 100$ Reward

[jadefalke]

Hi Guys,

i have here multiple Tesla P100 16 GB and try to get them working in a Mining Case with a Onda B250-D12P-D1 Motherboard.

Whole Setup:
2550Watt Power Supply
celeron cpu
8 GB DDR3 RAM
M2 Drive (64 gig)
8X Tesla P100 16 GB PCIE

If i add 2 Tesla P100 Cards it is working as expected, as soon as i add a 3rd card i have a black Screen and Rig does not boot up and hive is marking it as offline.

Bios has:

4 GB encoding active

CSM off

For the first who provides me a Solution to get the Rig fully populated with 8 Tesla P100 Cards working i will send out 100$ in USDT as a reward!

[1714233540]

1714233540

[1714233540]

1714233540

[1714233540]

1714233540

[1714233540]

1714233540

[1714233540]

1714233540

[1971ECPT]

Hi, try to use bigger SSD (256GB) and increase virtual memory at least as sum of entire quantity of  memory of all video cards used. By example if you have 10 cards of 16 GB video memory each, virtual memory must be at least 160GB. Also, update motherboad bios to the latest version available on producer website. Try to leave CSM on, too, with less UEFI and more legacy.Good luck!

[JayDDee]

Available PCIe lanes could also be an issue considering the CPU Is Celeron. Try with a SATA SSD instead of m2NVME and check if any other peripherals
share lanes with the PCIe slots. Also make sure all the motherboard's extra power connectors are powered.

I get the feeling that Celeron & Tesla are a bad combination, like putting cheap tires on a Ferrari.

[jadefalke]

Available PCIe lanes could also be an issue considering the CPU Is Celeron. Try with a SATA SSD instead of m2NVME and check if any other peripherals
share lanes with the PCIe slots. Also make sure all the motherboard's extra power connectors are powered.

I get the feeling that Celeron & Tesla are a bad combination, like putting cheap tires on a Ferrari.

good input, will try that out

[jadefalke]

update, replaced the M2. with SSD, no change, also CSM on or of does not change anything. also UEFI vs. Legacy seem to be not a factor. With 3 cards it won't post at all, just a black screen
t

[JayDDee]

update, replaced the M2. with SSD, no change, also CSM on or of does not change anything. also UEFI vs. Legacy seem to be not a factor. With 3 cards it won't post at all, just a black screen
t

You need to determine if it's a display issue or boot issue, Any difference in beep tones if your MB has a speaker? Does it respond to a ping?
I assume you've tried different cards in different slots to rule that out. Do your Teslas have display output? Check those if you have no ouput from iGPU.
Don't forget about VM. Try a different OS.

[jadefalke]

update, replaced the M2. with SSD, no change, also CSM on or of does not change anything. also UEFI vs. Legacy seem to be not a factor. With 3 cards it won't post at all, just a black screen
t

You need to determine if it's a display issue or boot issue, Any difference in beep tones if your MB has a speaker? Does it respond to a ping?
I assume you've tried different cards in different slots to rule that out. Do your Teslas have display output? Check those if you have no ouput from iGPU.
Don't forget about VM. Try a different OS.

Hey,

Tesla don't have a Video Output, and it is Hive installed, so even if no video output would be there, it would boot into hive, (which is not happening). I tried also Windows, but again it is not going to boot at all. It is all quite, does not beep and Monitor does not turn on at any time.

[1971ECPT]

Maybe the last option is bios update. Try it, just with integrated graphic video. Also, try to use better CPU, maybe Celeron is too weak.

[JayDDee]

It's not the mobo or the GPUs if any 2 cards work in any 2 slots and it's not the OS if both Win & HiveOS fail with > 2 GPUs.
I don't suspect BIOS unless it's been buggered with.
It's an incremental problem pointing to a resource exhaustion. That means not enough power or not enough lanes. VM is ruled out if it doesn't POST.
Celerons apparently work in minimalist large rigs with GTX but maybe Teslas are a different story.
I suggest try a different PSU and try installing >2 gaming GPUs if you have some to test with. Reflashing the BIOS or trying a bigger CPU would be
last thing to try but would be a long shot.

[jadefalke]

Bigger CPU is coming in today, but 4-Core is the max. for that board anyway. Bios update is not so easy, as i could not find any newer bios at all.

[jadefalke]

I gave up and moved to a IBM Server System looks like everything with a B250 Chipset and PCI-Lane splitting will not work for the P100, sad ;(

[FP91G]

I wish I had read this earlier. In such cases, you usually need to change the motherboard and processor. Many Tesla P100 video cards do not work on all older platforms. And for old video cards, your Onda B250-D12P-D1 Motherboard will still be relevant even with a old Celeron.

[batsonxl]

1.Try with risers:NO
2.Change motherboard

[666mrga999]

Not every celeron cpu is good for multi gpu mining. I remember from eth mining days when I bought wrong celeron for my 19xpcie motherboard. Celeron G3900 works with multiple (8+) gpus at x1 pcie.
Do not use first pcie slot, IMPORTANT it is x16 and it uses up most pcie lanes.

TLDR: DO NOT PUT GPU IN FIRST SLOT, LEAVE EMPTY

[JayDDee]

Not every celeron cpu is good for multi gpu mining. I remember from eth mining days when I bought wrong celeron for my 19xpcie motherboard. Celeron G3900 works with multiple (8+) gpus at x1 pcie.
Do not use first pcie slot, IMPORTANT it is x16 and it uses up most pcie lanes.

TLDR: DO NOT PUT GPU IN FIRST SLOT, LEAVE EMPTY

This makes sense, though using a Celeron for a large (> $5k) GPU rig is too minimalist IMO. Spend an extra $100 for a i3 or i5.
In this case there is still the issue of it being Tesla GPUs. They have much more VRAM to support and may have minimum lane requirements.
I'm not speaking from actual experience. I expect a motherboard designed for mining would work out of the box in most cases. The unknown
variable seems to be using Tesla GPUs in a large rig. It's also possible Nvidia deliberately borked Teslas to require higher end platforms. After
LHR I wouldn't put it past them.

[FP91G]

Not every celeron cpu is good for multi gpu mining. I remember from eth mining days when I bought wrong celeron for my 19xpcie motherboard. Celeron G3900 works with multiple (8+) gpus at x1 pcie.
Do not use first pcie slot, IMPORTANT it is x16 and it uses up most pcie lanes.

TLDR: DO NOT PUT GPU IN FIRST SLOT, LEAVE EMPTY

If you are using an x1 raiser, then there should be no problems with 8 video cards on a regular motherboard. And you should also not use NVMe drives.
If there are more video cards, then it is better to think about buying another processor in advance. But for me, the ideal number of video cards on a motherboard is 6, so that consumption is no more than 1 kilowatt per hour.

[jadefalke]

Just to keep you all in the loop, the Celeron theory was tested i bought i3-4 Core CPU, same result. i suspect the B250 Chipset and how the PCI Lanes are address is part of the Problem. Expensive Lesson for us, as we have to buy now Server ontop of the cases we got initially.
(we event tried two other different Mining mainboards, but they had also the B250 Chipsets and did not work out.)
We even were going as far as getting an Octominer Case, but that did not work either. Only working solition so far was a professional Server System.

[666mrga999]

Did you leave first PCIE slot empty, like I told you, this link has solved similar problem to yours with this MB

https://rsts11.com/2021/03/14/wrestling-with-an-onda-b250-d8p-d4-mining-motherboard/

[JayDDee]

Different Tesla,  same problem.

https://linustechtips.com/topic/1327883-folding-at-home-gpu-mining-rig-18-tesla-k20ms-a-little-help-needed/

[batsonxl]

Just to keep you all in the loop, the Celeron theory was tested i bought i3-4 Core CPU, same result. i suspect the B250 Chipset and how the PCI Lanes are address is part of the Problem. Expensive Lesson for us, as we have to buy now Server ontop of the cases we got initially.
(we event tried two other different Mining mainboards, but they had also the B250 Chipsets and did not work out.)
We even were going as far as getting an Octominer Case, but that did not work either. Only working solition so far was a professional Server System.

Is it works with riser?
Is it windows?or? Driver?