Bitcoin Forum
November 18, 2017, 09:08:17 AM *
News: Latest stable version of Bitcoin Core: 0.15.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: « 1 ... 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 [84] 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 ... 284 »
  Print  
Author Topic: [OS] nvOC easy-to-use Linux Nvidia Mining v0019-1.4  (Read 322220 times)
fullzero
Legendary
*
Offline Offline

Activity: 1092



View Profile
July 09, 2017, 03:40:37 AM
 #1661

I have had a lot of requests for this; so here is a new oneBash and modded switch file which implement full integration of SALFTER_NICEHASH_PROFIT_SWITCHING

see the OP for links:

Replace your current oneBash with the new one.

extract switch and move it to the:
Code:
/home/m1

directory

(the one which opens when you click the Files icon on the left)

configure the following in oneBash

Code:
SALFTER_NICEHASH_PROFIT_SWITCHING="YES"

# LOCAL will attach the mining process to the guake terminal
# REMOTE will leave it unattached / ready for SSH
LOCALorREMOTE="LOCAL"       # LOCAL  or  REMOTE

CURRENCY=USD
POWER_COST=0.10
MINIMUM_PROFIT=0.0
# this is salfters BTC address:
PAYMENT_ADDRESS=1TipsGocnz2N5qgAm9f7JLrsMqkb3oXe2
WORKER_NAME=nv$IP_AS_WORKER

daggerhashimoto_POWERLIMIT_WATTS=125
__daggerhashimoto_CORE_OVERCLOCK=100
daggerhashimoto_MEMORY_OVERCLOCK=100
_______daggerhashimoto_FAN_SPEED=75

equihash_POWERLIMIT_WATTS=125
__equihash_CORE_OVERCLOCK=100
equihash_MEMORY_OVERCLOCK=100
_______equihash_FAN_SPEED=75

neoscrypt_POWERLIMIT_WATTS=125
__neoscrypt_CORE_OVERCLOCK=100
neoscrypt_MEMORY_OVERCLOCK=100
_______neoscrypt_FAN_SPEED=75

lyra2rev2_POWERLIMIT_WATTS=125
__lyra2rev2_CORE_OVERCLOCK=100
lyra2rev2_MEMORY_OVERCLOCK=100
_______lyra2rev2_FAN_SPEED=75

lbry_POWERLIMIT_WATTS=125
__lbry_CORE_OVERCLOCK=100
lbry_MEMORY_OVERCLOCK=100
_______lbry_FAN_SPEED=75

pascal_POWERLIMIT_WATTS=125
__pascal_CORE_OVERCLOCK=100
pascal_MEMORY_OVERCLOCK=100
_______pascal_FAN_SPEED=75

remember to thank salfter if you use this  Smiley



Thanks for implementing this, but for some odd reason I keep getting two instances of the miner screen running which causes the system to crash, I will do some more testing to try and figure out what is going on.

EDIT: Actually the kill code does not seem to work causing multiple miner screens, this is how the system is crashing.

Are you using LOCAL or REMOTE?

Please walk me through how you got multiple instances running so I can recreate this myself.



I am using REMOTE.  I am connecting via SSH attaching the screen with "screen -r miner"
for testing purposes I am altering the algo speed manually in the switch file waiting 10 mins for it to change and I noticed it is hit and miss for the kill code to function correctly.  I am also experiencing for LBRY and LYRA the mining process is executed twice.

Maybe I should start from scratch...

I tested the same way; changing the speed of one algo to force a switch.  I also changed the timeout in oneBash to 10 seconds instead of 600.

I tested this out and found salfters logic works well for switching between Ethash and Equihash; but not any of the other algos.  

It does seem to endlessly spawn new ccminer instances as well.  

I don't want to spend a lot of time on this; so I implemented killing all mining processes every time: then launching a new one.  It should work without issue now.

I also edited the oneBash logic to conditionally add salfters IPv6 fix, and reattach the screen to the guake terminal every reinit when in local mode.

If you have added the cronjob:

Code:
0,10,20,30,40,50 * * * * (cd /media/m1/1263-A96E && python2.7 switch.py)

I would remove it; as the oneBash implementation doesn't use it: and it may be (most likely is) causing additional launches of salfters original switch.

Note: I uploaded a new oneBash and switch with these changes.  They are linked on the OP.


1510996097
Hero Member
*
Offline Offline

Posts: 1510996097

View Profile Personal Message (Offline)

Ignore
1510996097
Reply with quote  #2

1510996097
Report to moderator
1510996097
Hero Member
*
Offline Offline

Posts: 1510996097

View Profile Personal Message (Offline)

Ignore
1510996097
Reply with quote  #2

1510996097
Report to moderator
1510996097
Hero Member
*
Offline Offline

Posts: 1510996097

View Profile Personal Message (Offline)

Ignore
1510996097
Reply with quote  #2

1510996097
Report to moderator
A blockchain platform for effective freelancing
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1510996097
Hero Member
*
Offline Offline

Posts: 1510996097

View Profile Personal Message (Offline)

Ignore
1510996097
Reply with quote  #2

1510996097
Report to moderator
1510996097
Hero Member
*
Offline Offline

Posts: 1510996097

View Profile Personal Message (Offline)

Ignore
1510996097
Reply with quote  #2

1510996097
Report to moderator
1510996097
Hero Member
*
Offline Offline

Posts: 1510996097

View Profile Personal Message (Offline)

Ignore
1510996097
Reply with quote  #2

1510996097
Report to moderator
RMH
Newbie
*
Offline Offline

Activity: 3


View Profile
July 09, 2017, 03:44:24 AM
 #1662

First off thanks for all the work Fullzero!  

I've been fighting with Win10 on my rig for a month.  Can't get it to recognize any more than 3 of my 1070 cards, and one always is disabled with an error.  So I've decided to dump Win10.

I've just got nvoc 0017 running with my old USB 2.0 stick on my ASUS Prime Z270-A with 6 Geforce GTX 1070s rig and I am not sure now how to get the mining started.  

I guess I was under the impression that it would start automatically on boot but I must have missed something in the setup.  It's just sitting there with the terminal screen open.  

Also in the Nvidia X server settings it's also only recognizing 3 of the 1070 gpus.  I'm hoping this is something that can be easily worked out as I'd hate to think that 3 of the 6 1070's I have only had for a month are bad.

Thanks in advance for any help.

fullzero
Legendary
*
Offline Offline

Activity: 1092



View Profile
July 09, 2017, 03:45:39 AM
 #1663

Just tried nvOC v0017 on an ASROCK X370 Gaming K4 motherboard and it does not recognize the hardware.

I don't have a x370 mobo and haven't added support for the chipset. 

You can do this by:

Click Ubuntu button on top left and type:

u

Click on software updater

Install updates


If you can't get ubuntu to boot:

If the bios posts; you can access the grub loader menu by pressing

esc

continuously while booting (note holding it down doesn't usually work), then select boot in recovery mode. 

in recovery mode:

Enable networking

then install updates from the cmd prompt:
Code:
sudo apt-get update && sudo apt-get dist-upgrade --yes

and reboot

this should ensure your build has all known system files for your system.

lbrasi
Newbie
*
Offline Offline

Activity: 25


View Profile
July 09, 2017, 03:50:03 AM
 #1664

I have had a lot of requests for this; so here is a new oneBash and modded switch file which implement full integration of SALFTER_NICEHASH_PROFIT_SWITCHING

see the OP for links:

Replace your current oneBash with the new one.

extract switch and move it to the:
Code:
/home/m1

directory

(the one which opens when you click the Files icon on the left)

configure the following in oneBash

Code:
SALFTER_NICEHASH_PROFIT_SWITCHING="YES"

# LOCAL will attach the mining process to the guake terminal
# REMOTE will leave it unattached / ready for SSH
LOCALorREMOTE="LOCAL"       # LOCAL  or  REMOTE

CURRENCY=USD
POWER_COST=0.10
MINIMUM_PROFIT=0.0
# this is salfters BTC address:
PAYMENT_ADDRESS=1TipsGocnz2N5qgAm9f7JLrsMqkb3oXe2
WORKER_NAME=nv$IP_AS_WORKER

daggerhashimoto_POWERLIMIT_WATTS=125
__daggerhashimoto_CORE_OVERCLOCK=100
daggerhashimoto_MEMORY_OVERCLOCK=100
_______daggerhashimoto_FAN_SPEED=75

equihash_POWERLIMIT_WATTS=125
__equihash_CORE_OVERCLOCK=100
equihash_MEMORY_OVERCLOCK=100
_______equihash_FAN_SPEED=75

neoscrypt_POWERLIMIT_WATTS=125
__neoscrypt_CORE_OVERCLOCK=100
neoscrypt_MEMORY_OVERCLOCK=100
_______neoscrypt_FAN_SPEED=75

lyra2rev2_POWERLIMIT_WATTS=125
__lyra2rev2_CORE_OVERCLOCK=100
lyra2rev2_MEMORY_OVERCLOCK=100
_______lyra2rev2_FAN_SPEED=75

lbry_POWERLIMIT_WATTS=125
__lbry_CORE_OVERCLOCK=100
lbry_MEMORY_OVERCLOCK=100
_______lbry_FAN_SPEED=75

pascal_POWERLIMIT_WATTS=125
__pascal_CORE_OVERCLOCK=100
pascal_MEMORY_OVERCLOCK=100
_______pascal_FAN_SPEED=75

remember to thank salfter if you use this  Smiley



Thanks for implementing this, but for some odd reason I keep getting two instances of the miner screen running which causes the system to crash, I will do some more testing to try and figure out what is going on.

EDIT: Actually the kill code does not seem to work causing multiple miner screens, this is how the system is crashing.

Are you using LOCAL or REMOTE?

Please walk me through how you got multiple instances running so I can recreate this myself.



I am using REMOTE.  I am connecting via SSH attaching the screen with "screen -r miner"
for testing purposes I am altering the algo speed manually in the switch file waiting 10 mins for it to change and I noticed it is hit and miss for the kill code to function correctly.  I am also experiencing for LBRY and LYRA the mining process is executed twice.

Maybe I should start from scratch...

I tested the same way; changing the speed of one algo to force a switch.  I also changed the timeout in oneBash to 10 seconds instead of 600.

I tested this out and found salfters logic works well for switching between Ethash and Equihash; but not any of the other algos.  

It does seem to endlessly spawn new ccminer instances as well.  

I don't want to spend a lot of time on this; so I implemented killing all mining processes every time: then launching a new one.  It should work without issue now.

I also edited the oneBash logic to conditionally add salfters IPv6 fix, and reattach the screen to the guake terminal every reinit when in local mode.

If you have added the cronjob:

Code:
0,10,20,30,40,50 * * * * (cd /media/m1/1263-A96E && python2.7 switch.py)

I would remove it; as the oneBash implementation doesn't use it: and it may be (most likely is) causing additional launches of salfters original switch.

Note: I uploaded a new oneBash and switch with these changes.  They are linked on the OP.



Thank you, I use use the new switch now Smiley
fullzero
Legendary
*
Offline Offline

Activity: 1092



View Profile
July 09, 2017, 03:50:34 AM
 #1665


Started mining with nvOC v0017 on Asrock B250M Pro4 motherboard, 2x Zotac gtx1070 Mini.
During the first run, the Ubuntu loading screen hung for some time and did not go past the orange/white dots but after a restart the nvOC loaded properly and started mining zec at nicehash  about 400sol/s per card @103-105W per card.    Settings are:  power limit 105, core 200, mem 10

Keep up the good work Fullzero!  Smiley

Unless your power is very expensive; I recommend using a powerlimit of 125 or more when mining ZEC with 1070s.

fullzero
Legendary
*
Offline Offline

Activity: 1092



View Profile
July 09, 2017, 03:56:18 AM
 #1666

First off thanks for all the work Fullzero!  

I've been fighting with Win10 on my rig for a month.  Can't get it to recognize any more than 3 of my 1070 cards, and one always is disabled with an error.  So I've decided to dump Win10.

I've just got nvoc 0017 running with my old USB 2.0 stick on my ASUS Prime Z270-A with 6 Geforce GTX 1070s rig and I am not sure now how to get the mining started.  

I guess I was under the impression that it would start automatically on boot but I must have missed something in the setup.  It's just sitting there with the terminal screen open.  

Also in the Nvidia X server settings it's also only recognizing 3 of the 1070 gpus.  I'm hoping this is something that can be easily worked out as I'd hate to think that 3 of the 6 1070's I have only had for a month are bad.

Thanks in advance for any help.

If you had the same problem with windows only recognizing 3 out of 6 GPUs; this indicates there is most likely a hardware problem.

Have you tried swapping your risers?

How are you powering your risers?

What kind of risers are you using?

If you run with only 3x GPUs and it works:

swap out the 3 working GPUs with the 3 that aren't being recognized and see if they work.

fullzero
Legendary
*
Offline Offline

Activity: 1092



View Profile
July 09, 2017, 03:57:22 AM
 #1667

I have had a lot of requests for this; so here is a new oneBash and modded switch file which implement full integration of SALFTER_NICEHASH_PROFIT_SWITCHING

see the OP for links:

Replace your current oneBash with the new one.

extract switch and move it to the:
Code:
/home/m1

directory

(the one which opens when you click the Files icon on the left)

configure the following in oneBash

Code:
SALFTER_NICEHASH_PROFIT_SWITCHING="YES"

# LOCAL will attach the mining process to the guake terminal
# REMOTE will leave it unattached / ready for SSH
LOCALorREMOTE="LOCAL"       # LOCAL  or  REMOTE

CURRENCY=USD
POWER_COST=0.10
MINIMUM_PROFIT=0.0
# this is salfters BTC address:
PAYMENT_ADDRESS=1TipsGocnz2N5qgAm9f7JLrsMqkb3oXe2
WORKER_NAME=nv$IP_AS_WORKER

daggerhashimoto_POWERLIMIT_WATTS=125
__daggerhashimoto_CORE_OVERCLOCK=100
daggerhashimoto_MEMORY_OVERCLOCK=100
_______daggerhashimoto_FAN_SPEED=75

equihash_POWERLIMIT_WATTS=125
__equihash_CORE_OVERCLOCK=100
equihash_MEMORY_OVERCLOCK=100
_______equihash_FAN_SPEED=75

neoscrypt_POWERLIMIT_WATTS=125
__neoscrypt_CORE_OVERCLOCK=100
neoscrypt_MEMORY_OVERCLOCK=100
_______neoscrypt_FAN_SPEED=75

lyra2rev2_POWERLIMIT_WATTS=125
__lyra2rev2_CORE_OVERCLOCK=100
lyra2rev2_MEMORY_OVERCLOCK=100
_______lyra2rev2_FAN_SPEED=75

lbry_POWERLIMIT_WATTS=125
__lbry_CORE_OVERCLOCK=100
lbry_MEMORY_OVERCLOCK=100
_______lbry_FAN_SPEED=75

pascal_POWERLIMIT_WATTS=125
__pascal_CORE_OVERCLOCK=100
pascal_MEMORY_OVERCLOCK=100
_______pascal_FAN_SPEED=75

remember to thank salfter if you use this  Smiley



Thanks for implementing this, but for some odd reason I keep getting two instances of the miner screen running which causes the system to crash, I will do some more testing to try and figure out what is going on.

EDIT: Actually the kill code does not seem to work causing multiple miner screens, this is how the system is crashing.

Are you using LOCAL or REMOTE?

Please walk me through how you got multiple instances running so I can recreate this myself.



I am using REMOTE.  I am connecting via SSH attaching the screen with "screen -r miner"
for testing purposes I am altering the algo speed manually in the switch file waiting 10 mins for it to change and I noticed it is hit and miss for the kill code to function correctly.  I am also experiencing for LBRY and LYRA the mining process is executed twice.

Maybe I should start from scratch...

I tested the same way; changing the speed of one algo to force a switch.  I also changed the timeout in oneBash to 10 seconds instead of 600.

I tested this out and found salfters logic works well for switching between Ethash and Equihash; but not any of the other algos.  

It does seem to endlessly spawn new ccminer instances as well.  

I don't want to spend a lot of time on this; so I implemented killing all mining processes every time: then launching a new one.  It should work without issue now.

I also edited the oneBash logic to conditionally add salfters IPv6 fix, and reattach the screen to the guake terminal every reinit when in local mode.

If you have added the cronjob:

Code:
0,10,20,30,40,50 * * * * (cd /media/m1/1263-A96E && python2.7 switch.py)

I would remove it; as the oneBash implementation doesn't use it: and it may be (most likely is) causing additional launches of salfters original switch.

Note: I uploaded a new oneBash and switch with these changes.  They are linked on the OP.



Thank you, I use use the new switch now Smiley

Thanks for finding this problem; let me know if find any others.

fullzero
Legendary
*
Offline Offline

Activity: 1092



View Profile
July 09, 2017, 03:59:37 AM
 #1668

With the 12x biostar nearly here, I don't think this is the right change to make.  If the biostar can actually run 12x GPUs then I will for sure; not make this alteration.  If it can only support 9x or less GPUs then I will consider making the change.

Speaking of 12gpu motherboards, I've got an ASRock 13 GPU board arriving next week.  Any thoughts on how to configure nvOC with an ASRock to get as many as possible of the GPUs running NVidia?  (my current board is an ASRock z270 that loads 6 GPUs without too much trouble, but the new board is H110, not Z270)  I've got 9 GPUs on hand to try - 8 1070s and a 980 ti.

In theory v0017+ can support 14 GPUs (13 + 1 via m2 adapter with that ASRock mobo).

But the chipset might not currently be supported and there are likely other system changes needed.  I need one of those mobos to test.  Where did you order one from?



https://www.aliexpress.com/item/New-in-BOX-ASRock-Technology-H110-PRO-BTC-H110-mining-board-support-13-graphics-card-DDR4/32822043486.html?spm=2114.13010608.0.0.GrsVzE

With the 12x biostar nearly here, I don't think this is the right change to make.  If the biostar can actually run 12x GPUs then I will for sure; not make this alteration.  If it can only support 9x or less GPUs then I will consider making the change.

Speaking of 12gpu motherboards, I've got an ASRock 13 GPU board arriving next week.  Any thoughts on how to configure nvOC with an ASRock to get as many as possible of the GPUs running NVidia?  (my current board is an ASRock z270 that loads 6 GPUs without too much trouble, but the new board is H110, not Z270)  I've got 9 GPUs on hand to try - 8 1070s and a 980 ti.

In theory v0017+ can support 14 GPUs (13 + 1 via m2 adapter with that ASRock mobo).

But the chipset might not currently be supported and there are likely other system changes needed.  I need one of those mobos to test.  Where did you order one from?

I got mine from Newegg.. according to ASRock, Newegg is expecting another shipment to arrive next week so you might want to setup an autonotify on the website.
He also did not provide any other distributors that I could check for stock, so newegg might be the only source for them right now.


https://www.newegg.com/Product/Product.aspx?Item=N82E16813157781

Thanks for links.    Smiley

pixelizedchaos
Newbie
*
Offline Offline

Activity: 18

You only die when you are forgotten.


View Profile
July 09, 2017, 04:04:29 AM
 #1669

So while I am not mining ZEC at the moment, when using nanopool with Nvoc, where would I add my email so I can change payout on nanopool.

Also could you elaborate on the Minimum profit = 0.0? is it like a percentage?

Btw on my toes for v0018!!! I can't wait super excited!

I use [OS] nvOC easy-to-use Linux Nvidia Mining <3
lbrasi
Newbie
*
Offline Offline

Activity: 25


View Profile
July 09, 2017, 04:10:17 AM
 #1670

I have had a lot of requests for this; so here is a new oneBash and modded switch file which implement full integration of SALFTER_NICEHASH_PROFIT_SWITCHING

see the OP for links:

Replace your current oneBash with the new one.

extract switch and move it to the:
Code:
/home/m1

directory

(the one which opens when you click the Files icon on the left)

configure the following in oneBash

Code:
SALFTER_NICEHASH_PROFIT_SWITCHING="YES"

# LOCAL will attach the mining process to the guake terminal
# REMOTE will leave it unattached / ready for SSH
LOCALorREMOTE="LOCAL"       # LOCAL  or  REMOTE

CURRENCY=USD
POWER_COST=0.10
MINIMUM_PROFIT=0.0
# this is salfters BTC address:
PAYMENT_ADDRESS=1TipsGocnz2N5qgAm9f7JLrsMqkb3oXe2
WORKER_NAME=nv$IP_AS_WORKER

daggerhashimoto_POWERLIMIT_WATTS=125
__daggerhashimoto_CORE_OVERCLOCK=100
daggerhashimoto_MEMORY_OVERCLOCK=100
_______daggerhashimoto_FAN_SPEED=75

equihash_POWERLIMIT_WATTS=125
__equihash_CORE_OVERCLOCK=100
equihash_MEMORY_OVERCLOCK=100
_______equihash_FAN_SPEED=75

neoscrypt_POWERLIMIT_WATTS=125
__neoscrypt_CORE_OVERCLOCK=100
neoscrypt_MEMORY_OVERCLOCK=100
_______neoscrypt_FAN_SPEED=75

lyra2rev2_POWERLIMIT_WATTS=125
__lyra2rev2_CORE_OVERCLOCK=100
lyra2rev2_MEMORY_OVERCLOCK=100
_______lyra2rev2_FAN_SPEED=75

lbry_POWERLIMIT_WATTS=125
__lbry_CORE_OVERCLOCK=100
lbry_MEMORY_OVERCLOCK=100
_______lbry_FAN_SPEED=75

pascal_POWERLIMIT_WATTS=125
__pascal_CORE_OVERCLOCK=100
pascal_MEMORY_OVERCLOCK=100
_______pascal_FAN_SPEED=75

remember to thank salfter if you use this  Smiley



Thanks for implementing this, but for some odd reason I keep getting two instances of the miner screen running which causes the system to crash, I will do some more testing to try and figure out what is going on.

EDIT: Actually the kill code does not seem to work causing multiple miner screens, this is how the system is crashing.

Are you using LOCAL or REMOTE?

Please walk me through how you got multiple instances running so I can recreate this myself.



I am using REMOTE.  I am connecting via SSH attaching the screen with "screen -r miner"
for testing purposes I am altering the algo speed manually in the switch file waiting 10 mins for it to change and I noticed it is hit and miss for the kill code to function correctly.  I am also experiencing for LBRY and LYRA the mining process is executed twice.

Maybe I should start from scratch...

I tested the same way; changing the speed of one algo to force a switch.  I also changed the timeout in oneBash to 10 seconds instead of 600.

I tested this out and found salfters logic works well for switching between Ethash and Equihash; but not any of the other algos.  

It does seem to endlessly spawn new ccminer instances as well.  

I don't want to spend a lot of time on this; so I implemented killing all mining processes every time: then launching a new one.  It should work without issue now.

I also edited the oneBash logic to conditionally add salfters IPv6 fix, and reattach the screen to the guake terminal every reinit when in local mode.

If you have added the cronjob:

Code:
0,10,20,30,40,50 * * * * (cd /media/m1/1263-A96E && python2.7 switch.py)

I would remove it; as the oneBash implementation doesn't use it: and it may be (most likely is) causing additional launches of salfters original switch.

Note: I uploaded a new oneBash and switch with these changes.  They are linked on the OP.



Thank you, I use use the new switch now Smiley

Thanks for finding this problem; let me know if find any others.


No problem I will let you know if I find anything else.  Also do you believe killing the miner every 10 mins will impact performance/hash rates reporting to nicehash? Can this possibly mean less of a payout?
RMH
Newbie
*
Offline Offline

Activity: 3


View Profile
July 09, 2017, 04:18:11 AM
 #1671

First off thanks for all the work Fullzero!  

I've been fighting with Win10 on my rig for a month.  Can't get it to recognize any more than 3 of my 1070 cards, and one always is disabled with an error.  So I've decided to dump Win10.

I've just got nvoc 0017 running with my old USB 2.0 stick on my ASUS Prime Z270-A with 6 Geforce GTX 1070s rig and I am not sure now how to get the mining started.  

I guess I was under the impression that it would start automatically on boot but I must have missed something in the setup.  It's just sitting there with the terminal screen open.  

Also in the Nvidia X server settings it's also only recognizing 3 of the 1070 gpus.  I'm hoping this is something that can be easily worked out as I'd hate to think that 3 of the 6 1070's I have only had for a month are bad.

Thanks in advance for any help.

If you had the same problem with windows only recognizing 3 out of 6 GPUs; this indicates there is most likely a hardware problem.

Have you tried swapping your risers?

How are you powering your risers?

What kind of risers are you using?

If you run with only 3x GPUs and it works:

swap out the 3 working GPUs with the 3 that aren't being recognized and see if they work.


Actually hardware wise I've already change motherboards thinking it was the MSI Z170A SLI Plus that I started out with.  I sent it back to Amazon earlier this week and ordered the Asus.

The risers are directly powered by the SATA cable from the PSU.  These are the risers... https://www.amazon.com/Extender-Powered-Extension-Adapter-Card-Currency/dp/B072MFBYCM/ref=sr_1_10?s=electronics&ie=UTF8&qid=1499386958&sr=1-10&keywords=PCIe%2BPowered%2BRiser%2BAdapter%2BCard%2B-%2BUSB%2B3.0%2BPCIe%2B1x%2Bto%2B16x%2BExtender%2BCard%2B-%2B6-Pin%2BPCI-E%2Bto%2BSATA%2BPower%2BCable&th=1

Yes I have swapped them around many time, but when I was on win10, haven't yet with nvoc as I just got it going tonight.  All of them worked in that testing.

Just a side note here....  I shut down the rig after I posted the first post and just powered it back a few mins ago.  Getting this scrolling on the terminal now...

bash: /media/m1/1263-A96E/oneBash: Permission denied
dos2unix: /media/m1/1263-A96E/oneBash: No such file or directory
dos2unix: Skipping /media/m1/1263-A96E/oneBash, not a regular file.

I have to laugh at myself cause I've obviously dorked something up I guess.

TenaciousJ
Full Member
***
Offline Offline

Activity: 123


View Profile
July 09, 2017, 04:34:21 AM
 #1672



Thanks for links.    Smiley

You bet..

If you get a second, i posted a while back about an issue I'm having in 0017 where one card runs at 66% capacity while the others are at 100%.  I'm just wondering if that's something you've seen before, and if there might be a straight forward solution to it.  If not, I'll probably just wait until I swap boards in a few days to mess with it since otherwise it's running solid as a rock with all 6 gpus.

fullzero
Legendary
*
Offline Offline

Activity: 1092



View Profile
July 09, 2017, 04:58:34 AM
 #1673

First off thanks for all the work Fullzero!  

I've been fighting with Win10 on my rig for a month.  Can't get it to recognize any more than 3 of my 1070 cards, and one always is disabled with an error.  So I've decided to dump Win10.

I've just got nvoc 0017 running with my old USB 2.0 stick on my ASUS Prime Z270-A with 6 Geforce GTX 1070s rig and I am not sure now how to get the mining started.  

I guess I was under the impression that it would start automatically on boot but I must have missed something in the setup.  It's just sitting there with the terminal screen open.  

Also in the Nvidia X server settings it's also only recognizing 3 of the 1070 gpus.  I'm hoping this is something that can be easily worked out as I'd hate to think that 3 of the 6 1070's I have only had for a month are bad.

Thanks in advance for any help.

If you had the same problem with windows only recognizing 3 out of 6 GPUs; this indicates there is most likely a hardware problem.

Have you tried swapping your risers?

How are you powering your risers?

What kind of risers are you using?

If you run with only 3x GPUs and it works:

swap out the 3 working GPUs with the 3 that aren't being recognized and see if they work.


Actually hardware wise I've already change motherboards thinking it was the MSI Z170A SLI Plus that I started out with.  I sent it back to Amazon earlier this week and ordered the Asus.

The risers are directly powered by the SATA cable from the PSU.  These are the risers... https://www.amazon.com/Extender-Powered-Extension-Adapter-Card-Currency/dp/B072MFBYCM/ref=sr_1_10?s=electronics&ie=UTF8&qid=1499386958&sr=1-10&keywords=PCIe%2BPowered%2BRiser%2BAdapter%2BCard%2B-%2BUSB%2B3.0%2BPCIe%2B1x%2Bto%2B16x%2BExtender%2BCard%2B-%2B6-Pin%2BPCI-E%2Bto%2BSATA%2BPower%2BCable&th=1

Yes I have swapped them around many time, but when I was on win10, haven't yet with nvoc as I just got it going tonight.  All of them worked in that testing.

Just a side note here....  I shut down the rig after I posted the first post and just powered it back a few mins ago.  Getting this scrolling on the terminal now...

bash: /media/m1/1263-A96E/oneBash: Permission denied
dos2unix: /media/m1/1263-A96E/oneBash: No such file or directory
dos2unix: Skipping /media/m1/1263-A96E/oneBash, not a regular file.

I have to laugh at myself cause I've obviously dorked something up I guess.

This usually means there is a problem with the image.

I would re image the USB key.

Let me know if that works.

fullzero
Legendary
*
Offline Offline

Activity: 1092



View Profile
July 09, 2017, 05:04:25 AM
 #1674

So while I am not mining ZEC at the moment, when using nanopool with Nvoc, where would I add my email so I can change payout on nanopool.

Also could you elaborate on the Minimum profit = 0.0? is it like a percentage?

Btw on my toes for v0018!!! I can't wait super excited!

add to the end of the worker a / then your email like so:

Code:
ZEC_WORKER="nv$IP_AS_WORKER/EMAILHERE"


Code:
MINIMUM_PROFIT=0.0

means mining will stop if there is no coin that is profitable for you to mine (profit at 0.0 or less)

If you want to stop mining if profit is at -1$ you would set

Code:
MINIMUM_PROFIT=-1

fullzero
Legendary
*
Offline Offline

Activity: 1092



View Profile
July 09, 2017, 05:11:09 AM
 #1675

I have had a lot of requests for this; so here is a new oneBash and modded switch file which implement full integration of SALFTER_NICEHASH_PROFIT_SWITCHING

see the OP for links:

Replace your current oneBash with the new one.

extract switch and move it to the:
Code:
/home/m1

directory

(the one which opens when you click the Files icon on the left)

configure the following in oneBash

Code:
SALFTER_NICEHASH_PROFIT_SWITCHING="YES"

# LOCAL will attach the mining process to the guake terminal
# REMOTE will leave it unattached / ready for SSH
LOCALorREMOTE="LOCAL"       # LOCAL  or  REMOTE

CURRENCY=USD
POWER_COST=0.10
MINIMUM_PROFIT=0.0
# this is salfters BTC address:
PAYMENT_ADDRESS=1TipsGocnz2N5qgAm9f7JLrsMqkb3oXe2
WORKER_NAME=nv$IP_AS_WORKER

daggerhashimoto_POWERLIMIT_WATTS=125
__daggerhashimoto_CORE_OVERCLOCK=100
daggerhashimoto_MEMORY_OVERCLOCK=100
_______daggerhashimoto_FAN_SPEED=75

equihash_POWERLIMIT_WATTS=125
__equihash_CORE_OVERCLOCK=100
equihash_MEMORY_OVERCLOCK=100
_______equihash_FAN_SPEED=75

neoscrypt_POWERLIMIT_WATTS=125
__neoscrypt_CORE_OVERCLOCK=100
neoscrypt_MEMORY_OVERCLOCK=100
_______neoscrypt_FAN_SPEED=75

lyra2rev2_POWERLIMIT_WATTS=125
__lyra2rev2_CORE_OVERCLOCK=100
lyra2rev2_MEMORY_OVERCLOCK=100
_______lyra2rev2_FAN_SPEED=75

lbry_POWERLIMIT_WATTS=125
__lbry_CORE_OVERCLOCK=100
lbry_MEMORY_OVERCLOCK=100
_______lbry_FAN_SPEED=75

pascal_POWERLIMIT_WATTS=125
__pascal_CORE_OVERCLOCK=100
pascal_MEMORY_OVERCLOCK=100
_______pascal_FAN_SPEED=75

remember to thank salfter if you use this  Smiley



Thanks for implementing this, but for some odd reason I keep getting two instances of the miner screen running which causes the system to crash, I will do some more testing to try and figure out what is going on.

EDIT: Actually the kill code does not seem to work causing multiple miner screens, this is how the system is crashing.

Are you using LOCAL or REMOTE?

Please walk me through how you got multiple instances running so I can recreate this myself.



I am using REMOTE.  I am connecting via SSH attaching the screen with "screen -r miner"
for testing purposes I am altering the algo speed manually in the switch file waiting 10 mins for it to change and I noticed it is hit and miss for the kill code to function correctly.  I am also experiencing for LBRY and LYRA the mining process is executed twice.

Maybe I should start from scratch...

I tested the same way; changing the speed of one algo to force a switch.  I also changed the timeout in oneBash to 10 seconds instead of 600.

I tested this out and found salfters logic works well for switching between Ethash and Equihash; but not any of the other algos.  

It does seem to endlessly spawn new ccminer instances as well.  

I don't want to spend a lot of time on this; so I implemented killing all mining processes every time: then launching a new one.  It should work without issue now.

I also edited the oneBash logic to conditionally add salfters IPv6 fix, and reattach the screen to the guake terminal every reinit when in local mode.

If you have added the cronjob:

Code:
0,10,20,30,40,50 * * * * (cd /media/m1/1263-A96E && python2.7 switch.py)

I would remove it; as the oneBash implementation doesn't use it: and it may be (most likely is) causing additional launches of salfters original switch.

Note: I uploaded a new oneBash and switch with these changes.  They are linked on the OP.



Thank you, I use use the new switch now Smiley

Thanks for finding this problem; let me know if find any others.


No problem I will let you know if I find anything else.  Also do you believe killing the miner every 10 mins will impact performance/hash rates reporting to nicehash? Can this possibly mean less of a payout?

It might make a very marginal difference; less so, the less frequently you check profitability. 

You can always increase the sleep time (its in seconds) in oneBash to whatever you want.  It is currently set to 700 seconds.

Code:
  if [ $LOCALorREMOTE == "LOCAL" ]
  then
  python2.7 '/home/m1/switch'
  guake -r SALFTER_NICEHASH_PROFIT_SWITCHING -e "screen -r miner"
  echo ""
  echo "mining process in Guake Tab"
  echo ""
  sleep 700
  fi

  if [ $LOCALorREMOTE == "REMOTE" ]
  then
  python2.7 '/home/m1/switch'
  echo "ENTER:"
  echo ""
  echo "screen -r miner"
  echo ""
  echo "in a terminal to view mining process"
  sleep 700
  fi

I will make the timeout seconds a variable to make it easier to change. 

Ultimately this will be implemented as intended; either by salfter, someone else or myself.



fullzero
Legendary
*
Offline Offline

Activity: 1092



View Profile
July 09, 2017, 05:12:49 AM
 #1676



Thanks for links.    Smiley

You bet..

If you get a second, i posted a while back about an issue I'm having in 0017 where one card runs at 66% capacity while the others are at 100%.  I'm just wondering if that's something you've seen before, and if there might be a straight forward solution to it.  If not, I'll probably just wait until I swap boards in a few days to mess with it since otherwise it's running solid as a rock with all 6 gpus.



Is the hashrate lower with the GPU running at 66%?

DJ ACK
Newbie
*
Offline Offline

Activity: 11


View Profile
July 09, 2017, 05:34:34 AM
 #1677

First off thanks for all the work Fullzero!  

I've been fighting with Win10 on my rig for a month.  Can't get it to recognize any more than 3 of my 1070 cards, and one always is disabled with an error.  So I've decided to dump Win10.

I've just got nvoc 0017 running with my old USB 2.0 stick on my ASUS Prime Z270-A with 6 Geforce GTX 1070s rig and I am not sure now how to get the mining started.  

I guess I was under the impression that it would start automatically on boot but I must have missed something in the setup.  It's just sitting there with the terminal screen open.  

Also in the Nvidia X server settings it's also only recognizing 3 of the 1070 gpus.  I'm hoping this is something that can be easily worked out as I'd hate to think that 3 of the 6 1070's I have only had for a month are bad.

Thanks in advance for any help.

If you had the same problem with windows only recognizing 3 out of 6 GPUs; this indicates there is most likely a hardware problem.

Have you tried swapping your risers?

How are you powering your risers?

What kind of risers are you using?

If you run with only 3x GPUs and it works:

swap out the 3 working GPUs with the 3 that aren't being recognized and see if they work.


Actually hardware wise I've already change motherboards thinking it was the MSI Z170A SLI Plus that I started out with.  I sent it back to Amazon earlier this week and ordered the Asus.

The risers are directly powered by the SATA cable from the PSU.  These are the risers... https://www.amazon.com/Extender-Powered-Extension-Adapter-Card-Currency/dp/B072MFBYCM/ref=sr_1_10?s=electronics&ie=UTF8&qid=1499386958&sr=1-10&keywords=PCIe%2BPowered%2BRiser%2BAdapter%2BCard%2B-%2BUSB%2B3.0%2BPCIe%2B1x%2Bto%2B16x%2BExtender%2BCard%2B-%2B6-Pin%2BPCI-E%2Bto%2BSATA%2BPower%2BCable&th=1

Yes I have swapped them around many time, but when I was on win10, haven't yet with nvoc as I just got it going tonight.  All of them worked in that testing.

Just a side note here....  I shut down the rig after I posted the first post and just powered it back a few mins ago.  Getting this scrolling on the terminal now...

bash: /media/m1/1263-A96E/oneBash: Permission denied
dos2unix: /media/m1/1263-A96E/oneBash: No such file or directory
dos2unix: Skipping /media/m1/1263-A96E/oneBash, not a regular file.

I have to laugh at myself cause I've obviously dorked something up I guess.



As fullzero mentioned the image or partioning on your USB stick may have issues.   Depends on how you installed the image.  I used Etcher in Ubuntu and had no problems (yes having Linux on another box saves you a lot of trouble).  When I installed nvOC 17 on my SSD, I had to change the 2Unix file.  By default the oneBash sits on the 9Gig partition.  I copied over to the home directory where the 2unix file sits and just changed the 2Unix file pointing it to the new location.  Problem solved.

Swapping out my MSI Z170 Gaming M5 boards for the ASUS Z270 Prime-A boards solved a lot of instability problems that I thought were riser related.  Additionally these boards are awesome for running 9 GPUs or more.  I have 9 x 1050Tis (1506 Boost Clock versions) running solid on these boards (0 Core OC, 1550 Mem OC, power limit 65, 133 MH/s per rig).
TenaciousJ
Full Member
***
Offline Offline

Activity: 123


View Profile
July 09, 2017, 06:07:02 AM
 #1678



Thanks for links.    Smiley

You bet..

If you get a second, i posted a while back about an issue I'm having in 0017 where one card runs at 66% capacity while the others are at 100%.  I'm just wondering if that's something you've seen before, and if there might be a straight forward solution to it.  If not, I'll probably just wait until I swap boards in a few days to mess with it since otherwise it's running solid as a rock with all 6 gpus.



Is the hashrate lower with the GPU running at 66%?

yeah by about 100 sol/sec or so

EDIT: actually by about 130 sol/sec, and the power going to the card is only 2/3 most of the time... 145-150w for other cards, and only 85-110w for that one.
S9k
Newbie
*
Offline Offline

Activity: 8


View Profile
July 09, 2017, 07:44:14 AM
 #1679

First off thanks for all the work Fullzero!  

I've been fighting with Win10 on my rig for a month.  Can't get it to recognize any more than 3 of my 1070 cards, and one always is disabled with an error.  So I've decided to dump Win10.

I've just got nvoc 0017 running with my old USB 2.0 stick on my ASUS Prime Z270-A with 6 Geforce GTX 1070s rig and I am not sure now how to get the mining started.  

I guess I was under the impression that it would start automatically on boot but I must have missed something in the setup.  It's just sitting there with the terminal screen open.  

Also in the Nvidia X server settings it's also only recognizing 3 of the 1070 gpus.  I'm hoping this is something that can be easily worked out as I'd hate to think that 3 of the 6 1070's I have only had for a month are bad.

Thanks in advance for any help.

If you had the same problem with windows only recognizing 3 out of 6 GPUs; this indicates there is most likely a hardware problem.

Have you tried swapping your risers?

How are you powering your risers?

What kind of risers are you using?

If you run with only 3x GPUs and it works:

swap out the 3 working GPUs with the 3 that aren't being recognized and see if they work.


Actually hardware wise I've already change motherboards thinking it was the MSI Z170A SLI Plus that I started out with.  I sent it back to Amazon earlier this week and ordered the Asus.

The risers are directly powered by the SATA cable from the PSU.  These are the risers... https://www.amazon.com/Extender-Powered-Extension-Adapter-Card-Currency/dp/B072MFBYCM/ref=sr_1_10?s=electronics&ie=UTF8&qid=1499386958&sr=1-10&keywords=PCIe%2BPowered%2BRiser%2BAdapter%2BCard%2B-%2BUSB%2B3.0%2BPCIe%2B1x%2Bto%2B16x%2BExtender%2BCard%2B-%2B6-Pin%2BPCI-E%2Bto%2BSATA%2BPower%2BCable&th=1

Yes I have swapped them around many time, but when I was on win10, haven't yet with nvoc as I just got it going tonight.  All of them worked in that testing.

Just a side note here....  I shut down the rig after I posted the first post and just powered it back a few mins ago.  Getting this scrolling on the terminal now...

bash: /media/m1/1263-A96E/oneBash: Permission denied
dos2unix: /media/m1/1263-A96E/oneBash: No such file or directory
dos2unix: Skipping /media/m1/1263-A96E/oneBash, not a regular file.

I have to laugh at myself cause I've obviously dorked something up I guess.

This usually means there is a problem with the image.

I would re image the USB key.

Let me know if that works.

hi, it is not problems of image.
these are problems of the FAT32 file system which falls after each incorrect reset. You can find such line in dmesg:

120.350515] FAT-fs (sda1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
I don't trust FAT32 and have transferred oneBash to other place.
At me it is impossible to stabilize my system in any way. I observe a large number of restarts which I can't analyse.
Syslog doesn't contain anything useful.

m1@m1-desktop:~$ last reboot
reboot   system boot  4.4.0-83-generic Sun Jul  9 04:36   still running
reboot   system boot  4.4.0-83-generic Sun Jul  9 00:00   still running
reboot   system boot  4.4.0-83-generic Sat Jul  8 17:50   still running
reboot   system boot  4.4.0-83-generic Sat Jul  8 15:31   still running
reboot   system boot  4.4.0-83-generic Sat Jul  8 14:56   still running
reboot   system boot  4.4.0-83-generic Sat Jul  8 14:48   still running
reboot   system boot  4.4.0-83-generic Sat Jul  8 12:58   still running
reboot   system boot  4.4.0-83-generic Sat Jul  8 10:10   still running
reboot   system boot  4.4.0-83-generic Sat Jul  8 09:54   still running
reboot   system boot  4.4.0-83-generic Sat Jul  8 08:27   still running
reboot   system boot  4.4.0-83-generic Sat Jul  8 03:16   still running
reboot   system boot  4.4.0-83-generic Fri Jul  7 23:31   still running
reboot   system boot  4.4.0-83-generic Fri Jul  7 22:52   still running
reboot   system boot  4.4.0-83-generic Fri Jul  7 20:02   still running
reboot   system boot  4.4.0-83-generic Fri Jul  7 16:26   still running
reboot   system boot  4.4.0-83-generic Fri Jul  7 10:00   still running


I need methodology of a solution. How to find the reason of a collapse of an operating system and restart. What video card exceeds admissible values and is unstable? I think it it will be interesting to many. Thank you very much!
TenaciousJ
Full Member
***
Offline Offline

Activity: 123


View Profile
July 09, 2017, 08:59:29 AM
 #1680

First off thanks for all the work Fullzero!  

I've been fighting with Win10 on my rig for a month.  Can't get it to recognize any more than 3 of my 1070 cards, and one always is disabled with an error.  So I've decided to dump Win10.

I've just got nvoc 0017 running with my old USB 2.0 stick on my ASUS Prime Z270-A with 6 Geforce GTX 1070s rig and I am not sure now how to get the mining started.  

I guess I was under the impression that it would start automatically on boot but I must have missed something in the setup.  It's just sitting there with the terminal screen open.  

Also in the Nvidia X server settings it's also only recognizing 3 of the 1070 gpus.  I'm hoping this is something that can be easily worked out as I'd hate to think that 3 of the 6 1070's I have only had for a month are bad.

Thanks in advance for any help.

If you had the same problem with windows only recognizing 3 out of 6 GPUs; this indicates there is most likely a hardware problem.

Have you tried swapping your risers?

How are you powering your risers?

What kind of risers are you using?

If you run with only 3x GPUs and it works:

swap out the 3 working GPUs with the 3 that aren't being recognized and see if they work.


Actually hardware wise I've already change motherboards thinking it was the MSI Z170A SLI Plus that I started out with.  I sent it back to Amazon earlier this week and ordered the Asus.

The risers are directly powered by the SATA cable from the PSU.  These are the risers... https://www.amazon.com/Extender-Powered-Extension-Adapter-Card-Currency/dp/B072MFBYCM/ref=sr_1_10?s=electronics&ie=UTF8&qid=1499386958&sr=1-10&keywords=PCIe%2BPowered%2BRiser%2BAdapter%2BCard%2B-%2BUSB%2B3.0%2BPCIe%2B1x%2Bto%2B16x%2BExtender%2BCard%2B-%2B6-Pin%2BPCI-E%2Bto%2BSATA%2BPower%2BCable&th=1

Yes I have swapped them around many time, but when I was on win10, haven't yet with nvoc as I just got it going tonight.  All of them worked in that testing.

Just a side note here....  I shut down the rig after I posted the first post and just powered it back a few mins ago.  Getting this scrolling on the terminal now...

bash: /media/m1/1263-A96E/oneBash: Permission denied
dos2unix: /media/m1/1263-A96E/oneBash: No such file or directory
dos2unix: Skipping /media/m1/1263-A96E/oneBash, not a regular file.

I have to laugh at myself cause I've obviously dorked something up I guess.

This usually means there is a problem with the image.

I would re image the USB key.

Let me know if that works.

hi, it is not problems of image.
these are problems of the FAT32 file system which falls after each incorrect reset. You can find such line in dmesg:

120.350515] FAT-fs (sda1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
I don't trust FAT32 and have transferred oneBash to other place.
At me it is impossible to stabilize my system in any way. I observe a large number of restarts which I can't analyse.
Syslog doesn't contain anything useful.

m1@m1-desktop:~$ last reboot
reboot   system boot  4.4.0-83-generic Sun Jul  9 04:36   still running
reboot   system boot  4.4.0-83-generic Sun Jul  9 00:00   still running
reboot   system boot  4.4.0-83-generic Sat Jul  8 17:50   still running
reboot   system boot  4.4.0-83-generic Sat Jul  8 15:31   still running
reboot   system boot  4.4.0-83-generic Sat Jul  8 14:56   still running
reboot   system boot  4.4.0-83-generic Sat Jul  8 14:48   still running
reboot   system boot  4.4.0-83-generic Sat Jul  8 12:58   still running
reboot   system boot  4.4.0-83-generic Sat Jul  8 10:10   still running
reboot   system boot  4.4.0-83-generic Sat Jul  8 09:54   still running
reboot   system boot  4.4.0-83-generic Sat Jul  8 08:27   still running
reboot   system boot  4.4.0-83-generic Sat Jul  8 03:16   still running
reboot   system boot  4.4.0-83-generic Fri Jul  7 23:31   still running
reboot   system boot  4.4.0-83-generic Fri Jul  7 22:52   still running
reboot   system boot  4.4.0-83-generic Fri Jul  7 20:02   still running
reboot   system boot  4.4.0-83-generic Fri Jul  7 16:26   still running
reboot   system boot  4.4.0-83-generic Fri Jul  7 10:00   still running


I need methodology of a solution. How to find the reason of a collapse of an operating system and restart. What video card exceeds admissible values and is unstable? I think it it will be interesting to many. Thank you very much!


I had a problem with the fat32 partition on my images.  I just had to reimage it... use HDD Raw Copy.  It doesn't ALWAYS work, but it does most of the time.  Also, make sure your flash drive or hard drive is bigger than 16gb.  I ran into a problem with mine where the drive was just a hair shy of the required size for the image and that caused partition file table problems that made the first part unreadable.

You could also use gparted to try to repair the partition which works sometimes.  Or use TestDisk if you're savvy with partition file table manipulation at all.

http://www.cgsecurity.org/wiki/TestDisk_Download



Pages: « 1 ... 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 [84] 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 ... 284 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!