Bitcoin Forum
December 04, 2016, 08:24:11 PM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: [1] 2 »  All
  Print  
Author Topic: AMD Stream SDK 2.6 (Catalyst 11.12/12.1) - Get your performance back! (Phoenix)  (Read 13998 times)
deepceleron
Legendary
*
Offline Offline

Activity: 1470



View Profile WWW
January 04, 2012, 02:21:07 AM
 #1

I just did a whole bunch of benchmarking on my ATI Radeon 5830, with an overclocked core of 1060MHz. The goal was to find out all about the new AMD Stream SDK 2.6's OpenCL performance, and what miner optimizations may get performance back where it was with 2.5.775.2 and previous SDK runtimes.

The 5830 is basically a 5870 (the same chip die) with the 1600 stream processors cut down to 1120 (failed die yields or simply product segmentation). The stock core clock on the 5870 is 850MHz, while the 5830 is 800MHz.

My card's highest possible mhash with 2.4-2.5 are at 1060/366Mhz with Phoenix 1.7.2 (exe) phatk2 VECTORS AGGRESSION=13 FASTLOOP=False WORKSIZE=256. The odd 366MHz optimal RAM speed differs on 6xxx, 5850, and 57xx, where 300MHz is where the peak performance is usually found.

Lets see what we find, I was curious what Diapolo's new 2011-12-21 kernel could do, so lots of Phoenix 1.7.2 benchmarks with different kernels, worksize, and RAM speeds (the scaling is similar to standard phatk2), tests were ran for several minutes and shares (unless the setting was obviously poor), with the -a averaging option:

SDK 2.5/11.6
worksize:25612864
DiapoloVECTORS2366MHz340.87335.56324.79
DiapoloVECTORS21000MHz328.81335.56332.32
DiapoloVECTORS4366MHz200.94204.04200.97
DiapoloVECTORS41000MHz244.39250.09244.39
phatk2VECTORS366MHz345.32

Note: VECTORS3 is in the diapolo init file, but phoenix crashes with a python traceback error if it is used.

SDK 2.6/11.12 (12.1 is identical)
worksize:25612864
DiapoloVECTORS2366MHz332.99327.86321.16
DiapoloVECTORS21000MHz326.03327.90325.99
DiapoloVECTORS4366MHz219298329.74
DiapoloVECTORS41000MHz255278216
phatk2VECTORS366MHz307.37306.78297.25
phatk2VECTORS1000MHz300.87304.61298.81
phatk2VECTORS4366MHz217.42289.25288.78
phatk2VECTORS41000MHz262.69339.5340.21
phatkVECTORS366MHz326323.5315.3
phatkVECTORS1000MHz317321.2320.5

Conclusions:
-AMD seems to have fixed the "bug" where underclocking memory is required for best performance; there is no massive drop at full RAM clock speed in most cases.
-Performance using phatk2/VECTORS4/WORKSIZE 64 is 99% of the old maximum, but with the video card using more power because of not underclocking RAM
-The new driver/SDK fixes the CPU bug; maybe you will break even in power usage by not using 100% CPU too...
-The stock 1000MHz RAM speed is the optimum speed, from 300MHz-1200MHz, the highest performance was at 1000MHz, and no further tweaking could get more out of the best setting.
-If you need the latest driver, play games, don't want to underclock, now you have your setting! Dedicated miners still get more hashrate for less wattage with SDK 2.5 (until OpenCL kernel tweakers can figure out how to get even more out of 2.6...)

Soon to come: recompiling pyOpenCL against the 2.6 SDK. jedi95 might try this for his exe distribution. pyOpenCL-0.98 seems to get about 1% more than the newest pyOpenCL-2011.2, BTW, and appears to be the version rolled into the exe. done: no difference. Be sure to use PyOpenCL 0.98 for best performance if running from source/Linux.

edit: My 5770 give similar results - it's default memory clock is 1200MHz, but the performance peak is at 1000MHz. VECTORS4 AGGRESSION=12 FASTLOOP=False WORKSIZE=128 gives me 229Mhash/s, vs the best before at 230Mhash/s.



Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1480883051
Hero Member
*
Offline Offline

Posts: 1480883051

View Profile Personal Message (Offline)

Ignore
1480883051
Reply with quote  #2

1480883051
Report to moderator
1480883051
Hero Member
*
Offline Offline

Posts: 1480883051

View Profile Personal Message (Offline)

Ignore
1480883051
Reply with quote  #2

1480883051
Report to moderator
grue
Global Moderator
Legendary
*
Offline Offline

Activity: 1932



View Profile
January 04, 2012, 02:40:15 AM
 #2

tl;dr:
mine while gaming: 2.6
mine while browsing: 2.1-2.5

It is pitch black. You are likely to be eaten by a grue.

Tired of annoying signature ads? Ad block for signatures
deepceleron
Legendary
*
Offline Offline

Activity: 1470



View Profile WWW
January 04, 2012, 02:43:53 AM
 #3

tl;dr:
mine while gaming: 2.6
mine while browsing: 2.1-2.5

If you didn't read, you can get the highest gaming and mining performance with 2.6 by using optimal miner settings. No need to switch SDK or underclock.

DiabloD3
Legendary
*
Offline Offline

Activity: 1162


DiabloMiner author


View Profile WWW
January 04, 2012, 03:41:40 PM
 #4

I just did a whole bunch of benchmarking on my ATI Radeon 5830, with an overclocked core of 1060MHz. The goal was to find out all about the new AMD Stream SDK 2.6's OpenCL performance, and what miner optimizations may get performance back where it was with 2.5.775.2 and previous SDK runtimes.

The 5830 is basically a 5870 (the same chip die) with the 1600 stream processors cut down to 1120 (failed die yields or simply product segmentation). The stock core clock on the 5870 is 850MHz, while the 5830 is 800MHz.

My card's highest possible mhash with 2.4-2.5 are at 1060/366Mhz with Phoenix 1.7.2 (exe) phatk2 VECTORS AGGRESSION=13 FASTLOOP=False WORKSIZE=256. The odd 366MHz optimal RAM speed differs on 6xxx, 5850, and 57xx, where 300MHz is where the peak performance is usually found.

Lets see what we find, I was curious what Diapolo's new 2011-12-21 kernel could do, so lots of Phoenix 1.7.2 benchmarks with different kernels, worksize, and RAM speeds (the scaling is similar to standard phatk2), tests were ran for several minutes and shares (unless the setting was obviously poor), with the -a averaging option:

SDK 2.5/11.6
worksize:25612864
DiapoloVECTORS2366MHz340.87335.56324.79
DiapoloVECTORS21000MHz328.81335.56332.32
DiapoloVECTORS4366MHz200.94204.04200.97
DiapoloVECTORS41000MHz244.39250.09244.39
phatk2VECTORS366MHz345.32

Note: VECTORS3 is in the diapolo init file, but phoenix crashes with a python traceback error if it is used.

SDK 2.6/11.12 (12.1 is identical)
worksize:25612864
DiapoloVECTORS2366MHz332.99327.86321.16
DiapoloVECTORS21000MHz326.03327.90325.99
DiapoloVECTORS4366MHz219298329.74
DiapoloVECTORS41000MHz255278216
phatk2VECTORS366MHz307.37306.78297.25
phatk2VECTORS1000MHz300.87304.61298.81
phatk2VECTORS4366MHz217.42289.25288.78
phatk2VECTORS41000MHz262.69339.5340.21
phatkVECTORS366MHz326323.5315.3
phatkVECTORS1000MHz317321.2320.5

Conclusions:
-AMD seems to have fixed the "bug" where underclocking memory is required for best performance; there is no massive drop at full RAM clock speed in most cases.
-Performance using phatk2/VECTORS4/WORKSIZE 64 is 99% of the old maximum, but with the video card using more power because of not underclocking
-The new driver/SDK fixes the CPU bug; maybe you break even in power usage by not using 100% CPU too...
-The stock 1000MHz RAM speed is the optimum speed, from 300MHz-1200MHz, the highest performance was at 1000MHz, and no further tweaking could get more out of the best setting.
-If you need the latest driver, play games, don't want to underclock, now you have your setting! Dedicated miners still get more hashrate for less wattage with SDK 2.5 (until OpenCL kernel tweakers can figure out how to get even more out of 2.6...)

Soon to come: recompiling pyOpenCL against the 2.6 SDK. jedi95 might try this for his exe distribution. pyOpenCL-0.98 seems to get about 1% more than the newest pyOpenCL-2011.2, BTW, and appears to be the version rolled into the exe.

edit: My 5770 give similar results - it's default memory clock is 1200MHz, but the performance peak is at 1000MHz. VECTORS4 AGGRESSION=12 FASTLOOP=False WORKSIZE=128 gives me 229Mhash/s, vs the best before at 230Mhash/s.




I found the exact same results on DiabloMiner a few days ago. -v 4 -w 64 seems to give almost the same results, but not quite as much. However, I did the test with underclocked RAM, not full speed. However, I still recommend 2.1 on Radeon 5xxx hardware instead of 2.5, as that gets you the full performance.

grue
Global Moderator
Legendary
*
Offline Offline

Activity: 1932



View Profile
January 06, 2012, 10:59:19 PM
 #5

If you didn't read, you can get the highest gaming and mining performance with 2.6 by using optimal miner settings. No need to switch SDK or underclock.
lolwut?

Quote
2.5
phatk2   VECTORS   366MHz   345.32      
2.6
phatk2   VECTORS4   1000MHz   262.69   339.5   340.21
plus 2.5 allows to get max speeds at lower clocks = more efficiency.

It is pitch black. You are likely to be eaten by a grue.

Tired of annoying signature ads? Ad block for signatures
disclaimer201
Legendary
*
Offline Offline

Activity: 1316


View Profile
January 08, 2012, 01:56:06 PM
 #6

Would love to see some such tests for poclbm (Gui) versions as well! The performance loss from 2.5 to 2.6 is very noticeable, but burning up CPU cores over time and spending more electricity for the 100% bug isn't good, either.

wndrbr3d
Sr. Member
****
Offline Offline

Activity: 295


View Profile
January 09, 2012, 05:05:29 PM
 #7

I'm having issues keeping my 2nd GPU busy.  Huh

Using the above method with the updated miner script for 2.6, I'm able to keep GPU1 at 99% utilization, but GPU2 bounces between 91-98% on my graph. I've put both on their own CPU to see if that would help but it doesn't seem to have an impact.

GUIMiner Settings on both:
-k phatk AGGRESSION=12 FASTLOOP=false VECTORS2 WORKSIZE=64

Has anyone tried the above with multiple GPU's??
DiabloD3
Legendary
*
Offline Offline

Activity: 1162


DiabloMiner author


View Profile WWW
January 09, 2012, 05:55:44 PM
 #8

I'm having issues keeping my 2nd GPU busy.  Huh

Using the above method with the updated miner script for 2.6, I'm able to keep GPU1 at 99% utilization, but GPU2 bounces between 91-98% on my graph. I've put both on their own CPU to see if that would help but it doesn't seem to have an impact.

GUIMiner Settings on both:
-k phatk AGGRESSION=12 FASTLOOP=false VECTORS2 WORKSIZE=64

Has anyone tried the above with multiple GPU's??

Just switch to DiabloMiner or cgminer already.

wndrbr3d
Sr. Member
****
Offline Offline

Activity: 295


View Profile
January 09, 2012, 06:32:16 PM
 #9

Just switch to DiabloMiner or cgminer already.

Thanks for the troll!  Roll Eyes

cgminer hasn't implemented the fixes yet that allow me to get back to full speed after installing 12.1, so until that happens, I need to figure out a way to get this working right.

Fastest I can get of cgminer is 265 Mhash/sec, where as this fix gets back back up to the 290's. Additionally, the latest kernel from Diapolo won't compile in cgminer.

So any ACTUAL help is welcome.

Cheers! Grin Grin
DiabloD3
Legendary
*
Offline Offline

Activity: 1162


DiabloMiner author


View Profile WWW
January 09, 2012, 07:49:13 PM
 #10

Just switch to DiabloMiner or cgminer already.

Thanks for the troll!  Roll Eyes

cgminer hasn't implemented the fixes yet that allow me to get back to full speed after installing 12.1, so until that happens, I need to figure out a way to get this working right.

Fastest I can get of cgminer is 265 Mhash/sec, where as this fix gets back back up to the 290's. Additionally, the latest kernel from Diapolo won't compile in cgminer.

So any ACTUAL help is welcome.

Cheers! Grin Grin

Diapolo seems to be missing optimizations DiabloMiner has, and DiabloMiner works on 79xx (although in an unoptimized state). Some guy has already gotten 666 mhash out of his 7970 on it.

As for the 2.6 bug, on DiabloMiner, try -v 36 -w 64 with memory at full speed, or try your normal settings with memory at full speed. It should get at least most of it back. 2.6 seems to actually be using the hardware correctly, or at least doing something that requires much tighter memory latency.

Or, otoh, you can slink back to 2.1 or 2.5, the slowness is SDK related.

disclaimer201
Legendary
*
Offline Offline

Activity: 1316


View Profile
January 09, 2012, 09:55:21 PM
 #11

Will the CPU bug be gone with reversing to 2.1? How do I do that when I'm on 2.6 already? Heard all over the board going back isn't easy AT ALL.

Thx!

DiabloD3
Legendary
*
Offline Offline

Activity: 1162


DiabloMiner author


View Profile WWW
January 09, 2012, 10:07:58 PM
 #12

Will the CPU bug be gone with reversing to 2.1? How do I do that when I'm on 2.6 already? Heard all over the board going back isn't easy AT ALL.

Thx!

The CPU use bug currently is a flaw of the driver, not the SDK. I currently have no issues on 11.12 combined with any SDK, yet 10.7 through 11.10 cause me problems. SDK 2.2 and 2.3, however, had an identical CPU use bug and were the fault of the SDK (2.1 didn't do it, 2.2 and 2.3 do it with any driver revision); they may be identical bugs, but the source is different than the one people suffer from now. Don't get them confused.

On Windows, it is difficult to revert SDKs, on Linux it is easy. So, ymmv.

mikemacman
Newbie
*
Offline Offline

Activity: 7


View Profile
January 12, 2012, 01:29:19 AM
 #13

Has anyone tried this on a 5850? I'm curious if the same settings would be optimal?
deepceleron
Legendary
*
Offline Offline

Activity: 1470



View Profile WWW
January 14, 2012, 03:43:02 PM
 #14

I'm having issues keeping my 2nd GPU busy.  Huh

Using the above method with the updated miner script for 2.6, I'm able to keep GPU1 at 99% utilization, but GPU2 bounces between 91-98% on my graph. I've put both on their own CPU to see if that would help but it doesn't seem to have an impact.

GUIMiner Settings on both:
-k phatk AGGRESSION=12 FASTLOOP=false VECTORS2 WORKSIZE=64

Has anyone tried the above with multiple GPU's??

On Vista and Win7, try the command line
Start /REALTIME phoenix.exe ...
This sets the miner to the highest possible process priority, ensuring that other OS background stuff don't steal any CPU time from the miner sending work to your GPU.

Additionally, after the realtime option, you can add the command line option setting single-CPU affinity for each miner. /AFFINITY 01 = first CPU core
/AFFINITY 02 = second CPU core
/AFFINITY 04 = third CPU core
/AFFINITY 08 = fourth CPU core

Finally, if you have two console windows open, each with a miner, the selected/highlighted one will get more CPU. Click on the wallpaper so neither are selected. Alternately, under Control Panel -> System -> Advanced -> Performance settings, you can optimize for "background services" instead of "Programs" to avoid a foreground window hogging resources.

zvs
Legendary
*
Offline Offline

Activity: 1386



View Profile WWW
January 21, 2012, 07:17:52 PM
 #15

My card's highest possible mhash with 2.4-2.5 are at 1060/366Mhz with Phoenix 1.7.2 (exe) phatk2 VECTORS AGGRESSION=13 FASTLOOP=False WORKSIZE=256. The odd 366MHz optimal RAM speed differs on 6xxx, 5850, and 57xx, where 300MHz is where the peak performance is usually found.

Lets see what we find, I was curious what Diapolo's new 2011-12-21 kernel could do, so lots of Phoenix 1.7.2 benchmarks with different kernels, worksize, and RAM speeds (the scaling is similar to standard phatk2), tests were ran for several minutes and shares (unless the setting was obviously poor), with the -a averaging option:

SDK 2.6/11.12 (12.1 is identical)
worksize:25612864
DiapoloVECTORS2366MHz332.99327.86321.16
DiapoloVECTORS21000MHz326.03327.90325.99
DiapoloVECTORS4366MHz219298329.74
DiapoloVECTORS41000MHz255278216
phatk2VECTORS366MHz307.37306.78297.25
phatk2VECTORS1000MHz300.87304.61298.81
phatk2VECTORS4366MHz217.42289.25288.78
phatk2VECTORS41000MHz262.69339.5340.21
phatkVECTORS366MHz326323.5315.3
phatkVECTORS1000MHz317321.2320.5


5830, SDK 2.6 (10.0.851.6)

Diapolo from December 12th, 2011

1060 core, 385 memory (phoenix 1.7.2)

phoenix -k phatk device=1 AGGRESSION=11 fastloop=false VECTORS4 WORKSIZE=64 -a 1000

332.96

395 raises it another point or so.  Never tried 399.

Not going to run my card at 1060 core and 1000 memory setting.  I imagine the ~2% gain you have listed here is more than offset by energy costs associated w/ cooling (and wear and tear on the card, theoretically resulting in faster depreciation of the value of the equipment)

I usually run it at 1000/385.

(ed: for comparison, back in June or so when I put my score on the 'Mining Hardware comparison' wiki page, I got 334, with 1030 core and 385 memory...   but it's worth it for a multi-card setup, anyway...  don't have that core getting smashed 100% anymore)

(ed2:  same thing at 399 memory setting - [335.09 Mhash/sec] [18 Accepted] [0 Rejected] [RPC (+LP)])


Dacentec, best deals for US dedicated servers. They regularly restock $20-$25 Opterons with 8-16GB RAM & 2x1-2TB HDD's (ofc, usually lots of other good stuff to choose from).  I did a Serverbear benchmark of one of my $20/mo Opteron (June last year), it's here.  Have had about a half dozen different servers with Dacentec, & none have failed to sustain at least 40MB/s (burst higher). My favorite is a 12-month rent-to-own ZT Systems 2XL5520 16GB 2x2TB SATA for $40/month (got lucky with the 'off-brand', haven't seen a RTO 2xL5520 for under $50/mo since -- at least for monthly contracts).  wholesaleinternet.com has some ancient 2-core intel CPUs @ $10/mo sometimes (I got an Intel Core 2 6300 @ 1.86GHz, with a 250GB HDD with 46000 hours on it, LOL. $20 @ Dacentec is much better, if you can grab one). joesdatacenter.com (same location as Wholesale Internet) also occasionally has specials (or if you don't want to wait, it has an AMD Opteron 170 @ $16/mo).
zvs
Legendary
*
Offline Offline

Activity: 1386



View Profile WWW
January 21, 2012, 07:50:11 PM
 #16

I'm having issues keeping my 2nd GPU busy.  Huh

Using the above method with the updated miner script for 2.6, I'm able to keep GPU1 at 99% utilization, but GPU2 bounces between 91-98% on my graph. I've put both on their own CPU to see if that would help but it doesn't seem to have an impact.

GUIMiner Settings on both:
-k phatk AGGRESSION=12 FASTLOOP=false VECTORS2 WORKSIZE=64

Has anyone tried the above with multiple GPU's??

Just switch to DiabloMiner or cgminer already.

i still use phoenix rising  v1.3, hoho

Dacentec, best deals for US dedicated servers. They regularly restock $20-$25 Opterons with 8-16GB RAM & 2x1-2TB HDD's (ofc, usually lots of other good stuff to choose from).  I did a Serverbear benchmark of one of my $20/mo Opteron (June last year), it's here.  Have had about a half dozen different servers with Dacentec, & none have failed to sustain at least 40MB/s (burst higher). My favorite is a 12-month rent-to-own ZT Systems 2XL5520 16GB 2x2TB SATA for $40/month (got lucky with the 'off-brand', haven't seen a RTO 2xL5520 for under $50/mo since -- at least for monthly contracts).  wholesaleinternet.com has some ancient 2-core intel CPUs @ $10/mo sometimes (I got an Intel Core 2 6300 @ 1.86GHz, with a 250GB HDD with 46000 hours on it, LOL. $20 @ Dacentec is much better, if you can grab one). joesdatacenter.com (same location as Wholesale Internet) also occasionally has specials (or if you don't want to wait, it has an AMD Opteron 170 @ $16/mo).
zvs
Legendary
*
Offline Offline

Activity: 1386



View Profile WWW
February 10, 2012, 01:27:46 PM
 #17

I finally got a 5870 to mess around with, and, yeah, the "best" memory setting on that is diff than the 5830. 

At 1000/395, it gets 440mhash, at 1000/310, 448mhash, etc.

Dacentec, best deals for US dedicated servers. They regularly restock $20-$25 Opterons with 8-16GB RAM & 2x1-2TB HDD's (ofc, usually lots of other good stuff to choose from).  I did a Serverbear benchmark of one of my $20/mo Opteron (June last year), it's here.  Have had about a half dozen different servers with Dacentec, & none have failed to sustain at least 40MB/s (burst higher). My favorite is a 12-month rent-to-own ZT Systems 2XL5520 16GB 2x2TB SATA for $40/month (got lucky with the 'off-brand', haven't seen a RTO 2xL5520 for under $50/mo since -- at least for monthly contracts).  wholesaleinternet.com has some ancient 2-core intel CPUs @ $10/mo sometimes (I got an Intel Core 2 6300 @ 1.86GHz, with a 250GB HDD with 46000 hours on it, LOL. $20 @ Dacentec is much better, if you can grab one). joesdatacenter.com (same location as Wholesale Internet) also occasionally has specials (or if you don't want to wait, it has an AMD Opteron 170 @ $16/mo).
DiabloD3
Legendary
*
Offline Offline

Activity: 1162


DiabloMiner author


View Profile WWW
February 10, 2012, 01:29:54 PM
 #18

I finally got a 5870 to mess around with, and, yeah, the "best" memory setting on that is diff than the 5830. 

At 1000/395, it gets 440mhash, at 1000/310, 448mhash, etc.

1/3rd core speed is considered the rule. So, in your case, 1000/333.

zvs
Legendary
*
Offline Offline

Activity: 1386



View Profile WWW
February 10, 2012, 01:37:30 PM
 #19

I finally got a 5870 to mess around with, and, yeah, the "best" memory setting on that is diff than the 5830. 

At 1000/395, it gets 440mhash, at 1000/310, 448mhash, etc.

1/3rd core speed is considered the rule. So, in your case, 1000/333.
Small test size, but results were similar to 1000/310.  I know if I set my 5830 to 1000/333 instead of 1000/395, it drops it quite noticeably (agg 9, ~316mhash @ 395 to ~305mhash @ 333)

Dacentec, best deals for US dedicated servers. They regularly restock $20-$25 Opterons with 8-16GB RAM & 2x1-2TB HDD's (ofc, usually lots of other good stuff to choose from).  I did a Serverbear benchmark of one of my $20/mo Opteron (June last year), it's here.  Have had about a half dozen different servers with Dacentec, & none have failed to sustain at least 40MB/s (burst higher). My favorite is a 12-month rent-to-own ZT Systems 2XL5520 16GB 2x2TB SATA for $40/month (got lucky with the 'off-brand', haven't seen a RTO 2xL5520 for under $50/mo since -- at least for monthly contracts).  wholesaleinternet.com has some ancient 2-core intel CPUs @ $10/mo sometimes (I got an Intel Core 2 6300 @ 1.86GHz, with a 250GB HDD with 46000 hours on it, LOL. $20 @ Dacentec is much better, if you can grab one). joesdatacenter.com (same location as Wholesale Internet) also occasionally has specials (or if you don't want to wait, it has an AMD Opteron 170 @ $16/mo).
Joshwaa
Hero Member
*****
Offline Offline

Activity: 489



View Profile
February 10, 2012, 01:56:23 PM
 #20

Will the CPU bug be gone with reversing to 2.1? How do I do that when I'm on 2.6 already? Heard all over the board going back isn't easy AT ALL.

Thx!

Actually Quite easy.. Let me know if you want help with it.

Like what I said : 1JosHWaA2GywdZo9pmGLNJ5XSt8j7nzNiF
Don't like what I said : 1FuckU1u89U9nBKQu4rCHz16uF4RhpSTV
Don't Like BFL's Project Management : 1FuckbFLZpmWLuyHyFJw1RGkWm3yRM1L5D
Pages: [1] 2 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!