Bitcoin Forum
June 25, 2018, 10:39:11 AM *
News: Latest stable version of Bitcoin Core: 0.16.1  [Torrent]. (New!)
 
   Home   Help Search Donate Login Register  
Pages: « 1 ... 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 [771] 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 »
  Print  
Author Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool  (Read 2559325 times)
frodocooper
Moderator
Full Member
*
Offline Offline

Activity: 225
Merit: 262


View Profile
May 21, 2017, 12:12:51 AM
 #15401

I'm still trying to understand how P2Pool works. How do I find a list of P2Pool servers? The idea is that I make a node and connect to a server, right? I can't quite visualize how all of this works, despite looking on Google for information visualizations. The image I saw on the Bitcoin wiki didn't make much sense, seemed too technical.

This P2Pool guide should be able to help. It's a lot more beginner-friendly than the Bitcoin wiki's article.
And here is an up-to-date list of public P2Pool nodes that you can connect your miners to, if you decide to not run your own P2Pool node.

There are multiple servers, right?

Yes, in a sense. There are multiple P2Pool nodes, and all of them make up the P2Pool network. It is very similar to how the Bitcoin network works, where multiple Bitcoin full nodes make up the Bitcoin network.

Is there like one main server? Or do a bunch of nodes together make the server?

P2Pool is a decentralized pool, similar to how Bitcoin is a decentralized network. There is therefore no main or central server, nor nodes that make up a main or central server. There is only the network of P2Pool nodes.

In other words, try to look at it according to what in2tactics said: each P2Pool node is its own pool. The P2Pool network connects these nodes together through the P2Pool sharechain, similar to how the Bitcoin network connects every Bitcoin full node together through the Bitcoin blockchain. P2Pool can therefore also be described as a collection of solo miners that pool block payouts and distribute them accordingly, since each P2Pool node is essentially doing its own thing. Contrast this to a traditional pool, where the central pool server dictates what its miners do.
1529923151
Hero Member
*
Offline Offline

Posts: 1529923151

View Profile Personal Message (Offline)

Ignore
1529923151
Reply with quote  #2

1529923151
Report to moderator
1529923151
Hero Member
*
Offline Offline

Posts: 1529923151

View Profile Personal Message (Offline)

Ignore
1529923151
Reply with quote  #2

1529923151
Report to moderator
1529923151
Hero Member
*
Offline Offline

Posts: 1529923151

View Profile Personal Message (Offline)

Ignore
1529923151
Reply with quote  #2

1529923151
Report to moderator
The World's Betting Exchange

Bet with play money. Win real Bitcoin. 5BTC Prize Fund for World Cup 2018.

Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1001


View Profile WWW
May 21, 2017, 09:28:23 PM
 #15402

Quick reminder: the jtoomimnet 1mb_hardforked branch of p2pool uses about 2x as much RAM, since the share chain contains around 2x as many transactions. If you are using pypy, this means that your memory consumption may get up to 6 GB. If you're using CPython, it's a little under 1 GB. Please make sure that you have enough RAM in your nodes. If you don't have enough RAM, you will get massive swapping, and you might notice that bitcoind is unable to keep up with blocks and may fall behind, causing p2pool to stop working.

Lowering RAM usage is my next goal for p2pool.

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
tubexc
Hero Member
*****
Offline Offline

Activity: 501
Merit: 500


View Profile
May 21, 2017, 11:41:22 PM
 #15403

Quick reminder: the jtoomimnet 1mb_hardforked branch of p2pool uses about 2x as much RAM, since the share chain contains around 2x as many transactions. If you are using pypy, this means that your memory consumption may get up to 6 GB. If you're using CPython, it's a little under 1 GB. Please make sure that you have enough RAM in your nodes. If you don't have enough RAM, you will get massive swapping, and you might notice that bitcoind is unable to keep up with blocks and may fall behind, causing p2pool to stop working.

Lowering RAM usage is my next goal for p2pool.
That's good.
And what about the p2pool ltc node testing and then the
merge into only one p2pool powerfull bitcoin sharechain?
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1001


View Profile WWW
May 22, 2017, 04:27:32 AM
 #15404

And what about the p2pool ltc node testing
LTC now requires SegWit, which requires veqtrus's PR. It would take some more work to get LTC working with both my code and veqtrus's. It would need to be a different alt for testing.

At the moment, I'm more interested in the memory issues and the share size issues than in doing testing and merging. I'd also like to add some code that checks the system clock against NTP (when reachable) at startup in order to reduce the clock offset issues that people have been having, and I think that might be good to do before merging into p2pool master.

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
Meuh6879
Legendary
*
Offline Offline

Activity: 1512
Merit: 1000



View Profile
May 22, 2017, 07:55:32 PM
 #15405

I'd also like to add some code that checks the system clock against NTP (when reachable) at startup in order to reduce the clock offset issues

use a preformated NTP list : http://www.prunoqi.com/~probruno/WORD/Liste%20des%20serveurs%20de%20temps%20SNTP%20disponibles%20sur%20Internet.htm
KorbinDallas
Jr. Member
*
Offline Offline

Activity: 55
Merit: 0


View Profile
May 25, 2017, 10:17:27 PM
 #15406

Queue the block dance guy ......
frodocooper
Moderator
Full Member
*
Offline Offline

Activity: 225
Merit: 262


View Profile
May 26, 2017, 10:37:50 AM
 #15407

Empty block by jtoomimnet. Sad
in2tactics
Hero Member
*****
Offline Offline

Activity: 518
Merit: 500



View Profile
May 26, 2017, 12:30:29 PM
 #15408

Empty block by jtoomimnet. Sad
Well, there was only 24 seconds between block 468172 and 468173.

Casual Miner: 3x 2PAC and 3x Moonlander 2
Retired HW: 2x AntMiner S7-LN, 5x AntMiner U1, 2x ASICMiner Block Erupter Cube, 4x AntMiner S3, 4x AntMiner S1, GAW Black Widow, and ZeusMiner Thunder X6
frodocooper
Moderator
Full Member
*
Offline Offline

Activity: 225
Merit: 262


View Profile
May 26, 2017, 01:01:30 PM
 #15409

Empty block by jtoomimnet. Sad
Well, there was only 24 seconds between block 468172 and 468173.

With 110 MB of transactions waiting in the Bitcoin mempool during that time. jtoomimnet should've been able to fill its transaction cache back up to capacity with the share that found block 468173, but apparently did not.
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1001


View Profile WWW
May 26, 2017, 06:53:12 PM
 #15410

It's more an issue of processing and propagation latency than mempool size. P2pool (along with most other pools) has code in it to generate empty blocks when the pool has heard of a new block header on the network, but has not yet gotten a block template from bitcoind. bitcoind can't generate a new block template until after it has fully downloaded and verified the previous block, since otherwise it might include transactions twice or include double-spends. Since downloading a block header is faster than downloading and verifying a full block, there can sometimes be a substantial amount of time in between where the best you can do is mine an empty block. The idea is that if you have a miner hashing on something, it's better to hash on an empty block at the right height than a full block at the wrong height (which would result in an orphan race). This condition should only last for a few seconds, though. The 23 actual seconds that elapsed seems excessive. It's possible that I need to do something to improve block propagation latency on that node. It's also possible that p2pool didn't poll getblocktemplate frequently enough or something.

If this happens when blocks are found in the first 30 15 seconds of mining, that would correspond to a 5% 2.5% reduction in average transactions included. That seems undesirable, but not excessive. The alternative -- mining on the wrong block height for up to 15 seconds -- sounds worse. I wish I had time to look into this more and improve performance, but at the moment I don't.

Edit: It appears that the timestamps on the blocks are wrong. This is pretty common, as generally timestamps are set when the stratum job is issued to the miner, not when the miner actually finds the block. Here's what shows up in my bitcoind logs:

Code:
2017-05-26 05:03:32 UpdateTip: new best=0000000000000000019c0b9124dd971ca31d59deac39f23cac484495895eb2af  height=468
172  log2_work=86.489404  tx=226059198  date=2017-05-26 05:02:56 progress=1.000000  cache=144.7MiB(44861tx)
2017-05-26 05:03:32 CreateNewBlock(): total size 999899 txs: 2197 fees: 296309213 sigops 7618
2017-05-26 05:03:35 Acceptable block: ver:20000000 time:1495775000 size: 1623 Tx:1 Sig:41
2017-05-26 05:03:35 UpdateTip: new best=00000000000000000117d37aba3cd8d9d1811f4f719834f5b8f71c78f8c504d7  height=468173  log2_work=86.489438  tx=226059199  date=2017-05-26 05:03:20 progress=1.000000  cache=144.7MiB(44877tx)

Then, in my p2pool logs:

Code:
2017-05-25 22:03:20.093931 Skipping from block 156cb130cb9ee78a01dd6d5ed4f144fb05c222f117829e7 to block 19c0b9124dd971ca31d59deac39f23cac484495895eb2af!
2017-05-25 22:03:20.102054 New work for 1PXxBrUbWUMZemAQknEqDTSHKaKmVmhJCK! Diff: 65681.48 Share diff: 9893593.66 Block value: 12.50 BTC (0 tx, 0 kB)
2017-05-25 22:03:34.275890 Generating a share with 998819 bytes (348901 new) and 2197 transactions (678 new)
2017-05-25 22:03:34.316686 New work for 1PXxBrUbWUMZemAQknEqDTSHKaKmVmhJCK! Diff: 69877.30 Share diff: 9893593.66 Block value: 15.46 BTC (2197 tx, 999 kB)
2017-05-25 22:03:34.384701 Generating a share with 998819 bytes (348901 new) and 2197 transactions (678 new)
2017-05-25 22:03:34.788256
2017-05-25 22:03:34.788319 GOT BLOCK FROM MINER! Passing to bitcoind! https://blockchain.info/block/00000000000000000117d37aba3cd8d9d1811f4f719834f5b8f71c78f8c504d7
2017-05-25 22:03:34.788340
2017-05-25 22:03:34.791621 GOT SHARE! 1EyWF5ZQ9BHxbLAKuFj2MfQT9daE1sVsTx f8c504d7 prev 91af17f0 age 14.68s DEAD ON ARRIVAL
2017-05-25 22:03:34.802307

It's worth noting that this was a DOA share that happened to be a block. P2pool learned of the block at height 468172 at 3m20s, bitcoind learned of it 12 seconds later at 3m32s and had work for p2pool around that time, but p2pool didn't have it bundled into a stratum job until 3m34.7s. While p2pool was bundling the stratum job, a miner returned the block from the previous stratum job, and p2pool didn't process it until after it finished assigning the new work because p2pool is single-threaded. This means that the block was found at some point between 3m32s and 3m34.7s.

This node was running CPython, not pypy, because that server is running low on RAM. This may have contributed to the slow (2.7s) processing of new work, and increases the likelihood of a DOA share.

So in this case, we're talking about a 15 second time window, not 30 or 24. Of those 15 seconds, 12 seconds were due to bitcoind being a bit slow, and 3 seconds were due to p2pool being slow. Overall, I think finding an empty block in this scenario is a reasonable outcome, and do not consider this to be a priority for optimization right now.

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
kano
Legendary
*
Offline Offline

Activity: 2492
Merit: 1042


Linux since 1997 RedHat 4


View Profile
May 26, 2017, 10:40:40 PM
 #15411

It's more an issue of processing and propagation latency than mempool size. P2pool (along with most other pools) has code in it to generate empty blocks when the pool has heard of a new block header on the network, but has not yet gotten a block template from bitcoind.
...
This is based on the fact that LukeJr is a crappy coder and came up with the idea that a block change with transactions is slow, so do not put transaction in the first work.

This is of course bullshit.

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1001


View Profile WWW
May 27, 2017, 02:49:29 AM
 #15412

I'm having trouble following your assertion. Is 14.68 seconds not slow to you? Or is it bullshit that it's so slow? Or is it bullshit to mine transaction-less blocks during those 14.68 seconds? Or is it just bullshit because Luke-jr was involved?

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
kano
Legendary
*
Offline Offline

Activity: 2492
Merit: 1042


Linux since 1997 RedHat 4


View Profile
May 27, 2017, 05:33:35 AM
 #15413

I'm having trouble following your assertion. Is 14.68 seconds not slow to you? Or is it bullshit that it's so slow? Or is it bullshit to mine transaction-less blocks during those 14.68 seconds? Or is it just bullshit because Luke-jr was involved?
Takes me less than 1 second - so I guess you have the same problem as LukeJr.
(less than 1 second includes: from block arrives, processed, work generated, new work sent out to miners)

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1001


View Profile WWW
May 27, 2017, 06:16:03 AM
 #15414

Takes me less than 1 second - so I guess you have the same problem as LukeJr.
(less than 1 second includes: from block arrives, processed, work generated, new work sent out to miners)
We're not counting the same way. In my example above, it took 12 seconds (after p2pool received the header) for the block to arrive, less than one second for the block to be validated and a new block template to be pushed to p2pool, and 2.7 seconds (because CPython p2pool is slow) for p2pool to issue new work to the miners. You're not counting what took 12 seconds in the case above.

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
kano
Legendary
*
Offline Offline

Activity: 2492
Merit: 1042


Linux since 1997 RedHat 4


View Profile
May 27, 2017, 09:22:49 AM
 #15415

Takes me less than 1 second - so I guess you have the same problem as LukeJr.
(less than 1 second includes: from block arrives, processed, work generated, new work sent out to miners)
We're not counting the same way. In my example above, it took 12 seconds (after p2pool received the header) for the block to arrive, less than one second for the block to be validated and a new block template to be pushed to p2pool, and 2.7 seconds (because CPython p2pool is slow) for p2pool to issue new work to the miners. You're not counting what took 12 seconds in the case above.
Thus you are saying there's some problem with p2pool receiving blocks.
That 12 seconds doesn't exist for me.

Edit: here's a link to make you worry more about p2pool ... https://poolbench.antminer.link/

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
HeroC
Legendary
*
Offline Offline

Activity: 862
Merit: 1000


GPG: FA122C1A | IRC: HeroCC


View Profile WWW
May 27, 2017, 03:11:18 PM
 #15416

Takes me less than 1 second - so I guess you have the same problem as LukeJr.
(less than 1 second includes: from block arrives, processed, work generated, new work sent out to miners)
We're not counting the same way. In my example above, it took 12 seconds (after p2pool received the header) for the block to arrive, less than one second for the block to be validated and a new block template to be pushed to p2pool, and 2.7 seconds (because CPython p2pool is slow) for p2pool to issue new work to the miners. You're not counting what took 12 seconds in the case above.
Thus you are saying there's some problem with p2pool receiving blocks.
That 12 seconds doesn't exist for me.

Edit: here's a link to make you worry more about p2pool ... https://poolbench.antminer.link/

Maybe it's just my internet, but that link you sent never actually loaded for me. It's just been loading for like a minute.

EDIT: Another minute later it loaded, nevermind
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1001


View Profile WWW
May 27, 2017, 11:29:49 PM
 #15417

Thus you are saying there's some problem with p2pool receiving blocks.
That 12 seconds doesn't exist for me.
No, the 12 seconds was due to my bitcoind process, not p2pool, as you would know if you had read all of my original post. I'm not running Falcon or FIBRE on my node, just theblumatt's old relay network, which is probably why it took 12 seconds. If you think it is bullshit that it took 12 seconds to download that block, then I'm inclined to agree with you. If you think that it's bullshit that p2pool choose to mine an empty block during those 12+2.7 seconds, then I'm inclined to disagree with you.

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
kano
Legendary
*
Offline Offline

Activity: 2492
Merit: 1042


Linux since 1997 RedHat 4


View Profile
May 27, 2017, 11:32:49 PM
 #15418

Thus you are saying there's some problem with p2pool receiving blocks.
That 12 seconds doesn't exist for me.
No, the 12 seconds was due to my bitcoind process, not p2pool, as you would know if you had read all of my original post. I'm not running Falcon or FIBRE on my node, just theblumatt's old relay network, which is probably why it took 12 seconds. If you think it is bullshit that it took 12 seconds to download that block, then I'm inclined to agree with you. If you think that it's bullshit that p2pool choose to mine an empty block during those 12+2.7 seconds, then I'm inclined to disagree with you.
No, I'm saying there is no reason to mine empty blocks due to the first work generated on a block change.
If your stats say otherwise, then you need to fix your stats, not mine empty blocks to overcome a crappy setup or crappy code.

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1001


View Profile WWW
May 29, 2017, 12:55:28 AM
 #15419

I heard from one of the Nicehash miners. He abandoned p2pool because the DOA rates he was getting from Nicehash were significantly eating into his revenue, and he can get more by solo mining. I can think of three ways of addressing this issue.

The first one is to submit DOA shares to the network and have them compete with non-DOA shares, and possibly change the preference algorithm on the receiver's side to give DOA shares a significant and fair chance of winning the subsequent orphan race. This approach would be likely to increase the possibility and severity of selfish mining attacks, so I'm a bit reluctant to pursue it. It would be pretty simple in terms of code changes, though. Some more detailed thoughts on this below.

The second one is to add an uncle mechanism or DAG structure to the share chain. This would likely be much fairer, but it would require a p2pool hard fork and a substantial amount of new code. This would be the best option if developer time weren't an issue.

The third one is to try to increase the average share interval by increasing the default or minimum share difficulty. Changes to the minimum share difficulty require a hard fork, but a simple one. Changes to the default share difficulty just require that some nodes (especially the large ones) run different code. If the average share took 4x as long, then both orphan and DOA rates would be 1/4th their current values. Drawbacks of this approach are (1) that small miners would find it more difficult to get shares into the share chain (unless share difficulty were a function of the number of shares in the chain), and (2) that it's potentially not incentive-compatible, as the benefit of making high-difficulty shares goes to everyone who mines afterwards, and not to the person who mines the high-diff share. A person with a lot of hashrate has a greater chance of winning an orphan race if he works on low-diff shares than if he works on high-diff shares, since p2pool essentially just uses (1) the share height and (2) the time of arrival when determining which share to base its work off of, and ignores difficulty.

But maybe that isn't the best way to choose the share to base work off of? Ultimately, the problem we're having with fairness comes from the fact that some shares have greater expected revenue in the share chain than other shares, due to differences in orphan risk caused by differences in node performance and propagation times. If we no longer take time-of-arrival into account (or weight it less heavily), then the incentive to run a large, high-performance p2pool node dissipates. That might sound like a bad thing, since it could result in a higher block orphan rate for p2pool, but it's not really that bad. The issue comes from the fact that Bitcoin block intervals are 600 seconds, but shares are 30 seconds, which means that we are punishing low-performance nodes by 20x more than is fair.

Another problem with the existing algorithm for choosing which share to base work off of is that all shares are treated the same, regardless of their difficulty, which makes low-diff shares have an advantage for selfish mining attacks. Large-diff shares contribute more work to p2pool, so they should have a greater chance of winning an orphan race, ceteris paribus. But you can't just pick the highest-diff share of all shares in an orphan race, since that would cause a race-to-the-top scenario where the optimal difficulty to use for your shares is the highest diff that a competitor uses plus one. That would also allow a selfish miner to reliably win orphan races against siblings: instead of working on a child share of the best known share, you could just orphan it by creating a sibling with +1 difficulty. So it needs to be probabilistic.

The algorithm I'm thinking of for choosing the winner of an orphan race is this:

1. Take all of the shares at the best chain height (using the current 5th-gen-parent work done heuristic)
2. Find the time that the first of those candidate shares arrived, and call that start_time.
3. For each share, calculate the difference between that share's arrival time and start_time, and call that share_delay.
4. Calculate share_nonorphan_chance = 1 - share_delay/600 (for Bitcoin) for each share. This gives a correction factor for how much less likely than the best share this share would have been to not be orphaned as a bitcoin block. That is, a share that took 6 seconds longer than the fastest share would get a 1% higher orphan risk, and would get a share_nonorphan_chance of 0.99.
5. Calculate share_corr_share_diff = share_diff * share_nonorphan_chance.
6. Pick a share randomly, giving a chance of share_corr_share_diff/sum(share_corr_share_diffs) for each share.
7. Mine using that share as the parent.

If my math/logic is correct, this algorithm would give pretty much everyone a fair chance of winning a share orphan race, regardless of their node performance or share difficulty.

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
LevinSwe
Newbie
*
Offline Offline

Activity: 11
Merit: 0


View Profile
May 29, 2017, 07:00:54 PM
 #15420

Message written in incorrect english with a touch of swe..
I has spend some time to watch bitcoin-q and p2pool's prompt window.. and summary is:

1. Sometimes, core gets a new block, verify it and then p2pool prompt "new work for workers.. (value13.77btc 1200tx..example)" within a second. No empty block mining. The thing is when core gets the block, it's somtimes about 5-20 seconds old already.

2. Sometimes p2pool prompt, "new work for worker.. (value 12.5btc 0tx) and nothing happens in core for a few seconds.. then the block show up, about 5-20 seconds old, gets verified and within a second p2pool prompt "new work for worker.. (value 13,77btc 1200tx)

I'm a machine programmer that heats my house with miners, not an expert. But I can't see how the empty block mining problem can be solved if not the local bitcoin core gets the block before some other remote p2pool node heard of the block and tells the local nod that there is a new block. I don't think the problem is between p2pool and btc core, is more an internet problem that core needs to have the block faster and before p2pool heard of it. Maybe if p2pool can relay blocks to other nodes, but why should that be faster then core get the block from an other core node?...

And the ophran and doa thing, it's the same amount we inside the pool share, if all involved in this gets about 10% ophran shares, it does not mean that we earn 10% less.. otherwise I had get it all wrong with the share thing and now I'm going to open a beer and paint a ceiling Smiley

40TH house heating system.
Pages: « 1 ... 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 [771] 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!