[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[zvs]

i got this behavior on v11 that i don't recall ever seeing before:

2013-01-02 21:24:57.170986 > Failure: twisted.internet.defer.TimeoutError: in GenericDeferrer
2013-01-02 21:39:20.329566 > Failure: twisted.internet.defer.TimeoutError: in GenericDeferrer
2013-01-02 21:55:23.730752 > Failure: twisted.internet.defer.TimeoutError: in GenericDeferrer

2013-01-02 21:24:48.933495  Shares: 82 (2 orphan, 3 dead) Stale rate: ~6.1% (2-14%) Efficiency: ~103.8% (95-108%) Current payout: 0.5881 BTC
2013-01-02 21:55:30.091965  Shares: 87 (5 orphan, 3 dead) Stale rate: ~9.2% (4-18%) Efficiency: ~103.7% (94-109%) Current payout: 0.5825 BTC

(ed: DOA rates were similar, 3% to 2.2%, 27 peers, 7 incoming)

and after noticing that, i reset, and here are my stats after 6 minutes

2013-01-02 22:02:16.745949 P2Pool: 18169 shares in chain (18173 verified/18174 total) Peers: 22 (2 incoming)
2013-01-02 22:02:16.746018  Local: 8137MH/s in last 5.8 minutes Local dead on arrival: ~2.2% (1-5%) Expected time to share: 6.2 minutes
2013-01-02 22:02:16.746042  Shares: 2 (0 orphan, 0 dead) Stale rate: ~0.0% (0-66%) Efficiency: ~114.2% (39-115%) Current payout: 0.5892 BTC
2013-01-02 22:02:16.746085  Pool: 337GH/s Stale rate: 12.4% Expected time to block: 10.5 hours

[kitsura]

After running overnight I see latency has improved alot for v11.

[spiccioli]

After running overnight I see latency has improved alot for v11.

It still leaks memory, after less than a day I'm already around 1 Gb of ram used.



spiccioli

[rav3n_pl]

I not understand, why ppl hacking code so bad...

Code:

2013-01-04 11:22:16.984000 Outgoing connection to peer 82.224.192.145:9323 established. p2pool version: 1100 '9.4-45-g0981bdf'
2013-01-04 11:22:32.818000 Outgoing connection to peer 94.224.37.118:51333 established. p2pool version: 1000 '10.0-dirty'
2013-01-04 11:22:33.972000 Outgoing connection to peer 91.235.254.37:9323 established. p2pool version: 1100 '11.0-2-g0981bdf'
2013-01-04 11:23:41.505000 Outgoing connection to peer 46.105.103.55:9323 established. p2pool version: 1100 '9.4-45-g0981bdf'
2013-01-04 15:07:00.230000 Outgoing connection to peer 81.83.16.111:51333 established. p2pool version: 1000 '10.0-dirty'

Find 2 bad hacks.
This is on TRC nodes, will check other ones later.
It is so easy to keep things updated, why messing in code this way? It is always bad way...

Edit:
On BTC I found:

Code:

connection to peer 71.227.175.139:9333 established. p2pool version: 1000 '3.1-236-g2dffea0'
connection to peer 84.208.55.32:9333 established. p2pool version: 1100 '9.0-140-g5c1786a'
connection to peer 71.227.175.139:58337 established. p2pool version: 1100 '3.1-244-g0981bdf'

And few funny "version: 1100 'madness'" and lots of "unknowns".

On LTC side only some unknowns.

[rav3n_pl]

After running overnight I see latency has improved alot for v11.

It still leaks memory, after less than a day I'm already around 1 Gb of ram used.



spiccioli

Odd.

Code:

Node uptime: 1.065 days 

  1  [||||||||                                         12.1%]     Tasks: 39; 2 running
  2  [||||||||||||||||||||||||||||||||                 57.1%]     Load average: 0.77 0.84 0.68
  Mem[|||||||||||||||||||||||||||||||||||||||||||3699/3954MB]     Uptime: 3 days, 03:33:15
  Swp[|||||                                       180/2043MB]

  PID USER      PRI  NI  VIRT   RES   SHR S CPU% MEM%   TIME+  Command
    1 root       20   0 24332  1004   280 S  0.0  0.0  0:01.67 /sbin/init
11944 rav3n      20   0 2773M 2106M  3196 S  0.0 53.3 14:29.55 ├─ i0coind
11226 rav3n      20   0  793M  238M  2148 S  1.0  6.0 11:13.91 ├─ ixcoind
11063 rav3n      20   0  732M  114M  2792 S  0.0  2.9 11:09.80 ├─ namecoind
11004 rav3n      20   0  792M  155M  2660 S  1.0  3.9 12:39.08 ├─ devcoind
10333 rav3n      20   0  892M  122M  2488 S  1.0  3.1 16:38.70 ├─ litecoind
10138 rav3n      20   0  823M 35996  2484 S  0.0  0.9  8:45.10 ├─ terracoind
 9908 rav3n      20   0 1032M  202M 10764 S  3.0  5.1 30:20.22 ├─ bitcoind
 9703 rav3n      20   0 24800   492   348 S  0.0  0.0  0:11.57 ├─ SCREEN -U -d -m python /home/rav3n/p2pool/run_p2pool.py --merge
 9704 rav3n      20   0  613M  333M  2028 R 62.0  8.4  2h43:54 │  └─ python /home/rav3n/p2pool/run_p2pool.py --merged http://rav3
 9678 rav3n      20   0 24800   524   348 S  0.0  0.0  0:10.26 ├─ SCREEN -U -d -m python /home/rav3n/p2pool/run_p2pool.py -f 0.5
 9679 rav3n      20   0  450M 97292  1996 S  1.0  2.4 31:38.35 │  └─ python /home/rav3n/p2pool/run_p2pool.py -f 0.5 --irc-announc
 9673 rav3n      20   0 24800   492   300 S  0.0  0.0  0:07.20 ├─ SCREEN -U -d -m python /home/rav3n/p2pool/run_p2pool.py -f 0.5
 9674 rav3n      20   0  394M  185M  2020 S  0.0  4.7 45:34.11 │  └─ python /home/rav3n/p2pool/run_p2pool.py -f 0.5 --irc-announc

[Prattler]

It still leaks memory, after less than a day I'm already around 1 Gb of ram used.
spiccioli

I can confirm this. 2 GB (RES) in 2-3 days and then I'm forced to restart.

[K1773R]

It still leaks memory, after less than a day I'm already around 1 Gb of ram used.
spiccioli

I can confirm this. 2 GB (RES) in 2-3 days and then I'm forced to restart.

650MB after 2 days, which platform are u on?

[mdude77]

It still leaks memory, after less than a day I'm already around 1 Gb of ram used.
spiccioli

I can confirm this. 2 GB (RES) in 2-3 days and then I'm forced to restart.

650MB after 2 days, which platform are u on?

At 360mb after 2d 12hr on win 7 x64 (64-bit python, not executable).

M

[zvs]

I not understand, why ppl hacking code so bad...
And few funny "version: 1100 'madness'" and lots of "unknowns".

On LTC side only some unknowns.

i liked my 'crazed hookers' better

it does the unknown stuff on a ramdisk, i just changed what it said

oh, and someone earlier mentioned something about dropping incoming connections to 2... i think that's probably too low, but you do have to check which incoming (and outgoing) peers you have, since it seems like ones that are, say, 200ms+ are detrimental

oh, and more.. it's really easy to find fast peers with plenty of bandwidth in europe, but i've killed off a few incoming connections until i finally got a couple of 110-130ms connections in US, a 120ms in canada, and some 140ms or so in russia...   i have a decent connection latency wise, but i'm limited to jack and shit for upstream here in rural texas, so....    those seemed not to fluctuate much though

[zvs]

It still leaks memory, after less than a day I'm already around 1 Gb of ram used.
spiccioli

I can confirm this. 2 GB (RES) in 2-3 days and then I'm forced to restart.

couldnt this be related to the increase in incoming connections?

my p2pool is at 300MB, limited to 10 outgoing and 6 incoming connections

my bitcoind is limited to 900 outgoing and 1000 total and it's at 4GB (well, ok, 3.5)

[mdude77]

It still leaks memory, after less than a day I'm already around 1 Gb of ram used.
spiccioli

I can confirm this. 2 GB (RES) in 2-3 days and then I'm forced to restart.

couldnt this be related to the increase in incoming connections?

my p2pool is at 300MB, limited to 10 outgoing and 6 incoming connections

my bitcoind is limited to 900 outgoing and 1000 total and it's at 4GB (well, ok, 3.5)

I have port forwarding completely turned off for p2pool.

M

[K1773R]

It still leaks memory, after less than a day I'm already around 1 Gb of ram used.
spiccioli

I can confirm this. 2 GB (RES) in 2-3 days and then I'm forced to restart.

couldnt this be related to the increase in incoming connections?

my p2pool is at 300MB, limited to 10 outgoing and 6 incoming connections

my bitcoind is limited to 900 outgoing and 1000 total and it's at 4GB (well, ok, 3.5)

i have no port forwarding and now after 2.5 days uptime with 50 outgoing connection it only uses 611MB!
this is clearly not p2pools fault, its the python version (or py2exe) fault. maybe its even more horrible on winblows.

[AV]

If I have more than one network, I can:

1. In a one network run the bitcoind+p2pool and connect to it from all other computers.
2. Every network to run a separate bitcoind+p2pool.

As will be better?

[mdude77]

It still leaks memory, after less than a day I'm already around 1 Gb of ram used.
spiccioli

I can confirm this. 2 GB (RES) in 2-3 days and then I'm forced to restart.

couldnt this be related to the increase in incoming connections?

my p2pool is at 300MB, limited to 10 outgoing and 6 incoming connections

my bitcoind is limited to 900 outgoing and 1000 total and it's at 4GB (well, ok, 3.5)

i have no port forwarding and now after 2.5 days uptime with 50 outgoing connection it only uses 611MB!
this is clearly not p2pools fault, its the python version (or py2exe) fault. maybe its even more horrible on winblows.

50 outgoing connections?  all manual?

Only have 6 here, 1 of which is manual.

Maybe that's the source of your memory usage.

M

[AV]

Holliday, thanks.

[K1773R]

It still leaks memory, after less than a day I'm already around 1 Gb of ram used.
spiccioli

I can confirm this. 2 GB (RES) in 2-3 days and then I'm forced to restart.

couldnt this be related to the increase in incoming connections?

my p2pool is at 300MB, limited to 10 outgoing and 6 incoming connections

my bitcoind is limited to 900 outgoing and 1000 total and it's at 4GB (well, ok, 3.5)

i have no port forwarding and now after 2.5 days uptime with 50 outgoing connection it only uses 611MB!
this is clearly not p2pools fault, its the python version (or py2exe) fault. maybe its even more horrible on winblows.

50 outgoing connections?  all manual?

Only have 6 here, 1 of which is manual.

Maybe that's the source of your memory usage.

M

611MB isnt alot! i did modify some parts (for example the connection one to allow 50 outgoing cons) to reduce my stale, im atm at 0.21% stale.
someone else did confirm they have heavy RAM Usage, my guess is that they use winblows with py2exe and therefore it leaks or to be correct, the unused RAM isnt being freed.

[mdude77]

It still leaks memory, after less than a day I'm already around 1 Gb of ram used.
spiccioli

I can confirm this. 2 GB (RES) in 2-3 days and then I'm forced to restart.

couldnt this be related to the increase in incoming connections?

my p2pool is at 300MB, limited to 10 outgoing and 6 incoming connections

my bitcoind is limited to 900 outgoing and 1000 total and it's at 4GB (well, ok, 3.5)

i have no port forwarding and now after 2.5 days uptime with 50 outgoing connection it only uses 611MB!
this is clearly not p2pools fault, its the python version (or py2exe) fault. maybe its even more horrible on winblows.

50 outgoing connections?  all manual?

Only have 6 here, 1 of which is manual.

Maybe that's the source of your memory usage.

M

611MB isnt alot! i did modify some parts (for example the connection one to allow 50 outgoing cons) to reduce my stale, im atm at 0.21% stale.
someone else did confirm they have heavy RAM Usage, my guess is that they use winblows with py2exe and therefore it leaks or to be correct, the unused RAM isnt being freed.

I'm at 387MB on windows 7 using python x64.

M

[K1773R]

I'm at 387MB on windows 7 using python x64.

M

so you use python and not py2exe? i think the py2exe is the faulty thing. one time allocated RAM cant be freed anymore maybe, this is usual behavior of such tools.
give it a try with py2exe, if u got the same "leak" (even it isnt a leak) we would know why this is happening.

[powersync]

Whats with our super crappy luck the past 5 days?  Does it have some thing to do with Stratum support in V10 and v11 released 5 days ago?

PS

[mdude77]

Whats with our super crappy luck the past 5 days?  Does it have some thing to do with Stratum support in V10 and v11 released 5 days ago?

PS

I was wondering the same.  A bit too coincidental.  According to my local graph, 92% are on v11, the rest are on v9.  Something up with v11, or is the mix of v9/v11 causing problems?

M