Bitcoin Forum
December 11, 2016, 08:31:49 AM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: « 1 ... 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 [651] 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 ... 744 »
  Print  
Author Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool  (Read 2035627 times)
jonnybravo0311
Legendary
*
Offline Offline

Activity: 1008


Mine at Jonny's Pool


View Profile WWW
July 06, 2015, 08:10:26 PM
 #13001

Looks like the network is getting hammered with transactions... I turned off Matt's relay network and restarted everything.  Looking at blockchain.info, there's ver 11k unconfirmed transactions currently.

My node is currently trying to put together blocks with 2k transactions...

Code:
2015-07-06 16:11:19.107855 New work for worker! Difficulty: 1217.831707 Share difficulty: 5838904.174398 Total block value: 25.382547 BTC including 1995 transactions
2015-07-06 16:11:52.966132 New work for worker! Difficulty: 1089.745702 Share difficulty: 5838098.085605 Total block value: 25.386081 BTC including 2018 transactions

Jonny's Pool - Mine with us and help us grow!  Support a pool that supports Bitcoin, not a hardware manufacturer's pockets!  No SPV cheats.  No empty blocks.
1481445109
Hero Member
*
Offline Offline

Posts: 1481445109

View Profile Personal Message (Offline)

Ignore
1481445109
Reply with quote  #2

1481445109
Report to moderator
1481445109
Hero Member
*
Offline Offline

Posts: 1481445109

View Profile Personal Message (Offline)

Ignore
1481445109
Reply with quote  #2

1481445109
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481445109
Hero Member
*
Offline Offline

Posts: 1481445109

View Profile Personal Message (Offline)

Ignore
1481445109
Reply with quote  #2

1481445109
Report to moderator
1481445109
Hero Member
*
Offline Offline

Posts: 1481445109

View Profile Personal Message (Offline)

Ignore
1481445109
Reply with quote  #2

1481445109
Report to moderator
1481445109
Hero Member
*
Offline Offline

Posts: 1481445109

View Profile Personal Message (Offline)

Ignore
1481445109
Reply with quote  #2

1481445109
Report to moderator
nonnakip
Hero Member
*****
Offline Offline

Activity: 535



View Profile
July 06, 2015, 08:13:37 PM
 #13002

However, some are reporting high bitcoind CPU usage - P2Pool P2P network activity shouldn't be affecting bitcoind, so I'm not sure about that. Is that happening for Bitcoin users without P2Pool? It might be some similar post-BIP66 effect.

I noticed when I started P2Pool that my node is carrying about 6000 unverified shares (17354 verified but on start it loaded 23000). I assume these are shares from <14.0. Perhaps all these unverifiable shares are causing the 100% CPU usage and bitcoind work_poller delays?

Would it be better to delete my data directory and redownload share chain? Or just wait until we solve a block and maybe the problem goes away by itself?

p3yot33at3r
Sr. Member
****
Offline Offline

Activity: 266



View Profile
July 06, 2015, 08:16:03 PM
 #13003

Hmmm.......has anyone made contact with forrestv? It would be good to get his take of what's happening........

Hey, I've been watching. You are trying the recent P2Pool commit that blocks pre-v14 peers, right? If that's confirmed to help, I'll make a P2Pool release with it.

Some P2P network thrashing is normal after a fork since you have the unupgraded nodes madly trying to find peers and download the (invalid to them) sharechain. Normally, there are less unupgraded nodes, but this fork was rushed, so I think that that contributed to it.

However, some are reporting high bitcoind CPU usage - P2Pool P2P network activity shouldn't be affecting bitcoind, so I'm not sure about that. Is that happening for Bitcoin users without P2Pool? It might be some similar post-BIP66 effect.

Hi forrestv,

Thanks for chiming in buddy. Yes, using the latest p2pool repo. I'm seeing high cpu usage on both p2pool & bitcoind, but the cpu usage on p2pool is much more prolonged & constantly higher than bitcoind, which seems to have sudden peaks & bursts of high usage - resulting in very high latency issues.

It's a tricky one eh?  Smiley

I even tried re-downloading the p2pool sharechain......
xyzzy099
Legendary
*
Offline Offline

Activity: 941



View Profile
July 06, 2015, 08:30:32 PM
 #13004

Looks like someone is pounding the network with transactions:

https://bitcointalk.org/index.php?topic=1111811.msg11807542#msg11807542


Maybe this accounts for the problems you guys are seeing.

Libertarians:  Diligently plotting to take over the world and leave you alone.
p3yot33at3r
Sr. Member
****
Offline Offline

Activity: 266



View Profile
July 06, 2015, 08:58:49 PM
 #13005

That would explain a lot, the thought did occur to me earlier:


Is that plonker doing another "stress test" again?  Cheesy


If this is the case, it happening at around the same time as a forking would have to be intentional I reckon........is it from the same origin as the last time?
mdude77
Legendary
*
Offline Offline

Activity: 1358


View Profile
July 06, 2015, 09:12:21 PM
 #13006

I upgraded here as well.  Here's a message I don't recall seeing before:

Code:
2015-07-06 17:10:22.969017 Peer 104.193.40.248:9333 misbehaving, will drop and ban. Reason: first message was not version message
2015-07-06 17:10:22.969112 Bad peer banned: (u'104.193.40.248', 9333)

Otherwise I'm getting lots of bad peers.  For being at 95% on the new version, I sure do get connected to the 5% pretty easily.

M

MMinerMonitor author, monitor/auto/schedule reboots/alerts/remote/MobileMiner for Ants and Spondoolies! Latest (5.2). MPoolMonitor author, monitor stats/workers for most pools, global BTC stats (current/nxt diff/USD val/hashrate/calc)! Latest (v4.2) 
Buyer beware of Bitmain hardware and services.
squidicuz
Jr. Member
*
Offline Offline

Activity: 50


View Profile
July 06, 2015, 10:34:14 PM
 #13007

I upgraded here as well.  Here's a message I don't recall seeing before:

Code:
2015-07-06 17:10:22.969017 Peer 104.193.40.248:9333 misbehaving, will drop and ban. Reason: first message was not version message
2015-07-06 17:10:22.969112 Bad peer banned: (u'104.193.40.248', 9333)

Otherwise I'm getting lots of bad peers.  For being at 95% on the new version, I sure do get connected to the 5% pretty easily.

M
The bad peers are the remaining forked version 13 peers being removed.  Once your node ignores them all, stability will improve.  The high transaction volume on the bitcoin network combined with the persistent p2pool share fetching of version 13 nodes, along with some nodes timing out and further adding peer stress, is really hammering everyone.
Updating to the latest version of p2pool from github is needed in order to block the connection spam and stabilize the pool.
windpath
Legendary
*
Offline Offline

Activity: 938


View Profile WWW
July 07, 2015, 02:06:06 AM
 #13008


Things seem to be running smoothly again after the final update at around 3PM EDT today.

There has been a flood of TX's on the bitcoin network, and GBT latency is currently higher than I'd like to see it, but I believe P2Pool is functioning as it should with the latest version.

As Forrest mentioned, on the newest version it appears we are simply rejecting anything from a node with lower than p2pool version 14, I show 100% V 14 peers on my node...

Anyone monitoring how much of the p2pool network we left behind?

Here is the bitcoin transaction flood from today (source linked, click on "txInMemoryPool" on lower left):


srpmatt
Newbie
*
Offline Offline

Activity: 6


View Profile
July 07, 2015, 04:53:16 PM
 #13009

Well i finally have p2pool setup and working and having a payout earlier in the week all seems good, spent most of today getting my head round the merged mining side of thinks and finally have that setup with namecoin, just waiting on a block now Roll Eyes
Just want to say thanks to all on the forum that have posted guides e.t.c has been a big help
jonnybravo0311
Legendary
*
Offline Offline

Activity: 1008


Mine at Jonny's Pool


View Profile WWW
July 07, 2015, 05:04:28 PM
 #13010

You are welcome... and welcome to p2pool Smiley.

Jonny's Pool - Mine with us and help us grow!  Support a pool that supports Bitcoin, not a hardware manufacturer's pockets!  No SPV cheats.  No empty blocks.
jonnybravo0311
Legendary
*
Offline Offline

Activity: 1008


Mine at Jonny's Pool


View Profile WWW
July 07, 2015, 10:21:14 PM
 #13011

So it appears the problems continue... my node has been effectively offline for the majority of the day today.  P2Pool is pegging my CPU at 100% and I'm consistently seeing the "lost contact with bitcoind" message in the logs.  Is this solely because of the ridiculous number of unconfirmed transactions that are still flooding the network (nearly 13k at this point)?  Granted, my VPS isn't exactly a monster (but not horrible as a dual core droplet).

I'm on the latest version of p2pool and bitcoin core 0.10.2 compiled from source.

To the miners who are on my node, I apologize for the inconvenience this is likely causing you.  On the bright side, my miners fall back to ck.'s solo pool, so maybe I'll be lucky and they'll solve a block for me Tongue.

EDIT: when my node does appear online, and the miners are connected, they're reporting huge rejected hash percentages (25% or more).

Jonny's Pool - Mine with us and help us grow!  Support a pool that supports Bitcoin, not a hardware manufacturer's pockets!  No SPV cheats.  No empty blocks.
luthermarcus
Full Member
***
Offline Offline

Activity: 206



View Profile
July 07, 2015, 10:40:29 PM
 #13012

I had the problem of bitcoin core freezing (I would only notice it on p2pool log "lost connection" or through email saying my node has stalled) since i upgraded from 10.0 to 10.2 i thought it was my linksys router so i bought a switch should be arriving any day now. Could still be a coincidence but more and more people are speaking out about it. Bitcoin has been more stable now but does drop at peeks. Bitcoin uses a about 4 g ram which is 51% of a little under 8 g's and a quater to half my cpu on my intel 2 Quad CPU (Q6700 @ 2.66GHz). That's bitcoin alone. Once the switch comes in ill put p2pool back online and put out stats I should be getting 100mb/s but running a speedtest i was getting 7.5 which is slow so it could have been the connection timing out.
p3yot33at3r
Sr. Member
****
Offline Offline

Activity: 266



View Profile
July 07, 2015, 11:37:54 PM
 #13013

I dunno. It might be nothing, but I was going through my p2pool logs earlier & noticed the same IP over & over again that was connecting, submitting dodgy hash, getting banned for it then reconnecting again on a different port & doing the same thing all over again, constantly & repeatedly using a different port every time. My logs are full of the same IP, namely 219.239.31.11 - it's in China:

Code:
2015-07-06 06:14:57.151757 invalid hash for 219.239.31.11 'remember_tx' 748990 a9117513
2015-07-06 06:14:57.151874 Bad peer banned: ('219.239.31.11', 64888)
2015-07-06 06:14:57.155612 Lost peer 219.239.31.11:64888 - Connection was aborted locally, using.
2015-07-06 06:14:57.156126 > in download_shares:
2015-07-06 06:14:57.156218 > Traceback (most recent call last):
2015-07-06 06:14:57.156259 > Failure: twisted.internet.error.ConnectionAborted: Connection was aborted locally, using.

Then reconnect on different port, request shares, submit bad hash, get banned again:

Code:
2015-07-06 05:13:49.929600 Incoming connection to peer 219.239.31.11:33919 established. p2pool version: 1400 '14.0'
2015-07-06 05:13:54.675766 invalid hash for 219.239.31.11 'remember_tx' 292502 69c836d4
2015-07-06 05:13:54.675885 Bad peer banned: ('219.239.31.11', 33919)
2015-07-06 05:13:54.676322 Lost peer 219.239.31.11:33919 - Connection was aborted locally, using.

Over & over, closely followed by a bitcoind disconnect.

Note that whoever this is is using v14, so isn't getting banned by p2pool straight away. My logs are full of this IP using a different port every connection, submitting bad hash, requesting good then getting banned - so instead of waiting for 65535 banned connections to get through, I figured I'd just get rid, so I banned it at the router level firewall too.

Since I did that a few hours ago I haven't had a single disconnect. Yet. Maybe I'm clutching at straws, but it just seems a bit weird that some China IP keeps flooding my node with shit so close to losing contact with bitcoind. I'd be interested to hear if anyone else has this IP in their logs........ Wink

Or any other ideas of course, but check your logs anyway - see if you have the same IP as in my logs.......

EDIT: Still getting this though  Sad:

luthermarcus
Full Member
***
Offline Offline

Activity: 206



View Profile
July 08, 2015, 12:04:35 AM
 #13014

I dunno. It might be nothing, but I was going through my p2pool logs earlier & noticed the same IP over & over again that was connecting, submitting dodgy hash, getting banned for it then reconnecting again on a different port & doing the same thing all over again, constantly & repeatedly using a different port every time. My logs are full of the same IP, namely 219.239.31.11 - it's in China:

Code:
2015-07-06 06:14:57.151757 invalid hash for 219.239.31.11 'remember_tx' 748990 a9117513
2015-07-06 06:14:57.151874 Bad peer banned: ('219.239.31.11', 64888)
2015-07-06 06:14:57.155612 Lost peer 219.239.31.11:64888 - Connection was aborted locally, using.
2015-07-06 06:14:57.156126 > in download_shares:
2015-07-06 06:14:57.156218 > Traceback (most recent call last):
2015-07-06 06:14:57.156259 > Failure: twisted.internet.error.ConnectionAborted: Connection was aborted locally, using.

Then reconnect on different port, request shares, submit bad hash, get banned again:

Code:
2015-07-06 05:13:49.929600 Incoming connection to peer 219.239.31.11:33919 established. p2pool version: 1400 '14.0'
2015-07-06 05:13:54.675766 invalid hash for 219.239.31.11 'remember_tx' 292502 69c836d4
2015-07-06 05:13:54.675885 Bad peer banned: ('219.239.31.11', 33919)
2015-07-06 05:13:54.676322 Lost peer 219.239.31.11:33919 - Connection was aborted locally, using.

Over & over, closely followed by a bitcoind disconnect.

Note that whoever this is is using v14, so isn't getting banned by p2pool straight away. My logs are full of this IP using a different port every connection, submitting bad hash, requesting good then getting banned - so instead of waiting for 65535 banned connections to get through, I figured I'd just get rid, so I banned it at the router level firewall too.

Since I did that a few hours ago I haven't had a single disconnect. Yet. Maybe I'm clutching at straws, but it just seems a bit weird that some China IP keeps flooding my node with shit so close to losing contact with bitcoind. I'd be interested to hear if anyone else has this IP in their logs........ Wink

Or any other ideas of course, but check your logs anyway - see if you have the same IP as in my logs.......

EDIT: Still getting this though  Sad:



Here is an article which could explain what your seeing
https://bitcoinmagazine.com/21125/bitcoin-network-still-backlogged-tens-thousands-unconfirmed-transactions-causing-delays/
p3yot33at3r
Sr. Member
****
Offline Offline

Activity: 266



View Profile
July 08, 2015, 12:11:39 AM
 #13015

Yeah, I'm aware of the attack - been following it:

https://www.reddit.com/r/Bitcoin/comments/3cfext/bitcoin_under_attack/

&

https://www.reddit.com/r/Bitcoin/comments/3cg95d/bitcoin_is_resilient/

But I don't think that has anything to do with the IP flooding my node.

BTW - love this quote:

Quote
Now the interesting thing is that some core devs (paging Peter Todd, Luke-Jr, Gregory Maxwell) (some with notable direct conflicts of interest by working at Blockstream) do not wish to allow bitcoin to grow......

QFT.  Grin
p3yot33at3r
Sr. Member
****
Offline Offline

Activity: 266



View Profile
July 08, 2015, 02:53:39 AM
 #13016

BLOCK!!

Phew....... Tongue
squidicuz
Jr. Member
*
Offline Offline

Activity: 50


View Profile
July 08, 2015, 03:44:33 AM
 #13017

I dunno. It might be nothing, but I was going through my p2pool logs earlier & noticed the same IP over & over again that was connecting, submitting dodgy hash, getting banned for it then reconnecting again on a different port & doing the same thing all over again, constantly & repeatedly using a different port every time. My logs are full of the same IP, namely 219.239.31.11 - it's in China:

Code:
2015-07-06 06:14:57.151757 invalid hash for 219.239.31.11 'remember_tx' 748990 a9117513
2015-07-06 06:14:57.151874 Bad peer banned: ('219.239.31.11', 64888)
2015-07-06 06:14:57.155612 Lost peer 219.239.31.11:64888 - Connection was aborted locally, using.
2015-07-06 06:14:57.156126 > in download_shares:
2015-07-06 06:14:57.156218 > Traceback (most recent call last):
2015-07-06 06:14:57.156259 > Failure: twisted.internet.error.ConnectionAborted: Connection was aborted locally, using.

Then reconnect on different port, request shares, submit bad hash, get banned again:

Code:
2015-07-06 05:13:49.929600 Incoming connection to peer 219.239.31.11:33919 established. p2pool version: 1400 '14.0'
2015-07-06 05:13:54.675766 invalid hash for 219.239.31.11 'remember_tx' 292502 69c836d4
2015-07-06 05:13:54.675885 Bad peer banned: ('219.239.31.11', 33919)
2015-07-06 05:13:54.676322 Lost peer 219.239.31.11:33919 - Connection was aborted locally, using.

Over & over, closely followed by a bitcoind disconnect.

Note that whoever this is is using v14, so isn't getting banned by p2pool straight away. My logs are full of this IP using a different port every connection, submitting bad hash, requesting good then getting banned - so instead of waiting for 65535 banned connections to get through, I figured I'd just get rid, so I banned it at the router level firewall too.

Since I did that a few hours ago I haven't had a single disconnect. Yet. Maybe I'm clutching at straws, but it just seems a bit weird that some China IP keeps flooding my node with shit so close to losing contact with bitcoind. I'd be interested to hear if anyone else has this IP in their logs........ Wink

Or any other ideas of course, but check your logs anyway - see if you have the same IP as in my logs.......

EDIT: Still getting this though  Sad:



That node likely does not have the most recent update to block v13 nodes. So it will get bad shares from its bad peers and then try to pass them on.  For some reason this is creating a timeout within p2pool at times, which then causes a disconnect to bitcoind and the other [p2pool] peers.

The best course of action is to manually ban it, like you did.

With a bunch of nodes now updated to 14.0-6, the pool seems to be recovering.  Just a few hiccups here and there.  Even still, we managed to score a block tonight!
Meuh6879
Legendary
*
Offline Offline

Activity: 1092



View Profile
July 08, 2015, 07:24:33 AM
 #13018

Yes ... above 0.9.1 bitcoin core, RPC of Bitcoin Core (or bitcoind) crash because of the P2Pool server ...
It's not new.

I have an automated routine to restart Bitcoin Core every 30min~1h for this.

French ... but not so much   ---===---   P2P ... it's people at the end   ---===---   P2Pool (10,9 GH/s).
Comment miner des bitcoins ? Un tutoriel est là : https://bitcointalk.org/index.php?topic=1114415.0
Bitcoin change everything ... an explain of this fact : https://www.youtube.com/watch?v=joITmEr4SjY
nonnakip
Hero Member
*****
Offline Offline

Activity: 535



View Profile
July 08, 2015, 08:38:40 AM
 #13019

I run a simple test of calling "bitcoin-cli getblocktemplate" in a loop. Normally this request is relative fast. But every 6 seconds it takes nearly 2 seconds to complete and causes 100% CPU load on a bitcoind thread during the 2 seconds. The bitcoind thread with the load changes each time so it is probably the bitcoind thread that happens to process the cli request.

I do not have time right now to profile bitcoind to isolate this issue. But I thought it was interesting that the requests take so long (busy waiting?) at nearly exact 6 second intervals.

I also continue to see bitcoind connection issues every hour or so. Connection is lost between 1 and 25 minutes! I need to make some time so I can debug the exact scenario. It occurs often enough.

p3yot33at3r
Sr. Member
****
Offline Offline

Activity: 266



View Profile
July 08, 2015, 09:15:50 AM
 #13020

I also continue to see bitcoind connection issues every hour or so

Hi nonnakip,

It's been nearly 12 hours since I implemented the bad IP block & I've had no bitcoind disconnects since doing so. I presume you've checked your logs for a similar repeat offender? Worth a try......

I just noticed Matt has released a new "dirty hack" relaynetworkclient for p2pool also - https://github.com/TheBlueMatt/RelayNode

gonna try & compile it now & see if it helps any......

EDIT: Nope, it still won't run on 14.04 64bit - anyone else had any luck?
Pages: « 1 ... 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 [651] 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 ... 744 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!