Bitcoin Forum
October 23, 2017, 11:15:34 PM *
News: Latest stable version of Bitcoin Core: 0.15.0.1  [Torrent]. (New!)
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 [19] 20 21 »  All
  Print  
Author Topic: How (and why) pools (and all miners) should use the Relay Network  (Read 270104 times)
WBF1
Sr. Member
****
Offline Offline

Activity: 372


View Profile
February 25, 2016, 04:16:56 AM
 #361

also, does this currently not relay blocks mined on bitcoin classic? seems to cause an issue for me whenever one is being mined (for example, block 399914).

Without relaying this block, bitcoind tells me the relay client is misbehaving when it tries to relay the next block.


nevermind it was a miscofiguration on my end.
1508800534
Hero Member
*
Offline Offline

Posts: 1508800534

View Profile Personal Message (Offline)

Ignore
1508800534
Reply with quote  #2

1508800534
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
o_solo_miner
Hero Member
*****
Offline Offline

Activity: 784


-> morgen, ist heute, schon gestern <-


View Profile
March 10, 2016, 03:54:32 PM
 #362

 Huh looks like that the public.eu.relay.mattcorallo.com Node is down, as I got this message:

[2016-03-10 12:39:25.858+00] 00000000000000000149a9d6ccc79d07f460bbe888bc220e9ed643b3ba0226ec sent, size 998129 with 882427 bytes on the wire
[2016-03-10 12:39:25.868+00] public.02.relay.mattcorallo.com Disconnect: failed to read message header (Broken pipe)


I manualy changed to US-WEST and that works.
Strange, as it went down after the blockfound message.

Anybody else with problems on eu?

http://ckpool.org "THE Pool" from the creator of CGMiner & CKPool / Payout System:SPLNS / ZERO FEE!
------------------------------------------- join now -----------------------------------------------
http://solo.ckpool.org for Solominers with the best block notify system
Matt Corallo
Hero Member
*****
Offline Offline

Activity: 755


View Profile
March 10, 2016, 08:23:39 PM
 #363

Yea, eu went down for a bit...should be good now.

Bitcoin Ubuntu PPA maintainer - donate to me personally: 1JBMattRztKDF2KRS3vhjJXA7h47NEsn2c
http://bitcoinrelaynetwork.org maintainer
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
o_solo_miner
Hero Member
*****
Offline Offline

Activity: 784


-> morgen, ist heute, schon gestern <-


View Profile
March 10, 2016, 08:28:05 PM
 #364

Thank you matt
now it works ok.

I was just suprised, never got an error before.





http://ckpool.org "THE Pool" from the creator of CGMiner & CKPool / Payout System:SPLNS / ZERO FEE!
------------------------------------------- join now -----------------------------------------------
http://solo.ckpool.org for Solominers with the best block notify system
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2310


Ruu \o/


View Profile WWW
March 10, 2016, 08:36:36 PM
 #365

Hey Matt. Any update on what the plans are for your invaluable network? Thanks.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
Matt Corallo
Hero Member
*****
Offline Offline

Activity: 755


View Profile
March 10, 2016, 10:39:23 PM
 #366

I was just suprised, never got an error before.
Servers go down sometimes Smiley.

Hey Matt. Any update on what the plans are for your invaluable network? Thanks.
Until other things that are much higher priority (ie the current drama) is dealt with, I will keep restarting servers when they go down but no other changes are gonna happen...I'll revisit things in a month or three.

Bitcoin Ubuntu PPA maintainer - donate to me personally: 1JBMattRztKDF2KRS3vhjJXA7h47NEsn2c
http://bitcoinrelaynetwork.org maintainer
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
MeatPopsicle
Jr. Member
*
Offline Offline

Activity: 48


View Profile
March 13, 2016, 11:44:06 PM
 #367

If worst comes to worst could you throw together a short docu on how the server components fit together? Trying to piece together how and with what each of them talk to by digging through the C code is less than optimal.
mmeijeri
Hero Member
*****
Offline Offline

Activity: 714

Martijn Meijering


View Profile
March 16, 2016, 04:28:41 PM
 #368

I didn't want to wade through 19 pages of text and I know this has been answered before, but why would it be a bad idea to integrate this into the P2P network on a bilateral basis? Wouldn't it still help a bit?

ROI is not a verb, the term you're looking for is 'to break even'.
kano
Legendary
*
Offline Offline

Activity: 2240


Linux since 1997 RedHat 4


View Profile
March 16, 2016, 09:56:54 PM
 #369

I didn't want to wade through 19 pages of text and I know this has been answered before, but why would it be a bad idea to integrate this into the P2P network on a bilateral basis? Wouldn't it still help a bit?
Yeah I asked that already ... the problem is that at least some of the bitcoin devs are morons.
https://github.com/bitcoin/bitcoin/issues/7049

... especially when you get the part where gmaxwell considers exceptionally useful code black magic Tongue

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
Matt Corallo
Hero Member
*****
Offline Offline

Activity: 755


View Profile
May 05, 2016, 04:11:06 AM
 #370

I didn't want to wade through 19 pages of text and I know this has been answered before, but why would it be a bad idea to integrate this into the P2P network on a bilateral basis? Wouldn't it still help a bit?
See https://github.com/TheBlueMatt/bips/blob/master/bip-TODO.mediawiki for something similar which is more targeted at the P2P nature of Bitcoin. See-also https://github.com/TheBlueMatt/bitcoin/tree/udp-wip for something which is intended to eventually replace the Relay Network - a whitelisted-peers-only secondary relay channel that is easy for miners to use between their own nodes (ie its just an additional flag) and will also be much faster than the relay network today in its worst-case (and probably average) relay times.

Bitcoin Ubuntu PPA maintainer - donate to me personally: 1JBMattRztKDF2KRS3vhjJXA7h47NEsn2c
http://bitcoinrelaynetwork.org maintainer
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
Matt Corallo
Hero Member
*****
Offline Offline

Activity: 755


View Profile
May 27, 2016, 09:08:54 PM
 #371

For those paying attention...The relay network donation address was recently nearly entirely depleted to pay for a new set of servers...The network is going to switch entirely to a new infrastructure based on bitcoind with UDP-FEC (ie no more delays from TCP lag) which allows for cut-through routing by rumoring UDP-FEC packets around the entire network before receiving the entire block. You can find the source for this patch at https://github.com/TheBlueMatt/bitcoin/tree/udp-wip...The API to it is being refactored to enable the cut-through routing optimizations, and note that it includes some LGPL components in the binary, so its probably all secretly under LGPL, but IANAL, please dont sue me.

Bitcoin Ubuntu PPA maintainer - donate to me personally: 1JBMattRztKDF2KRS3vhjJXA7h47NEsn2c
http://bitcoinrelaynetwork.org maintainer
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
Matt Corallo
Hero Member
*****
Offline Offline

Activity: 755


View Profile
July 07, 2016, 09:31:25 PM
 #372

So I'm sure everyone has seen, by now, my blog post on the future of the relay network. I do have a test-network based on FIBRE which has shown incredibly good performance, but I'm not quite sure if I will allow public connections to it (it currently only connects to random nodes on the public P2P network and to the original relay network) or allow others to use FIBRE to take up the mantle of running high-speed relay networks. For now, the original relay network continues to run and the FIBRE network running side-by-side is only making it faster!
If you want any help running your own FIBRE-based network, I'm more than happy to help out!

Bitcoin Ubuntu PPA maintainer - donate to me personally: 1JBMattRztKDF2KRS3vhjJXA7h47NEsn2c
http://bitcoinrelaynetwork.org maintainer
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
kano
Legendary
*
Offline Offline

Activity: 2240


Linux since 1997 RedHat 4


View Profile
July 07, 2016, 10:23:20 PM
 #373

I block all UDP and run no UDP services coz it's been the majority cause of DDoS that occurred since I started the pool ...
... since there is no 'network security' applied to UDP on the internet.

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
Matt Corallo
Hero Member
*****
Offline Offline

Activity: 755


View Profile
July 07, 2016, 11:01:57 PM
 #374

I block all UDP and run no UDP services coz it's been the majority cause of DDoS that occurred since I started the pool ...
... since there is no 'network security' applied to UDP on the internet.
If you're talking about running your own FIBRE-based network: No need to enable incoming UDP on pool servers to run FIBRE. TCP/Compact Block relay to servers only a few ms away is plenty fast, so you could just as easily put a relay network server on a separate server (and probably should, given that its still beta and based on Bitcoin Core master just before segwit was merged).

If you're talking about connecting to someone else's: I never really envisioned it as running over UDP between miners and the first-hop server. While this is definitely an option (and FIBRE has an "untrusted" mode for this), there is little advantage to spending 10ms calculating FEC data over just eating the 10ms RTT for a dropped packet or two if the servers are nearby. FIBRE is great for long-haul links, but Compact Blocks work just as well if you're talking about short-hops.

General note: FIBRE's UDP does have a shared-key HMAC in each packet, so that packets are easily and trivially authenticated.

Bitcoin Ubuntu PPA maintainer - donate to me personally: 1JBMattRztKDF2KRS3vhjJXA7h47NEsn2c
http://bitcoinrelaynetwork.org maintainer
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
kano
Legendary
*
Offline Offline

Activity: 2240


Linux since 1997 RedHat 4


View Profile
July 07, 2016, 11:14:36 PM
 #375

I block all UDP and run no UDP services coz it's been the majority cause of DDoS that occurred since I started the pool ...
... since there is no 'network security' applied to UDP on the internet.
If you're talking about running your own FIBRE-based network: No need to enable incoming UDP on pool servers to run FIBRE. TCP/Compact Block relay to servers only a few ms away is plenty fast, so you could just as easily put a relay network server on a separate server ...
Yep good solution.
One collection of 'satellites' I have are sub 1ms (around 1/3ms)

Quote
(and probably should, given that its still beta and based on Bitcoin Core master just before segwit was merged).
Well looks like I'll be behind the times for a while if it's been merged, coz yeah not really interested in adding code to centralise bitcoin and give away mining fees to those centralising it so they can take those fees.

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
Matt Corallo
Hero Member
*****
Offline Offline

Activity: 755


View Profile
July 07, 2016, 11:17:53 PM
 #376

If anyone is looking to set up a FIBRE-based network, I'm happy to share the slow-as-fuck bash script which generates the slow-to-render pretty graphs on test network stats page.

Bitcoin Ubuntu PPA maintainer - donate to me personally: 1JBMattRztKDF2KRS3vhjJXA7h47NEsn2c
http://bitcoinrelaynetwork.org maintainer
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2310


Ruu \o/


View Profile WWW
September 18, 2016, 12:36:31 PM
 #377

This has been misbehaving for the last week or so. Any chance it could get a kick?

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
Matt Corallo
Hero Member
*****
Offline Offline

Activity: 755


View Profile
October 18, 2016, 03:46:36 PM
 #378

Hi folks,

I'd like to turn off the relay network on December 1st. For that to happen, I'd like to first have a public relay network of nodes based on FIBRE available ASAP.

In order to make the network much more reliable/lower-latency than the original relay network, I'd like to use real servers with good,
dedicated, bandwidth. I speced out a network which would cost about $1550/month and would have public peering nodes in Hong Kong, Tokyo, Seattle, Washington DC, Beijing, and one of London/Amsterdam/Frankfurt. Additionally, there would be (at least) one node in Siberia to utilize the TEA Cable route.

A few folks have already agreed to chip in to make this a reality, but I'd love to see a few more so that we can make it incredibly cheap for
everyone involved. If you'd like to help, I'm looking for people willing to make commitments to cover a fair share of the above costs, and would list the sponsors on bitcoinrelaynetwork.org (which would be updated with new info).

Bitcoin Ubuntu PPA maintainer - donate to me personally: 1JBMattRztKDF2KRS3vhjJXA7h47NEsn2c
http://bitcoinrelaynetwork.org maintainer
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
kano
Legendary
*
Offline Offline

Activity: 2240


Linux since 1997 RedHat 4


View Profile
October 18, 2016, 11:36:34 PM
 #379

Well ... a few things.

I take it that the current relay network has been crappy for a while coz you decided you weren't getting enough sponsorship?
i.e. when the sponsorship isn't enough in your (unstated) opinion you just let it go to crap?
(No I've never sponsored it - but I do connect to every relay ... even the long dead hk one - our ckpool block relay is faster than using the relay to distribute to other relay nodes so I connect to all nodes to get our blocks out faster)

Unless you have the top pools connected (i.e. every pool over a few % of the network) it's value is marginal so I'd think you should actually list the pools that are sponsoring and are connected - and update it when that changes - since that's not a large list.
I'd also suggest you whitelist access to most of it. That way you know who is connected and know it is still useful.
e.g. f2pool wasn't on the relay until long after the relay existed, and I've no idea if all of the 3 bitmain pools use it. Bitfury's high orphan rate at times makes me wonder if they have always used it.

Well I'm certainly no where near 7% of the network but I'll put forward ~$100 in BTC Smiley a month towards it (as long as the biggest pools are on it)
N.B. none of my servers but the single main back end cost me even half of that Smiley So I guess this network is either using an expensive provider or high end servers on every node Smiley

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
Matt Corallo
Hero Member
*****
Offline Offline

Activity: 755


View Profile
October 19, 2016, 06:11:39 PM
 #380

I take it that the current relay network has been crappy for a while coz you decided you weren't getting enough sponsorship?
i.e. when the sponsorship isn't enough in your (unstated) opinion you just let it go to crap?
(No I've never sponsored it - but I do connect to every relay ... even the long dead hk one - our ckpool block relay is faster than using the relay to distribute to other relay nodes so I connect to all nodes to get our blocks out faster)
Yea, the HK one died after the host decided to wipe the drive on the server when the NIC failed....yes, you read that right. I didn't bother to re-install since a) I dont actually remember how to reinstall that stuff, and havent had time to look it up (FIBRE is so much easier...) and b) I want to turn the thing off for SegWit (though getting a public FIBRE network up has taken longer than expected :/.

Unless you have the top pools connected (i.e. every pool over a few % of the network) it's value is marginal so I'd think you should actually list the pools that are sponsoring and are connected - and update it when that changes - since that's not a large list.
I'd also suggest you whitelist access to most of it. That way you know who is connected and know it is still useful.
e.g. f2pool wasn't on the relay until long after the relay existed, and I've no idea if all of the 3 bitmain pools use it. Bitfury's high orphan rate at times makes me wonder if they have always used it.
Indeed, one of the things I'm planning on doing is listing those sponsoring on the site by putting their logo up. I assume if folks are paying 2-300$/month for something they will use it. Indeed, it was an uphill battle to get people to use it initially, but its much better these days. Bitfury still has some insanity occasionally, but I'm working with them to solve it.

Well I'm certainly no where near 7% of the network but I'll put forward ~$100 in BTC Smiley a month towards it (as long as the biggest pools are on it)
N.B. none of my servers but the single main back end cost me even half of that Smiley So I guess this network is either using an expensive provider or high end servers on every node Smiley
Yea, they're all dedicated servers since the random lag introduced by VPSes was incredibly annoying. Sadly, for dedicated servers, unlike virtual ones, it can be hard to get them in good locations in some places (APAC especially), and its also very nice to have them all on one network, so they're all on Softlayer. They're a bit more expensive than neccessary, but they have really, really good latency between any two points on their network (with the exception of JPY<->EU as they dont have TEA bandwidth), as well as solid peering in Tokyo/HK with PCCW/Pacnet, which is important for China.

Bitcoin Ubuntu PPA maintainer - donate to me personally: 1JBMattRztKDF2KRS3vhjJXA7h47NEsn2c
http://bitcoinrelaynetwork.org maintainer
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 [19] 20 21 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!