Peer Discovery (STUN/ICE/TURN)

[bbqsamich]

I'm curious why not implement STUN/ICE/TURN functionality instead (or on top of) DNS discovery?

Here's a basic overview of what STUN/ICE/TURN tries to accomplish. It should be noted, I found this by researching how browsers plan to enable P2P communications via WebRTC.

As such, if we follow a standard implementation for peer discovery, isn't this to the benefit of the bitcoin community? Wouldn't this open up the world to some additional peers stuck behind firewalls and behind proxies? If/when browsers fully support P2P, if we are already following their standard, adding wallet support to a webpage could become extremely simple, allowing further ease of access for bitcoin in the digital world! Any cons to this? What about other positives I'm not considering?

I'm posting this here because its a topic starter and a bit of a noob question to the experts out there. (I also don't want to screw up which forum it goes into).

[K1773R]

Your link appears to be broken.

is it so hard removing the "%22" from the start and end of the link and putting ":" between http and //? ah wait, for some ppl it is...
here we go:
http//www.quora.com/Internet-Security/How-does-ICE-STUN-work-in-the-context-of-firewall-penetration

[gmaxwell]

STUN/ICE/TURN help hosts punch holes for UDP through firewalls, they don't provide presence services. Not terribly applicable unless bitcoin were to start using UDP in order to get inbound through firewalls, or to imitate webrtc (DTLS in UDP) in order to get past ignorant filtering — but it doesn't seem necessary to do so.

[bbqsamich]

My point was that, since it helps with discovery and firewall punching, why not use this instead of DNS discover? Additionally, it seems that turn servers could be used to help mitigate the firewall issues that generally happen on the peering ports for bitcoin, opening up more peers for the network as a whole. Finally, peers can dynamically be included with these services, removing the crazy complicated functionality for dns discovery.

Now, I'm no expert on this stuff, but at minimum it seems to me that this is on par with DNS discovery with far more advantages than disadvantages.

[gmaxwell]

My point was that, since it helps with discovery and firewall punching, why not use this instead of DNS discover?

Because— as far as I know— it can't be used for discovery of the kind that dns discovery does. The discovery that it does is discovering your own IP address, which we don't (and couldn't) use DNS for. (We do, however use UPNP for that).

opening up more peers for the network as a whole.

Yes, but the cost is that we'd effectively need our own userspace TCP in UDP implementation. This is a whole lot of risky network exposed code, and we don't currently appear to be short peers (esp since we got UPNP working).

removing the crazy complicated functionality for dns discovery.

Huh? DNS discovery itself is basically a dozen trivial lines of code and a small static table. Part of the reason we adopted it is because its very simple, safe, and scalable.

seems to me that this is on par with DNS discovery with far more advantages than disadvantages.

I'll gladly review your patch.

[maxihatop]

EmerCoin uses STUN for obtain public IP. See their press-release:


Emercoin  0.2.2 is released.

Major improvement:

  For the first time in the history of cryptocurrencies, is used STUN protocol to obtaining an external IP address, instead of WEB-service.

  This problem is already discussed in the cryptocurrency community, for example, here:
http://www.reddit.com/r/Bitcoin/comments/29zx7z/bitcoin_core_uses_showmyip_as_a_centralized_hard/

But, without any positive outcome.

Problem description:

  Bitcoin and it’s descendants (PPC, QRK, etc) – all of them uses same public WEB-server for obtain external IP.
There is two public servers, hardcoded into a wallet program:
- http://checkip.dyndns.org
- http://www.showmyip.com

For source code, see https://github.com/bitcoin/bitcoin/blob/master/src/net.cpp, function GetMyExternalIP()

  What is interesting: 2nd server http://www.showmyip.com is out of service nowadays. So, all Bitcoin and altcoin networks currently
dependent  on single WEB-server: checkip.dyndns.org. If this server will gone, then coin network will not died, but dramatically lost stability.
This will happening, because of peers wouldn't able to advertise  (represent) another peers into network, and peer network
connectivity will be decreased.

  Moreover, current WEB-mechanism for obtain external IP is based on non-standardized human-oriented text protocol.
Therefore, if site owners will modify output format, (for instance, for better reading), then all coin wallets will be unable
to parse and understand answers, and results will be as same as server down.

  In addition, dyndnd.org’s owners able to analyze WEB-server logs, and collect list of IP addresses, where is running coin wallets
and request timestamps, if is not doing this right now (request signature is hardcoded and well known, and easy to distinct).
If we remember, users usually runs wallet just before start some transactions, then we aware:
analyst can correlate wallet start and transaction time, and link it to IP address.
This is serious compromise of coin users anonymity.

Our solution:

  We substituted current non-standard, hard-weight (based on TCP/HTTP) protocol to protocol, based on mature standard RFC3489 (STUN),
designed especially for this purpose. See: http://en.wikipedia.org/wiki/STUN . This is lightweight, UDP-based protocol, used in SIP-based
applications (VOIP, IP-video) for 10+ years. And since it is standardized, we can use unlimited list of  servers for request, and always be
sure: EmerCoin wallet can parse/understand answer from anyone.

  Currently, new EMC-wallet for obtain IP address uses 47 STUN-servers, distributed worldwide, and request sequence is random. Therefore,
nobody can collect significant list if our wallets. Moreover, we performed some steps for randomize signature request, and for server is difficulty
to distinct EMC-wallet request from request, sent by IP phone or softswitch.

  Therefore, we reached our goals: removed dependency from centralized server, increased network stability and provide real  anonymity.

  EmerCoin - cryptocurrency of future. Download/upgrade a new wallet here: http://emercoin.com/downloads/

[gmaxwell]

Thats a really foolish method. The Bitcoin protocol already has a facility to identify the peers address without any reliance on a trusted remote parties at all.

Ignoring that, the address discovery is also moderately unimportant: Normally the address is either the interface address (in the no nat case), or it's discovered via UPNP. If the host is behind nat but UPNP is not used the port is likely not forwarded at all. (And, of course, the users can always override if they have an unusual configuration).

Also, your post is barely on topic and you've bumped a pretty old thread. Please don't do that.

[maxihatop]

> Thats a really foolish method.
> The Bitcoin protocol already has a facility to identify the peers address without any reliance on a trusted remote parties at all.

Really foolish? are you sure?

1. Regarding UPNP.

  It is not always works correctly. For instance, it works incorrect, if wallet runs behind chain of NATs (corp or regional networks).
And, UPNP just can be unsupported by router, or disabled here. In additional, if you build headless wallet (for servers), it build
with flags USE_UPNP=0. This is reasonable, because of server usually connected to the Internet directly, and no UPNP-source for it.
Therefore, in this case, used only method [2].

2. Regarding GetMyExternalIP/dyndns.org:

  I am running BitcoinCore v 0.9.1.0-g026a939-beta (64 bit). Relative fresh, are you?
Run behind the NAT device with UPNP.

See file debug.log:

2014-08-04 22:23:59 UPnP: ExternalIPAddress = 228.284.111.72
2014-08-04 22:23:59 AddLocal(228.284.111.72:8333,3)
2014-08-04 22:23:59 UPnP Port Mapping successful.
2014-08-04 22:24:01 ERROR: GetMyExternalIP() : connection to 91.198.22.70:80 failed
2014-08-04 22:24:01 GetMyExternalIP() received [] [::]:0
2014-08-04 22:24:02 GetMyExternalIP() received [228.284.111.72] 228.284.111.72:0
2014-08-04 22:24:02 GetMyExternalIP() returned 228.284.111.72
2014-08-04 22:24:02 AddLocal(228.284.111.72:8333,4)

As you see, modern Bitcoin uses both: UPNP and dyndns.org. Therefore, reports about it's presence is sent to dyndns.org.
You can check your log, too.

Of course, you can think and doing everything you wish. If you feel comfortable, when your wallet reports about your presence - this is your choice.
We going our own way, and do not urge anyone to make any decision.

We just answer to the subject of this topic:
 - Is anyone wish/use STUN/ICE/TURN for discover external IP?
 - Yes, we do. We using.

This is foolish by your opinion? OK, no problem. Don't use it.

[gmaxwell]

> Thats a really foolish method.
> The Bitcoin protocol already has a facility to identify the peers address without any reliance on a trusted remote parties at all.

Really foolish? are you sure?

1. Regarding UPNP.

  It is not always works correctly. For instance, it works incorrect, if wallet runs behind chain of NATs (corp or regional networks).
And, UPNP just can be unsupported by router, or disabled here. In additional, if you build headless wallet (for servers), it build
with flags USE_UPNP=0. This is reasonable, because of server usually connected to the Internet directly, and no UPNP-source for it.
Therefore, in this case, used only method [2].

If you are behind a chain of NATs inbound connectivity is not going to work with high likelihood, so discovery is irrelevant.

If you are connected to the internet directly then it gets the correct address off the interface, and again— discovery is irrelevant.

Of course, you can think and doing everything you wish. If you feel comfortable, when your wallet reports about your presence - this is your choice.

My comments were about the importance of discovery and the wisdom of your proposed solution. In cases where discovery isn't important you can disable it (via -discovery=0 or -externalip=).

We going our own way, and do not urge anyone to make any decision.
We just answer to the subject of this topic:
 - Is anyone wish/use STUN/ICE/TURN for discover external IP?
 - Yes, we do. We using.

This is foolish by your opinion? OK, no problem. Don't use it.

Yes. It adds a _new_ dependency on centeralized servers (though more widely spread ones). This is completely unnecessary because the Bitcoin protocol already has a facility for discovering your address from the peers you're already connecting to built into it.  There is no need to add support for an additional complex (think attack surface) protocol with more hosts that could monitor users just to support figuring out what your address might be.

[maxihatop]

As I stated before, my current version: BitcoinCore v 0.9.1.0-g026a939-beta (64 bit) reports about my IP to dyndns.org.

Can you provide reference to Bitcoin-core wallet program, which does not report about me to dyndns.org?

This is really would be helpful.

[gmaxwell]

As I stated before, my current version: BitcoinCore v 0.9.1.0-g026a939-beta (64 bit) reports about my IP to dyndns.org.
Can you provide reference to Bitcoin-core wallet program, which does not report about me to dyndns.org?
This is really would be helpful.

Sure, https://github.com/gmaxwell/bitcoin/tree/extip3

[btchris]

I'm all for removing the check to dyndns.org, but it seems like the two pull requests which attempted this were trickier to get exactly "right" then was first imagined... it's too bad, but sometimes there are bigger fish to fry.

Thanks to gmaxwell and laanwj for giving it a try, though.

[gmaxwell]

I'm all for removing the check to dyndns.org, but it seems like the two pull requests which attempted this were trickier to get exactly "right" then was first imagined... it's too bad, but sometimes there are bigger fish to fry.

It's a trivial change, but one that requires some testing and no one has had any chance to test it. It has previously been ~zero priority. (the second pull request isn't a separate one, it's just a rebase). It's somewhat annoying to test because you really want to test it from an IP which hasn't previously run a node so you can verify that it successfully gets connections.  If it's interesting you please test it and provide feedback.

[btchris]

It's a trivial change, but one that requires some testing and no one has had any chance to test it. It has previously been ~zero priority. (the second pull request isn't a separate one, it's just a rebase). It's somewhat annoying to test because you really want to test it from an IP which hasn't previously run a node so you can verify that it successfully gets connections.  If it's interesting you please test it and provide feedback.

I am interested, but only if you think such feedback would be helpful (not necessarily sufficient, but at least helpful).

I was thinking of a simple test scenario such as this:

Linux-VM-w/Bitcoin  -----  VM-NAT-router-w/Public-IP ----- mainnet

Two tests with different Public IPs (likely never running Bitcoin before*), one where VM-NAT-router is running UPnP, one without. I could do a testnet-in-a-box, but it doesn't seem any easier to me...

If I have extra time, I might do an additional tests with both 32 and 64-bit versions (so a total of 4 tests on 4 different IPs).

I might be able to run a single Win64 or Win32 test as well, behind a residential cable modem NAT box, again likely never running Bitcoin before. I don't have the resources to run any OS X tests.

Before each test, I should verify that UPnP is working (or is not working) as expected.

I'd need to verify the existence of inbound connections, preferably on the VM-NAT-router. In the non-UPnP case, I'd need to manually forward a port first. I'll save the log files for future reference. I should probably save a packet capture as well. Anything else? Do I have to wait for a full blockchain download (I wasn't planning to)?

For building, I'll probably rebase your branch from a month ago to master.

* All of these IPs are dynamic, so although none have intentionally been running a Bitcoin node, I can't be sure. For the VM tests, I can leave the VM-NAT router running for an hour or so before beginning each test and check that no inbound connections are occurring before the test starts.

[K1773R]

Emercoin has implemented this with STUN, worth taking a look at it.

[e4xit]

Emercoin has implemented this with STUN, worth taking a look at it.

As discussed (and debunked?) above.

[K1773R]

Emercoin has implemented this with STUN, worth taking a look at it.

As discussed (and debunked?) above.

if you refer to https://bitcointalk.org/index.php?topic=134731.msg8335303#msg8335303, yes.
its a copy of this post/thread https://bitcointalk.org/index.php?topic=736721.0 (the post was in emercoins thread at the beginning).

didnt see someone else already mentioned it (and my post in the beginning). seems to be a thread made of posts from other threads.

[spin]

Sure, https://github.com/gmaxwell/bitcoin/tree/extip3

Is there a compiled version of this?  Happy to test.