Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: hhanh00 on March 14, 2015, 02:04:24 PM



Title: How does the protocol broadcast hidden services?
Post by: hhanh00 on March 14, 2015, 02:04:24 PM
When I run a regular node, my listening endpoint is part of the version msg and is relayed through addr messages.
But when I run it as a Tor hidden service, is it advertised or people who want to connect must know my service onion address?


Title: Re: How does the protocol broadcast hidden services?
Post by: theymos on March 14, 2015, 07:16:12 PM
They're encoded as special IPv6 addresses and then passed around normally. IPv6-encoded hidden service addresses start with the prefix FD87:D87E:EB43. If you run Bitcoin through Tor with -externalip=___.onion and -listen=1, you should eventually get incoming connections.


Title: Re: How does the protocol broadcast hidden services?
Post by: ca333 on March 14, 2015, 11:18:31 PM
They're encoded as special IPv6 addresses and then passed around normally. IPv6-encoded hidden service addresses start with the prefix FD87:D87E:EB43. If you run Bitcoin through Tor with -externalip=___.onion and -listen=1, you should eventually get incoming connections.

so the client in the TOR-network can itself discover peers when connected to the TOR (with -onlynet=tor) with the above externalip parameter("-externalip=___.onion")?

#EDIT: in tor-talk mailing list i read a talk (from Alex Biryukov and Ivan Pustogarov) about this and it say without .onion id in .config there is no chance for the client to advertise to other peer..  


Title: Re: How does the protocol broadcast hidden services?
Post by: theymos on March 15, 2015, 12:08:33 AM
so the client in the TOR-network can itself discover peers when connected to the TOR (with -onlynet=tor) with the above externalip parameter("-externalip=___.onion")?

#EDIT: in tor-talk mailing list i read a talk (from Alex Biryukov and Ivan Pustogarov) about this and it say without .onion id in .config there is no chance for the client to advertise to other peer.. 

By "-externalip=___.onion", I mean that you have to replace ___.onion with a hidden service address that you already set up with Tor. If you also have listen=1 set, Bitcoin will then advertise your hidden service to the network automatically.

If you don't do this, then Bitcoin will still work through Tor, and you might automatically make outgoing connections to hidden services, but you won't get any incoming connections. Bitcoin doesn't set up a hidden service for itself automatically.


Title: Re: How does the protocol broadcast hidden services?
Post by: hhanh00 on March 15, 2015, 04:32:15 AM
Thanks, with your help I found a post that documents the encoding[1].

[1]: https://lists.torproject.org/pipermail/tor-talk/2012-June/024591.html


Title: Re: How does the protocol broadcast hidden services?
Post by: gmaxwell on March 15, 2015, 09:19:52 PM
If you don't do this, then Bitcoin will still work through Tor, and you might automatically make outgoing connections to hidden services, but you won't get any incoming connections. Bitcoin doesn't set up a hidden service for itself automatically.
Yep. Just so.

It can't setup a hidden service for itself. We've asked the tor project for some kind of ability to control HS setup from socks and/or the control port and they have a feature request for it (and have for a number of years), but it isn't there yet.

Same reason you need to tell bitcoin what your onion address is: there is no way for Bitcoin to find out on its own... only systems with effective access control (e.g. stock tor install on most Linux distros) it can't even read the relevant files to go find out for itself.

The file doc/tor.md included with Bitcoin Core describes the settings.

Once set up it will do automatic discovery just fine. There is no need to use that fallback node list on the Bitcoin wiki ever.


Title: Re: How does the protocol broadcast hidden services?
Post by: hhanh00 on March 16, 2015, 12:15:10 AM
When the core client runs through Tor and looks for seed nodes. Is DNS still used? And if so, does it go through the Tor proxy too?


Title: Re: How does the protocol broadcast hidden services?
Post by: theymos on March 16, 2015, 12:56:52 AM
When the core client runs through Tor and looks for seed nodes. Is DNS still used? And if so, does it go through the Tor proxy too?

Yes, if you configure Bitcoin to use Tor then it will use Tor for the DNS seeds.

A good thing that BitcoinJ does is that it makes sure that it connects to the DNS seeds via at least two(?) different Tor exit nodes. I don't think that Bitcoin Core does this, though it might be a good idea.


Title: Re: How does the protocol broadcast hidden services?
Post by: gmaxwell on March 16, 2015, 04:01:28 AM
When the core client runs through Tor and looks for seed nodes. Is DNS still used? And if so, does it go through the Tor proxy too?
DNS can't simply be used over Tor. What it does is it "connects" to the DNSseed names like they were regular peers and gets addrs from them and disconnects, causing the tor network to do the dns resolution and randomly select endpoints.  It's not great.

Quote
I don't think that Bitcoin Core does this, though it might be a good idea.
There is an open PR on making it use separate tor circuits to reduce the incidence of using the same exit node (but not preventing it: there isn't a way to prevent it without having a very low level interface with tor, AFAIK).  I think we previously got sidetracked with discussion on how to avoid breaking non-tor proxies. (The way you get different circuits for different connections in tor is to send different usernames; which doesn't work so well if you're using a non-tor socks proxy and it won't accept a username). I'll be in the next release in any case.


Title: Re: How does the protocol broadcast hidden services?
Post by: hhanh00 on March 16, 2015, 01:13:03 PM
Thanks, I was wondering how one could tunnel a DNS request through the socks 5 proxy. It turns out, we can't.


Title: Re: How does the protocol broadcast hidden services?
Post by: belcher on March 12, 2016, 04:02:03 PM
Found this topic, it's worth a bump for new information.

Bitcoin 0.12 now makes bitcoin-over-tor much easier.

https://bitcoin.org/en/release/v0.12.0#automatically-use-tor-hidden-services

Quote
Starting with Tor version 0.2.7.1 it is possible, through Tor’s control socket API, to create and destroy ‘ephemeral’ hidden services programmatically. Bitcoin Core has been updated to make use of this.

This means that if Tor is running (and proper authorization is available), Bitcoin Core automatically creates a hidden service to listen on, without manual configuration. Bitcoin Core will also use Tor automatically to connect to other .onion nodes if the control socket can be successfully opened. This will positively affect the number of available .onion nodes and their usage.

This new feature is enabled by default if Bitcoin Core is listening, and a connection to Tor can be made. It can be configured with the -listenonion, -torcontrol and -torpassword settings. To show verbose debugging information, pass -debug=tor.

https://github.com/bitcoin/bitcoin/pull/6639
https://github.com/bitcoin/bitcoin/pull/7090


Title: Re: How does the protocol broadcast hidden services?
Post by: Pkzone on March 12, 2016, 05:27:58 PM
When the core client runs through Tor and looks for seed nodes. Is DNS still used? And if so, does it go through the Tor proxy too?
DNS can't simply be used over Tor. What it does is it "connects" to the DNSseed names like they were regular peers and gets addrs from them and disconnects, causing the tor network to do the dns resolution and randomly select endpoints.  It's not great.

Quote
I don't think that Bitcoin Core does this, though it might be a good idea.
There is an open PR on making it use separate tor circuits to reduce the incidence of using the same exit node (but not preventing it: there isn't a way to prevent it without having a very low level interface with tor, AFAIK).  I think we previously got sidetracked with discussion on how to avoid breaking non-tor proxies. (The way you get different circuits for different connections in tor is to send different usernames; which doesn't work so well if you're using a non-tor socks proxy and it won't accept a username). I'll be in the next release in any case.

Link to the PR?


Title: Re: How does the protocol broadcast hidden services?
Post by: Polyatomic on March 13, 2016, 07:51:38 AM
Link to the PR?

Seems to be in docs Pkzone

git grep 'different exit node'
Code:
doc/release-notes/release-notes-0.11.0.md:different exit node. That significantly reduces the chance to get unlucky and

git blame -L '/different exit node/',-15 ./doc/release-notes/release-notes-0.11.0.md
Code:
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 191) 
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 192) One such application is selective Tor usage, where the node runs on the normal
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 193) internet but transactions are broadcasted over Tor.
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 194)
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 195) For an example script see [bitcoin-submittx](https://github.com/laanwj/bitcoin-submittx).
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 196)
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 197) Privacy: Stream isolation for Tor
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 198) ----------------------------------
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 199)
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 200) This release adds functionality to create a new circuit for every peer
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 201) connection, when the software is used with Tor. The new option,
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 202) `-proxyrandomize`, is on by default.
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 203)
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 204) When enabled, every outgoing connection will (potentially) go through a
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 205) different exit node. That significantly reduces the chance to get unlucky and



Title: Re: How does the protocol broadcast hidden services?
Post by: Pkzone on March 14, 2016, 05:00:51 PM
Link to the PR?

Seems to be in docs Pkzone

git grep 'different exit node'
Code:
doc/release-notes/release-notes-0.11.0.md:different exit node. That significantly reduces the chance to get unlucky and

git blame -L '/different exit node/',-15 ./doc/release-notes/release-notes-0.11.0.md
Code:
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 191) 
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 192) One such application is selective Tor usage, where the node runs on the normal
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 193) internet but transactions are broadcasted over Tor.
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 194)
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 195) For an example script see [bitcoin-submittx](https://github.com/laanwj/bitcoin-submittx).
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 196)
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 197) Privacy: Stream isolation for Tor
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 198) ----------------------------------
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 199)
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 200) This release adds functionality to create a new circuit for every peer
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 201) connection, when the software is used with Tor. The new option,
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 202) `-proxyrandomize`, is on by default.
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 203)
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 204) When enabled, every outgoing connection will (potentially) go through a
75a88039 (Casey Rodarmor 2015-08-05 11:22:14 -0400 205) different exit node. That significantly reduces the chance to get unlucky and


OK I get it, will look out.


Title: Re: How does the protocol broadcast hidden services?
Post by: bananasandalmondmilk on April 16, 2016, 08:06:38 PM
I'm trying to do "Automatically listen on Tor"
https://github.com/bitcoin/bitcoin/blob/master/doc/tor.md

"Starting with Tor version 0.2.7.1 it is possible, through Tor's control socket API, to create and destroy 'ephemeral' hidden services programmatically. Bitcoin Core has been updated to make use of this.

This means that if Tor is running (and proper authorization is available), Bitcoin Core automatically creates a hidden service to listen on, without manual configuration. This will positively affect the number of available .onion nodes.

This new feature is enabled by default if Bitcoin Core is listening, and a connection to Tor can be made. It can be configured with the -listenonion, -torcontrol and -torpassword settings. To show verbose debugging information, pass -debug=tor."


I tried:
bitcoin-cli -listenonion

but I get error:

error: too few parameters

so how can I run bitcoin on Tor?


Title: Re: How does the protocol broadcast hidden services?
Post by: achow101 on April 16, 2016, 08:26:12 PM
I'm trying to do "Automatically listen on Tor"
https://github.com/bitcoin/bitcoin/blob/master/doc/tor.md

"Starting with Tor version 0.2.7.1 it is possible, through Tor's control socket API, to create and destroy 'ephemeral' hidden services programmatically. Bitcoin Core has been updated to make use of this.

This means that if Tor is running (and proper authorization is available), Bitcoin Core automatically creates a hidden service to listen on, without manual configuration. This will positively affect the number of available .onion nodes.

This new feature is enabled by default if Bitcoin Core is listening, and a connection to Tor can be made. It can be configured with the -listenonion, -torcontrol and -torpassword settings. To show verbose debugging information, pass -debug=tor."


I tried:
bitcoin-cli -listenonion

but I get error:

error: too few parameters

so how can I run bitcoin on Tor?
You have to do that to the startup command for starting bitcoind it bitcoin-qt.


Title: Re: How does the protocol broadcast hidden services?
Post by: bananasandalmondmilk on April 16, 2016, 09:22:48 PM
thanks so much usually the startup command for me is

bitcoin-server &

but I seem to have trouble passing -listenonion, -rescan, or anything to bitcoin-cli or bitcoin-server

If I can figure out what I'm doing fundamentally wrong, when I see all the guides talk about passing in these options, maybe I can figure it out!


Title: Re: How does the protocol broadcast hidden services?
Post by: achow101 on April 16, 2016, 09:28:55 PM
thanks so much usually the startup command for me is

bitcoin-server &

but I seem to have trouble passing -listenonion, -rescan, or anything to bitcoin-cli or bitcoin-server

If I can figure out what I'm doing fundamentally wrong, when I see all the guides talk about passing in these options, maybe I can figure it out!
Is bitcoin-server supposed to be bitcoind renamed? If so, then you need to have the options before the ampersand (&). Otherwise, I'm not sure if you are actually using bitcoind. Perhaps that command is actually for a script?


Title: Re: How does the protocol broadcast hidden services?
Post by: bananasandalmondmilk on April 28, 2016, 05:09:45 AM
it looks like one is the daemon and one is the RPC client, same exact version numbers

bitcoin-server --version
Bitcoin Core Daemon version v0.12.0.0-g188ca9c

bitcoind --version
Bitcoin Core RPC client version v0.12.0.0-g188ca9c