Bitcoin Forum
December 09, 2016, 02:08:46 AM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: [1] 2 »  All
  Print  
Author Topic: Getting the chain faster - more than 8 outbound connections  (Read 4361 times)
Realpra
Hero Member
*****
Offline Offline

Activity: 819


View Profile
March 27, 2012, 08:05:05 AM
 #1

Hey,

would it be possible to edit the source code so that more than 8 outbound connections were allowed?

(See this thread: https://bitcointalk.org/index.php?topic=41726.msg507980#msg507980)

Could anyone point me to the part of the source code I need to edit?

I think this should be standard, at least until you have the block chain - this will decrease the annoyance experienced by new users.
(or people like me with multiple computers)

I don't want to trust an online wallet.

Is there any selfish thin clients out yet? Something I could easily put on new computers?

Cheap and sexy Bitcoin card/hardware wallet, buy here:
http://BlochsTech.com
1481249326
Hero Member
*
Offline Offline

Posts: 1481249326

View Profile Personal Message (Offline)

Ignore
1481249326
Reply with quote  #2

1481249326
Report to moderator
1481249326
Hero Member
*
Offline Offline

Posts: 1481249326

View Profile Personal Message (Offline)

Ignore
1481249326
Reply with quote  #2

1481249326
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
Revalin
Hero Member
*****
Offline Offline

Activity: 728


165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g


View Profile
March 27, 2012, 08:16:39 AM
 #2

The blockchain download speed is mostly limited by the time it takes the database to commit to the disk.  Extra connections won't help.

      War is God's way of teaching Americans geography.  --Ambrose Bierce
Bitcoin is the Devil's way of teaching geeks economics.  --Revalin 165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g
Realpra
Hero Member
*****
Offline Offline

Activity: 819


View Profile
March 27, 2012, 08:25:32 AM
 #3

The blockchain download speed is mostly limited by the time it takes the database to commit to the disk.  Extra connections won't help.

That is blatantly false, the size was what 500 mb in 2011 and now maybe its double that - I could copy that from disk to disk in a few minutes.

I have been waiting days with the client running - its crazy.

The 8 connections only is the problem.

Cheap and sexy Bitcoin card/hardware wallet, buy here:
http://BlochsTech.com
JoelKatz
Legendary
*
Offline Offline

Activity: 1386


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 27, 2012, 08:28:22 AM
 #4

The blockchain download speed is mostly limited by the time it takes the database to commit to the disk.  Extra connections won't help.

That is blatantly false, the size was what 500 mb in 2011 and now maybe its double that - I could copy that from disk to disk in a few minutes.
Copying != committing. Committing takes about three spins of the drive per object committed.

Quote
I have been waiting days with the client running - its crazy.

The 8 connections only is the problem.
It's more likely the time it take to commit each block and transaction to disk and verify all the transactions in it. How many blocks per minute are you getting? What's your CPU load look like? What CPU do you have?

I am an employee of Ripple.
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
Revalin
Hero Member
*****
Offline Offline

Activity: 728


165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g


View Profile
March 27, 2012, 08:42:33 AM
 #5

To maintain database integrity it writes out a journal of the transaction to be peformed, syncs the disk, writes the transaction, syncs again, then deletes the journal.  This hits the disk much harder than a simple copy and it has to be done millions of times.

If you want more connections you can open the inbound port and people will connect to you.  If you want to force more connections you can edit MAX_OUTBOUND_CONNECTIONS in net.cpp.

If you actually want it to go faster, try the beta.  The database settings have been adjusted for better performance: https://bitcointalk.org/index.php?topic=74214.0

If you want to skip the download process you can download a blockchain from here:  http://eu1.bitcoincharts.com/blockchain/

      War is God's way of teaching Americans geography.  --Ambrose Bierce
Bitcoin is the Devil's way of teaching geeks economics.  --Revalin 165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g
Realpra
Hero Member
*****
Offline Offline

Activity: 819


View Profile
March 27, 2012, 08:48:32 AM
 #6

Copying != committing. Committing takes about three spins of the drive per object committed.
Even so this seems to be taking too long unless the database commits are being done in horrendous manor...

Quote
It's more likely the time it take to commit each block and transaction to disk and verify all the transactions in it. How many blocks per minute are you getting? What's your CPU load look like? What CPU do you have?
I get about 50 a minute, thats about 41 kb/s.

My load is around 5-10% average. Memory usage is a bit high right now since I doing programming in visual studio, but I have plenty to spare.

CPU is AMD64 dual core 5400+ 2.81 GHz. I run most heavy games though not at full graphics anymore.

I'm in Scandinavia so I don't know how good the node coverage is here.


I have modified the source code now, there was a constant in the top of file src/Net.cpp with MAX_OUTBOUND or something. Was 8 and I changed it to 80.

Program didn't seem to recompile though.

Cheap and sexy Bitcoin card/hardware wallet, buy here:
http://BlochsTech.com
Realpra
Hero Member
*****
Offline Offline

Activity: 819


View Profile
March 27, 2012, 08:50:50 AM
 #7

If you want more connections you can open the inbound port and people will connect to you.  If you want to force more connections you can edit MAX_OUTBOUND_CONNECTIONS in net.cpp.

Yeah did that before, how do I make it recompile? (I still get 8 connections only)

Cheap and sexy Bitcoin card/hardware wallet, buy here:
http://BlochsTech.com
Revalin
Hero Member
*****
Offline Offline

Activity: 728


165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g


View Profile
March 27, 2012, 08:56:14 AM
 #8

Have you read src/doc/build-msw.txt ?

It should build under mingw.  I don't know if VS works.

      War is God's way of teaching Americans geography.  --Ambrose Bierce
Bitcoin is the Devil's way of teaching geeks economics.  --Revalin 165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g
Realpra
Hero Member
*****
Offline Offline

Activity: 819


View Profile
March 27, 2012, 08:59:51 AM
 #9

Have you read src/doc/build-msw.txt ?

It should build under mingw.  I don't know if VS works.
Hmm I don't see that folder in src.

I'm not building with VS thats just homework Smiley

I kinda thought the bitcoin client was built each time you ran it - hence why the source files where there...

Cheap and sexy Bitcoin card/hardware wallet, buy here:
http://BlochsTech.com
Revalin
Hero Member
*****
Offline Offline

Activity: 728


165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g


View Profile
March 27, 2012, 09:01:34 AM
 #10

Nope.  The source is there to encourage people to look at it.

      War is God's way of teaching Americans geography.  --Ambrose Bierce
Bitcoin is the Devil's way of teaching geeks economics.  --Revalin 165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g
theymos
Administrator
Legendary
*
expert
Offline Offline

Activity: 2506


View Profile
March 27, 2012, 09:02:36 AM
 #11

Bitcoin only downloads the chain from one of your peers at a time, so connecting to more will only increase your chances of connecting to a fast one. You could get the same effect by "-connect"ing to a node you know is fast.

(As others have mentioned, network speed isn't usually the limiting factor, anyway.)

1NXYoJ5xU91Jp83XfVMHwwTUyZFK64BoAD
JoelKatz
Legendary
*
Offline Offline

Activity: 1386


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 27, 2012, 10:12:13 AM
 #12

I get about 50 a minute, thats about 41 kb/s.

My load is around 5-10% average. Memory usage is a bit high right now since I doing programming in visual studio, but I have plenty to spare.
To clarify, you are seeing CPU load of around 5-10% while you are getting blocks at about 50 a minute. Is that correct?

I am an employee of Ripple.
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
Pieter Wuille
Legendary
*
qt
Offline Offline

Activity: 1036


View Profile WWW
March 27, 2012, 11:31:09 AM
 #13

As I've said before in another thread, the initial synchronisation process consists of three steps:
  • Downloading the data: limited by your and your peer's network speed
  • Maintaining the database: mostly limited by disk latency
  • Verifying the chain: limited by CPU speed

Before 0.6.0, maintaining the database almost always outweighed the other two. With 0.6.0rc5 the caching settings have been tweaked, resulting in a remarkable speedup for the database. This means that now the downloading time may indeed become significant, and it may be worth looking at improving the download process, for example downloading from several peers at once.

Increasing the number of outgoing connections will not help you get the chain faster: even when you use 0.6.0rc5, the chain is still downloaded from a single peer, and frequently still not the slowest step. In fact, increasing this number is a bad idea for the network, as peers with open incoming ports are not too abundant.

aka sipa, core dev team

Tips and donations: 1KwDYMJMS4xq3ZEWYfdBRwYG2fHwhZsipa
Realpra
Hero Member
*****
Offline Offline

Activity: 819


View Profile
March 27, 2012, 12:43:09 PM
 #14

I get about 50 a minute, thats about 41 kb/s.

My load is around 5-10% average. Memory usage is a bit high right now since I doing programming in visual studio, but I have plenty to spare.
To clarify, you are seeing CPU load of around 5-10% while you are getting blocks at about 50 a minute. Is that correct?

Yes:

50 blocks/minute
Estimated datatransfer speed 41 kb/s (with each block = 50 kb)
5-10% average CPU load
1.5 gb of 2 gb ram allocated (windows 7... seems to load a lot of things - no slowdowns though)

EDIT: My client is the one from bitcoin.org, version 0.5.2

Cheap and sexy Bitcoin card/hardware wallet, buy here:
http://BlochsTech.com
wabber
Member
**
Offline Offline

Activity: 85


View Profile
March 27, 2012, 02:48:09 PM
 #15

The number of connections really doesn't matter. I've downloaded the chain at pretty much the same speed with 120 connections.
If you don't believe us forward the bitcoin port to your pc in your router. It doesn't matter if the additional connections are incoming or outgoing but it's better for the network health to have incoming connections and you wont have to recompile your client.

The limiting factor is almost always the harddisk and it has been improved in the new beta release AFAIK. Again if you don't believe us create a ramdisk and download the blockchain there. You should see a significant speed up since the RAM is much faster than your harddisk.
Mushoz
Hero Member
*****
Offline Offline

Activity: 686


Bitbuy


View Profile WWW
March 27, 2012, 03:22:20 PM
 #16

As I've said before in another thread, the initial synchronisation process consists of three steps:
  • Downloading the data: limited by your and your peer's network speed
  • Maintaining the database: mostly limited by disk latency
  • Verifying the chain: limited by CPU speed

Before 0.6.0, maintaining the database almost always outweighed the other two. With 0.6.0rc5 the caching settings have been tweaked, resulting in a remarkable speedup for the database. This means that now the downloading time may indeed become significant, and it may be worth looking at improving the download process, for example downloading from several peers at once.

Increasing the number of outgoing connections will not help you get the chain faster: even when you use 0.6.0rc5, the chain is still downloaded from a single peer, and frequently still not the slowest step. In fact, increasing this number is a bad idea for the network, as peers with open incoming ports are not too abundant.

Thank you very much for tweaking the caching settings! Tremendous improvement in synchronize speed! I too believe downloading from multiple peers at once is going to be the next logical step in speeding the initial download up. Do you happen to know how hard this would be to implement? It's not as necessary as improving the caching was, but it would most definitely help and would be very welcome Smiley

www.bitbuy.nl - Koop eenvoudig, snel en goedkoop bitcoins bij Bitbuy!
Diapolo
Hero Member
*****
Offline Offline

Activity: 769



View Profile WWW
March 27, 2012, 04:13:28 PM
 #17

If you can, put your datadir on a SSD Cheesy!

Dia

Liked my former work for Bitcoin Core? Drop me a donation via:
1PwnvixzVAKnAqp8LCV8iuv7ohzX2pbn5x
bitcoin:1PwnvixzVAKnAqp8LCV8iuv7ohzX2pbn5x?label=Diapolo
dooglus
Legendary
*
Offline Offline

Activity: 2002



View Profile
March 27, 2012, 10:29:15 PM
 #18

I timed the initial blockchain download with v0.6.0rc4 and rc5.  rc5 is faster, but still gets very slow after a while.  Notice both charts have a flat point at around 93k blocks, and the shapes are otherwise very similar.

The graph shows blocks downloaded against time hh:mm


JoelKatz
Legendary
*
Offline Offline

Activity: 1386


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 27, 2012, 11:33:01 PM
 #19

That's probably because the average block starts to contain transactions that require looking at more blocks that are, on average, further back. Also, the average block contains more transactions.

I am an employee of Ripple.
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
dooglus
Legendary
*
Offline Offline

Activity: 2002



View Profile
March 28, 2012, 01:47:27 AM
 #20

I tried again, this time having ~/.bitcoin/ be a ram disk.

Code:
$ mv .bitcoin .bitcoin.real
$ mkdir .bitcoin
$ sudo mount -t tmpfs -o size=1600M tmpfs .bitcoin

I thought 1600 megabytes would be big enough, but forgot to take the database logfiles into account, and it ran out of space after about downloading 136k blocks in 26 minutes.  The shape of the curve is still the same.  The CPU very rarely was flat out during the download, and so I'm thinking the limit now is how quickly the blocks are coming over the network.  The blocks get bigger with time, so take longer to transmit.


Pages: [1] 2 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!