Bitcoin Forum

Bitcoin => Bitcoin Technical Support => Topic started by: bitcoinex on November 01, 2010, 01:38:04 PM



Title: This server is overloaded?
Post by: bitcoinex on November 01, 2010, 01:38:04 PM
Approximately every 3-5 request falls off due to a timeout. Especially in the evening and when writing messages.

Especially insulting when you write a post - you press send and nothing happens. You may lose the message text


Title: Re: This server is overloaded?
Post by: sirius on November 01, 2010, 02:21:19 PM
Server rebooted. Any better now?


Title: Re: This server is overloaded?
Post by: bitcoinex on November 01, 2010, 02:31:16 PM
Not sure. This post I wrote from first time. But the problem was long enough.


Title: Re: This server is overloaded?
Post by: ShadowOfHarbringer on November 05, 2010, 08:39:16 PM
Especially insulting when you write a post - you press send and nothing happens. You may lose the message text

BTW.
You can use this firefox extension (https://addons.mozilla.org/en-US/firefox/addon/6984/) to recover any form (and any post) You lost due to connection failure.
It's pretty cool.

Going back to the topic: yeah the bitcoin.org servers seem overloaded by traffic.
Satoshi should invest in better servers soon or there could be a problem.

With the amounts of bitcoins he has, that shouldn't be an issue.


Title: Re: This server is overloaded?
Post by: ShadowOfHarbringer on November 05, 2010, 09:05:13 PM
Going back to the topic: yeah the bitcoin.org servers seem overloaded by traffic.
Satoshi should invest in better servers soon or there could be a problem.

With the amounts of bitcoins he has, that shouldn't be an issue.

Sorry Shadow, but this seems a bit naive. How do you know it's not a killer server generating coins at the same time? As an open source project, the community should contribute to a server if it is needed. In all reality, this is probably simply hosted on shared hosting for a minuscule dollar amount.

If bitcoin.org needs faster / better server for hosting, let me know and I'll gladly toss it up.

Hey, that's even better.

But i just thought that only satoshi runs this server, so nobody else has access to do such thing.


Title: Re: This server is overloaded?
Post by: ribuck on November 05, 2010, 09:37:41 PM
Do you mean the server which hosts this forum? This is the fastest forum I visit, by far.

If there's a speed problem, I think it must be somewhere between the server and the users who are getting slow performance.


Title: Re: This server is overloaded?
Post by: theymos on November 06, 2010, 12:19:12 AM
In all reality, this is probably simply hosted on shared hosting for a minuscule dollar amount.

Bitcoin.org accepts Bitcoin connections, so it must be at least a VPS.


Title: Re: This server is overloaded?
Post by: bitcoinex on November 06, 2010, 07:13:46 AM
The problem is not with the passage of traffic into remote corners of the planet. The only problem when you write or somehow change the information on site.

I think that the number of Apache preforks is little or (if there is a separate php fasttcgi) php workers are few or leaked and ate memory or something like that. Or just DB is overloaded?

I understand correctly that you are using bare Apache? There is not reason to use Apache unless you use its functions as an application server (LDAP auth, for example).

Try to put nginx or lighttpd and  php as a fastcgi. In any way this will save 90% of resources.
I am ready to assist with this.

Theese configs from btcex.com (Debian Linux):

Code:
$ cat /etc/nginx/sites-enabled/btcex.com 
log_format  withservname   '$host:$server_port $remote_addr $remote_user [$time_local] '
                           '"$request" $status $body_bytes_sent '
                           '"$http_referer" "$http_user_agent"';

access_log  /var/log/nginx/access.log  withservname;

server_tokens  off;

server {
    server_name  www.btcex.com;
    listen  80;
    rewrite  ^(.*)$ http://btcex.com$1 permanent;
}

server {
    server_name  btcex.com;
    listen  80;
    rewrite  ^(.*)$ https://btcex.com$1 permanent;
}

server {
server_name  btcex.com;

#listen  80;
listen  443 default ssl;

keepalive_timeout    70;

ssl                  on;
ssl_certificate      /etc/ssl/certs/startssl_btcex.com.pem;
ssl_certificate_key  /etc/ssl/private/btcex.com.key;
ssl_session_cache    shared:SSL:10m;
ssl_session_timeout  10m;

location / {
    try_files  $uri  $uri/ @yii;
    root   /var/www/btcex.com/htdocs;
    index  index.html index.htm index.php;

    location  ~ /\.ht {
deny  all;
    }

    location  /var/www/btcex.com/htdocs/protected/ {
deny  all;
    }

# pass the PHP scripts to FastCGI server
# (not used on btcex.com really)
location ~ \.php$ {
    fastcgi_index  index.php;
    #fastcgi_pass localhost:999;
    fastcgi_pass  unix:/var/run/php-fcgi.sock;
    fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
    include fastcgi_params;
}

    location  ~* \.(gif|jp(|e)g|png|mp(|e)g|avi|flv|swf)$ {
expires  1d;
    }
}

# yii is our web framework:
location @yii {
    root  /var/www/btcex.com/htdocs;

    include fastcgi_params; # в нaчaлe, чтoбы нe влиялo нa cлeдyющиe oпции
    fastcgi_pass  unix:/var/run/php-fcgi.sock;
    fastcgi_param  SCRIPT_NAME      /index.php;
    # нe oчeнь xopoшo чтo index.php лeжит в диpeктopии caйтa нo пycть пoкa тaк бyдeт:
    fastcgi_param  SCRIPT_FILENAME  $document_root/index.php;
    fastcgi_param  QUERY_STRING     $args;
}

}

and fastcgi php daemon ("runit" script, debian not contain ready script for fastcgi php in its repository http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=426780 ):

Code:
$ cat /etc/service/php-fcgi/run 
#!/bin/sh
# Use this as a ./run script with daemontools or runit
# You should replace xxx with the user you want php to run as (and www-data with the user lighty runs as)

exec 2>&1
PHP_FCGI_CHILDREN=8 \
PHP_FCGI_MAX_REQUESTS=1500 \
exec /usr/bin/spawn-fcgi -n -s /var/run/php-fcgi.sock -n -u www-data -- /usr/bin/php-cgi


Title: Re: This server is overloaded?
Post by: bitcoinex on November 06, 2010, 07:36:53 AM
However, first I change browser for 1-2 days.

I have a lot of plugins in it. Once was so that the plugin prevented switching to Tor, and this time some such reason may be.


Title: Re: This server is overloaded?
Post by: bitcoinex on November 07, 2010, 06:46:25 PM
However, first I change browser for 1-2 days.

I have a lot of plugins in it. Once was so that the plugin prevented switching to Tor, and this time some such reason may be.

problem is still  persists. Anyone else seen it?

upd:
Now do not opens topics that changes were just made. Can be a problem in the database?

Do not leave me!


Title: Re: This server is overloaded?
Post by: ShadowOfHarbringer on November 16, 2010, 09:08:28 PM
However, first I change browser for 1-2 days.

I have a lot of plugins in it. Once was so that the plugin prevented switching to Tor, and this time some such reason may be.

problem is still  persists. Anyone else seen it?

upd:
Now do not opens topics that changes were just made. Can be a problem in the database?

Do not leave me!

No i haven't seen it. The servers are blazingly fast for me.
Perhaps your proxy/tunnel/vpn/tor/whatever is malfunctioning.

If you're getting bad transfers from russia, perhaps You should try tunneling to some fast-internet country as USA, france or germany. There are some VPN's & SSH accounts for tunneling for sale there.

Also, check Your DNSes - the problems may be caused by slow domain resolving.


Title: Re: This server is overloaded?
Post by: bitcoinex on November 18, 2010, 10:21:42 PM
However, first I change browser for 1-2 days.

I have a lot of plugins in it. Once was so that the plugin prevented switching to Tor, and this time some such reason may be.

problem is still  persists. Anyone else seen it?

upd:
Now do not opens topics that changes were just made. Can be a problem in the database?

Do not leave me!

No i haven't seen it. The servers are blazingly fast for me.
Perhaps your proxy/tunnel/vpn/tor/whatever is malfunctioning.

If you're getting bad transfers from russia, perhaps You should try tunneling to some fast-internet country as USA, france or germany. There are some VPN's & SSH accounts for tunneling for sale there.

Also, check Your DNSes - the problems may be caused by slow domain resolving.


I think, this is problem in my ISP, who apparently has established a poor configured transparent proxy.

after moving to https problem has completely disappeared


Title: Re: This server is overloaded?
Post by: ShadowOfHarbringer on November 19, 2010, 08:29:47 AM
I think, this is problem in my ISP, who apparently has established a poor configured transparent proxy.

after moving to https problem has completely disappeared

Yeah, i would also check if Your ISP is not spying on You / sniffing Your traffic.
In Soviet Russia, everything is possible :P


Title: Re: This server is overloaded?
Post by: bitcoinex on November 19, 2010, 01:22:40 PM
I think, this is problem in my ISP, who apparently has established a poor configured transparent proxy.

after moving to https problem has completely disappeared

Yeah, i would also check if Your ISP is not spying on You / sniffing Your traffic.
In Soviet Russia, everything is possible :P

I'm sure what ISP does it - the law obliges to do it.


Title: Re: This server is overloaded?
Post by: ShadowOfHarbringer on November 19, 2010, 01:37:59 PM
I think, this is problem in my ISP, who apparently has established a poor configured transparent proxy.

after moving to https problem has completely disappeared

Yeah, i would also check if Your ISP is not spying on You / sniffing Your traffic.
In Soviet Russia, everything is possible :P

I'm sure what ISP does it - the law obliges to do it.

So we have the explanation.

Just use some good (and paid) VPN service for all Your traffic. Your problems should disappear.


Title: Re: This server is overloaded?
Post by: bitcoinex on November 19, 2010, 10:04:28 PM
I think, this is problem in my ISP, who apparently has established a poor configured transparent proxy.

after moving to https problem has completely disappeared

Yeah, i would also check if Your ISP is not spying on You / sniffing Your traffic.
In Soviet Russia, everything is possible :P

I'm sure what ISP does it - the law obliges to do it.

So we have the explanation.

Just use some good (and paid) VPN service for all Your traffic. Your problems should disappear.

The problem occurs only with this site! I think the KGB is watching us. :)


Title: Re: This server is overloaded?
Post by: wumpus on November 20, 2010, 12:23:11 PM
it's way overdue anyway for the entire web to switch to https


Title: Re: This server is overloaded?
Post by: ribuck on November 20, 2010, 12:31:23 PM
By the way, who pays for bitcoin.org and for the forum server?

How about a "donate" button? Every time I see those checksums at the bottom of the bitcoin.org home page (http://bitcoin.org) I have to resist the urge to send some bitcoins to them.


Title: Re: This server is overloaded?
Post by: The Madhatter on November 20, 2010, 12:45:40 PM
it's way overdue anyway for the entire web to switch to https

That will never happen.

I used to think that too; until I flipped some of my extremely high traffic sites to https and my bandwidth consumption jumped. Turns out ISPs that used to be able to cache my images are unable to now. :P

https is great, but it sucks on sites with lots and lots of images. :(


Title: Re: This server is overloaded?
Post by: wumpus on November 20, 2010, 12:58:53 PM
it's way overdue anyway for the entire web to switch to https

That will never happen.

I used to think that too; until I flipped some of my extremely high traffic sites to https and my bandwidth consumption jumped. Turns out ISPs that used to be able to cache my images are unable to now. :P

https is great, but it sucks on sites with lots and lots of images. :(
I agree that plain http is better for static non-restricted stuff such as images, css, js, which is perfectly cachable. The main problem is that you can't use http images in https sites without creating browser warnings (the reason for this being insertion/xss attacks). A compromised cached proxy server could insert arbitrary images/css/js (and thus, scripts) into your site. (This could be solved if http supported content signing and checking on import, but that'd require browser and protocol changes)

Eventually security will trump bandwidth and CPU concerns, as people will trust more of their life to internet, and it becomes easier and easier for laymen to sniff plaintext connections and hijack connections (firesheep et al).
You can see it now with gmail, hotmail switching to https. That's only the beginning, many more are on the verge of switching.




Title: Re: This server is overloaded?
Post by: The Madhatter on November 20, 2010, 01:19:44 PM
I agree that plain http is better for static non-restricted stuff such as images, css, js, which is perfectly cachable. The main problem is that you can't use http images in https sites without creating browser warnings (the reason for this being insertion/xss attacks). A compromised cached proxy server could insert arbitrary images/css/js (and thus, scripts) into your site. (This could be solved if http supported content signing and checking on import, but that'd require browser and protocol changes)

I know all of this already. You're preaching to the choir. :)

Eventually security will trump bandwidth and CPU concerns, as people will trust more of their life to internet, and it becomes easier and easier for laymen to sniff plaintext connections and hijack connections (firesheep et al).
You can see it now with gmail, hotmail switching to https. That's only the beginning, many more are on the verge of switching.

Oh sure. Webmail should have always had SSL. I'm surprised they went this long without it. I'm sure that back when Hotmail started the CPU overhead would have killed them. I know this isn't the case now.

SSL is a PITA for content delivery. It just won't fly. If I can't let ISPs cache my content I need 10x the amount of servers. Not to mention my site slows down because the ability for ISPs to cache my content local to a particular geographic region isn't possible.

SSL also doesn't stop MITM/sniffing problems since no browser cares when SSL certs change. Until browsers ship with a plugin like Certificate Patrol (for FF), SSL won't save anyone from these attacks.

Food for thought.. :)


Title: Re: This server is overloaded?
Post by: wumpus on November 20, 2010, 01:27:20 PM
SSL is a PITA for content delivery. It just won't fly. If I can't let ISPs cache my content I need 10x the amount of servers. Not to mention my site slows down because the ability for ISPs to cache my content local to a particular geographic region isn't possible.
Well, we don't really have ISP-based caching servers here in Europe, and it works OK. Most companies do their own frontend/backend caching.

And for paid content you want security in place anyway. Although my argument was not for youtube-like services. More like forums, communication, chat, mail, identification, etc... The things for which security matters.
Quote
SSL also doesn't stop MITM/sniffing problems since no browser cares when SSL certs change. Until browsers ship with a plugin like Certificate Patrol (for FF), SSL won't save anyone from these attacks.
It doesn't stop all attacks but is significantly more secure than plaintext. Nothing ever is fully secure, that is not an argument for lower security, ever!


Title: Re: This server is overloaded?
Post by: The Madhatter on November 20, 2010, 01:36:22 PM
Well, we don't really have ISP-based caching servers here in Europe, and it works OK. Most companies do their own frontend/backend caching.

Are you sure? Tons of ISPs do transparent caching.

And for paid content you want security in place anyway.

Security != encryption. For paid content (small static files), I'd choose http over https any day. For handling logins/sensitive information I'd use SSL. There is no reason to SSL an entire site. Especially when the content is public anyway.

It doesn't stop all attacks but is significantly more secure than plaintext. Nothing ever is fully secure, that is not an argument for lower security, ever!

That's not what I meant. I was pointing out that SSL isn't a "cure all" for MITM/sniffing attacks. If the browsers got their $hit together when it came to SSL it would be far better than it currently is.


Title: Re: This server is overloaded?
Post by: The Madhatter on November 20, 2010, 01:52:42 PM
it's way overdue anyway for the entire web to switch to https

Although my argument was not for youtube-like services. More like forums, communication, chat, mail, identification, etc... The things for which security matters.

These two statements are contradictory.


Title: Re: This server is overloaded?
Post by: wumpus on November 20, 2010, 01:57:00 PM
Yes, it might be. I just remembered that even digital cable TV is encrypted these days.

Even with public files, one does not necessarily want everyone to know he/she is accessing them. And someone that is sniffing can see all http requests, including full path.

You are only reasoning from the side of the service provider, not the side of your customer. In my opinion, there is no reason to not simply encrypt everything. Better be safe than sorry.




Title: Re: This server is overloaded?
Post by: The Madhatter on November 20, 2010, 02:14:31 PM
Yes, it might be. I just remembered that even digital cable TV is encrypted these days.

Sure. That is their form of access control. They broadcast their streams 24/7 to everyone. (Sometimes even to those who don't need/want the signal.) To prevent theft they scramble all of the content and rent you a box to decode it.

The Internet doesn't work the same way. The Internet has a lot of public content that has no need for encryption (you already mentioned YouTube) and you propose that we encrypt it and make the Internet slower? Why exactly? Just 'because we can'?

Even with public files, one does not necessarily want everyone to know he/she is accessing them. And someone that is sniffing can see all http requests, including full path.

With all of the SSL MITM/sniffing stuff aside, the IP connections can still be logged and the data flows measured. What you are doing can still be assumed. You can still be implicated for connecting to a naughty site, etc.

For example, say you connect to an adult site from work and this site is entirely https. Your boss still knows you were looking at porn on company time. He cares not which individual image or video you downloaded.