Holy F***, what is happening? Most Users Online today since 1 and 1/2 Years!

[cedivad]

Is mysql query cache enabled? AFAIK it is disabled by default and can really improve DB performance.

You can check with a SQL query:

Code:

Show variables like 'query_cache_size';

(0=disabled)

Thanks, I'll try enabling that!

Disable it.

I have no words to describe how dumb its to enable the query cache on such a website. Every time there is an update to a table the whole cache is invalidated (removed from the ram). Every time someone access the website the new cache is created, to be invalidate one second after.

It's counter productive. It only works with websites with no/infrequent upgrades.

If you are tiny on budget (why the hell are you using softlayer/rackspace?) and you want to improve the database performance, convert everything to InnoDB and set innodb_buffer_pool to your free ram. I suppose that the free ram should be bigger than the size of the database. If not, buy a new dedicated. Even an ovh.com/us should do, you can buy 2 monster servers for the price you pay now and setup a failover to withstand "poor" hardware iusses.

I could do that for you, I enjoy doing these things, but I'm sure you wouldn't trust me enought. So please study and fix this.

[exxe]

I have no words to describe how dumb its to enable the query cache on such a website. Every time there is an update to a table the whole cache is invalidated (removed from the ram). Every time someone access the website the new cache is created, to be invalidate one second after.

From dev.mysql.com: "When tables are modified, any relevant entries in the query cache are flushed." Not the whole cache ... but maybe I'm too dumb to understand your point of a complete cache invalidation when someone just accesses the forum.

However, I would suggest to: Enable it; Check and optimize the hit ratio; See how it performs; Disable it if it makes things worse

Cheers!

[theymos]

25% of queries are delivered from the query cache, which seems pretty good.

[cedivad]

25% of queries are delivered from the query cache, which seems pretty good.

http://www.mysqlperformanceblog.com/2011/04/10/should-we-give-a-mysqlquery-cache-a-second-chance/

Over last few years I’ve been suggesting more people to disable Query Cache than to enable it. It can cause contention problems as well as stalls and due to coarse invalidation is not as efficient as it could be.

25% of queries delivered from cache doesn't necessarily means better performances.

I don't know about how optimized the indexes are for this decade-old board, but i'm sure that you would see a huge benefit converting to a 2013 board (xenforo is my bet, IPB of VB if you wish) and using InnoDB.

[ShadowOfHarbringer]

Why the hell is the forum unable to cope with the traffic given how much has been donated to upgrade the software?  Seriously, the software upgrade seems like vapor-ware at this point.

the forum needs to be running on http://www.litespeedtech.com/ instead of apache

which i have offered up a license 100% free for use on the forum (i'd even pay the support/updates contract)  (for sale in this thread https://bitcointalk.org/index.php?topic=122271.0)

Seriously ?
Why not simply use NGINX with PHP-FPM instead (If not already using it) ?

NGINX is already used by many large sites and is blazing fast - I don't think there will be much debate over it.

PHP-FPM is slightly faster than SPAWN-FCGI or custom script dispatcher, but consumes much less memory (as all php processes share memory).

[cedivad]

Why the hell is the forum unable to cope with the traffic given how much has been donated to upgrade the software?  Seriously, the software upgrade seems like vapor-ware at this point.

the forum needs to be running on http://www.litespeedtech.com/ instead of apache

which i have offered up a license 100% free for use on the forum (i'd even pay the support/updates contract)  (for sale in this thread https://bitcointalk.org/index.php?topic=122271.0)

Seriously ?
Why not simply use NGINX with PHP-FPM instead (If not already using it) ?

NGINX is already used by many large sites and is blazing fast - I don't think there will be much debate over it.

PHP-FPM is slightly faster than SPAWN-FCGI or custom script dispatcher, but consumes much less memory (as all php processes share memory).

+1

[ShadowOfHarbringer]

Why the hell is the forum unable to cope with the traffic given how much has been donated to upgrade the software?  Seriously, the software upgrade seems like vapor-ware at this point.

the forum needs to be running on http://www.litespeedtech.com/ instead of apache

which i have offered up a license 100% free for use on the forum (i'd even pay the support/updates contract)  (for sale in this thread https://bitcointalk.org/index.php?topic=122271.0)

Seriously ?
Why not simply use NGINX with PHP-FPM instead (If not already using it) ?

NGINX is already used by many large sites and is blazing fast - I don't think there will be much debate over it.

PHP-FPM is slightly faster than SPAWN-FCGI or custom script dispatcher, but consumes much less memory (as all php processes share memory).

+1

Also, moving from MySQL to MariaDB could be helpful. "Normal" MySQL is not really developed anymore, as Oracle does not give a duck about it - they only want your money by selling you closed source extensions.

I have already moved my 2 servers with 50+ million views/month to MariaDB and I am serving ~35 million views with single quad-core server (but of course heavy caching is involved).

[cedivad]

Why the hell is the forum unable to cope with the traffic given how much has been donated to upgrade the software?  Seriously, the software upgrade seems like vapor-ware at this point.

the forum needs to be running on http://www.litespeedtech.com/ instead of apache

which i have offered up a license 100% free for use on the forum (i'd even pay the support/updates contract)  (for sale in this thread https://bitcointalk.org/index.php?topic=122271.0)

Seriously ?
Why not simply use NGINX with PHP-FPM instead (If not already using it) ?

NGINX is already used by many large sites and is blazing fast - I don't think there will be much debate over it.

PHP-FPM is slightly faster than SPAWN-FCGI or custom script dispatcher, but consumes much less memory (as all php processes share memory).

+1

Also, moving from MySQL to MariaDB could be helpful. "Normal" MySQL is not really developed anymore, as Oracle does not give a duck about it - they only want your money by selling you closed source extensions.

I have already moved my 2 servers with 50+ million views/month to MariaDB and I am serving ~35 million views with single quad-core server (but of course heavy caching is involved).

I use XtraDB instead of the standard InnoDB. I serve 40 req/s med and 100 req/s top on a single server. It heavily depends on the patter of the visits.

[farlack]

So is there any word on where the hits came from?
Can admin let us know where the hits came from?

[ShadowOfHarbringer]

Why the hell is the forum unable to cope with the traffic given how much has been donated to upgrade the software?  Seriously, the software upgrade seems like vapor-ware at this point.

the forum needs to be running on http://www.litespeedtech.com/ instead of apache

which i have offered up a license 100% free for use on the forum (i'd even pay the support/updates contract)  (for sale in this thread https://bitcointalk.org/index.php?topic=122271.0)

Seriously ?
Why not simply use NGINX with PHP-FPM instead (If not already using it) ?

NGINX is already used by many large sites and is blazing fast - I don't think there will be much debate over it.

PHP-FPM is slightly faster than SPAWN-FCGI or custom script dispatcher, but consumes much less memory (as all php processes share memory).

+1

Also, moving from MySQL to MariaDB could be helpful. "Normal" MySQL is not really developed anymore, as Oracle does not give a duck about it - they only want your money by selling you closed source extensions.

I have already moved my 2 servers with 50+ million views/month to MariaDB and I am serving ~35 million views with single quad-core server (but of course heavy caching is involved).

I use XtraDB instead of the standard InnoDB. I serve 40 req/s med and 100 req/s top on a single server. It heavily depends on the patter of the visits.

MariaDB contains XtraDB (it replaces InnoDB "drop in", plus it is binary compatibile, or at least MariaDB devs claim so)

[rasputin]

The right approach to handling load spikes on a forum like this would be full-page caching in the web layer, using a caching reverse proxy in front of whatever webserver you use now. I do not know the guest:logged in user ratios here, but you could probably get away with caching only guest contents (probably less headaches that way).

Look at Varnish, which is written specifically for this job, or at Nginx which should also work. Here is an example and some numbers for you.

[furrycoat]

We need more servers!

[theymos]

So is there any word on where the hits came from?
Can admin let us know where the hits came from?

reddit and Hacker News.

[ShadowOfHarbringer]

The right approach to handling load spikes on a forum like this would be full-page caching in the web layer

It is a good idea, however it will not be enough.
There should also PHP-side cache based on Memcached to offload MySQL/MariaDB servers.

I don't even use Varnish and still I can serve hundereds of requests per second on single server. You can get performance increase of thousand percent and more. Been there, done that.

Check this out:

Code:

$ mysqladmin status
Uptime: 3154789  Threads: 16  Questions: 3738552556  Slow queries: 112796  Opens: 27041477  Flush tables: 1  Open tables: 1482  Queries per second avg: 1185.040

This is a single Quad Core server, it contains MariaDB, NGINX, Apache and PHP - all on a single server. It serves total of ~1.400.000 views/day (when you count google/yahoo bots).
And i still think i can squeeze out much more speed using NoSQL.

Also, my MariaDB tables contain ~15GiB of data, with total of 65 million records.

[wtfvanity]

Seriously ?
Why not simply use NGINX with PHP-FPM instead (If not already using it) ?

NGINX is already used by many large sites and is blazing fast - I don't think there will be much debate over it.

PHP-FPM is slightly faster than SPAWN-FCGI or custom script dispatcher, but consumes much less memory (as all php processes share memory).

He already stated he would eventually move to nginx if apache became the bottleneck.

Think about what servers you are on now.

https://www.ovh.com/us/dedicated-servers/

OVH allows huge bandwidth, you can get a full 100 Mbps line, intel e3 1245v2 processor with 2x SSD 120GB drives for under $100 a month. That would be a heck of a DB server. Mirror raid that for read performance and data security. Have that sit behind the main server as the DB server, and toss another one out in front with nginx as the main server. for $180 a month... you've got a solid set up. Each server has 32 GB of ram in it.

Their virtual rack price is a little bit expensive compared to most solutions (But their servers are much cheaper) Virtual rack you can communicate between the two servers on a local lan and not use the public bandwidth and shield the DB server a little bit. Local connection is at a gig. It appears even without the virtual rack though, that you can communicate with local IPs to other servers. Have not tested that and not as secure as a virtual rack.

If you're hosting with softlayer, you are probably already paying for a similar processor for about $200 a month, and tapped at 5 TB bandwidth with a fraction of the ram.

[mufa23]

It may have something to do with the fact that Moot accepts Bitcoins for his 4chan Pass and says so under every post form. Thus millions of people see the word Bitcoin everytime they write a post on 4chan. CES is nothing compared to that.

well how long has that been there

Late November 2012, I think? I doubt it's related. I remember the passes being on sale, with Bitcoin, right before the holidays.