What happened to bitcointalk.org today? Crashed? Hacked?

[ForceField]

When trying to access bitcointalk.org today, the following error message was displayed on a blank white page:

"Table './bitcoin/smf_sessions' is marked as crashed and should be repaired"

[1714799959]

1714799959

[1714799959]

1714799959

[1714799959]

1714799959

[1714799959]

1714799959

[1714799959]

1714799959

[Grouver (BtcBalance)]

Got an "Got error 134 from storage engine" message.
So I assume the database crashed or was full?

[davout]

Usually happens to MySQL when the partition it sits on is full.

[John (John K.)]

Who brought it back online? theymos is off IIRC.

[Maged]

Who brought it back online? theymos is off IIRC.

Most likely it was Mark (MagicalTux).

Also, based on when the posts stopped, we either reverted to a DB backup, or this was the result of the backup. I don't have a tracker watching when the forum is down (since I can't do anything anyway), so I don't know which it was.

[Stephen Gornick]

With current traffic exceeding two million page views per week, this forum is getting quite a workout!

[ForceField]

Site was down again at 10:54PM EST. New error message:

"Connection Problems
Sorry, SMF was unable to connect to the database. This may be caused by the server being busy. Please try again later."

[John (John K.)]

I got another one:

Database Error: Query execution was interrupted
File: /var/www/bitcoin/smf/Sources/BoardIndex.php
Line: 84Database Error: MySQL server has gone away
File: /var/www/bitcoin/smf/Sources/Load.php
Line: 2040

[pirateat40]

From what I've gathered, Maged slipped on a stale hot pocket (pepperoni) and spilled his Malava Novocaine drink all over the back of the server.  It just needed to dry out a bit before the system came back up.

Warning! Smoke signals are often mistranslated.

 

[theymos]

I have a guess about what caused all of the problems today. While writing watchlist.php today, I ran an SQL query that seemed to really confuse MySQL. It was something like this:

Code:

select t.ID_TOPIC as ID_TOPIC, t.ID_BOARD as ID_BOARD, 
t.ID_LAST_MSG, m.subject AS subject from smf_watchlist w LEFT JOIN
smf_topics t ON (t.ID_TOPIC = t.ID_TOPIC) LEFT JOIN smf_messages m ON
(m.ID_MSG = t.ID_FIRST_MSG) where w.ID_MEMBER = 35;

I believe it was the ON (t.ID_TOPIC = t.ID_TOPIC) part that caused problems. (This isn't actually what I wanted to do -- I meant to compare ID_TOPIC in two different tables.) I think this query caused the database to lock up completely and refuse all connections, even after I stopped loading the page which ran this query. Maybe it also created huge temporary files which caused the other problems -- I didn't notice.

[apetersson]

[01:27]   yellowhat   theymos: are you sure its not a self-dos due to new addons (maybe badly written) addons?
[01:29]   yellowhat   is the source available? I could review the sql maybe
[01:30]   yellowhat   maybe its doing something dumb as querying without index or anything

/pats myself on back for remote debugging ninja-senses