SMF was unable to connect to the database?

[innocent93]

I can't access the forum for few hours and it keeps on this pages that says there are connection problems. Anyone?

[Bizmark13]

I can't access the forum for few hours and it keeps on this pages that says there are connection problems. Anyone?

I saw the same error as well. There's more information in this thread:

http://bitcointalk.org/index.php?topic=932315

Basically, the hardware (or more specifically the hard disks) that the site was hosted on had some problems and some data was lost. After recovering the site data from a backup, theymos has been tweaking and fixing a few things so some downtime over the next few days is to be expected:

There will some periodic downtime over the next few days (a few hours in total) as we get everything reconfigured/settled. A few things might be broken. Tell me if you see any bugs.

[deluxeCITY]

Recent downtime is actually unrelated to the HDDs failing recently (although both downtimes do make the forum look very bad IMO).

IIRC the recent downtime is due to some obscure DB failing when most people (who have admin access to the forum) were asleep/away.

I am really not 100% sure why theymos has not migrated to amazon cloud computing as the forum has likely grown to be large enough for this to make sense, plus the fact that AWS would likely protect us from DDOS attacks in the future. 

[theymos]

AWS would likely protect us from DDOS attacks in the future. 

If by "protect" you mean "allow attackers to use a limitless amount of money"... AWS doesn't have much built-in DDoS protection AFAIK.

While you can definitely use AWS as part of a very scalable and stable architecture, it's easy for costs to go out of control. For example, a site I used called inkblazers.com is apparently going to shut down very soon because they're spending over $60,000 per month on their AWS hosting. The forum's hosting costs a few thousand dollars per month. Their Alexa rank is 118,092, whereas bitcointalk.org's Alexa rank is 4,568. (They probably have to deliver a lot more data, but that cost is still absolutely ridiculous.)

And AWS doesn't guarantee uptime. Reddit, for example, is apparently based on AWS, and temporary overload errors are very common there. On an average day, bitcointalk.org is usually more stable than Reddit, I think.

It's far easier to do things mostly-right if you just use a traditional single server. This also allows more control and better security. (Amazon doesn't have the best reputation for protecting customers.)

Responding to things that other people keep saying elsewhere: Yes, the forum has a lot of money. But this is mostly due to BTC value increases, and it would quickly be depleted if costs increased much higher. Spending at the level of most VC-backed startups would be absolutely reckless. I don't think that the forum can afford more than maybe one additional full-time employee, for example. (Currently the only full-time employees are Slickage. I and the moderators are paid so little that we're basically volunteers. There are also a few part-time contractors.)

Believe me, no one finds it more annoying than I do when the forum is down -- I'm usually the one who has to fix it and try to prevent it from happening again... In this case, things are breaking without any obvious cause, so it might take some time to figure this out and get things rock-solid again.

[deluxeCITY]

AWS would likely protect us from DDOS attacks in the future. 

If by "protect" you mean "allow attackers to use a limitless amount of money"... AWS doesn't have much built-in DDoS protection AFAIK.

I would say that AWS would be one step away from cloudflare (which has it's own security concerns along with other draw backs). The infrastructure that amazon has in place would at least make it more difficult/expensive for attackers to successfully DDoS the forum.

While you can definitely use AWS as part of a very scalable and stable architecture, it's easy for costs to go out of control. For example, a site I used called inkblazers.com is apparently going to shut down very soon because they're spending over $60,000 per month on their AWS hosting. The forum's hosting costs a few thousand dollars per month. Their Alexa rank is 118,092, whereas bitcointalk.org's Alexa rank is 4,568. (They probably have to deliver a lot more data, but that cost is still absolutely ridiculous.)

I am not familiar with inkblazers so I really cannot comment on why they were spending so much money on their hosting. I do know that there have been several reports of people having ridiculous bills because their accounts were "hacked" and were used to mine either bitcoin or altcoins.

I would imagine that the forum would be able to negotiate some terms with Amazon to either keep costs in line or to get some kind of volume discount.

I would not be surprised if sites like instagram (which I believe to at least previously was hosted on AWS) spend more then $60k on hosting every month.

It should be noted that the forum gives away one free advertising spot worth ~4 BTC per week to PIA which would add ~$4,000 to current hosting costs. I am not sure if we get more out of this then the 4 grand per month we give away in free advertising or not, however I imagine that we do.

And AWS doesn't guarantee uptime. Reddit, for example, is apparently based on AWS, and temporary overload errors are very common there. On an average day, bitcointalk.org is usually more stable than Reddit, I think.

There is way too much junk on reddit for me to be a regular user of it, but I will take your word for it.

It's far easier to do things mostly-right if you just use a traditional single server. This also allows more control and better security. (Amazon doesn't have the best reputation for protecting customers.)

I believe that google offers somewhat of a similar service that amazon offers and they seem to score pretty well on your chart.

Responding to things that other people keep saying elsewhere: Yes, the forum has a lot of money. But this is mostly due to BTC value increases, and it would quickly be depleted if costs increased much higher. Spending at the level of most VC-backed startups would be absolutely reckless. I don't think that the forum can afford more than maybe one additional full-time employee, for example. (Currently the only full-time employees are Slickage. I and the moderators are paid so little that we're basically volunteers. There are also a few part-time contractors.)

Agreed. However there are ways to potentially change this, for example by lowering requirements to become a donator and/or creating a lower donation rank and/or offering an option to pay to remove the 360 second jail/limit (among other ways to increase revenue).

(slightly off topic) - There is no real reason to treat yourself and the other mods like volunteers (granted some are very early adopters of Bitcoin and can afford to volunteer and would benefit financially if their efforts were to positively impact bitcoin) as the forum is kept generally clean with few, if any major issues. Adding revenue would allow the mods to receive a more "fair" income (especially considering the intelligence/qualifications of the majority of the moderators on here).

Believe me, no one finds it more annoying than I do when the forum is down -- I'm usually the one who has to fix it and try to prevent it from happening again... In this case, things are breaking without any obvious cause, so it might take some time to figure this out and get things rock-solid again.

I think the a cost benefit analysis of using AWS (and google's equivalent) should at least be done. Granted it may not be a better solution then what we have now, however I think the amount of hardware/hosting related problems we have had in recent past would make the few hours it would take to do this be time well spent. The worse case scenario is that you have one additional argument against people who claim bitcointalk is run in a less then professional manner, and the best case scenario is that we have a cost effective way to host the forum that results in an overall better experience for users.

[Blazr]

however I think the amount of hardware/hosting related problems we have had in recent past would make the few hours it would take to do this be time well spent.

There was only one hardware failure recently, a complete RAID failure, where two drives failed simultaneously (it can also happen with cheap RAID controllers), which is a highly unlikely thing to happen (although I personally have seen it happen a few times, maybe I'm bad luck!).

A single server setup is better than a cloud setup from a security/control/privacy/cost side but not from an uptime/performance side. I think theymos wants the security, control, privacy and cost-saving that a single server setup gives.

[deluxeCITY]

however I think the amount of hardware/hosting related problems we have had in recent past would make the few hours it would take to do this be time well spent.

There was only one hardware failure recently, a complete RAID failure, where two drives failed simultaneously (it can also happen with cheap RAID controllers), which is a highly unlikely thing to happen (although I personally have seen it happen a few times, maybe I'm bad luck!).

A single server setup is better than a cloud setup from a security/control/privacy/cost side but not from an uptime/performance side. I think theymos wants the security, control, privacy and cost-saving that a single server setup gives.

My understanding is that banks, and government agencies charged with securing top secret information use cloud computing.

You are probably right about control.

I would say privacy may be improved at the margins as theymos have more power to fight search warrants and subpoenas, however at the end of the day a valid court request is just that (I don't think the ability to fight search warrants/subpoenas is trivial in any respect).

There are obviously some price points where it would make more sense to have cloud computing over owning physical servers otherwise services like AWS would go out of business.

[funtotry]

Yea occasionally Im getting these as well. Sucks not being able to browse the forums but gives me an excuse to get off the computer.

[Blazr]

My understanding is that banks, and government agencies charged with securing top secret information use cloud computing.

Oh because those never have data breaches do they?

A number of the biggest Bitcoin hacks that occurred were done by social engineering the hosting company into giving the hacker root access to the server. This is a very common technique and it is so effective it's not even funny anymore, it's beyond ridiculous. At least when you have a physical server you can secure it to prevent against this.

There are obviously some price points where it would make more sense to have cloud computing over owning physical servers otherwise services like AWS would go out of business.

Cloud computing is almost always more expensive, but if you need scalability or high uptime and redundancy it is obviously a better choice. This is why people use it. It's also much more convenient to manage.

[deluxeCITY]

My understanding is that banks, and government agencies charged with securing top secret information use cloud computing.

Oh because those never have data breaches do they?

LOL they do, but so has the forum.

A number of the biggest Bitcoin hacks that occurred were done by social engineering the hosting company into giving the hacker root access to the server. This is a very common technique and it is so effective it's not even funny anymore, it's beyond ridiculous. At least when you have a physical server you can secure it to prevent against this.

The alleged hacks. I would argue this is less effective then you think because some were probably the owner faking a hack to steal customer money, however you do have a good point here. I am fairly certain there is a way to protect yourself from this, however I am not sure what it is yet.

There are obviously some price points where it would make more sense to have cloud computing over owning physical servers otherwise services like AWS would go out of business.

Cloud computing is almost always more expensive, but if you need scalability or high uptime and redundancy it is obviously a better choice. This is why people use it. It's also much more convenient to manage to.

Well the forum does have a lot of business conducted on here. Large amounts of data loss would likely result in scams and financial loss to the people that conduct business on here (unless you saved a PGP signed message for a payment address, it would be difficult to prove you were told to send funds/goods to a certain address in the event data is lost).

It also may be more expensive, however if you were to ignore payments to slickage (which are temporary) then the forum is cash flow positive.