..

[vm1990]

Donators funds have never been touched.

How do the donators feel about that? I guess its better knowing they're still somewhere than not knowing what was done with them.

id be pissed if donations are just sitting there then whats the point in donating? even worse when they could be being used to improve/tighten security.

[SaltySpitoon]

not going to say much but donations and stuff are there to support and fix issues in time of need. i dont know maybe everyones passwords getting stolen is a time of need. dont know why theymos sits on a large stack of BTCBTCBTC maybe he wants his own island or something. as for mod payments i dont want to upset you but recently its not the hardest job   but you still do a good job please dont shout qt me

Yep, donations and such are to fix issues in time of need. Some issues though aren't fixed by throwing all of your money at them. For example, this recent hack, there is no reasonable solution that Theymos could have done to prevent this. If you know the solution, let us know  

I didn't say that moderating was hard, it just takes time. My point was that moderators are still technically volunteers, the staff (Theymos included) aren't bleeding the forums dry of funds. Donations have never been used to pay any Moderators/Admins, and funds or lack of funds isn't the issue. People hack websites that are targets, Bitcointalk is a target. The fact that the hackers didn't find a vulnerability in the site, and instead targeted the host says a lot. Facebook was hacked with a 1.05 Billion dollar private datacenter. We could spend every last penny to set up a tiny data center on an isolated island and hire one armed guard to prevent this same issue from happening again, but we can't really say that is reasonable solution.

*edit*

Donators funds have never been touched.

How do the donators feel about that? I guess its better knowing they're still somewhere than not knowing what was done with them.

id be pissed if donations are just sitting there then whats the point in donating? even worse when they could be being used to improve/tighten security.

Donators were told that Donations would be used for new forum software, which is in production. I meant that donator's funds have never been touched by Staff/Admins.

Again, how would you increase security? There isn't a hole in which to throw money that gives you what you want. New more secure forum software is in production, but it wouldn't have helped in this case.

iv actually been through ddoss suggestions on here in the past. dont know if they didnt like my idea or was just ignored. id set up a few cheap vps load balancers. set software can only respond to certain requests so it filters out damaging traffic to the main website. i know ddos attacks are getting bigger and more complex but so are defenses. and in fairness this isnt a massive site so dosnt attract the worst ddos or hackers mostly because its a forum and little info/money to be gained from it. i do however respect the fact passwords where atleast encrypted... see  alot bigger sites fall at that point

The forums does have multiple hosts to help mitigate DDOS attacks. One of those hosts is what allowed the latest hack to happen.

[2112]

For example, this recent hack, there is no reasonable solution that Theymos could have done to prevent this. If you know the solution, let us know  

I posted the simple solutions elsewhere, I'll repost it here:

https://bitcointalk.org/index.php?topic=1069837.msg11453289#msg11453289

Easily preventable on two levels:

1) collocate your own equipment in a remote data center. The customer service staff will simply have no access to it besides being able to press buttons on the box.

2) use non-commodity hardware like Oracle SPARC or IBM POWER or HP Integrity/Itanium.  Then even if they manage to steal it they most likely will not be able to get the data off of it without specialized assistance.

Also, don't run Linux on those machines, but their native OS: Solaris, AIX, HP/UX respectively.

[vm1990]

For example, this recent hack, there is no reasonable solution that Theymos could have done to prevent this. If you know the solution, let us know  

I posted the simple solutions elsewhere, I'll repost it here:

https://bitcointalk.org/index.php?topic=1069837.msg11453289#msg11453289

Easily preventable on two levels:

1) collocate your own equipment in a remote data center. The customer service staff will simply have no access to it besides being able to press buttons on the box.

2) use non-commodity hardware like Oracle SPARC or IBM POWER or HP Integrity/Itanium.  Then even if they manage to steal it they most likely will not be able to get the data off of it without specialized assistance.

Also, don't run Linux on those machines, but their native OS: Solaris, AIX, HP/UX respectively.

just had a look at the article and all i can say is what the hell. either root password was piss poor or he got lucky. and ontop of that no white list for ip login? thats asking to get ass raped. all ROOT logins should be white listed its a basic security feature or even 2 levels of security... like 2 passwords... be interesting to know if it was being brute forced to? and if it was how that many attempts went unnoticed maybe a failed login attempt warning would be a good idea

theres loads of defenses out there and now iv read the article im pretty shocked. i was expecting a datacenter backdoor being used not walking in the front door

https://bitcointalk.org/index.php?topic=1067985

a few suggesting it was an inside job by someone xd think its unlikely unless they where blackmailed into it. but still get a damn white list in place and do it now maybe i should come up with ideas for security its fun

maybe have a table in a secret location all mods get a key and only 2 mods turning the keys at the same time can change any forum code or access the servers... ooo with hand scanners and retinal scanners to confirm the changes... just a thought

[SaltySpitoon]

I posted the simple solutions elsewhere, I'll repost it here:

https://bitcointalk.org/index.php?topic=1069837.msg11453289#msg11453289

Easily preventable on two levels:

1) collocate your own equipment in a remote data center. The customer service staff will simply have no access to it besides being able to press buttons on the box.

2) use non-commodity hardware like Oracle SPARC or IBM POWER or HP Integrity/Itanium.  Then even if they manage to steal it they most likely will not be able to get the data off of it without specialized assistance.

Also, don't run Linux on those machines, but their native OS: Solaris, AIX, HP/UX respectively.

I suppose its not entirely out of the question, but collocating our own equipment probably isn't the best idea either. It would be less than cost effective, and forum up time and reliability would be far less than it is with professional datacenters. What country to place the datacenter would be another issue, and hiring employees to manage it doesn't sound too appealing. It sounds like a complete mess, and something unnecessary for a forum. This is a website, a large one at that, but the Bitcoin forum isn't Google.

[2112]

I suppose its not entirely out of the question, but collocating our own equipment probably isn't the best idea either. It would be less than cost effective, and forum up time and reliability would be far less than it is with professional datacenters. What country to place the datacenter would be another issue, and hiring employees to manage it doesn't sound too appealing. It sounds like a complete mess, and something unnecessary for a forum. This is a website, a large one at that, but the Bitcoin forum isn't Google.

You are just bullshitting. I've been doing exactly that professionally (mostly as a consultant) for many years. Yeah, it is somewhat more expensive, especially in the upfront capital cost, but the operating expenses are frequently actually lower. It is a perfect solution for "a website" even with much less traffic than this one.

In particular the reliability is better if the owner of the equipment is conscientious and willing to learn, because there is no blaming "somebody else". Also, the customer service staff for the physical collocation customers is typically way more responsible and conscientious.

The "professional datacenters" that have equipment leasing included in their rental fees are the dreck of the datacenter business, because they by necessity serve mostly fly-by-nights. It seem to me like you've never owned the equipment in any datacenter so you don't really have a base to make a real judgment.

[SaltySpitoon]

You are just bullshitting. I've been doing exactly that professionally (mostly as a consultant) for many years. Yeah, it is somewhat more expensive, especially in the upfront capital cost, but the operating expenses are frequently actually lower. It is a perfect solution for "a website" even with much less traffic than this one.

In particular the reliability is better if the owner of the equipment is conscientious and willing to learn, because there is no blaming "somebody else". Also, the customer service staff for the physical collocation customers is typically way more responsible and conscientious.

The "professional datacenters" that have equipment leasing included in their rental fees are the dreck of the datacenter business, because they by necessity serve mostly fly-by-nights. It seem to me like you've never owned the equipment in any datacenter so you don't really have a base to make a real judgment.

Tell me then, how much would it cost to set up a datacenter in a couple of countries, buy the equipment, and hire employees? I think you are vastly overestimating how much the advertising revenue brought in could support.

[2112]

Tell me then, how much would it cost to set up a datacenter in a couple of countries, buy the equipment, and hire employees? I think you are vastly overestimating how much the advertising revenue brought in could support.

I'm not going to give you a quote just to prove myself, I charge for such services and I'm positive that you are neither serious nor authorized to purchase anything.

But one thing is worth mentioning: "hire employees". For a physical collocation "remote hands" services are usually available in increments of 15 minutes. What I'm positive is that after buying and paying for "remote hands" a couple of times, which normally involves a telephone/facetime/skype conversation with the remote contractor, the possibility "social engineering" essentially disappears.

The worst "hack"s that did happen on my watch was nothing more than an equipment destruction or theft (for wipe & resale).

[SaltySpitoon]

I think you are vastly overestimating how much the advertising revenue brought in could support.

Hope the advertising revenue is at least supporting the $100k a month Theymos is spending to develop new forums !

* Xian01 ducks

Heh, no it would not had the forums not had years of reserves. What it brings in for advertising revenue is public, just check the closed auction threads to see what it could support.

I'm not going to give you a quote just to prove myself, I charge for such services and I'm positive that you are neither serious nor authorized to purchase anything.

But one thing is worth mentioning: "hire employees". For a physical collocation "remote hands" services are usually available in increments of 15 minutes. What I'm positive is that after buying and paying for "remote hands" a couple of times, which normally involves a telephone/facetime/skype conversation with the remote contractor, the possibility "social engineering" essentially disappears.

The worst "hack"s that did happen on my watch was nothing more than an equipment destruction or theft (for wipe & resale).

I wasn't asking for something that you spent 20 minutes or more figuring out, I meant a rough figure, because for someone who knows what they are doing, you are either grossly overestimating the forum's budget or underestimating the cost of setting up multiple datacenters in various countries and the unpleasantness that would come with that. The forum could support a single full time employee perhaps, not multiple + building expenses + interesting tax implications for owning physical property, and more tax issues for having physical property in multiple countries. I'm no expert on the matter, but even with absurdly and unreasonably low cost assumptions, we are still vast sums of money apart. I could set up a datacenter in a shed in my backyard for $5k. If Theymos wants to take me up on that offer, I'd be happy to oblige.

[ElectricMucus]

Hey Mods, do you realize some people work in IT, some at small businesses who are renting servers too, knowing that it's not that big of a deal these days?

[freedomno1]

You have to realize that hackers hate BCT. They've been hacked and had long downtimes quite a lot in the history.

Yeah, Bitcointalk is one of the larger forums in the world, I don't know the actual statistic, but I'm sure it gets a lot more attempted attacks than is publicly known. I dont think the forum's track record is all that bad though, two or three hacks come to mind in 5 years. Some DDOS too, but you can't really prevent that.

It's a pretty good record, that said the hackers could have made some coin from the bounties but I guess they thought hacking a userbase was a better ROI in the long run either way it is a lot of work
It does make me wonder if this is the last hack per se of the old forum, the new forum software's launch is getting closer as will a ton of holes in all likelihood.

[Lineranger]

is there something missing ? since bitcointalk forum hacked?
what is the major effect on this attack?

[favdesu]

is there something missing ? since bitcointalk forum hacked?
what is the major effect on this attack?

you should change your password.

other effects may be old accounts coming back to life...

[vm1990]

***wonders if iv yet again been ignored on suggestions for forum security*** white list ips that can access the server or very least who can use root.

again very simple solution to a not so complex hack and add email alerts to every single attempted login and successful login

[Blazr]

***wonders if iv yet again been ignored on suggestions for forum security*** white list ips that can access the server or very least who can use root.

That really won't help much against something like this. The hackers would just ask the hosting company to change the whitelisted IP's too, they already reset the root password for the hackers, I don't see why they wouldn't change the whitelisted IP's too.

[BitUsher]

Tell me then, how much would it cost to set up a datacenter in a couple of countries, buy the equipment, and hire employees? I think you are vastly overestimating how much the advertising revenue brought in could support.

Giving you a quote is dependent upon many variables but as a rule of thumb Colocation is more secure and less expensive in the long term in almost all cases. Leasing a dedicated server is less expensive initially because you don't need to purchase the hardware, install it and have some backup parts on hand/or have funds set aside for a smart hand. Colocation has higher upfront costs and complexities but most small businesses can pay 50-150USD a month on colocation fees.

Example-
Leasing a dedicated server may cost ~150 USD a month for the first 6 month promotion and than ~250 USD a month thereafter. You could purchase a refurbished 2u server for a couple grand and spend 80 dollars a month in colocation fees.

A higher traffic site like Bitcointalk would need more expensive servers purchased and higher colocation fees  but would have much higher levels of savings as the dedicated server lease fees and bandwidth costs would be much more as well.

Colocation would likely offer better security and large savings in the longterm.

[2112]

I wasn't asking for something that you spent 20 minutes or more figuring out, I meant a rough figure, because for someone who knows what they are doing, you are either grossly overestimating the forum's budget or underestimating the cost of setting up multiple datacenters in various countries and the unpleasantness that would come with that. The forum could support a single full time employee perhaps, not multiple + building expenses + interesting tax implications for owning physical property, and more tax issues for having physical property in multiple countries. I'm no expert on the matter, but even with absurdly and unreasonably low cost assumptions, we are still vast sums of money apart. I could set up a datacenter in a shed in my backyard for $5k. If Theymos wants to take me up on that offer, I'd be happy to oblige.

Dude, what can I say? You are not only a compulsive bullshit artist, but you've also mostly lost touch with reality. What buildings? What full time employees? What tax implications? One is true:

I'm no expert on the matter

.

I am an expert, but I'm not really interested in learning the finances of this forum. It is up to theymos to scan his tax returns for bitcointalk and call Dell Small Business (or any other large reseller of electronics) financial department and ask them how much credit he's going to get for his non-profit organization. Literally millions of small business' owners done that before him.

I could then discuss various technical details and options, but I'm too ethical to even joke about $5k hosting in a shed.

All I have to say is my school had lots of wisdom retaining and maintaining the old mainframe. It allowed us to learn not only the technical details of virtualization (it was called VM/370 then, not KVM or whatever) but also experience first hand the bullshit from the time-share salesmen. The "cloud" terminology was not invented then, everyone used "time-share". Nowadays the "time-share" is a dirty word related to the vacation package sales. But the infectious anti-technical sales bullshit permeating the business is the same as it was through the 1960-1980 when it was popular.

But before he's going to even scan the tax returns for the forum he'll need to ask himself a question "Do I give a flying fuck on a rolling donut about the information security of the members of this forum?" Maybe the true answer really is "I like to have a convenient 'scapegoat in the cloud'. I can always point to the sky and say 'It was their fault, not mine!'".

[redsn0w]

There are some very clever hackers hiding on this forum and in bitcoin-world in general. You need to stay on your toes at all times.

You can always leave and also the OP can always leave . However, yes this time due a social engineering attack... really impressive how it was easy.

Actually Theymos already said it wasn't a social engineering attack, they just havent said what it was yet.

im betting theymos's password is "theymos is AWESOME" but seriously the amount this forum has earned for him you think hed keep on his toes about stuff. ddos attacks and hacking.

what would you suggest for preventing both? I'm sure theymos would be very grateful if you could lend your ideas. he might have money to invest but it doesn't grant him unlimited knowledge. its nt as simple as you make it sound. many multi million websites have been exploited and ddos

iv actually been through ddoss suggestions on here in the past. dont know if they didnt like my idea or was just ignored. id set up a few cheap vps load balancers. set software can only respond to certain requests so it filters out damaging traffic to the main website. i know ddos attacks are getting bigger and more complex but so are defenses. and in fairness this isnt a massive site so dosnt attract the worst ddos or hackers mostly because its a forum and little info/money to be gained from it. i do however respect the fact passwords where atleast encrypted... see  alot bigger sites fall at that point

But in this case it was 'used' also a soc. eng. practice... and you can build the security that you want but if an employee will reset the root password it will be really a problem  .

However I think (all) we are waiting more info. from theymos about this situation.

[SaltySpitoon]

Dude, what can I say? You are not only a compulsive bullshit artist, but you've also mostly lost touch with reality. What buildings? What full time employees? What tax implications? One is true:

I'm no expert on the matter

.

We are discussing the feasibility of creating our own hosting location, having our own facilities, not giving another random 3rd party access to the server. I thought you were aware that the hosting company staff were the weak link in this hack. The way to get around that, is to change to a facility operated by an employee of the Bitcointalk. That would involve building our own infrastructure, hiring staff to monitor its physical location, etc. That would also involve owning property to build on.

If we just rent server space from an already established company, we face the same issues. Not having complete trust of the people who have access to the server. So if we are talking about just changing hosts to something that isn't a large operation in a giant datacenter to a shared location with a couple of other people, we still have to worry about the human factor.

[redsn0w]

Dude, what can I say? You are not only a compulsive bullshit artist, but you've also mostly lost touch with reality. What buildings? What full time employees? What tax implications? One is true:

I'm no expert on the matter

.

We are discussing the feasibility of creating our own hosting location, having our own facilities, not giving another random 3rd party access to the server. I thought you were aware that the hosting company staff were the weak link in this hack. The way to get around that, is to change to a facility operated by an employee of the Bitcointalk. That would involve building our own infrastructure, hiring staff to monitor its physical location, etc. That would also involve owning property to build on.

If we just rent server space from an already established company, we face the same issues. Not having complete trust of the people who have access to the server. So if we are talking about just changing hosts to something that isn't a large operation in a giant datacenter to a shared location with a couple of other people, we still have to worry about the human factor.

Ecatly, I have proposed in the past days the creation of an "home made" server to hosting the forum but I do not know if it is a really *possibilty or not.


*With all the money spent in the creation of the epochtalk forum software.