Bitcoin Forum
September 30, 2016, 06:36:14 PM *
News: Latest stable version of Bitcoin Core: 0.13.0 (New!) [Torrent]. Make sure you verify it.
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 [3] 4 »  All
  Print  
Author Topic: Yes, the Bitcoin.org Site and Forum were Down  (Read 7895 times)
srb123
Jr. Member
*
Offline Offline

Activity: 59


View Profile
December 14, 2010, 07:45:25 AM
 #41

Anybody here ever used Collanos, it might hold the answer.

I havent used it for 2-3years, but it is P2p, distributed etc. (Very similar to Microsoft Groove, now know as Sharepoint Workspace.)

Cheers

EDIT: Had a download and play, and it is probably not appropriate for what we want to do.
1475260574
Hero Member
*
Offline Offline

Posts: 1475260574

View Profile Personal Message (Offline)

Ignore
1475260574
Reply with quote  #2

1475260574
Report to moderator
1475260574
Hero Member
*
Offline Offline

Posts: 1475260574

View Profile Personal Message (Offline)

Ignore
1475260574
Reply with quote  #2

1475260574
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1475260574
Hero Member
*
Offline Offline

Posts: 1475260574

View Profile Personal Message (Offline)

Ignore
1475260574
Reply with quote  #2

1475260574
Report to moderator
1475260574
Hero Member
*
Offline Offline

Posts: 1475260574

View Profile Personal Message (Offline)

Ignore
1475260574
Reply with quote  #2

1475260574
Report to moderator
1475260574
Hero Member
*
Offline Offline

Posts: 1475260574

View Profile Personal Message (Offline)

Ignore
1475260574
Reply with quote  #2

1475260574
Report to moderator
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470


Bringing Legendary Har® to you since 1952


View Profile
December 14, 2010, 11:18:01 AM
 #42

Automated backups of the database to a remote location would be useful if something unexpected happens with the hosting. Any ideas how to store the backups? Maybe encrypt them and distribute to multiple servers around the world. Distributing them through a single gateway (Dropbox, Tahoe-LAFS?) would save bandwidth.

That would be a good start. Tahoe-LAFS would be nice to use also. (There's also possible synergy between Tahoe and Bitcoin)

I think we can handle trusting the seniors not to abuse the database data.

Any possible timetable?

Why so difficult ? Why make it complicated ?

Make it simple. RSYNC, a simple-yet-advanced Linux program can distribute almost anything (even whole Linux partition) over SSH from server to server using public keys (passwords are not needed).
An automated script ( about 10-30 lines of code) can automatically ping main server, restore last backup and start backup forum once the main forum fails.

RSYNC is blazingly fast. It can distribute whole directory structure over SSH, being waaaaaaaaaaaaaaay much faster than any FTP.

Quite easy to do actually.

ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470


Bringing Legendary Har® to you since 1952


View Profile
December 14, 2010, 11:34:27 AM
 #43

Since I know something about Linux administration (and i have done something similiar already), i think can help here.

OK, here is what needs to be done:

1. Create a dedicated read-only MySQL or PostgreSQL (whichever is being used) database slave on the main bitcoin server.
This slave will be used for doing quick full backups every hour or half an hour. That backups will be automatically packaged and put in a certain folder on the server.

2. Create a bash or php script (about 10 lines) that will
a) Backup the database and pack it to .tar.gz
b) Backup directory structure of the webserver and pack it to tar.gz
Every X minutes.

3. Create a low-privileged user on the main Bitcoin server, using which RSYNC/SSH will connect to the server and download backups.

4. Setup another backup webserver - with apache/nginx/mysql/postrgre/whatever.

5. Create a script that will regularly download current backups from the main server using RSYNC over SSH.

6. Create a script that will ping/wget the main server to check if it works. If not, then the script will unpack last posessed backup, and then mstart apache/nginx/mysql

7. Setup SSH public key authentication so that RSYNC can connect.

8. ?? ??

9. PROFIT !

kiba
Legendary
*
Offline Offline

Activity: 980


View Profile
December 16, 2010, 02:15:55 AM
 #44

I think we could implement Shadow's idea.

Sirius?

bitcoinex
Sr. Member
****
Offline Offline

Activity: 350


probiwon.com


View Profile WWW
December 16, 2010, 07:49:02 AM
 #45

The main site could be mirrored, but keeping the forum database in sync on multiple servers sounds difficult. Automated backups of the database to a remote location would be useful if something unexpected happens with the hosting. Any ideas how to store the backups? Maybe encrypt them and distribute to multiple servers around the world. Distributing them through a single gateway (Dropbox, Tahoe-LAFS?) would save bandwidth.

Will be sufficient to WAL-log of database and send log files to gmail account at fixed intervals

It is a reliable way, even in the case of hacking the server, this backup can not be deleted.

New bitcoin lottery: probiwon.com
- Может, ты ещё и в Невидимую Руку Рынка веруешь? - Зачем же веровать в то, что можно наблюдать непосредственно?
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470


Bringing Legendary Har® to you since 1952


View Profile
December 16, 2010, 10:24:06 AM
 #46

The main site could be mirrored, but keeping the forum database in sync on multiple servers sounds difficult. Automated backups of the database to a remote location would be useful if something unexpected happens with the hosting. Any ideas how to store the backups? Maybe encrypt them and distribute to multiple servers around the world. Distributing them through a single gateway (Dropbox, Tahoe-LAFS?) would save bandwidth.

Will be sufficient to WAL-log of database and send log files to gmail account at fixed intervals

It is a reliable way, even in the case of hacking the server, this backup can not be deleted.

1. Gmailing backups is too complicated and slow.

2. RSYNCed Backups on 10 other servers also cannot be deleted. RSYNC is an extremely powerful, stable, fast & totally command-line-scriptable app. You can use it to create a system of webserver backups.
In fact it is widely used for Linux mirrors (Gentoo Linux uses this in its package management system).

ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470


Bringing Legendary Har® to you since 1952


View Profile
December 16, 2010, 10:25:43 AM
 #47

I think we could implement Shadow's idea.

Sirius?

I can help writing some rsync scripts as i am experienced in it, if you're interested.

bitcoinex
Sr. Member
****
Offline Offline

Activity: 350


probiwon.com


View Profile WWW
December 16, 2010, 01:37:40 PM
 #48

The main site could be mirrored, but keeping the forum database in sync on multiple servers sounds difficult. Automated backups of the database to a remote location would be useful if something unexpected happens with the hosting. Any ideas how to store the backups? Maybe encrypt them and distribute to multiple servers around the world. Distributing them through a single gateway (Dropbox, Tahoe-LAFS?) would save bandwidth.

Will be sufficient to WAL-log of database and send log files to gmail account at fixed intervals

It is a reliable way, even in the case of hacking the server, this backup can not be deleted.

1. Gmailing backups is too complicated and slow.

5 Mb attached WAL-log file per day can be sended very fast

Quote
2. RSYNCed Backups on 10 other servers also cannot be deleted.

why?

New bitcoin lottery: probiwon.com
- Может, ты ещё и в Невидимую Руку Рынка веруешь? - Зачем же веровать в то, что можно наблюдать непосредственно?
sirius
Bitcoiner
Staff
Sr. Member
****
Offline Offline

Activity: 429



View Profile
December 16, 2010, 09:44:46 PM
 #49

I'll look more into this next weekend. Rsync sounds good.

Identifi - Decentralized address book with trust ratings
I'm not a forum admin - please contact theymos instead.
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470


Bringing Legendary Har® to you since 1952


View Profile
December 17, 2010, 03:55:54 PM
 #50

Quote
2. RSYNCed Backups on 10 other servers also cannot be deleted.

why?

1. Simple. Because the main server will not have access to the mirror servers. Only the mirror servers need access to root server, not the other way around.
Neither the main server, or possible hackers don't need to know anything about what mirror servers are there, or how many of them is out there.

2. Once you do one backup, you can create a chain of backup servers, where server A backups ROOT server, server B backups A server, and server C backups B server, making it nearly impossible to hack or even trace where all the backups are. All of this can be done with few simple RSYNC scripts.

If you are thinking about government trying to hack or interfere with main site & mirror servers to bring them down, then I'm telling you: not a chance. Just look at the wikileaks case. There are thousands of wikileaks mirrors now. And it is quite easy to create new ones. Very powerful people had interest in killing wikileaks, yet they failed.
Internet is already far more powerful & resilient than everybody expected.

Even Russia government with all their dirty methods would not be able to make all wikileaks mirrors go offline if they wanted to. The only way to do that is shut down the internet completely.

bitcoinex
Sr. Member
****
Offline Offline

Activity: 350


probiwon.com


View Profile WWW
December 18, 2010, 12:04:06 AM
 #51

Quote
2. RSYNCed Backups on 10 other servers also cannot be deleted.

why?

1. Simple. Because the main server will not have access to the mirror servers. Only the mirror servers need access to root server, not the other way around.
Neither the main server, or possible hackers don't need to know anything about what mirror servers are there, or how many of them is out there.

2. Once you do one backup, you can create a chain of backup servers, where server A backups ROOT server, server B backups A server, and server C backups B server, making it nearly impossible to hack or even trace where all the backups are. All of this can be done with few simple RSYNC scripts.

If you are thinking about government trying to hack or interfere with main site & mirror servers to bring them down, then I'm telling you: not a chance. Just look at the wikileaks case. There are thousands of wikileaks mirrors now. And it is quite easy to create new ones. Very powerful people had interest in killing wikileaks, yet they failed.
Internet is already far more powerful & resilient than everybody expected.

Even Russia government with all their dirty methods would not be able to make all wikileaks mirrors go offline if they wanted to. The only way to do that is shut down the internet completely.

The problem is that source of "creation_a_new_WAL_log_file" event is DB daemon, not a remote backup server.

And to secure WAL-log (if its required, which is doubtful) it is better to use a pgp encryption

WikiLeaks not only involved in the dissemination of information but also promote themselves for profit. Otherwise they could just put all files in eMule and gradually publish passwords to them without any kind of 'mirrors'.

It is also very strange to hear about what methods of intelligence agencies of our countries are very different. Smiley

New bitcoin lottery: probiwon.com
- Может, ты ещё и в Невидимую Руку Рынка веруешь? - Зачем же веровать в то, что можно наблюдать непосредственно?
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470


Bringing Legendary Har® to you since 1952


View Profile
December 18, 2010, 07:43:42 AM
 #52

And to secure WAL-log (if its required, which is doubtful) it is better to use a pgp encryption

You can use simple automated script to compress the files, encrypt them with PGP or Truecrypt, and then RSYNC them.
Simple enough, no need to involve any emails in this.

Mahkul
Sr. Member
****
Offline Offline

Activity: 420


Be silent, or be silenced.


View Profile WWW
December 19, 2010, 02:55:47 AM
 #53

Sorry to interrupt the discussion, only to let you guys know that I have just created another forum:

http://bitcoin.ie/forum/

It is going to be mostly focused on the Irish Bitcoin community, however it's all in English so always another place to talk about Bitcoins in case the main forum fails again. I am still in the process of setting it up, so there are no rules posted yet etc, but it should be more or less fully functional now.

Thanks.

Bitalo.com coming soon!

1MAHKULzqZb4evFFg9157LvnJhJQQbeYo7
bitcoinex
Sr. Member
****
Offline Offline

Activity: 350


probiwon.com


View Profile WWW
December 19, 2010, 09:34:55 AM
 #54

And to secure WAL-log (if its required, which is doubtful) it is better to use a pgp encryption

You can use simple automated script to compress the files, encrypt them with PGP or Truecrypt, and then RSYNC them.
Simple enough, no need to involve any emails in this.

Quote
The problem is that source of "creation_a_new_WAL_log_file" event is DB daemon, not a remote backup server.

New bitcoin lottery: probiwon.com
- Может, ты ещё и в Невидимую Руку Рынка веруешь? - Зачем же веровать в то, что можно наблюдать непосредственно?
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470


Bringing Legendary Har® to you since 1952


View Profile
December 19, 2010, 02:31:21 PM
 #55

And to secure WAL-log (if its required, which is doubtful) it is better to use a pgp encryption

You can use simple automated script to compress the files, encrypt them with PGP or Truecrypt, and then RSYNC them.
Simple enough, no need to involve any emails in this.

Quote
The problem is that source of "creation_a_new_WAL_log_file" event is DB daemon, not a remote backup server.

So ?

bitcoinex
Sr. Member
****
Offline Offline

Activity: 350


probiwon.com


View Profile WWW
December 19, 2010, 10:48:12 PM
 #56

And to secure WAL-log (if its required, which is doubtful) it is better to use a pgp encryption

You can use simple automated script to compress the files, encrypt them with PGP or Truecrypt, and then RSYNC them.
Simple enough, no need to involve any emails in this.

Quote
The problem is that source of "creation_a_new_WAL_log_file" event is DB daemon, not a remote backup server.

So ?

Hence, the original server pushes backups to other servers. And backups will be erased when the original server has been hacked.

New bitcoin lottery: probiwon.com
- Может, ты ещё и в Невидимую Руку Рынка веруешь? - Зачем же веровать в то, что можно наблюдать непосредственно?
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470


Bringing Legendary Har® to you since 1952


View Profile
December 19, 2010, 11:16:30 PM
 #57

And to secure WAL-log (if its required, which is doubtful) it is better to use a pgp encryption

You can use simple automated script to compress the files, encrypt them with PGP or Truecrypt, and then RSYNC them.
Simple enough, no need to involve any emails in this.

Quote
The problem is that source of "creation_a_new_WAL_log_file" event is DB daemon, not a remote backup server.

So ?

Hence, the original server pushes backups to other servers. And backups will be erased when the original server has been hacked.

That is not a problem when using RSYNC.
By default, RSYNC does not delete files that are deleted from remote server.

bitcoinex
Sr. Member
****
Offline Offline

Activity: 350


probiwon.com


View Profile WWW
December 19, 2010, 11:42:44 PM
 #58

And to secure WAL-log (if its required, which is doubtful) it is better to use a pgp encryption

You can use simple automated script to compress the files, encrypt them with PGP or Truecrypt, and then RSYNC them.
Simple enough, no need to involve any emails in this.

Quote
The problem is that source of "creation_a_new_WAL_log_file" event is DB daemon, not a remote backup server.

So ?

Hence, the original server pushes backups to other servers. And backups will be erased when the original server has been hacked.

That is not a problem when using RSYNC.
By default, RSYNC does not delete files that are deleted from remote server.

This implies that the attacker have permissions to delete. Hence it is - a threat.

New bitcoin lottery: probiwon.com
- Может, ты ещё и в Невидимую Руку Рынка веруешь? - Зачем же веровать в то, что можно наблюдать непосредственно?
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470


Bringing Legendary Har® to you since 1952


View Profile
December 20, 2010, 10:33:16 AM
 #59

This implies that the attacker have permissions to delete. Hence it is - a threat.

The only thing that implies is that you do not understand how rsync works.
RSYNC uses privileges of given SSH user when accessing remote server. So if you create an unprivileged user, only give SSH public keys only to trusted people, and make all backups read - only, really nothing can happen.

Also, you can setup a separate server with SSH daemon, which will be used for storing backups... So potential attacker will never have access to the main server.

If you are really really really paranoid, You can even set up a chain of servers ROOT -> A -> B -> C, where ROOT SSH-connects to A and stores backups there, then B connects to A to download the backups, and C connects to B to download backup copies. And you only give access to C to the outside world.
But i seriously doubt such a security measure will be ever needed.

bitcoinex
Sr. Member
****
Offline Offline

Activity: 350


probiwon.com


View Profile WWW
December 20, 2010, 01:18:18 PM
 #60

nightmare

New bitcoin lottery: probiwon.com
- Может, ты ещё и в Невидимую Руку Рынка веруешь? - Зачем же веровать в то, что можно наблюдать непосредственно?
Pages: « 1 2 [3] 4 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!