So the procedure is basically is that you would be voted off the island.
If you were to go full rogue, then the forum would likely permanently the ~10k
BTC that you are holding on it's behalf, although this should be okay because the forum appears to be self sustaining via ad sales. This might mean that we won't see the new forum depending on a number of factors, including who specifically who the counter-party of the contract with Slickage is.
I think the more pressing concern is what would happen from a technological standpoint if you were "voted off the island". If you were to go full rogue, then I am fairly certain that you would be able to do a significant amount of damage in even a short amount of time. Does someone have the ability to revoke your various admin accesses in the event that you would be voted out? Does someone have the ability to grant someone new the various admin accesses in the event that someone new would be voted to be a new admin, assuming you were unable/unwilling to grant such access? The formal process to have you removed is fairly straight forward (although a large number of people who would be voting are really no longer active in the community from their main accounts), however the technological process is somewhat more complex.
While I do think the chances of you going "full rogue" are fairly slim, there is a chance that something would happen that would cause you to be unable to carry out your admin duties. Some examples of things that might prevent you from carrying out your admin duties would include getting hit by a bus (or getting struck by lightning, or other freak of nature type accidents), getting arrested (or otherwise being under government duress), loosing access to your various private keys (and backups), being too cold (or getting hit by a giant snowball - you do live in a state that borders the great lakes right? -- sorry but I had to poke fun of you a little bit, I am not a fan of the cold weather).
I understand that sirius controls the domain bitcointalk.org, although I believe you have partial (?) control over it as well. I believe that I have read that some people have access to unencrypted backups of the forum, although I am not sure who these people are nor if they would be willing to honor any vote regarding your removal. I am not sure if anyone else has the ability to grant/revoke admin access, nor am I sure what it would take to revoke that person's ability to make these changes.
I think the chances of a vote to remove you ever happening would be extremely slim at best, however I know that many bitcoineers like to have nearly every scenario accounted for in order to protect themselves.