It's more a question of "if" not "when". Currently we have no plans on adding Ethereum and we probably won't consider it until the fork situation clarifies.

We'll prepare a more official post-mortem post in a few days. It'll sum up what exactly went wrong both before and during the incident and what we're planning to do to prevent it in the future.

However here's a quick summary of what happened today:

1. the initial backup restoration finished at about 9:00 UTC
2. upon inspection we discovered the backup was broken. One of tables crashed due to running out of disk space and our backup tool just silently skipped this table and all tables after that. It included: multiple not-important cache/log tables, configuration of Safety Limits, our outgoing transactions, users (emails, passwords etc), audit of login attempts, configuration of ACLs.
3. to restore missing tables, we recovered an earlier backup. That means that changes made to these tables in up to 2 hours before our yesterday's crash could be lost. If you did any changes, please verify that they're still present (especially Safety Limits, changed passwords or ACLs).
4. after restoring missing tables from older backup, we need to rebuilt all caches. That took about an hour.
5. we made sure that everything looks correct and we enabled access to FaucetBOX.com website. In case we missed any inconsistencies, we waited before enabling sends from faucets.
6. we reenabled sends from faucets.
7. we'll skip processing payouts today, just in case there's still something we've missed. If we don't find any errors, payouts should be processed tomorrow evening. If you see any inconsistencies (missing deposits, too much / too low balance), please let me know through PM here.

Weird trivia: currently we're processing 10-20% more sends per second than usual. Looks like users didn't run away yet

Last update today, time to sleep.

The server we've ordered was successfully delivered, although 6 hours late...
We've provisioned it and currently we're waiting until database backup gets restored. I estimate it will take up to 9 12 hours.

Assuming it would be weaker or the same as our original, that wouldn't help. It would have run out of disk space in the same moment or sooner.

We're talking about using asymmetric setup, with master database server being weaker and having smaller storage than slave replica.
That would have helped, but I'm not sure we can afford that.

Current status:

Trying to work with limited disk space crashed our current database server, so it will be impossible to bring it up even temporarily. I've made better estimates and it will take about 10 hours to load backup on new server. Server was ordered 5 hours ago and it was supposed to be activated in 2 minutes, but we're still waiting... We've contacted our provider and they told us there is some problem, but that server "should be available today".

Hi guys,

you deserve an update. This unexpected maintenance will be long. It has nothing to do with hardware replacement. Monitoring of free disk space on database server failed. Only one (root) partition was monitored. Today we run out of disk space on a second partition that contains actual data. Previous predictions were that it will be enough for a couple more months, but FaucetBOX growing much faster than expected.

A new server was already ordered, but we're still waiting for it's provision. I estimate that the migration will take about 8 hours once the server is delivered.

It's possible that we'll be able to reclaim some disk space and restore the service, but that would be a temporary solution, we'll still have to migrate in a few days.

IMPORTANT

FaucetBOX.com will be down for maintenance from around 25th July 23:00 UTC to 26th July 1:00 UTC.

Our master database server needs a cooling system replacement.

Payouts are being processed right now, first DASH transaction was already sent. BTC should finish in about 2-3 hours.

PM me the actual IP address.

1. make a backup of all files
2. make a backup of database
3. update all files except config.php and templates/ directory

Are you sure you're checking the same address you're using on faucets? What address are you using on faucets? What faucets do you use, are you sure they're legit?

48 hours didn't pass yet since you reached the threshold, 1 more hour left. However we're having "Transaction commit failed" issues with Bitcoin daemon, so payouts are delayed and you probably won't receive coins in time. It's an issue with Bitcoin Core.

We're working on a fix. Until then payouts for addresses with low number of accumulated satoshis (about less than 20000 satoshi) can be delayed and unreliable. Sorry

AdSense ToS forbids disclosing this. Unfortunately

Your hosting is missing PDO extension for PHP.

It's not a temporary discount. We're going to keep deposits fee at the same level we're paying for network fees for our outgoing transactions.

We're having some random problems with bitcoin daemon failing to send coins ("Transaction commit failed", whatever that means), which can delay payments, but 4 days seems too long. What's your address?

Can you send me samples of these IP addresses? I'll see if they're something I can ban in NastyHosts.

Tweak libs/check.js to only include the claim button. You're interested in lines 7 and 8:


Currently they include all <input type="submit"> and all <button>. If you're using default template, you can probably change it to something like this:

Change the type of "value" column in Faucetinabox_Settings from "text" to "longtext". Look for phpMyAdmin or something similar to manage your database on your hosting panel.

You can extract it from $data["paid"].