Surf here, and I thought I'd take a minute to recap the events leading up to the downtime this last weekend. We've been experiencing lag and poor response from our host for a while, which contributed to the issues.

XxFlufferssxX and TnT_Lover had been playing on our server a bit, being noobs, not doing anything really bad...but apparently they were scouts for a team of avo-lovers (they weren't avo themselves...they get no ice cream).

Beginning 8/16/2011 at exactly 7:40 PM, these guys got on and started a full chat spam event. This caused issues server-side...as our log file has been having size issues over the last few weeks due to changes with servercraft...the file almost tripled in size in under 20 minutes. The server literally had trouble managing the process of writing changes to the log. A flood of noobs from this team came on, with the intent to spam and grief (with TnT_Lover saying he found an admin's house). This attack continued until 8:03 PM, when if I'm correct Gorg caught it happening from the backend and just slammed the server shut to gain control of the situation.

All would have been well, except there were still some issues. Due to the spam...the server tripped itself up on the log writes and when it tried to come back on, it hung. The server had begun running, but it never finished loading properly so we couldn't connect. Additionally, this crash cascaded to McMyAdmin which was querying the server for information, meaning McMyAdmin couldn't load either. The only way for us to reboot or control the server would have been to get the host to reboot it for us...which took close to 48 hours. If you read the log below, you'll see that this whole downtime the server was running in a very limited state (due to a java "out of memory" error), basically just repeating that something was wrong. The server initiation is actually through a console command, and the server tried to unsuccessfully execute that command 750,000 times! That means the log file was well over 1,000,000 lines long, and over 160 MB in size. No wonder the server had trouble loading it!

Since then, we've banned everyone involved in the attack. Also, I'm looking into some spam-control plugins, something where if it detects the same message 5+ times in a row it auto-bans or something. I may have to write it myself. Either way, we learned a bunch from this and a combination of no owners immediately available, plus our increasingly unreliable host, contributed to the downtime. I posted the ENTIRE abridged log of the incident below, full with usernames and IP addresses of the attackers. You'll see I cleaned up the epic repeating spam messages, it's an interesting read.