Times are approximations until an automated monitoring system is set up with the means to update the necessary pages.
- 2019.03.01 00:00 UTC - 2019.03.01 14:36 UTC
- TLS Cert was expired, renewal script had broken, repaired by @omnipotens
- 2019.04.09 ~20:00 UTC - 2019.04.10 ~01:00 UTC
- Flood attack detected by upstream firewall, inbound traffic was dropped in defense.
NOTE: After a quick inspection, it appears someone was trying to gain root access, but was unable to do so. We're looking into additional
hardening means to ensure we remain secure in the event of future attack.
- 2019.04.20 2350 UTC - 2019.04.21 04:30 UTC
- Root DNS issue was preventing DNS lookups from working properly across several resolvers.
- 2019.06.15 2111 UTC - 2019.06.17 1230 UTC
- The time is the last known time that the server was functioning, there's a combination issue with DNS and
a server failure of some sort. Recovery attempts are under way, but may be delayed due to work.
- 2019.07.15 - 2019.07.17
- An unknown filesystem error caused a forced remount as read-only, somehow this corrupted several pages of data in the database. Unfortunately, prior to this event,
our backup script had been encountering errors. This has been corrected to make the recovery process easier and less time consuming in the event a similar issue happens in the future.
After digging through several different tables and attempts to salvage the database, the most recent backup was used to build a known good foundation. From there, the newer information was
copied out of the corrupted database, prioritizing information directly related to the users of LinuxRocks. The data restoration is not yet complete at this time of writing (2019-07-18 01:11 UTC),
however, the server is working normally at this time, just missing data that was not prioritized in the restoration process.