Forum downtime for maintenance?

I noticed that yesterday and today the forum was down for several hours.
Just curious: Was that scheduled maintenance?

If so: Could that maintenance happen during low traffic hours?

hmmm. I wondered about the same thing, the reason for the maintenance status. But if you do not know it, then probably only @jow knows the reason.

Seems the disk ran full (yay systemd and its f...ine logging system) and discourse switched itself into maintenance mode when it had troubles doing things.

I'll schedule a maintenance window for tonight to increase the disk size. Sadly it seems that there is no real low traffic period, nightly traffic appears to be 60% of the daily traffic volumes.

@jow - Need any help here? I have the time this evening.
While you're at it, we could use an update to the forum software.

@thess - if you want to take over I'd greatly appreciate it :slight_smile:

Fine with me. I'll plan on taking the site down at 01:00 UTC (21:00 EDT). I don't know how long it will take to migrate to larger storage though. It looks like we can get an additional 10G at no cost - should I get more?

Software upgrades usually put the site offline for less that 30min.

I'd say jump to the $40/Mo one, we'll probably need the additional space soon considering the current traffic growth rates.

All done - 4GB RAM, 2 CPU, 60GB of storage (~1/2 full). This went faster than I thought.
Upgraded Discourse to latest..
All Debian updates applied (one more reboot).

Excellent work, as always. Thank you! :smile:

Sounds like you guys should throw monit on your boxes so they yell at you before shit completely hits the fan.

We definitely should professionalize the hosting, main holdup there is lack of time on my side.

Basic fs and CPU checks are in the default config file. And Google is covered with checks for other services (httpd, postgres/mysql).

I could write the config file if you want. I just need to know what services you care about.

Yup, we definitely need some active monitoring here. At a minimum, we should at least check the stats we already collect and those which are available on the DO control panel every now and then. We were going to setup something a while back (I don't recall which package it was) but that sort of petered out due to other activities.

Sad part is we even set aside a DO system to mount it on. Maybe I can pick it up in the next couple of weeks. In any event, we have plenty of additional resource for now.