admin @admin@hax0rbana.social

Ahh, the unplanned outages. This time it was a couple days of no Internet. We are adopting a more ...aggressive... escalation process.

We apologize for our extended downtime (a bit over a week, I think). We had an unexpected 15-hour power outage at our data center, and our recovery processes still need some work. We have updated our processes and there is even drastic talk of trying them BEFORE the next unplanned incident. Stay tuned.

@admin Downtime turned out to be about 3 minutes to actually do the migration. Everything should be solid now. If you experience any issues, let me know.

@admin backed everything up and then decided to improve (and test) the deployment scripts. Up for now, will go down briefly later today.

Migrating a VM to a new server and expect our instance to be down for about half an hour. Sorry for the late notice.

Apparently not everything came back up after an unexpected power outage, and it took us several days to figure out what was down. *embarrassed face* We think we are visible to the Internet again. Sorry for the outage; we are making plans for monitoring to help us catch down services instantly.

@admin Mystery solved! It was an image size limit issue. Now posting up to 20MB files should work. If you want to post something larger, let us know.

Turns out uploading photos works fine in browsers and some mobile apps, but not all of them. Doesn't work on Tusky (Android), but it does work on Tootle (iOS), Tootdon (iOS), and Linky (iOS)... at least sometimes. The investigation continues...

Things the administrators know are broken on our Mastodon instance:
- Confirmation emails (possibly all emails)
- Uploading photos (possibly all media)
- No terms of service or code of conduct