Storage Issues
Incident Report for Servers Australia
Resolved
The platform's performance is back to normal.
If you're still noticing any problems, please reach out in a support case.
Posted Mar 29, 2024 - 15:31 AEDT
Monitoring
The storage vendor has applied fixes to the platform which have resolved connection issues. We're still seeing a degraded performance while the cluster rebuilds and rebalances, however. We will continue to monitor closely and will update the status with any changes.
Posted Mar 29, 2024 - 02:03 AEDT
Update
The storage vendor is applying a fix to the platform currently.
Posted Mar 28, 2024 - 21:16 AEDT
Update
Further issues have been identified that are causing some clients to be unable to connect.

We are still working on the issue with the storage vendor's engineers and will update once we have further information.
Posted Mar 28, 2024 - 20:23 AEDT
Identified
Storage connection issues are resolved but we're still noticing degraded performance on some volumes, and are continuing to work on the issue.

If you're still having problems mounting your storage, please restart the Ceph/RBD services.

Windows servers with certain versions of the Ceph binaries may need to be rebooted if you're unable to reconnect by restarting the Ceph service.
Posted Mar 28, 2024 - 17:22 AEDT
Investigating
We have identified issues with the following storage products:

S3
CephFS
RBD
NFS

The issues are preventing some clients being able to connect.

We are currently working with the storage vendor directly to resolve the problems.

Nimble storage products are NOT affected.
Posted Mar 28, 2024 - 15:59 AEDT
This incident affected: Services (Network Storage).