Emergency Maintenance: sw12.vlt control plane unresponsive

The switch rebooted and upgraded successfully and is once again fully operational.

This concludes the emergency maintenance.

Apologies for the inconvenience caused.

The switch rebooted ok - firmware upgrade commencing now.

The scheduled maintenance is now underway. We'll keep you updated on our progress.

The control plane (management interface/services ; ssh, snmp, etc) on the customer access switch sw12.vlt (in our Volta PoP) has become unresponsive, and while it is currently forwarding traffic OK we currently have no visibility into performance, utilisation, and no ability to effect configuration changes etc

This customer access switch has a number of dedicated and co-location customers connected to it.

This could just be a software issue which is resolved by powering the switch off and on again, or it could be a hardware fault which means that if the switch loses power, it will not boot successfully again.

As we cannot therefore safely power cycle it remotely, we will shortly attend site with a spare switch on hand and perform an 'attended reboot' and, if successful, a firmware upgrade. If all goes well, this will cause two short ~2-5 minute interruptions to service for all connected devices over a ~15-20 minute period.

In the event that the switch is actually physically defective and does not boot after being power cycled, we will need to swap the switch out for the spare and restore the configuration from backup. This will take between 30 and 90 minutes, but as we'll already be on site that work can commence immediately.