rt0.the Emergency Firmware Upgrade and Reload

Sunday, 23 February 1 hour and 59 minutes
Complete
Complete

This maintenance window is completed. Thanks for your patience and apologies again for the disruption to your evening.

Avatar for
Underway

The scheduled maintenance is about to start. We'll keep you updated on our progress.

Avatar for
Scheduled

We have some sort of TCAM programming fault with rt0.the which is preventing the proper operation of IP access-lists. The effect of this is that we are unable to properly restrict traffic from flowing where it should not.

Despite all best efforts to try and "unstick" the situation it seems the only remaining option is to carry out a full reload (reboot) of the affected device, and whilst we are having to reboot it anyway, we're taking the opportunity to upgrade the software on the device at the same time.

Unfortunately, due to the critical nature of this issue, we are going to have to perform this as emergency maintenance this evening.

Customers not directly connected or singlehomed to rt0.the should only notice some routing instablility whilst the maintenance window is occurring.

Broadband customers currently connected to the Telehouse East LNS will see their lines reset and may see some short additional delay reconnecting as we ensure you are shifted over to the LNS in Volta for the least amount of disruption.

Customers who are directly connected, or who have singlehomed services to or through rt0.the will see a total loss of service whilst the router is cleared of traffic and then rebooted.

The total disruption/outage is not expected to exceed 20 minutes (the boot time of the router being some 10-15 minutes of this), but out of an abundance of caution given both the nature of the fault and the software upgrade, we are scheduling a 2 hour window with staff on site throughout.

We apologise for the short notice for this maintenance window, but given the critical security nature of this issue we have to act as soon as possible.

Avatar for
Began at:

Affected components
  • Core Network Functions
    • Layer 3 (THE)
    • Layer 2 (THE)
  • Connectivity
    • Backhaul Services
    • Broadband
    • Leased Lines, EFM & EoFTTC
  • Hosting
    • cPanel Shared & Reseller Hosting
    • Dedicated Servers
    • Managed Shared Hosting
    • Virtual Server Platform (VLT)