Webdock - Rescheduled: Electrics maintenance for single rack in DK DC1 Thursday January 30th – Maintenance details

All systems operational

JAN30

Rescheduled: Electrics maintenance for single rack in DK DC1 Thursday January 30th

CompletedJanuary 30, 2025 at 7:00 AM – 8:11 AMUTC

Affects

Denmark: General Infrastructure

Under maintenance from 7:00 AM to 8:11 AM

Updates
  • Completed
    January 30, 2025 at 8:11 AMUTC
    Completed
    January 30, 2025 at 8:11 AMUTC

    The maintenance for today is complete. All customer VPS servers are up and running. Unfortunately we saw another outage in the rack but that helped us diagnose the issue, which was a faulty power transfer switch as we had surmised. The switch has been replaced and thus the issue has been solved. This means that in case of an unplanned power outage, all our emergency power systems in that rack will behave properly. After the initial crash we did repeated tests and everything was nominal for all racks and equipment in the DC.

    We apologize for any inconvenience caused here this morning.

  • In progress
    January 30, 2025 at 7:00 AMUTC
    In progress
    January 30, 2025 at 7:00 AMUTC
    Maintenance is now in progress
  • Planned
    January 30, 2025 at 7:00 AMUTC
    Planned
    January 30, 2025 at 7:00 AMUTC

    This maintenance has been rescheduled for Thursday January 30th, due to some required equipment not making it to us in time.

    On Thursday we will be performing some electrics related maintenance for the rack which tripped fuses when we did our generator upgrade and power outage tests the other day. The maintenance consists of putting less sensitive fuses on both of our circuits and potentially replacing a power transfer switch. We hope that the work will happen without incident, but there is a chance that power may go away from that rack again - we cannot guarantee this will not happen. This will impact most of our Ryzen workloads, our Epyc workloads and our core services such as our dashboard. If a power incident occurs, we will be operational again within about 5-10 minutes and our entire team will be standing by to make sure everything comes up as quickly as possible after a potential failure.

    We absolutely need to solve this issue as the situation is now, then if we have an unplanned power outage there is a chance this rack will go down, and we don't want that. In fact, everything we've designed in our DC is designed to prevent exactly this. We hope to get this issue resolved fully on thursday and hopefully without incident.