Today at 5:08:48AM (Pacific Time), the network in chi01 is becoming unreliable. The outage was spanning from 2 to 20 minutes. At 8:15:48AM, it's down completely.
This outage is purely caused by a failed distribution switch in chi01. After replaced and configured a new Cisco switch, the network is backed up at 11:05:48AM.
This outage affected customers that have servers located in chi01 only. All customer's servers in chi02 were not experienced this outage.
We know that you are very frustrated with the outage. We're working on a solution to prevent this issue from happening again.
At this time, everything seems to be backed normal. We apologize for any inconvenience which may caused your business. If you are still experiencing the issue or have any questions, please don't hesitate to contact us at [email protected]
Updated: (12/08/09)
As said, we will have a solution to prevent some customers from outage when a distribution switch fails. The solution is that we installed a redundant uplink for all switches. This uplink is connected to a different distribution switch. When a distribution switch fails, the traffic will flow to the redundant uplink while we replace the failed switch. This job was done today. All of our switches are now have a redundant uplink. We guarantee this issue will never happen again!!
We also would like to let you know that we are testing a new Client Portal, which you can manage your account, servers, submit/view tickets, pay invoices, reboot your servers, view bandwidth usage, etc. We expected this system is ready for use at the end of this month. We will let you know and email you the account login info when the system is ready.
One again, we apologize for the outage yesterday. We felt your frustrated. We will continue to work harder to make sure our services are more reliable and better. Thank you for being our valued and loyal customer!
Monday, December 7, 2009
