13 May 2015

Follow-up on previous incident at AMS-IX platform


Update: Thursday, May 21

Following the incident we had on Wednesday, May 13, which occurred during maintenance activities, we have evaluated our internal procedures. The measures we have taken define a more strict communication between the engineers doing on-site work and the NOC department and better clarity of responsibilities during maintenance.

Update: Wednesday, May 13, at 8:00pm

Wednesday afternoon (at 12:20pm) there was a problem at the AMS-IX platform, which occurred during maintenance activities by one of our engineers. The problem has been solved since approximately 12:30pm. As a result of the failure, part of the Internet traffic on our platform was hindered. The engineer accidentally placed a loop on the platform, causing problems for our systems. As of this, many parties could not exchange traffic with one another through our platform and therefore a number of websites weren’t accessible. 

It is possible that some parties needed to restart their sessions to be fully up and running again. This could have taken longer than 10 minutes.

Because our Internet traffic graphs couldn’t be updated during the failure, it seemed as if all Internet traffic was affected, but this wasn’t the case. 

Our apologies for the inconvenience this problem may have caused.

To prevent this from happening again, we will evaluate our procedures and make sure to tighten these if necessary.