13 March 2017

First Implementation of the Brocade SLX 9850 Switch Hardware

Blogpost-image_paper-3

In September 2016, I wrote about the replacement of the AMS-IX Amsterdam network’s core switches. At the time, I mentioned that we had selected the Brocade SLX 9850, expecting to begin implementation in Q4 of 2016. We managed to migrate one of the four main core switches from MLXe-32 to the SLX 9850.

The figure below shows the initial topology of the network:

Picture1

There are three sets of core (P) routers: one in euNetworks data centre, one in Global Switch and a pair distributed in Equinix AM2 and Equinix AM5 (former TeleCity AMS5). Each of these core routers is an MLXe-32 with a utilisation rate of over 50 100GE ports (out of 64). The lower row of switches is a pair of access switches in each data centre where AMS-IX offers services.

The extra pair of core routers at Equinix AM2 and Equinix AM5 were built to offload the core switches at euNetworks and Global Switch. These were reaching their limit of 64 100GE interfaces to connect to all the edge switches.

The new SLX 9850 was introduced into the network through the following procedure:

  1. The new switch was installed
  2. Tests were performed over all interfaces for the duration of a few days (snake)
  3. A connection (LSPs) was defined between edge (PE) switches to be moved to the new hardware
  4. The PE to PE connections were moved to their backup paths
  5. Physical connections were moved from the EUN 302 P router to the new EUN 306 P router
  6. The selected PE to PE connections were redefined to run over the new P router

Picture2

The figure above shows a number of physical connections being moved over during an intermediate stage.

At the end of the process, all connections had been moved over to the new EUN 306 P router. This meant that we were able to remove the old EUN 302 MLXe-32 core router.

Picture3

The graph below shows the traffic for all interfaces on the new EUN 306 SLX 9850 P router. We can see the traffic increase over the two-week migration period as we move an increasing number of connections onto the new switch. We were very careful with this initial migration which turned out to be a very positive experience. We have not encountered any service-affecting issues since the switch became operational and started forwarding over 1.6 Tbps of traffic.

Picture4

Migration of the other three switches at euNetworks and Global Switch will take place between late February and early March. Once this has been successfully completed we will take down the core routers at Equinix AM2 and AM5. The final network, which we plan to implement in the summer of 2017, will look like this:

Picture5

The migration will allow us to improve a number of important aspects:

  1. Significant reduction in network complexity
  2. Major increase in network core scalability by introducing the SLX9850
    a. Four time the number of 100GE interfaces
    b. Strong reduction in space and power usage allowing further scaling up in the same footprint.
  3. Cost savings by having two fewer core sites
    a. Lower operational expense through reduced space and power usage
  4. Cost savings as a result of more efficient use of dark fiber infrastructure between core and access switches

This development is helping us prepare our infrastructure for the future. This not only supports ongoing growth, but also fulfils the need for more bandwidth. It will help us manage the increasing amount of traffic, deliver more 100G-ports and get things done in a more cost-efficient way. This, in turn, will allow us to reduce prices further.

Henk Steenman, CTO AMS-IX