The Cornell Science Network

Performance Testing

For testing performance, we leave the host coscin-host-ithaca1 fixed while moving the host coscintest-host-nyc1 (called the Test Box, since the IP address varies and we therefore we can’t give it a permanent host name) at various points on the NYC side.

The Test Box runs Ubuntu Server 14.04 LTS. There are 4 Gigabit Ethernet copper interfaces on the Test box, generally assigned eth0-eth3. Two Ten-Gigabit interfaces are named eth4 and eth5. Traditionally, we plug the management network into eth0 or eth4, and use eth5 for the actual performance testing.

Changing the IP address is done in /etc/network/interfaces. Currently the setup for the Test Box at Cornell Tech is:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
#auto eth0
#iface eth0 inet dhcp
auto eth0
iface eth0 inet static
  address 128.253.80.134/26
  gateway 128.253.80.129

# Don't enable IP on eth1.  It will be sending utilization packets
auto eth1
iface eth1 inet manual
  pre-up ifconfig $IFACE up
  pre-down ifconfig $IFACE down

auto eth5
iface eth5 inet static
# address 192.168.57.100/24
  address 128.253.80.133/26
  gateway 128.253.80.129

Change the management interfaces on eth0 (or move it to eth4 if you wish) by changing the address and gateway lines for eth0. Then do the same for the production data network on eth5.

By default, Internet traffic will go over the default route using the lowest numbered Ethernet interface, which will be the management NIC. That’s fine, except you want CoSciN traffic to go over eth5. To do so, add a route for the Ithaca side of the CoSciN net:

sudo ip route add 132.236.137.96/27 via GATEWAY dev eth5 src NEWIP

Where GATEWAY and NEWIP are the values you used in the gateway and address lines for eth5 in the interfaces file.

You also need to set up the correct route on coscin-host-ithaca so it goes over the 10GB interface as well.

sudo ip route add NEWNET via 132.236.137.97 dev eth5 src 132.236.137.98

Where NEWNET is the network that the NEWIP is on (the network of the test box).

To make sure you have connectivity, ping both sides of the network - in the above case, ping NEWIP from the Ithaca host and 132.236.137.98 from the Test box. It generally takes a minute or two for the pings to actually work, and it tends to work best when you send it from both sides at about the same time. (This is really coincidence, but it generally seeds the ARP caches at the right time.)

Finally, to perform the actual test, do this on coscin-host-ithaca1:

iperf3 -s

And this on the test box:

iperf3 -c 132.236.137.98

If you get bandwidth over 1 Gbit/sec, you pretty much know everything is configured correctly. If it’s less then 1 Gbit/sec, make sure the traffic is going over the right interfaces. On both sides of the network, use ifconfig to verify the TX and RX packets of eth5 are increasing after an iperf3 test. If they are, the numbers you are seeing are correct.

Turning SDN On and Off

These notes are for internal use. For performance testing, we’ve been leaving SDN off by running the HP switches in traditional mode. (Actually we leave OpenFlow on, we just move all the ports to a VLAN which doesn’t use OpenFlow.)

To turn SDN back on:

Do the same on coscin-sw-nyc

Turn on controller software

Do the same for coscin-ctrl-nyc1

To turn it off,just reverse the steps above (move ports back to vlan 10, then stop the coscin-app-ryu service on the controllers)