Enabling Enhanced Networking with the Intel 82599 VF Interface on Linux Instances in a VPC

To enable enhanced networking (EBS-backed instances)

1. Connect to your instance.

2. From the instance, run the following command to update your instance with the newest kernel and kernel modules, including ixgbevf:

   [ec2-user ~]$ sudo yum update

3. From your local computer, reboot your instance using the Amazon EC2 console or the following command: reboot-instances (AWS CLI).

4. Connect to your instance again and verify that the ixgbevf module is installed and at the minimum recommended version using the modinfo ixgbevf command from Testing Whether Enhanced Networking with the Intel 82599 VF Interface is Enabled.

5. From your local computer, stop the instance using the Amazon EC2 console or the following command: stop-instances (AWS CLI). If your instance is managed by AWS OpsWorks, you should stop the instance in the AWS OpsWorks console so that the instance state remains in sync.

   Important

   If you are using an instance store-backed instance, you can't stop the instance. Instead, proceed to To enable enhanced networking (instance store-backed instances).

6. From your local computer, enable the enhanced networking attribute using the following command.

   Warning

   There is no way to disable the enhanced networking attribute after you've enabled it.

   Warning

   Enhanced networking is supported only for HVM instances. Enabling enhanced networking with a PV instance can make it unreachable. Setting this attribute without the proper module or module version can also make your instance unreachable.

   • modify-instance-attribute (AWS CLI)

     $ aws ec2 modify-instance-attribute --instance-id instance_id --sriov-net-support simple

7. (Optional) Create an AMI from the instance, as described in Creating an Amazon EBS-Backed Linux AMI . The AMI inherits the enhanced networking attribute from the instance. Therefore, you can use this AMI to launch another instance with enhanced networking enabled by default.

8. From your local computer, start the instance using the Amazon EC2 console or the following command: start-instances (AWS CLI). If your instance is managed by AWS OpsWorks, you should start the instance in the AWS OpsWorks console so that the instance state remains in sync.

9. Connect to your instance and verify that the ixgbevf module is installed and loaded on your network interface using the ethtool -i ethn command from Testing Whether Enhanced Networking with the Intel 82599 VF Interface is Enabled.

To enable enhanced networking on Ubuntu (EBS-backed instances)

1. Connect to your instance.

2. Update the package cache and packages.

   ubuntu:~$ sudo apt-get update && sudo apt-get upgrade -y

   Important

   If during the update process, you are prompted to install grub, use /dev/xvda to install grub onto, and then choose to keep the current version of /boot/grub/menu.lst.

3. Install the dkms package so that your ixgbevf module is rebuilt every time your kernel is updated.

   ubuntu:~$ sudo apt-get install -y dkms

4. Download the source for version 2.16.4 of the ixgbevf module on your instance from Sourceforge at http://sourceforge.net/projects/e1000/files/ixgbevf%20stable/.

   Note

   Earlier versions of ixgbevf, including the minimum recommended version, 2.14.2, do not build properly on some versions of Ubuntu. The 2.16.4 version of ixgbevf should be used for Ubuntu instances.

   ubuntu:~$ wget "sourceforge.net/projects/e1000/files/ixgbevf stable/2.16.4/ixgbevf-2.16.4.tar.gz"

5. Decompress and unarchive the ixgbevf package.

   ubuntu:~$ tar -xzf ixgbevf-2.16.4.tar.gz

6. Move the ixgbevf package to the /usr/src/ directory so dkms can find it and build it for each kernel update.

   ubuntu:~$ sudo mv ixgbevf-2.16.4 /usr/src/

7. Create the dkms configuration file with the following values, substituting your version of ixgbevf.

   1. Create the file.

      ubuntu:~$ sudo touch /usr/src/ixgbevf-2.16.4/dkms.conf

   2. Edit the file and add the following values.

      ubuntu:~$ sudo vim /usr/src/ixgbevf-2.16.4/dkms.conf
      PACKAGE_NAME="ixgbevf"
      PACKAGE_VERSION="2.16.4"
      CLEAN="cd src/; make clean"
      MAKE="cd src/; make BUILD_KERNEL=${kernelver}"
      BUILT_MODULE_LOCATION[0]="src/"
      BUILT_MODULE_NAME[0]="ixgbevf"
      DEST_MODULE_LOCATION[0]="/updates"
      DEST_MODULE_NAME[0]="ixgbevf"
      AUTOINSTALL="yes"

8. Add, build, and install the ixgbevf module on your instance with dkms.

   1. Add the module to dkms.

      ubuntu:~$ sudo dkms add -m ixgbevf -v 2.16.4

   2. Build the module with dkms.

      ubuntu:~$ sudo dkms build -m ixgbevf -v 2.16.4

   3. Install the module with dkms.

      ubuntu:~$ sudo dkms install -m ixgbevf -v 2.16.4

9. Rebuild the initramfs so the correct module is loaded at boot time.

   ubuntu:~$ sudo update-initramfs -c -k all

10. Verify that the ixgbevf module is installed and at the minimum recommended version using the modinfo ixgbevf command from Testing Whether Enhanced Networking with the Intel 82599 VF Interface is Enabled.

    ubuntu:~$ modinfo ixgbevf
    filename:       /lib/modules/3.13.0-74-generic/updates/dkms/ixgbevf.ko
    version:        2.16.4
    license:        GPL
    description:    Intel(R) 10 Gigabit Virtual Function Network Driver
    author:         Intel Corporation, <[email protected]>
    srcversion:     759A432E3151C8F9F6EA882
    alias:          pci:v00008086d00001515sv*sd*bc*sc*i*
    alias:          pci:v00008086d000010EDsv*sd*bc*sc*i*
    depends:
    vermagic:       3.13.0-74-generic SMP mod_unload modversions
    parm:           InterruptThrottleRate:Maximum interrupts per second, per vector, (956-488281, 0=off, 1=dynamic), default 1 (array of int)

11. From your local computer, stop the instance using the Amazon EC2 console or the following command: stop-instances (AWS CLI). If your instance is managed by AWS OpsWorks, you should stop the instance in the AWS OpsWorks console so that the instance state remains in sync.

    Important

    If you are using an instance store-backed instance, you can't stop the instance. Instead, proceed to To enable enhanced networking on Ubuntu (instance store-backed instances).

12. From your local computer, enable the enhanced networking sriovNetSupport attribute using the following command. Note that there is no way to disable this attribute after you've enabled it.

    Warning

    Enhanced networking is supported only for HVM instances. Enabling enhanced networking with a PV instance can make it unreachable. Setting this attribute without the proper module or module version can also make your instance unreachable.

    • modify-instance-attribute (AWS CLI)

      $ aws ec2 modify-instance-attribute --instance-id instance_id --sriov-net-support simple

13. (Optional) Create an AMI from the instance, as described in Creating an Amazon EBS-Backed Linux AMI . The AMI inherits the enhanced networking sriovNetSupport attribute from the instance. Therefore, you can use this AMI to launch another instance with enhanced networking enabled by default.

14. From your local computer, start the instance using the Amazon EC2 console or the following command: start-instances (AWS CLI). If your instance is managed by AWS OpsWorks, you should start the instance in the AWS OpsWorks console so that the instance state remains in sync.

15. (Optional) Connect to your instance and verify that the module is installed.

To enable enhanced networking on Linux (EBS-backed instances)

1. Connect to your instance.

2. Download the source for version 2.14.2 of the ixgbevf module on your instance from Sourceforge at http://sourceforge.net/projects/e1000/files/ixgbevf%20stable/. This is the minimum version recommended for enhanced networking.

   Note

   Earlier versions of ixgbevf, including the minimum recommended version, 2.14.2, do not build properly on some Linux distributions, including certain versions of Ubuntu. If you receive build errors, you may try a newer version, such as 2.16.4 (which fixes the build issue on affected Ubuntu versions).

3. Compile and install the ixgbevf module on your instance.

   If your distribution supports dkms, then you should consider configuring dkms to recompile the ixgbevf module whenever your system's kernel is updated. If your distribution does not support dkms natively, you can find it in the EPEL repository (https://fedoraproject.org/wiki/EPEL) for Red Hat Enterprise Linux variants, or you can download the software at http://linux.dell.com/dkms/. Use Step 6 through Step 8 in To enable enhanced networking on Ubuntu (EBS-backed instances) for help configuring dkms.

   Warning

   If you compile the ixgbevf module for your current kernel and then upgrade your kernel without rebuilding the driver for the new kernel, your system may revert to the distribution-specific ixgbevf module at the next reboot, which could make your system unreachable if the distribution-specific version is incompatible with enhanced networking.

4. Run the sudo depmod command to update module dependencies.

5. Update the initramfs on your instance to ensure that the new module loads at boot time.

6. Determine if your system uses predictable network interface names by default. Systems that use systemd or udev versions 197 or greater can rename Ethernet devices and they do not guarantee that a single network interface will be named eth0. This behavior can cause problems connecting to your instance. For more information and to see other configuration options, see Predictable Network Interface Names/ on the freedesktop.org website.

   1. You can check the systemd or udev versions on RPM-based systems with the following command:

      [ec2-user ~]$ rpm -qa | grep -e '^systemd-[0-9]\+\|^udev-[0-9]\+'
      systemd-208-11.el7_0.2.x86_64

      In the above Red Hat 7 example, the systemd version is 208, so predictable network interface names must be disabled.

   2. Disable predictable network interface names by adding the net.ifnames=0 option to the GRUB_CMDLINE_LINUX line in /etc/default/grub.

      [ec2-user ~]$ sudo sed -i '/^GRUB\_CMDLINE\_LINUX/s/\"$/\ net\.ifnames\=0\"/' /etc/default/grub

   3. Rebuild the grub configuration file.

      [ec2-user ~]$ sudo grub2-mkconfig -o /boot/grub2/grub.cfg

7. From your local computer, stop the instance using the Amazon EC2 console or the following command: stop-instances (AWS CLI). If your instance is managed by AWS OpsWorks, you should stop the instance in the AWS OpsWorks console so that the instance state remains in sync.

   Important

   If you are using an instance store-backed instance, you can't stop the instance. Instead, proceed to To enabled enhanced networking (instance store–backed instances)

8. From your local computer, enable the enhanced networking attribute using the following command. Note that there is no way to disable the networking attribute after you've enabled it.

   Warning

   Enhanced networking is supported only for HVM instances. Enabling enhanced networking with a PV instance can make it unreachable. Setting this attribute without the proper module or module version can also make your instance unreachable.

   • modify-instance-attribute (AWS CLI)

     $ aws ec2 modify-instance-attribute --instance-id instance_id --sriov-net-support simple

9. (Optional) Create an AMI from the instance, as described in Creating an Amazon EBS-Backed Linux AMI . The AMI inherits the enhanced networking attribute from the instance. Therefore, you can use this AMI to launch another instance with enhanced networking enabled by default.

   Important

   If your instance operating system contains an /etc/udev/rules.d/70-persistent-net.rules file, you must delete it before creating the AMI. This file contains the MAC address for the Ethernet adapter of the original instance. If another instance boots with this file, the operating system will be unable to find the device and eth0 may fail, causing boot issues. This file is regenerated at the next boot cycle, and any instances launched from the AMI create their own version of the file.

10. From your local computer, start the instance using the Amazon EC2 console or the following command: start-instances (AWS CLI). If your instance is managed by AWS OpsWorks, you should start the instance in the AWS OpsWorks console so that the instance state remains in sync.

11. (Optional) Connect to your instance and verify that the module is installed.