Skip to content

DPDK and OpenLI

Jump to bottom
Shane Alcock edited this page Feb 11, 2020 · 2 revisions

The following instructions are aimed at helping you get the OpenLI collector working with DPDK. DPDK can be a fickle beast, so I can only say that what is offered here is based on what works in my own development environment. Different versions of DPDK, different Linux distros, different phases of the moon: all of these may render these instructions useless :/

The current instructions were written based on DPDK 18.11, on a standard x86-64 Linux host. Every command needs to be run on your OpenLI collector host(s).

Note: these instructions will configure DPDK for the current boot-cycle -- getting DPDK automatically configured on boot is left as an exercise for the reader.

Installing DPDK packages

If you haven't already, install both the DPDK package and the package for the igb_uio kernel module.

For Debian, Ubuntu:

# apt-get install dpdk dpdk-igb-uio-dkms

Setup DPDK environment variables

# export RTE_TARGET="x86_64-default-linuxapp-gcc"
# export RTE_SDK="/usr/share/dpdk"
# export RTE_INCLUDE="/usr/include/dpdk"

Insert the modules for the igb_uio driver

# modprobe uio
# modprobe igb_uio

Configure your system to support hugepages

I've typically found 256 to be a good number of hugepages, but feel free to tweak this as necessary for your particular deployment (some people use 2048, for instance).

# for n in `ls -d /sys/devices/system/node/node*`; do \
      echo 256 > ${n}/hugepages/hugepages-2048kB/nr_hugepages \
  done

Figure out the PCI address of the interface you want to bind to DPDK

The best way to work this out to run:

$ /usr/bin/dpdk-devbind.py --status

This will list all of the network devices on your host. Find the line in the output where the if= field matches the name of the interface you want to give to DPDK. The PCI address is the first item in that line and will look something like 0000:XX:00.X.

IMPORTANT: once you bind this interface to DPDK, it will no longer be visible or usable by the kernel. This means it will no longer be able to pass traffic on to applications (including SSH!), so do NOT bind the interface that you are using to talk to the collector host.

Bind the interface to DPDK

 # /usr/bin/dpdk-devbind.py -b igb_uio <interface PCI address>

Replace <interface PCI address> with the PCI address you found in the previous step (e.g. 0000:XX:00.X).

You can re-run dpdk-devbind.py --status to check that the interface you bound appears in the list of devices using a DPDK-compatible driver. The interface should also no longer appear in the list that is returned if you run ip link.

To unbind the interface and re-bind it to the regular kernel driver:

# /usr/bin/dpdk-devbind.py -u <interface PCI address>
# /usr/bin/dpdk-devbind.py -b ixgbe <interface PCI address>

Using your DPDK interface in OpenLI

In your OpenLI collector config (i.e. /etc/openli/collector-config.yaml), you can tell OpenLI to use your DPDK interface by adding the following to your inputs list:

inputs:
  - uri: dpdk:<interface PCI address>
    threads: 4
Clone this wiki locally