netbenches

FreeBSD network forwarding performance benchmark results

This repository includes bench scripts (need netmap pkt-gen and nanobsd like), bench configuration files, raw results and gnuplot graphs on few setups. Bench script documentation is available too.

Hardware benched are:

• SuperMicro X10SRW-F (16 cores Intel Xeon E5-2697A v4 @ 2.60GHz) and: 40G Chelsio T580-LP-CR, 100G Mellanox ConnectX-4 MCX416A-CCAT
• Dell PowerEdgre R630 (2x12 cores Intel Xeon E5-2650 v4 @ 2.2GHz) and: Intel 82599, Chelsio T520-CR, Mellanox ConnectX-3 and ConnectX-4 (MCX4121A-XCA)
• HP ProLiant DL360p Gen8 (8 cores Intel Xeon E5-2650 @ 2.60GHz) and 10-Gigabit Chelsio T540-CR
• SuperMicro X9SRL-F (10 cores Intel Xeon E5-2650L v2 @ 1.70GHz) and 10-Gigabit Chelsio T540-CR
• SuperMicro 5018A-FTN4 (8 cores Atom C2758 @ 2.40GHz) and 10-Gigabit Chelsio T540-CR
• SuperMicro 5018A-FTN4 (8 cores Atom C2758 @ 2.40GHz) and 10-Gigabit Intel 82599
• Netgate RCC-VE 4860 (4 cores Intel Atom C2558 @ 2.40GHz) and quad port Intel i350 (the dual port Intel i211 are not benched)
• PC Engines APU2C4 (4 cores AMD GX-412TC @ 1GHz) and 4 Intel i210AT Gigabit NIC

Specific benches:

• Impact of firewalls (ipfw/pf/ipf) on forwarding performance on:

  • 16 cores Xeon E5-2697A & 100G Mellanox ConnectX-4 MCX416A-CCAT
  • 16 cores Xeon E5-2697A & 40G Chelsio T580
  • 2x12 cores Xeon E5-2650 & 10G Mellanox ConnectX-4 Lx
  • 2x12 cores Xeon E5-2650 & 10G Chelsio T520-CR
  • 2x12 cores Xeon E5-2650 & 10G Intel 82599ES
  • 10 cores Xeon E5-2650L & 10G Chelsio T540
  • 8 cores Xeon E5-2650 & 10G Chelsio T540
  • 8 cores Atom C2758 & 10G Chelsio T540
  • 8 cores Atom C2758 & 10G Intel 82599
  • 4 cores Atom C2558 & Intel i350
  • 4 cores AMD GX-412TC & Intel i210
  • (old)4 cores Xeon L5630
  • (old)2 cores AMD G-T40E

• One year evolution:

  • One year (2017) of forwarding performance with 8 cores Xeon E5-2650 & Chelsio T540
  • One year (2017) of forwarding performance with 4 cores Atom 2558 & Intel i350
  • One year (2016) of forwarding performance with 8 cores Atom 2758 & Chelsio T540
  • One year (2016) of forwarding performance with 4 cores Atom 2558 & Intel i350
  • One year (2016) of forwarding performance with 4 cores AMD GX-412TC & Intel i210
  • One year (2015) of fastforwarding performance evolution with with 8 cores Xeon E5-2650
  • One year (2015) of fastforwarding performance evolution with 4 cores Atom C2558

• Hyper Threading impact:

  • 16 cores Xeon E5-2697A & 100G Mellanox ConnectX-4
  • 10 cores Xeon E5-2650L & Chelsio T540
  • 8 cores Xeon E5-2650 & Chelsio T540

• IPFW-at-NIC-level (DDoS protection)

  • 16 cores Xeon E5-2697A & 100G Mellanox ConnectX-4
  • 16 cores Xeon E5-2697A & 100G Chelsio T580
  • 10 cores Xeon E5-2650L & Chelsio T540

• comparing 12-head and 11-stable:

  • fbsd12-head vs 11-stable r318516 with 4 cores AMD GX-412TC

• comparing 11.0 and 11.1-BETA1:

  • FreeBSD 11.0 vs 11.1-BETA1 on a 8 cores Xeon E5-2650 with Chelsio T540
  • FreeBSD 11.0 vs 11.1-BETA1 on a 4 cores AMD GX-412TC with Intel i210

• Impact of entropy sources on FreeBSD 11.1 forwarding performance:

  • 8 cores Xeon E5-2650
  • 8 cores Atom C2758
  • 4 cores Atom C2558
  • 4 cores AMD GX-412TC

• Impact of adding VIMAGE support (without using it) on FreeBSD 11.1 forwarding&firewalling performance:

  • 8 cores Xeon E5-2650
  • 8 cores Atom C2758
  • 4 cores Atom C2558
  • 4 cores AMD GX-412TC
  • 4 cores Xeon L5630

• Impact of Intel's TurboBoost on FreeBSD 11-STABLE r327668:

  • 8 cores Atom C2758 & Chelsio T540
  • 8 cores Atom C2758 & Intel 82599

• Impact of if_bridge on FreeBSD 11.1:

  • 8 cores cores Xeon E5-2650
  • 8 cores Atom C2758 & Chelsio T540

• Impact of enabling ALTQ on FreeBSD 11-STABLE r332393:

  • 4 cores Atom C2558
  • 4 cores AMD GX-412TC

• Impact of number of static routes on FreeBSD 10.2:

  • 4 cores Xeon L5630

• Mellanox ConnectX-4 tuning:

  • Impact of queue number on forwarding performance with 2x12 cores Xeon E5-2650

• Chelsio NIC T540-CR tuning:

  • Impact of queue number on forwarding performance with 8 cores Xeon E5-2650
  • Impact of queue number on forwarding performance with 8 cores Atom C2758
  • Impact of disabling cxgbe.toecaps_allowed on forwarding performance with 8 cores Xeon E5-2650
  • Impact of pinning IRQ queue to CPU with 8 cores Xeon E5-2650
  • Impact of pinning IRQ queue to CPU with 8 cores Atom C2758

• Intel 82599EB NIC tuning:

  • Impact of Intel 82599EB queue number (1 queue per core) on forwarding performance with 4 cores Xeon L5630
  • Impact of random.harvest.mask and Intel 82599EB queue number (1 queue per core) on forwarding performance with 4 cores Xeon L5630
  • Impact of Intel 82599EB AIM on forwarding performance with 4 cores Xeon L5630
  • Impact of Intel 82599EB Rx|Tx process limit on forwarding performance with 4 cores Xeon L5630
  • Impact of Intel 82599EB descriptors per queue on forwarding performance with 4 cores Xeon L5630
  • Impact of AIM on Intel 82599EB

• Intel i350 tuning:

  • Impact of Intel i350 number of queues(1 queue per core) on forwarding performance with 4 cores Atom C2558
  • Impact of AIM on Intel i350

• Packet filter (pf) tuning:

  • Impact of states_hashsize table on pf performance on FreeBSD 12-head r332390 on Atom 8 core
  • Impact of states_hashsize table on pf performance on FreeBSD 12-head r332390 on AMD GX
  • Impact of number of flows on pf performance on FreeBSD 11.1-yandex

• ipfw tuning:

  • Impact of number of flows with maximum ipfw hash table on FreeBSD 11.1-yandex

• IPsec performance:

  • 4 cores AMD GX-412TC (PC Engines APU2C4)
  • 4 cores Atom C2558 (Netgate RCC-VE 4860)
  • 8 cores Atom C2758 & Chelsio T540
  • 8 cores Xeon E5-2650 & Chelsio T540
  • 2x12 cores Xeon E5-2650v4 & Chelsio T520
  • (old) 4 cores Intel Xeon L5630 (IBM x3550-M3)
  • (old) 2 cores AMD G-T40E (PC Engines APU1D)

• OpenVPN performance:

  • 4 cores AMD GX-412TC (PC Engines APU2C4)
  • 4 cores Atom C2558 (Netgate RCC-VE 4860)
  • 8 cores Atom C2758 & Chelsio T540
  • 4 cores Intel Xeon L5630 (IBM x3550-M3)
  • (old)2 cores AMD G-T40E (PC Engines APU1D)
  • Impact of OpenVPN fast-io (PC Engines APU2C4)

• Wireguard kernel vs userland:

  • PC Engines APU2C4 (4 cores AMD GX-412TC @ 1GHz) and 4 Intel i210AT Gigabit NIC
  • Netgate RCC-VE 4860 (4 cores Intel Atom C2558 @ 2.40GHz) and quad port Intel i350
  • 8 cores Atom C2758 & Chelsio T540

• Tinc performance:

  • 4 cores AMD GX-412TC (PC Engines APU2C4)

Synthesis:

• 
• 
• 
• 
• 
•