Documentation on network tools useful for quick performance analysis by ordinary developers. These are the some of the tools a network engineer may ask you to perform first when reporting an issue. These area curated set of common tools for diagnosising network issues between two servers that you suspect there may be a problem at.
These tools are directional, so the results x->y is typically different than y->x. Ideally these tools are performed on both x->y and y->x for a proper analysis.
- How much data can get there and how much gets lost? What is the network speed between two points?
- iperf3 - Creates client/server between two network computers and measures throughput
- dd Netcat - Copies files between two network computers and measures file transfer (throughput)
- Is there a route from x to y and what are the stops between them? What path is taken and is there an issue inside or outside of the network?
- traceroute - Tracks all computers (typically routers) between two locations and measures round trip of sending three packets to each one
- mtr - Sends packets to all computers (typically routers) between two locations and reports packet loss and round trip statistics. Considered a combination of ping and traceroute.
- ping - Simple tool that sends packets to a remote host and reports round trip time for packets sent to that host.
- nc - Check if ip and ports are open and available for traffic
- These methods are more effective with the public internet than internal networks. For example, internal networks that use MPLS (Multi Protocol Label Switching) do not play well with traceroutes and by design, traceroutes can not see each path while traversing them and will return * * * as results.
example:
1 router1 4.261 ms 4.184 ms 5.198 ms
2 * * *
3 * * *
4 * * *
- Sometimes a single device may show packet loss. This is potentially not true packet loss, but 'superficial' packet loss, as the loss does not continue down on each hop, but is isolated to a single device. True packet loss will begin at one hop, and not only occur at each hope down to the end destination but will increase at each hop as well. So again, as long as these drops are isolated to the one device and not consistent across all subsequent hops, then what is observed is most likely a symptom of ICMP being deprioritized (superficial) by that one router for more important transit traffic.
example: Loss% Snt Last Avg Best Wrst StDev
3.|-- router3 0.0% 10 0.8 0.6 0.5 0.8 0.0
4.|-- router4 80.0% 10 4.7 3.7 2.7 4.7 1.4 <-
5.|-- router5 0.0% 10 1.0 1.1 1.0 1.2 0.0
- Traditional network tool used to examine network throughput performance between two servers. When run, it returns data transfered, bandwidth, packet loss and related statistics over a TCP connection.
- Language - C
- Authors - Esnet @ Berkeley
$ sudo apt install iperf3 #Debian/Ubuntu
$ sudo yum install iperf3 #RHEL/CentOS
$ sudo dnf install iperf3 #Fedora 22+ - Start an iperf3 server in the suspected machine
iperf3 -s- Connect to the server FROM your laptop or some other client and observe network performance statistics
iperf3 -c <server ip>
- Mature tool, been around since 2008
- Can be difficult to install on older servers which do not support iperf dependencies in thier kernel Centos < 7
- dd & Netcat
- Copies data between two servers and measures data transfered
dd typically comes installed in Linux
$ sudo apt install netcat #Debian/Ubuntu
$ sudo yum install netcat #RHEL/CentOS
$ sudo dnf install netcat #Fedora 22+ - Start an netcat server in the suspected machine
nc -v -l 2222 > /dev/null- Connect to the server FROM your laptop or some other client and observe network performance statistics
dd if=/dev/zero bs=1024K count=512 | nc -v <server ip> 2222- Observe file transfered. I had to interrupt transfer with Control C due to slow speed
27262976 bytes (27 MB, 26 MiB) copied, 53.4426 s, 510 kB/s- More likely to be installed already
- Limited in functionality, truly basic test.
- Tracks steps or hops between the source computer and the ip/url entered. Sends three packets to each computer and measures return time.
- Language - C
- Authors - Linux/OpenBSD
$ sudo apt install traceroute #Debian/Ubuntu
$ sudo yum install traceroute #RHEL/CentOS
$ sudo dnf install traceroute #Fedora 22+ - Run a traceroute to the server url or ip in question
traceroute 8.8.8.8- Oberve and track path. The output will resemble the following:
- The key to track this output:
| Hop # | Name/IP Address | RTT 1 (round trip time) | RTT 2 | RTT 3 |
|---|---|---|---|---|
| 1 | 174.111.103.xx | 44.904 ms | 44.885 ms | 44.855 ms |
- Hop number - hop along the route, in this case the first hop
- Domain/IP - IP address or domain of the router. Can appear as * * * if router will not reveal itself
- RTT columns - Display round trip time (RTT) for the packet to reach router and return to your computer. Listed in three columns and sent three times to guage consistency. Look for wildly different times when looking for problems.
- Likely to be already installed
- Typically difficult to act on knowledge if hops are in a network outside of your control.
- More detail
- Combines traceroute and ping. Sends packets to all computers between two locations and reports lost percentage and statistics on packet round trip times. Typically one must perform a mtr from both computers (A -> B) and (B -> A) to observe where packets are being dropped.
- Language - C
- Authors - https://github.com/traviscross
$ sudo apt install mtr-tiny #Debian/Ubuntu
$ sudo yum install mtr #RHEL/CentOS
$ sudo dnf install mtr #Fedora 22+ - Run a mtr to the server url or ip in question. MTR will continue to run unless control-C is pressed.
mtr 8.8.8.8- Alternatively, a report can be generated
mtr --report 8.8.8.8which will resemble the following:
- Great statistics on network performance
- Typically have to install on destination servers
- More detail
- Simple tool that sends packets to a remote host and reports round trip time for packets sent. You will have to hit control+c to interrupt the process when you are done with the results.
- Language - C
- Authors - Linux
Ping will typically be already installed in most Linux distros
$ sudo apt install iputils-ping #Debian/Ubuntu
$ sudo yum install iputils #RHEL/CentOS
$ sudo dnf install iputils #Fedora 22+ - Run a ping to the server url or ip in question. Ping will continue to run unless control-C is pressed.
ping 8.8.8.8which will resemble the following:
- Lets you know if a server is reachable and what the round time is for a packet
- Very basic information and more detail is often needed
- More detail
nc -z -v [hostname/IP address] [port number]
Example
nc -z -v 192.168.10.12 22
Connection to 192.118.20.95 22 port [tcp/ssh] succeeded!
- More detail