Learn what netstat command and some of the real-time examples are.
netstat (network statistics) is a command-line tool that displays network connections (both incoming and outgoing), routing tables, and a number of network interface statistics.
It is available on Linux, Unix-like, and Windows operating systems. netstat is powerful and can be a handy tool to troubleshoot network-related issues and verify connection statistics.
If you type netstat -help, you will get the following usage guidelines.
[root@lab ~]# netstat -help
usage: netstat [-vWeenNcCF] [<Af>] -r netstat {-V|--version|-h|--help}
netstat [-vWnNcaeol] [<Socket> ...]
netstat { [-vWeenNac] -I[<Iface>] | [-veenNac] -i | [-cnNe] -M | -s [-6tuw] } [delay]
-r, --route display routing table
-I, --interfaces=<Iface> display interface table for <Iface>
-i, --interfaces display interface table
-g, --groups display multicast group memberships
-s, --statistics display networking statistics (like SNMP)
-M, --masquerade display masqueraded connections
-v, --verbose be verbose
-W, --wide don't truncate IP addresses
-n, --numeric don't resolve names
--numeric-hosts don't resolve host names
--numeric-ports don't resolve port names
--numeric-users don't resolve user names
-N, --symbolic resolve hardware names
-e, --extend display other/more information
-p, --programs display PID/Program name for sockets
-o, --timers display timers
-c, --continuous continuous listing
-l, --listening display listening server sockets
-a, --all display all sockets (default: connected)
-F, --fib display Forwarding Information Base (default)
-C, --cache display routing cache instead of FIB
-Z, --context display SELinux security context for sockets
<Socket>={-t|--tcp} {-u|--udp} {-U|--udplite} {-S|--sctp} {-w|--raw}
{-x|--unix} --ax25 --ipx --netrom
<AF>=Use '-6|-4' or '-A <af>' or '--<af>'; default: inet
List of possible address families (which support routing):
inet (DARPA Internet) inet6 (IPv6) ax25 (AMPR AX.25)
netrom (AMPR NET/ROM) ipx (Novell IPX) ddp (Appletalk DDP)
x25 (CCITT X.25)
[root@lab ~]#Let me show you some of the examples of the command. The following are tested on RHEL/CentOS, but I don’t see any reason not to work on another distro like Ubuntu.
Established Connection
If you are looking for all established connections from the server.
[root@lab ~]# netstat -natu | grep 'ESTABLISHED'
tcp 0 21 68.183.37.102:22 222.186.31.135:21714 ESTABLISHED
tcp 0 36 68.183.37.102:22 52.148.155.182:49859 ESTABLISHED
tcp 0 0 68.183.37.102:22 61.177.142.158:55481 ESTABLISHED
[root@lab ~]#If you many established connections and interested in looking for one of the IPs, then you can use another grep.
[root@lab ~]# netstat -natu | grep 'ESTABLISHED' | grep 61.177.142.158
tcp 0 1280 68.183.37.102:22 61.177.142.158:33932 ESTABLISHED
[root@lab ~]#Listening Connection
Let’s say you’ve started some service, and that is supposed to listen on a particular IP:Port, this would be handy to verify.
[root@lab ~]# netstat -an | grep 'LISTEN'
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp6 0 0 :::111 :::* LISTEN
tcp6 0 0 :::80 :::* LISTEN
tcp6 0 0 :::22 :::* LISTEN
[root@lab ~]#Or, you can use -l argument to show all the listening sockets.
[root@lab ~]# netstat -l
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 localhost:smtp 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:sunrpc 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:ssh 0.0.0.0:* LISTEN
tcp6 0 0 [::]:sunrpc [::]:* LISTEN
tcp6 0 0 [::]:webcache [::]:* LISTEN
tcp6 0 0 [::]:ssh [::]:* LISTEN
udp 0 0 0.0.0.0:805 0.0.0.0:*
udp 0 0 0.0.0.0:sunrpc 0.0.0.0:*
udp 0 0 localhost:323 0.0.0.0:*
udp6 0 0 [::]:805 [::]:*
udp6 0 0 [::]:sunrpc [::]:*
udp6 0 0 ip6-localhost:323 [::]:*
Active UNIX domain sockets (only servers)
Proto RefCnt Flags Type State I-Node Path
unix 2 [ ACC ] STREAM LISTENING 15108 /run/dbus/system_bus_socket
unix 2 [ ACC ] STREAM LISTENING 8202 /run/systemd/journal/stdout
unix 2 [ ACC ] SEQPACKET LISTENING 12813 /run/udev/control
unix 2 [ ACC ] STREAM LISTENING 17542 public/pickup
unix 2 [ ACC ] STREAM LISTENING 15165 /var/run/rpcbind.sock
unix 2 [ ACC ] STREAM LISTENING 17546 public/cleanup
unix 2 [ ACC ] STREAM LISTENING 15605 /var/lib/gssproxy/default.sock
unix 2 [ ACC ] STREAM LISTENING 12706 /run/systemd/private
unix 2 [ ACC ] STREAM LISTENING 17549 public/qmgr
unix 2 [ ACC ] STREAM LISTENING 17571 public/flush
unix 2 [ ACC ] STREAM LISTENING 17553 private/tlsmgr
unix 2 [ ACC ] STREAM LISTENING 17586 public/showq
unix 2 [ ACC ] STREAM LISTENING 17556 private/rewrite
unix 2 [ ACC ] STREAM LISTENING 17559 private/bounce
unix 2 [ ACC ] STREAM LISTENING 17562 private/defer
unix 2 [ ACC ] STREAM LISTENING 17565 private/trace
unix 2 [ ACC ] STREAM LISTENING 17568 private/verify
unix 2 [ ACC ] STREAM LISTENING 17574 private/proxymap
unix 2 [ ACC ] STREAM LISTENING 17577 private/proxywrite
unix 2 [ ACC ] STREAM LISTENING 17580 private/smtp
unix 2 [ ACC ] STREAM LISTENING 17583 private/relay
unix 2 [ ACC ] STREAM LISTENING 17589 private/error
unix 2 [ ACC ] STREAM LISTENING 17592 private/retry
unix 2 [ ACC ] STREAM LISTENING 17595 private/discard
unix 2 [ ACC ] STREAM LISTENING 17598 private/local
unix 2 [ ACC ] STREAM LISTENING 17601 private/virtual
unix 2 [ ACC ] STREAM LISTENING 17604 private/lmtp
unix 2 [ ACC ] STREAM LISTENING 17607 private/anvil
unix 2 [ ACC ] STREAM LISTENING 17610 private/scache
unix 2 [ ACC ] STREAM LISTENING 15606 /run/gssproxy.sock
[root@lab ~]#Take advantage of grep to filter the results.
Port Number used by PID
You know your application started and aware of PID (Process Identifier) but not sure what’s the port number it’s using. Below example is for PID 3937
[root@lab ~]# netstat -anlp |grep 3937
tcp6 0 0 :::80 :::* LISTEN 3937/httpd
unix 3 [ ] STREAM CONNECTED 2442387 3937/httpd
[root@lab ~]#As you can see, port 80 is being used for PID 3937.
All Protocols Statistics
Having frequent disconnections due to packet discarded? -s argument will show you overall stats where you can pay attention to packets discarded messages.
[root@lab ~]# netstat -s
Ip:
731422 total packets received
0 forwarded
0 incoming packets discarded
731399 incoming packets delivered
787732 requests sent out
16 dropped because of missing route
Icmp:
5277 ICMP messages received
120 input ICMP message failed.
InCsumErrors: 6
ICMP input histogram:
destination unreachable: 193
timeout in transit: 16
echo requests: 5060
echo replies: 2
9355 ICMP messages sent
0 ICMP messages failed
ICMP output histogram:
destination unreachable: 4295
echo replies: 5060
IcmpMsg:
InType0: 2
InType3: 193
InType8: 5060
InType11: 16
OutType0: 5060
OutType3: 4295
Tcp:
42 active connections openings
35226 passive connection openings
1693 failed connection attempts
645 connection resets received
2 connections established
646705 segments received
648037 segments send out
99463 segments retransmited
27377 bad segments received.
150893 resets sent
InCsumErrors: 27377
Udp:
74547 packets received
4814 packets to unknown port received.
56 packet receive errors
74584 packets sent
0 receive buffer errors
0 send buffer errors
InCsumErrors: 56
UdpLite:
TcpExt:
177 invalid SYN cookies received
1693 resets received for embryonic SYN_RECV sockets
316 TCP sockets finished time wait in fast timer
3 packets rejects in established connections because of timestamp
70248 delayed acks sent
6 delayed acks further delayed because of locked socket
Quick ack mode was activated 3082 times
17 SYNs to LISTEN sockets dropped
28179 packets directly queued to recvmsg prequeue.
9802 bytes directly received in process context from prequeue
72106 packet headers predicted
94182 acknowledgments not containing data payload received
40094 predicted acknowledgments
332 times recovered from packet loss by selective acknowledgements
8 congestion windows recovered without slow start by DSACK
1173 congestion windows recovered without slow start after partial ack
1029 timeouts after SACK recovery
8 timeouts in loss state
329 fast retransmits
3 forward retransmits
32 retransmits in slow start
44785 other TCP timeouts
TCPLossProbes: 9763
TCPLossProbeRecovery: 1732
54 SACK retransmits failed
3144 DSACKs sent for old packets
4 DSACKs sent for out of order packets
695 DSACKs received
1 DSACKs for out of order packets received
44 connections reset due to unexpected data
76 connections reset due to early user close
6079 connections aborted due to timeout
TCPDSACKIgnoredNoUndo: 448
TCPSpuriousRTOs: 5
TCPSackShiftFallback: 465
IPReversePathFilter: 11
TCPRcvCoalesce: 32369
TCPOFOQueue: 4313
TCPOFOMerge: 4
TCPChallengeACK: 2
TCPSynRetrans: 43670
TCPOrigDataSent: 208010
TCPACKSkippedSeq: 12
IpExt:
InNoRoutes: 12
InOctets: 133789295
OutOctets: 151093769
InNoECTPkts: 731338
InECT1Pkts: 3
InECT0Pkts: 1568
InCEPkts: 108
[root@lab ~]#Kernel routing information
Having a routing issue? or, connectivity is not working as expected due to connection is traveling through a different route?
Quickly check the routing table.
[root@lab ~]# netstat -r
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
default gateway 0.0.0.0 UG 0 0 0 eth0
10.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
68.183.32.0 0.0.0.0 255.255.240.0 U 0 0 0 eth0
link-local 0.0.0.0 255.255.0.0 U 0 0 0 eth0
[root@lab ~]#PID used by Port Number
Very handy to troubleshoot port conflict issue. Lets’s say you are trying to start Apache or Nginx server, which listens on port 80 but can’t because some other process already using port 80.
[root@lab ~]# netstat -anlp |grep 80 | grep LISTEN
tcp6 0 0 :::80 :::* LISTEN 3937/httpd
[root@lab ~]#And, you can see the PID 3937 is using that port.
If you are using AIX, then
netstat -Aan | grep $portnumberThis will display the address of the Protocol Control Block in hexadecimal
Once you have hexadecimal, then can execute below to get wich process is holding a port number.
rmsock $address_of_pcb tcpcbList of network interfaces
Having multiple ethernet interfaces? or not sure and want to find out?
[root@lab ~]# netstat -i
Kernel Interface table
Iface MTU RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0 1500 793026 0 0 0 849443 0 0 0 BMRU
lo 65536 6 0 0 0 6 0 0 0 LRU
[root@lab ~]#Continuous Listening
An excellent option when troubleshooting services crash related issues. Let’s say an application is crashing randomly every few minutes. But, not sure when exactly. You can use -c argument which will continuously show the results.
[root@lab ~]# netstat -anlpc |grep 8080
tcp6 0 0 :::8080 :::* LISTEN 11766/httpd
tcp6 0 0 :::8080 :::* LISTEN 11766/httpd
tcp6 0 0 :::8080 :::* LISTEN 11766/httpd
tcp6 0 0 :::8080 :::* LISTEN 11766/httpdWhen it stops updating, then you know its crashed.
Conclusion
netstat is one of the widely used commands by sysadmin and I hope the above examples give you an idea about what you can do with it. If you are looking to learn more about Linux administration, then check out this Udemy course.