Tcpdump is a great tool for testing those strange issues where nothing seem to make sense. Using this tool we can track incoming request and verify the server is receiving a packet and how the server sends out the data, we can also use it to track request originated on the server and what may of ended up with the packets. Below I will go over some basic examples.
Before we get to in depth with how Tcpdump works I thought it may be beneficial to review how a TCP connection is handled, Remember TCP is a stateful protocol unlike UDP which is stateless.
# tcpdump -nn -i eth0 port 80
Flags:
-nn Don't convert protocol and port numbers etc. to names either.
-i Listen on interface.
$ tcpdump -nn -i eth0 port 80 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes 15:21:47.709984 IP 64.39.0.68.12275 > 98.129.169.14.80: S 3338814818:3338814818(0) win 5840 <mss 1380,sackOK,timestamp 63005548 0,nop,wscale 7> 15:21:47.709984 IP 98.129.169.14.80 > 64.39.0.68.12275: S 1890161241:1890161241(0) ack 3338814819 win 5792 <mss 1460,sackOK,timestamp 1030385409 63005548,nop,wscale 5> 15:21:47.729984 IP 64.39.0.68.12275 > 98.129.169.14.80: . ack 1 win 46 <nop,nop,timestamp 63005550 1030385409> 15:21:51.489984 IP 64.39.0.68.12275 > 98.129.169.14.80: P 1:17(16) ack 1 win 46 <nop,nop,timestamp 63005927 1030385409> 15:21:51.489984 IP 98.129.169.14.80 > 64.39.0.68.12275: . ack 17 win 181 <nop,nop,timestamp 1030385787 63005927> 15:21:52.419984 IP 64.39.0.68.12275 > 98.129.169.14.80: P 17:19(2) ack 1 win 46 <nop,nop,timestamp 63006020 1030385787> 15:21:52.419984 IP 98.129.169.14.80 > 64.39.0.68.12275: . ack 19 win 181 <nop,nop,timestamp 1030385880 63006020> 15:21:52.469984 IP 98.129.169.14.80 > 64.39.0.68.12275: . 1:2737(2736) ack 19 win 181 <nop,nop,timestamp 1030385885 63006020> 15:21:52.469984 IP 98.129.169.14.80 > 64.39.0.68.12275: . 2737:4105(1368) ack 19 win 181 <nop,nop,timestamp 1030385885 63006020> 15:21:52.469984 IP 98.129.169.14.80 > 64.39.0.68.12275: FP 4105:4778(673) ack 19 win 181 <nop,nop,timestamp 1030385885 63006020> 15:21:52.479984 IP 64.39.0.68.12275 > 98.129.169.14.80: . ack 1369 win 67 <nop,nop,timestamp 63006026 1030385885> 15:21:52.479984 IP 64.39.0.68.12275 > 98.129.169.14.80: . ack 2737 win 89 <nop,nop,timestamp 63006026 1030385885> 15:21:52.479984 IP 64.39.0.68.12275 > 98.129.169.14.80: . ack 4105 win 110 <nop,nop,timestamp 63006026 1030385885> 15:21:52.479984 IP 64.39.0.68.12275 > 98.129.169.14.80: F 19:19(0) ack 4779 win 132 <nop,nop,timestamp 63006026 1030385885> 15:21:52.479984 IP 98.129.169.14.80 > 64.39.0.68.12275: . ack 20 win 181 <nop,nop,timestamp 1030385886 63006026> ^C 15 packets captured 15 packets received by filter 0 packets dropped by kernel
The general format of a tcp protocol line is:
src > dst: flags data-seqno ack window urgent options
Src and dst are the source and destination IP addresses and
ports. Flags are some combination of S (SYN), F (FIN), P
(PUSH), R (RST), W (ECN CWR) or E (ECN-Echo), or a single '.'
(no flags). Data-seqno describes the portion of sequence space
covered by the data in this packet (see example below). Ack is
sequence number of the next data expected the other direction
on this connection. Window is the number of bytes of receive
buffer space available the other direction on this connection.
Urg indicates there is 'urgent' data in the packet. Options
are tcp options enclosed in angle brackets (e.g., <mss 1024>).
Now that we have a better understanding of tcpdump lets go over a few examples. Have a look at the dump below and, what do you think happened to this TCP connection?
tcpdump -nn -i eth0 port 80 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes 15:43:42.679984 IP 64.39.0.68.31226 > 98.129.169.14.80: S 1829087099:1829087099(0) win 5840 <mss 1380,sackOK,timestamp 63137047 0,nop,wscale 7> 15:43:42.679984 IP 98.129.169.14.80 > 64.39.0.68.31226: S 1032427262:1032427262(0) ack 1829087100 win 5792 <mss 1460,sackOK,timestamp 1030516906 63137047,nop,wscale 5> 15:43:42.699984 IP 64.39.0.68.31226 > 98.129.169.14.80: . ack 1 win 46 <nop,nop,timestamp 63137049 1030516906> 15:43:47.699984 IP 98.129.169.14.80 > 64.39.0.68.31226: F 1:1(0) ack 1 win 181 <nop,nop,timestamp 1030517408 63137049> 15:43:47.709984 IP 64.39.0.68.31226 > 98.129.169.14.80: F 1:1(0) ack 2 win 46 <nop,nop,timestamp 63137550 1030517408> 15:43:47.709984 IP 98.129.169.14.80 > 64.39.0.68.31226: . ack 2 win 181 <nop,nop,timestamp 1030517409 63137550> ^C 6 packets captured 6 packets received by filter 0 packets dropped by kernel
Have a look at this dump, what do you think may be happening?
$ tcpdump -nn -i eth0 port 80 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes 15:48:00.639984 IP 64.39.0.68.40858 > 98.129.169.14.80: S 3026500468:3026500468(0) win 5840 <mss 1380,sackOK,timestamp 63162843 0,nop,wscale 7> 15:48:00.639984 IP 98.129.169.14.80 > 64.39.0.68.40858: S 783066446:783066446(0) ack 3026500469 win 5792 <mss 1460,sackOK,timestamp 1030542702 63162843,nop,wscale 5> 15:48:03.629984 IP 64.39.0.68.40858 > 98.129.169.14.80: S 3026500468:3026500468(0) win 5840 <mss 1380,sackOK,timestamp 63163143 0,nop,wscale 7> 15:48:03.629984 IP 98.129.169.14.80 > 64.39.0.68.40858: S 783066446:783066446(0) ack 3026500469 win 5792 <mss 1460,sackOK,timestamp 1030543001 63162843,nop,wscale 5> 15:48:04.489984 IP 98.129.169.14.80 > 64.39.0.68.40858: S 783066446:783066446(0) ack 3026500469 win 5792 <mss 1460,sackOK,timestamp 1030543087 63162843,nop,wscale 5> 15:48:09.629984 IP 64.39.0.68.40858 > 98.129.169.14.80: S 3026500468:3026500468(0) win 5840 <mss 1380,sackOK,timestamp 63163743 0,nop,wscale 7> 15:48:09.629984 IP 98.129.169.14.80 > 64.39.0.68.40858: S 783066446:783066446(0) ack 3026500469 win 5792 <mss 1460,sackOK,timestamp 1030543601 63162843,nop,wscale 5> 15:48:10.489984 IP 98.129.169.14.80 > 64.39.0.68.40858: S 783066446:783066446(0) ack 3026500469 win 5792 <mss 1460,sackOK,timestamp 1030543687 63162843,nop,wscale 5> 15:48:13.289984 IP 98.129.169.14.80 > 64.39.0.68.42551: S 280296214:280296214(0) ack 1624128497 win 5792 <mss 1460,sackOK,timestamp 1030543967 63159568,nop,wscale 5> 15:48:13.299984 IP 64.39.0.68.42551 > 98.129.169.14.80: R 1:1(0) ack 1 win 5792 15:48:21.629984 IP 64.39.0.68.40858 > 98.129.169.14.80: S 3026500468:3026500468(0) win 5840 <mss 1380,sackOK,timestamp 63164943 0,nop,wscale 7> 15:48:21.629984 IP 98.129.169.14.80 > 64.39.0.68.40858: S 783066446:783066446(0) ack 3026500469 win 5792 <mss 1460,sackOK,timestamp 1030544801 63162843,nop,wscale 5> 15:48:22.489984 IP 98.129.169.14.80 > 64.39.0.68.40858: S 783066446:783066446(0) ack 3026500469 win 5792 <mss 1460,sackOK,timestamp 1030544887 63162843,nop,wscale 5> ^C 13 packets captured 13 packets received by filter 0 packets dropped by kernel
As you can see tcpdump is a very powerful tool, below lets go over some more examples on flags.
# tcpdump -nn -i eth0 host 64.39.0.68
# tcpdump -nn -i eth0 port 80 and host 64.39.0.68
# tcpdump -nn -i eth0 host 64.39.0.68 and not 22
# tcpdump -i eth0 -w dumpfile
Date: 2010-03-22 10:12:39 CDT
HTML generated by org-mode 6.21b in emacs 23