Automating Google Workspace (GWS) & more with Apps Script
Ngrep commands
1. ngrep
Rishu Seth
rishu-seth@hotmail.com
15th February,2011
1 What is it + what it does?
• network sniffer
• from command line
• network grep ← unix grep tool + network
• many protocols supported (IP ICMP UDP TCP ETHERNET ...)
• !! allows to specify regular or hexadecimal expressions to match against
data payloads
• common use:
– debug plaintext protocols like FTP,HTTP,SNMP
– id and analyze anomalous net activity (virus/zombies/authentication
exploits)
– hacking/security
2 Synopsis + examples of commands
2.1 Syntax:
ngrep <-hNXViwqpevxlDtTRM> <-IO pcap_dump > < -n num > < -d dev > < -A
num > < -s snaplen > < -S limitlen > < -W normal|byline|single|none > <
-c cols > < -P char > < -F file > < match expression > < bpf filter >
for the detailed description of the options see man page. (man ngrep)
1
2. 2.2 Example of commands:
• Example: Basic Packet Sniffing
– To listen for traffic from x.x.x.x on port 25:
:: ngrep host x.x.x.x and port 25
– Monitor all activity crossing source or destination port 25 (SMTP):
:: ngrep -d any port 25
– Monitor any network-based syslog traffic for the occurrence of the
word “error”. ngrep knows how to convert service port names (on
UNIX, located in “/etc/services”) to port numbers.
:: ngrep -d any ’error’ port syslog
– Monitor any traffic crossing source or destination port 21 (FTP),
looking case-insensitively for the words “user” or “pass”, matched
as word-expressions (the match term(s) must have non-alphanumeric,
delimiting characters surrounding them).
:: ngrep -wi -d any ’user|pass’ port 21
3 Useful commands (from experimentation)
4 Example output sessions
see http://ngrep.sourceforge.net/usage.html for more detailed examples
4.1 Example: Debugging HTTP interactions
In certain scenarios it is desirous to see how web browsers communicate
with web servers, and to inspect the HTTP headers and possibly cookie
values that they are exchanging. In this example, we run an ngrep on a
webserver. Since it only has one interface, eth0, we omit specifying the
interface manually on the command line and allow ngrep to choose the
default interface for us, for convenience.
# ngrep port 80
interface: eth0 (64.90.164.72/255.255.255.252)
filter: ip and ( port 80 )
####
T 67.169.59.38:42167 -> 64.90.164.74:80 [AP]
GET / HTTP/1.1..User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux i
686) Opera 7.21 [en]..Host: www.darkridge.com..Accept: text/html, applicat
ion/xml;q=0.9, application/xhtml+xml;q=0.9, image/png, image/jpeg, image/gi
f, image/x-xbitmap, */*;q=0.1..Accept-Charset: iso-8859-1, utf-8, utf-16, *
2
3. ;q=0.1..Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0..Cookie: SQ
MSESSID=5272f9ae21c07eca4dfd75f9a3cda22e..Cookie2: $Version=1..Connection:
Keep-Alive, TE..TE: deflate, gzip, chunked, identity, trailers....
##
T 64.90.164.74:80 -> 67.169.59.38:42167 [AP]
HTTP/1.1 200 OK..Date: Mon, 29 Mar 2004 00:44:40 GMT..Server: Apache/2.0.49
(Unix)..Last-Modified: Tue, 04 Nov 2003 12:09:41 GMT..ETag: "210e23-326-f8
200b40"..Accept-Ranges: bytes..Vary: Accept-Encoding,User-Agent..Content-En
coding: gzip..Content-Length: 476..Keep-Alive: timeout=15, max=100..Connect
ion: Keep-Alive..Content-Type: text/html; charset=ISO-8859-1..Content-Langu
age: en..............}S]..0.|...........H...8........@......(.....Dw.%.,..
;.k.....Y>q<........d ...........3.i..kdm.u@d{.Q......@..B1.0.2YI^..R.....
....X......X..y........,..(........1...g.......*...j..a.‘._@.W....0.....?.
.R.K.j..Y.....>...;kw*U.j.<...0Tn.l.:......>Fs....’....h.’...u.H4..’.6.vID
I.......N.r.O...}...I.w. ...mX...L.s..{.L.R..-...e....~nu..t.3...H..#..J...
.u.?..]....^..2.....e8v/gP.....].48...qD!..........#y...m}..>/?..#........I
..I..4.P......2:...n8l.......!.Yr&...
##
As you can see, all headers and aspects of the HTTP transmission are
exposed in their gory detail. It’s a little hard to parse though, so let’s see
what happens when “-W byline” mode is used:
# ngrep -W byline port 80
interface: eth0 (64.90.164.72/255.255.255.252)
filter: ip and ( port 80 )
####
T 67.169.59.38:42177 -> 64.90.164.74:80 [AP]
GET / HTTP/1.1.
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux i686) Opera ...
Host: www.darkridge.com.
Accept: text/html, application/xml;q=0.9, application/xhtml+xml;q=0.9 ...
Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1.
Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0.
Cookie: SQMSESSID=5272f9ae21c07eca4dfd75f9a3cda22e.
Cookie2: $Version=1.
Cache-Control: no-cache.
Connection: Keep-Alive, TE.
TE: deflate, gzip, chunked, identity, trailers.
.
##
T 64.90.164.74:80 -> 67.169.59.38:42177 [AP]
HTTP/1.1 200 OK.
3
4. Date: Mon, 29 Mar 2004 00:47:25 GMT.
Server: Apache/2.0.49 (Unix).
Last-Modified: Tue, 04 Nov 2003 12:09:41 GMT.
ETag: "210e23-326-f8200b40".
Accept-Ranges: bytes.
Vary: Accept-Encoding,User-Agent.
Content-Encoding: gzip.
Content-Length: 476.
Keep-Alive: timeout=15, max=100.
Connection: Keep-Alive.
Content-Type: text/html; charset=ISO-8859-1.
Content-Language: en.
.
..........}S]..0.|...........H...8........@......(.....Dw.%.,..;.k.. ...
.;kw*U.j.<...0Tn.l.:......>Fs....’....h.’...u.H4..’.6.vIDI.......N.r ...
..H..#..J....u.?..]....^..2.....e8v/gP.....].48...qD!..........#y...m ...
####
(Content visually truncated for display purposes.) “-W byline” mode
tells ngrep to respect embedded line feeds when they occur. You’ll note
from the output above that there is still a trailing dot (“.”) on each line,
which is the carriage-return portion of the CRLF pair. Using this mode,
now the output has become much easier to visually parse
5 Different ways of using ngrep
There are also various additional twists to theway ngrep can be used for ex-
like the ability to include libpcap style packet filtering. Libpcap provides
fairly simple language for filtering traffic.
Filters are written by combining primitives with conjunctions( and ,or).
Primitives can be preceeded with term ’not’. Primitives are normally formed
with an id(can be numeric or symbolic name followed by one or more Qual-
ifiers.
There are three kind of qualifiers :
• Type
• Direction
• Protocol
5.1 Type Qualifiers
It describes what does the id refer to. Allowed options are :
• Host
4
5. • Net
• Port
Various examples are host crashdummy test, net 192.168.10.2, port 80.
5.2 Directional Qualifiers
It indicates the direction where traffic is flowing. Allowed qualifiers are :
• src(source)
• dst(destination)
5.3 Protocol Qualifiers
It limits the capture packets to those of a single protocol. If it is not used
then all ip packets are captured. For ex tcp, icmp or udp packets can be
filtered. Also primitives can be negated and combined to develop more com-
plex filters. For ex - if you want to see all traffic to ’rose’ except ’telnet’ and
’ftp-data ’you can use following filters :
host dst rose and not port telnet and not port ftp - data
6 Some Command Line Switches
Various common command line switches that are used with ngrep are as
following :
• -e : show empty packets
• -n : match number of packets and then exit
• -x : show packet in alternate hexa and ascii style
7 Some Examples with Data
I tried various commands and got respective data’s for every command which
are written and explained below :
• sudo ngrep -d wlan0 port 80 - It gives us the data after listening to
traffic on port 80 that is basically ’http’ traffic.
T 10.17.236.12:32838 -> 74.125.230.81:80 [AP]
GET /gen_204?atyp=i&ct=backbutton&cad=&ei=aFdaTfXPHJK14AbrkO2vDA&zx=1297750
513062 HTTP/1.1..Host: www.google.com..User-Agent: Mozilla/5.0 (X11; U; Lin
5
6. ux x86_64; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.04 (lucid) Firefox/
3.6.13..Accept: image/png,image/*;q=0.8,*/*;q=0.5..Accept-Language: en-us,e
n;q=0.5..Accept-Encoding: gzip,deflate..Accept-Charset: ISO-8859-1,utf-8;q=
0.7,*;q=0.7..Keep-Alive: 115..Connection: keep-alive..Referer: http://www.g
oogle.com/..Cookie: PREF=ID=94848ae271e094ae:U=89526b1f9a9b5069:FF=0:LD=en:
CR=2:TM=1295223931:LM=1297438900:GM=1:S=xLOHwIcMJsimEVBc; NID=43=hD06P-eYjH
T-tGvUAaU4lyBgcKBAkKRc_NYH-367UsZ3fORwE-d73fGCndgp2jFoTmSbO1RUoh78UmiWoD3x1
7XQkAggnPtElRV9FgAN5yH7eWIZkI9tv8PathJ5d1vz; GMAIL_RTT=37....
#
T 74.125.230.81:80 -> 10.17.236.12:32838 [AP]
HTTP/1.1 204 No Content..Content-Type: text/html; charset=UTF-8..Date: Tue,
15 Feb 2011 10:45:25 GMT..Server: gws..Content-Length: 0..X-XSS-Protection
: 1; mode=block....
• Same command with grep for word ’Server’ and it separates out the
data with word ’Server’ :
sudo ngrep -d port 80 |grep Server
Encoding: gzip..Server: gws..Content-Length: 8066..X-XSS-Protection: 1; mod
eb 2011 10:47:22 GMT..Server: sffe..X-XSS-Protection: 1; mode=block....
eb 2011 10:47:22 GMT..Server: sffe..X-XSS-Protection: 1; mode=block....
HTTP/1.1 304 Not Modified..Date: Tue, 15 Feb 2011 10:47:22 GMT..Server: gws
HTTP/1.1 304 Not Modified..Date: Tue, 15 Feb 2011 10:47:22 GMT..Server: gws
15 Feb 2011 10:47:23 GMT..Server: gws..Content-Length: 0..X-XSS-Protection
• Same command with grep for word ’Mozilla’ and it searches out the
data with word ’Mozilla’ :
Encoding: gzip..Server: gws..Content-Length: 8066..X-XSS-Protection: 1; mod
eb 2011 10:47:22 GMT..Server: sffe..X-XSS-Protection: 1; mode=block....
eb 2011 10:47:22 GMT..Server: sffe..X-XSS-Protection: 1; mode=block....
HTTP/1.1 304 Not Modified..Date: Tue, 15 Feb 2011 10:47:22 GMT..Server: gws
HTTP/1.1 304 Not Modified..Date: Tue, 15 Feb 2011 10:47:22 GMT..Server: gws
15 Feb 2011 10:47:23 GMT..Server: gws..Content-Length: 0..X-XSS-Protection
• Now with command - sudo ngrep -x -d wlan0 port 80 >xx.txt
I got following data with hexadecimal and ascii formats.
a0 5a 97 59 96 b2 04 55 bd 47 57 07 ef 22 34 a9 .Z.Y...U.GW.."4.
0a 02 e7 29 27 a8 2d 38 41 a2 1b 44 d5 2d 51 c6 ...)’.-8A..D.-Q.
b1 3a a6 25 cc 09 ca b8 41 ed 68 78 94 bc 9e dc .:.%....A.hx....
6
7. 9e 02 35 55 17 2f 9d 28 01 3b 42 e2 77 bb d4 f9 ..5U./.(.;B.w...
3b 57 eb a9 59 d2 26 55 01 f3 b4 6e e3 e9 e8 fe ;W..Y.&U...n....
98 b9 33 72 9f d0 13 da 3a a7 45 9a b0 73 68 d4 ..3r....:.E..sh.
0e d4 80 6e 06 6e 17 fb 98 c9 92 9e c5 db e4 ba ...n.n..........
e2 29 3e fe d9 81 3a a8 f8 c0 81 13 c0 08 29 b4 .)>...:.......).
ee df e9 c0 06 01 88 16 88 a6 d3 08 36 47 a3 ef ............6G..
ec e9 52 32 93 b8 28 be c7 3a c6 d5 94 a0 f0 fb ..R2..(..:......
b7 dc 28 dd de fd 5d fa dd 91 d9 6f a3 c6 7f eb ..(...]....o....
ec b7 db 2c dd 52 aa 5e 1a cc 19 ea c5 b7 eb 10 ...,.R.^........
31 16 65 f4 67 aa 05 c9 d9 b7 c8 4b 51 fb 63 df 1.e.g......KQ.c.
de c3 2c 60 5d 0b 9d c8 8b f9 9e 82 11 35 0e 1d ..,‘]........5..
d5 55 2c fb 1b e3 74 59 26 d4 0e ab d5 5a 8b bd .U,...tY&....Z..
08 59 8b ef 8d cc 3d a3 44 bd 5a ed 35 3b cd b8 .Y....=.D.Z.5;..
f4 2a 04 fe 2c cd 58 b4 5c b0 f3 bc 84 9d 1e a0 .*..,.X........
87 48 80 d6 51 ae 09 a8 00 eb b1 21 c4 ae b0 16 .H..Q......!....
2d ba 62 b8 79 13 8f d7 b3 c2 6e 95 95 22 1b 4e -.b.y.....n..".N
93 1e ac 8e 89 02 ab b7 a1 69
This command shows data in hexadecimal and ascii format and from
above data i can see that in ascii format there are many dots n we
are not able to figure out what kind of data it is but in hexadecimal
format it is having some value and may be if we have a hexadecimal
interpreter we can try and figure out what actually it means.
• I saved this data in file called xx.txt and now with awk command we
can separate out the whole column or intended lines out of it with help
of command :
cat xx.txt |awk -F ’ ’ ’print $3’
It will print the third column for me.
A.{,.6t......(..
}8.l....J..]...
..9j....~..N/2r.
Z)P.x..g.......f
..j...n..R$.R.L.
.Z.Y...U.GW.."4.
...)’.-8A..D.-Q.
.:.%....A.hx....
..5U./.(.;B.w...
;W..Y.&U...n....
..3r....:.E..sh.
...n.n..........
.)>...:.......).
7
![2.2 Example of commands:
• Example: Basic Packet Sniffing
– To listen for traffic from x.x.x.x on port 25:
:: ngrep host x.x.x.x and port 25
– Monitor all activity crossing source or destination port 25 (SMTP):
:: ngrep -d any port 25
– Monitor any network-based syslog traffic for the occurrence of the
word “error”. ngrep knows how to convert service port names (on
UNIX, located in “/etc/services”) to port numbers.
:: ngrep -d any ’error’ port syslog
– Monitor any traffic crossing source or destination port 21 (FTP),
looking case-insensitively for the words “user” or “pass”, matched
as word-expressions (the match term(s) must have non-alphanumeric,
delimiting characters surrounding them).
:: ngrep -wi -d any ’user|pass’ port 21
3 Useful commands (from experimentation)
4 Example output sessions
see http://ngrep.sourceforge.net/usage.html for more detailed examples
4.1 Example: Debugging HTTP interactions
In certain scenarios it is desirous to see how web browsers communicate
with web servers, and to inspect the HTTP headers and possibly cookie
values that they are exchanging. In this example, we run an ngrep on a
webserver. Since it only has one interface, eth0, we omit specifying the
interface manually on the command line and allow ngrep to choose the
default interface for us, for convenience.
# ngrep port 80
interface: eth0 (64.90.164.72/255.255.255.252)
filter: ip and ( port 80 )
####
T 67.169.59.38:42167 -> 64.90.164.74:80 [AP]
GET / HTTP/1.1..User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux i
686) Opera 7.21 [en]..Host: www.darkridge.com..Accept: text/html, applicat
ion/xml;q=0.9, application/xhtml+xml;q=0.9, image/png, image/jpeg, image/gi
f, image/x-xbitmap, */*;q=0.1..Accept-Charset: iso-8859-1, utf-8, utf-16, *
2](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)