JUG Louvain - HTTP revisited

1. HTTP revisited
& some Java networking
Java User Group
Louvain-La-Neuve @ EPHEC
20/11/2014
Marc Tritschler
24/10/2014 Copyrigth (c) Marc Tritschler 1

2. Program
1.Introduction
2.Internet Stack (reminder ?)
3.Java and the Internet stack
4.Coding time 
PLEASE PLEASE INTERRUPT ME
(IRQ-0 or any other )
24/10/2014 Copyrigth (c) Marc Tritschler 2

3. 1. Introduction
Already heard of Gopher ?
Internet = HTTP
24/10/2014 Copyrigth (c) Marc Tritschler 3

4. Internet = HTTP
• Google
• Facebook
• Gmail
• Yahoo
HTTP
• Youtube
• Twitter
• Amazon
• …
24/10/2014 Copyrigth (c) Marc Tritschler 4

5. Almost EVERYTHING runs over HTTP
• HTTP ~ 75 % of traffic (http://www.caida.org/publications/papers/1998/Inet98/Inet98.html MUST
READ)
– WebServices (SOAP & REST)
– HTML
– AJAX
– Email (webmail)
• Exceptions
• Email (smtp/imap/pop3)
• DNS
• FTP
• WebSocket which 'upgrades' from HTPP (previous JUG)
24/10/2014 Copyrigth (c) Marc Tritschler 5

6. HTML, JS, GIF, MP4 … over HTTP
24/10/2014 Copyrigth (c) Marc Tritschler 6

7. 2. The Internet Stack
Forget about
the
7 layers OSI model
24/10/2014 Copyrigth (c) Marc Tritschler 7

8. The Internet Stack (4 layers)
My App
HTTP
TCP/IP familly
Physical Layer
SSL
80 443
In the JRE.
Java
Part of OS.
C/C++
Number of Job
& Products
Opportunities
Electronics
Assembly
24/10/2014 Copyrigth (c) Marc Tritschler 8

9. Where's HTML in this Stack ???
DO NOT MIX DATA, API and PROTOCOL
•Data (= contents = payload = BYTES)
– Binary vs Text
– HTML, CSS, XML, JavaScript, JPEG, MP4, …
– Text Data  Encodings (UTF-8)
•API  vertical links (no bytes on the wire)
•Protocol horizontal links
•AJAX = JavaScript performing HTTP requests
24/10/2014 Copyrigth (c) Marc Tritschler 9

10. TCP ports
http://fr.wikipedia.org/wiki/Liste_de_ports_logiciels
Well Known (0 – 1024)
20, 21 FTP
22 SSH
23 Telnet
25, 110 SMTP/POP3
80 HTTP
53 DNS
137 … 139 NETBIOS
389 LDAP
443 HTTPS
Others (1025-65535)
1521 Oracle DB
8080 http proxies, Tomcat
24/10/2014 Copyrigth (c) Marc Tritschler 10

11. HTTP versions
• HTTP 1.0 @DEPRECATED
– each request/response  new TCP connection (= exchange of 3 TCP packets
(SYN, SYN/ACK, ACK))
• HTTP 1.1 CURRENT
– Keep TCP session
• HTTP 2.0 FUTURE (around DEC 2014)
– Negotiation (1.1, 2.0, other protocols)
– Close to 1.1 (methods, status codes, …)
– Server Push
– Fix HOL problem
– Loads page elements in parallel over single TCP connection
http://en.wikipedia.org/wiki/HTTP/2 for more info
24/10/2014 Copyrigth (c) Marc Tritschler 11

12. HTTP Refresher 
• RFC/IETF Standards (read this only if …)
• Simple request/response
• Header + [Body]
• Stateless
• Bytes and Chars (use UTF-8 encoding)
• Synchronous HALF-DUPLEX (request ALWAYS
initiated by the client remeber the
problems for interactive games
• Can be verbose (http headers) (~600 bytes for simple Hello World)
24/10/2014 Copyrigth (c) Marc Tritschler 12

13. HTTP Overview
REQUEST (GET, POST, …)
Client Server
RESPONSE (CODE + [DATA])
1xx : Informational - Request received, continuing process
2xx : Success - The action was successfully received, understood, and accepted
3xx : Redirection - Further action must be taken in order to complete the request
4xx : Client Error - The request contains bad syntax or cannot be fulfilled
5xx : Server Error - The server failed to fulfill an apparently valid request
24/10/2014 Copyrigth (c) Marc Tritschler 13

14. HTTP Request : methods
http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html
• Safe (GET/HEAD) & Idempotents methods
• GET, HEAD
• OPTIONS
• POST, PUT
• DELETE
• TRACE
• CONNECT FREEDOM 
24/10/2014 Copyrigth (c) Marc Tritschler 14

15. HTTP Responses : Status Codes
• 200 OK
• 400 Bad Request
• 401 Unauthorized (WWW-Authenticate header)
• 403 Forbidden
• 404 Not Found
• 407 Proxy Authentication Required (Proxy-Authenticate header)
• 500 Internal Server Error
• Complete List
http://www.w3.org/Protocols/rfc2616/rfc2616-sec6.html#sec6.1.1
24/10/2014 Copyrigth (c) Marc Tritschler 15

16. HTTP Headers
http://en.wikipedia.org/wiki/List_of_HTTP_header_fields
• A lot of "standards" and "non standards"
defined … a little bit messy 
• Firefox Dev console
24/10/2014 Copyrigth (c) Marc Tritschler 16

17. HTTP Request Example
POST http://sghrsot.cc.cec.eu.int:1045/hermes/Proxy/1.17/DocumentWebServicePS HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: ""
User-Agent: Jakarta Commons-HttpClient/3.1
Host: host1.domain1.company :1045
Content-Length: 585
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:typ="http://xx.xxxxxx.eu/sg/hrs/types">
<soapenv:Header/>
<soapenv:Body>
<typ:getDocument>
<typ:header>
<typ:userName>xyz</typ:userName>
<typ:ticket>onetimeticket</typ:ticket>
<typ:applicationId>myapp</typ:applicationId>
</typ:header>
<typ:documentId>080166e48102103b</typ:documentId>
</typ:getDocument>
</soapenv:Body>
</soapenv:Envelope>
24/10/2014 Copyrigth (c) Marc Tritschler 17

18. HTTP Response Example
HTTP/1.1 200 OK
Date: Mon, 20 Oct 2014 16:12:22 GMT
Content-Length: 9159
Content-Type: text/xml; charset=utf-8
<?xml version="1.0" encoding="UTF-8"?>
<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">
<S:Body>
<typ:getDocumentResponse
xmlns:typ="http://xx.xxxxxx.eu/sg/hrs/types">
<typ:document><typ:documentId>080166e48102103b</typ:documen
tId> … (stripped)
</typ:getDocumentResponse></S:Body></S:Envelope>
24/10/2014 Copyrigth (c) Marc Tritschler 18

19. Quizz Time
Guess number of HTTP requests
per web site …
24/10/2014 Copyrigth (c) Marc Tritschler 19

20. Example 1 : www.lesoir.be
HOW Many HTTP requests ?
24/10/2014 Copyrigth (c) Marc Tritschler 20

21. Example 2: mon-programmetv.be
HOW Many HTTP requests ?
24/10/2014 Copyrigth (c) Marc Tritschler 21

22. Example 3: www.google.be
HOW Many HTTP requests ?
24/10/2014 Copyrigth (c) Marc Tritschler 22

23. Number of HTTP requests per
single web site visited
1. 200 requests/responses for www.lesoir.be
OMG !!!
2. It's full of advertisements (visible) and
invisible personal tracking systems (cookies,
javascript, re-directs, …)
3. js is evill
Conclusion : YOU ARE NOT ANONYMOUS
24/10/2014 Copyrigth (c) Marc Tritschler 23

24. How your browser gets its proxy ?
• Web Proxy Autodiscovery Protocol
24/10/2014 Copyrigth (c) Marc Tritschler 24

25. HTTP Advanced
• Authentication
• HTTP Proxies
• HTTP Tunnelling
• HTTP Pipelining
• HTTPS
24/10/2014 Copyrigth (c) Marc Tritschler 25

26. HTTP Authentication
(RFCs 2616, 2617, 7235)
Basic
The client sends the user name and password as unencrypted base64 encoded text. It should only be used with HTTPS, as the
password can be easily captured and reused over HTTP.
Digest
The client sends a hashed form of the password to the server. Although, the password cannot be captured over HTTP, it may be
possible to replay requests using the hashed password.
NTLM (Windows)
This uses a secure challenge/response mechanism that prevents password capture or replay attacks over HTTP.
24/10/2014 Copyrigth (c) Marc Tritschler 26

27. HTTP Authentication
401 – Access Denied
GET /securefiles/ HTTP/1.1
HTTP/1.1 401 Access Denied
WWW-Authenticate: Basic realm="My Server"
Content-Length: 0
GET /securefiles/ HTTP/1.1
Host: www.httpwatch.com
Authorization: Basic aHR0cHdhdGNoOmY=
Client
(browser)
Server
User types
his/her
password
24/10/2014 Copyrigth (c) Marc Tritschler 27

28. HTTP Authentication
407 – Proxy Authentication Required
• Same as 401 excepted proxy MUST return a
Proxy-Authenticate header
• Browser asks user to type his/her password
24/10/2014 Copyrigth (c) Marc Tritschler 28

29. HTTP Proxy/Reverse Proxy
• Proxy : local net  internet
• Reverse Proxy: internet  local net
Direct Connection
HTTP
Client Client
HTTP
Proxyied Connection
HTTP
Client Proxy Server
24/10/2014 Copyrigth (c) Marc Tritschler 29

30. HTTP Tunnelling
HTTP
TCP
Client Proxy Server
CONNECT
Port
forwarding
24/10/2014 Copyrigth (c) Marc Tritschler 30

31. HTTP Pipelining
http://en.wikipedia.org/wiki/HTTP_pipelining
24/10/2014 Copyrigth (c) Marc Tritschler 31

32. HTTPS
• HTTP over SSL
• Secure Browsing ?
– HeartBleed
– SSL3.0 recently found weak
– TLS 1.0 min
– Root certificate
24/10/2014 Copyrigth (c) Marc Tritschler 32

33. 3. Java & The Internet Stack
?
24/10/2014 Copyrigth (c) Marc Tritschler 33

34. Java and Internet
• Java is (my favorite) language to work @
application layer, up to TCP/IP … (wait next
slide )
• Java has no access to protocols below IP
(needs call to native libs, not in the HTTP
scope)
• Don't underestimate the complexity of SSL
interactions, even in Java !!!
24/10/2014 Copyrigth (c) Marc Tritschler 34

35. Java and the Internet Stack
ONLY FOCUS My Application My App
ON YOUR
BUSINESS
F Web
D
SMTP/POP3
T
Socke
N
P
t
S
Web
Services
53
80/443
25, 110
JavaMail
javax.mail
Web
Browser
Socket API (java.net) or JSSE (javax.ssl)
TCP/UDP
HTTP
IPv4 and IPv6
ICMP, ARP, DHCP, …
Physical Layer
Available in Java SE
Open Source or future
Implemented in the OS.
Java has limited access
via API
Implemented in
OS or hardware.
No 'direct' access
JRE
Linux
24/10/2014 Copyrigth (c) Marc Tritschler 35

36. API vs Protocol
• API  vertical links (no bytes on the wire)
• Protocol horizontal links
24/10/2014 Copyrigth (c) Marc Tritschler 36

37. Socket API
(java.net)
• Most important (access
• Server Sockets
• Client sockets
• Base for YOUR protocol !
• Base for HTTP, SMTP, …
24/10/2014 Copyrigth (c) Marc Tritschler 37

38. Socket API - Main Classes
• Socket & ServerSocket
https://docs.oracle.com/javase/7/docs/api/java/net/Socket.html (Java 7)
https://docs.oracle.com/javase/8/docs/api/java/net/Socket.html (Java 8 :-))
• URL
• URLConnection
• HttpURLConnection
• …
• java.net package http://docs.oracle.com/javase/8/docs/api/java/net/package-summary.html
• Stack properties http://docs.oracle.com/javase/8/docs/technotes/guides/net/properties.html
24/10/2014 Copyrigth (c) Marc Tritschler 38

39. SMTP/POP3
(java mail)
https://javamail.java.net/nonav/docs/api/com/sun/ma
il/smtp/package-summary.html
24/10/2014 Copyrigth (c) Marc Tritschler 39

40. SSL/TLS
(java.net javax.ssl)
• Socket API(java.net)  JSSE (javax.ssl)
• Sockets
– (Client) Socket  SSLSocket
– ServerSocket  SSLServerSocket
• HttpUrlConnection  HttpsUrlConnection
24/10/2014 Copyrigth (c) Marc Tritschler 40

41. Others
• WebSocket
– http://www.websocket.org/
• Java Specifics
– RMI
– JMX
• Web Services
– SOAP  JAX-WS
– REST  JAX-RS
24/10/2014 Copyrigth (c) Marc Tritschler 41

42. Part 3: Code Time
WARNING
Several packages and many classes

challenge is to use the right classes
24/10/2014 Copyrigth (c) Marc Tritschler 42

43. Setup - Toolbox
• Developer
– Java JDK (of course)
– Editor (Eclipse, NetBeans, …)
• Client Side
– Putty
– Web Browser + DEV console !
(Chrome, IE, FireFox, …)
– soapUI (Web Services)
• Server Side
– Apache HTTP server (min)
– Apache Tomcat
(recommended)
– Full JEE (GlassFish, WildFly, …)
• Cloud
– Red Hat OpenShift
– …
https://github.com/tritschler/LLN_JUG/tree/master/2014_11_20
24/10/2014 Copyrigth (c) Marc Tritschler 43

44. Example 1 – Echo protocol
(ClientSocket & ServerSocket)
• No HTTP, directly over TCP
https://docs.oracle.com/javase/tutorial/displayCode.html?
code=https://docs.oracle.com/javase/tutorial/networking/sockets/examples/Echo
Server.java
DON'T DO THIS IN REAL LIFE
24/10/2014 Copyrigth (c) Marc Tritschler 44

45. Example 1 - Echo
Echo (Client)
JVM
Socket API (java.net)
TCP
IPv4
Physical Layer
Echo (Server)
JVM
Socket API (java.net)
TCP
IPv4
Physical Layer
Hello
logical flow
Hello IP
real data flow
24/10/2014 Copyrigth (c) Marc Tritschler 45

46. Example 2 – Basic Web Crawler
(URL, HttpUrlConnection)
•Example 1 : no proxy
•Example 2 : proxy + basic http authentication
DON'T DO THIS IN REAL LIFE
24/10/2014 Copyrigth (c) Marc Tritschler 46

47. Java HTTP Client App
My Application
…
Socket API
HTTP
(JVM)
TCP/UDP
IP
Physical Layer
ANY HTTP Server
(Apache, Nginex,
Tomcat, Jboss, Microsoft
IIS, …) implemented in
any programming
language (Java, PHP, C,
…)
ANY OS (Linux,
Windows, Mac OS, …)
24/10/2014 Copyrigth (c) Marc Tritschler 47

48. Example 3 – Servlet
No networking code on the Server Side
• Servlet = java spec for writing the HTTP server side
• No networking code ! (thanks to your AS)
• Web.xml + class extends HttpServlet 
1. Browser – Servlet
2. Browser – HttpTrace – Servlet
3. HttpUrlConnection (no proxy) – Servlet
4. HttpUrlConnection – HttpTrace – Servlet
DON'T DO THIS IN REAL LIFE
24/10/2014 Copyrigth (c) Marc Tritschler 48

49. Java HTTP Client App – Java Servlet
ANY HTTP Server
+ Servlet Container
Apache Tomcat
ANY OS (Linux,
Windows, Mac OS, …)
ANY HTTP Client
(Web Browser, …)
ANY OS (Linux,
Windows, Mac OS, …)
24/10/2014 Copyrigth (c) Marc Tritschler 49

50. Example 4 – HTTP proxy
• Start local Tomcat
• Start HttpTrace
• Start Browser and point to localhost
• Launch httpclient
24/10/2014 Copyrigth (c) Marc Tritschler 50

51. Resources
(on the web of course, over HTTP )
24/10/2014 Copyrigth (c) Marc Tritschler 51