SlideShare ist ein Scribd-Unternehmen logo
1 von 70
WebTalk
Implementing Web
Services with a dedicated
Java daemon
Geert Van Pamel – HP-Interex Belux
Mannheim, Germany 10 November 2008
Talking to
the web
Introduction
• A Java application behaving as a web server
• Java & the HTTP protocol is the base
• and IP TCP + UDP programming
• KISS – Keep it simple, but fully functional
• Virtual file system
– Not about Applets, Servlets or JSP
– Not about Jakarta or Tomcat
– No SOAP, no BPEL, no WSDL
Why this Project?
• Interoperability of networked clients & servers
• Using Open standards
• HTTP protocol
• Network socket programming (the network is the system)
• Interfacing with middleware (building blocks)
• Any client supporting the HTTP protocol
• Reliable and performance
• 24/7 daemon written in Java
Why Java?
• Using standard Java
• Extensible
• Can run on any platform – CORE (code once run everywhere)
• Easy testing
• Easy implementation (distribute byte code)
• Can easily interface with other Java classes
• Use with any backend supporting Java
• Stable, resilient & very good exception handling
• Well documented, tools available
• Modern language – many programmers available
Architecture
WebTalk
Client
WebTalk
Server
WebTalk
Client
Legacy
server
Legacy
server
Management
Station
Alert
Useful? Let me know!
• Home use
• Enterprise use
• You manage everything
• No dependencies with other packages
• It is free, no licenses
• Just invest your own few mandays
• Download the template!
The Basic Algorithm
• Initialise the program
• Verify that it runs on one of the registered servers
• Verify the command parameters, overruling default parameters
• Open the log file if requested
• Create the main server listening socket
• Perform the main server loop (endless)
– Listen to the network port and wait for the next client request
– Get the client identification
– Check access authorisation
– Get the client capabilities
– Get the client input parameters
– Parse input parameters
– Validate input parameters
– Perform a server transaction
– Convert the status and results to the client capabilities
– Log the request and the results in the error log
– Return status and results to the client program
– Close the client network connection
A daemon
never dies
Program Header
package webtalk;
import java.io.*;
import java.net.*;
import java.util.*;
public class WebTalk {
// Creates a new instance of WebTalk
public WebTalk() {
}
// main program
}
Program initialization (1)
private static int tcpPort = 2006; // P1 = listen port: 2006
private static String logDirName = "/tmp";
private static FileOutputStream logFile; // P3 = logging mode: log,sms
private static PrintWriter log = null;
public static void main(String[] args) {
String crlfStr = System.getProperty("line.separator");
String runModeDefault = "normal"; // P2 = run mode: normal
if (crlfStr.equals("rn")) logDirName = "c:temp";
if (args.length >= 1 && !args[0].equals("")) tcpPort = Integer.parseInt(args[0]); // P1: listen port
logFileName = "webtalk-" + tcpPort + ".log";
if (args.length < 3 || args[2].indexOf("nolog") < 0) try { // P3: Open the log file at startup
logFile = new FileOutputStream(new File(logDirName, logFileName), true);
log = new PrintWriter (logFile, true);
} catch (FileNotFoundException e) {
writeLog(e.toString()); // But we continue processing, since not fatal
}
if (args.length >= 2 && !args[1].equals("")) runModeDefault = args[1]; // P2: Set trace mode
if (!runModeDefault.equals("normal")) writeLog("Enable " + runModeDefault);
}
First
implemented in
2006
Program initialization (2)
// P3: Open the log file at startup
if (args.length >= 3 && args[2].indexOf("log") >= 0) try {
logFile = new FileOutputStream(new File(logDirName,
logFileName), true);
log = new PrintWriter (logFile, true);
} catch (FileNotFoundException e) {
writeLog(e.toString());
// But we continue processing, since not fatal
}
default
nolog
Blocking Unauthorized Server Implementation
try {
serverAddress = InetAddress.getLocalHost();
remoteHost = serverAddress.getHostAddress();
serverHost = remoteHost;
serverName = serverAddress.getCanonicalHostName();
} catch (UnknownHostException e) {
sendPanic(e.toString());
System.exit(8);
}
if (!allowRemoteHost (remoteHost)) {
sendPanic("Reject host"); // Unauthorized server (address)
System.exit(2);
}
Open main Server Socket
try {
server = new ServerSocket(tcpPort);
} catch (IOException e) {
sendPanic(e.toString());
System.exit(1);
}
private final static String versionString =
"WebTalk/20081013 (Java/HTTP)";
sendPanic("Starting " + versionString);
Server up
and running
Waiting the next Client
while (true) {
try {
client = server.accept();
} catch (IOException e) {
sendPanic(e.toString());
System.exit(4);
}
remoteAddress = client.getRemoteSocketAddress().toString();
remoteHost = remoteAddress.substring(1, remoteAddress.indexOf(':', 1));
try{
in = new BufferedReader(
new InputStreamReader(client.getInputStream(), webCharset));
out = new PrintStream(client.getOutputStream(), false, webCharset);
} catch (IOException e) {
sendPanic(e.toString());
System.exit(5);
}
// main processing logic here
}
A daemon
never dies
Client Access Control
private final static int MAXIP = 50;
private static int clientCount[] = new int
[1+MAXIP];
private static String allowAddress[] = new
String [1+MAXIP];
…
allowAddress[14] =
nslookup("client1.mshome.net");
allowAddress[15] = "192.168.210.34";
…
if (!allowRemoteHost (remoteHost)) {
sendPanic("Reject host: " +
remoteHost);
}
static boolean allowRemoteHost (String
remoteAddress) {
boolean allowIP;
allowIP = false;
for (int i = 1; i < allowAddress.length; i++)
{
if
(remoteAddress.equals(allowAddress[i])) {
allowIP = true;
clientCount[i]++;
break;
}
}
return allowIP;
}
Getting Browser Input
String webTalkIn;
webTalkIn = "n-- Start request";
// Get the client request
while (webTalkIn != null && !webTalkIn.equals("")) {
// Here comes the parsing of the browser request
// Get the next HTTP header until empty line encountered
webTalkIn = in.readLine();
}
TCP does
not have
EOF
Parsing the Client URL
String webRequest[]; // GET /urlPath/...?webParam=value&... HTTP/1.1
String mainPath[];
String urlPath[];
String webParam[];
String paramPair[];
writeLog(remoteHost + " " + gmTime + " " + webTalkIn);
webRequest = webTalkIn.split(" ");
mainPath = webRequest[1].split("[?]", 2);
urlPath = mainPath[0].split("/");
webParam = mainPath[1].split("&");
/* Here we should map the key/value pairs into the application logic */
for (int c = 0; c < webParam.length; c++) {
paramPair = webParam[c].split("=", 2);
paramPair[0] = paramPair[0].toUpperCase();
//System.out.println(paramPair[0]);
//System.out.println(paramPair[1]);
}
Parsing HTTP Headers
• User-Agent
• Via
• Accept-Language
• HTTP/1.x
• Authentication
• Connection: keep-alive
Identifying the Browser
/* set default userAgent */
if (userAgent == unkownUserAgent
&& webRequest.length >= 2
&& webRequest[0].equals("User-Agent:")) {
if (runMode.equals("trace")) writeLog(webTalkIn);
if (webRequest[1].startsWith("Mozilla")) {
// Firefox, Internet Explorer
userAgent = mozillaUserAgent;
} else {
// Lynx, Opera
userAgent = otherUserAgent;
}
}
The main Server Logic
Some examples we have implemented with success in production are:
• Validating Web forms by calling WebTalk via TCP socket
programming
• .Net calling the WebTalk server to validate customer orders
• Validating customer orders via a DTMF and a text to speech
system
• Oracle PL/SQL calling the WebTalk server to validate customer
orders via the UTL_HTTP and UTL_TCP package [6]
• Storing database transactions via HTTP protocol (avoid locking)
• Apache Embedded Perl calling WebTalk for order validation
Returning Server Results
static void writeBody (String bodyText) {
int bodyLength;
String headerStr;
if (userAgent > unknownUserAgent) {
bodyLength = 2 + bodyText.length();
if (userAgent >= otherUserAgent) bodyLength += 6; // <PRE> tag
headerStr = "HTTP/1.1 200 OKnDate: " + httpTime
+ "nServer: " + versionString
+ "nExpires: -1nCache-Control: no-cache, no-store,
max-age=0, must-revalidatenPragma: no-cache"
+ "nContent-Length: " + bodyLength
+ "nConnection: closenContent-Type: text/htmln“;
if (userAgent >= otherUserAgent) headerStr += "n<pre>";
if (runMode.equals("debug")) writeLog(headerStr);
headerStr = headerStr.replaceAll("n","rn") + "r";
out.println(headerStr);
}
if (runMode.equals("debug")) writeLog(bodyText);
if (userAgent > unknownUserAgent)
bodyText = bodyText.replaceAll("n","rn") + "r"; // Avoid protocol error
out.println(bodyText);
}
HTTP
needs rn
GMT Time in HTTP Format
01234567890123456789012345678
Fri Nov 03 13:51:56 GMT 2006 standard GMT format (gmTime)
Fri, 03 Nov 2006 14:51:56 GMT internet GMT format (httpTime)
private static String gmTime;
private static String httpTime;
Date now;
TimeZone tz = TimeZone.getTimeZone("GMT");
tz.setDefault(tz);
now = new java.util.Date();
gmTime = now.toString();
httpTime = gmTime.substring(0,3) + ','
+ gmTime.substring(7,10)
+ gmTime.substring(3,7)
+ gmTime.substring(23,28)
+ gmTime.substring(10,23);
HTTP =
WWW thus
GMT time
Disabling Client Browser Cache
Our application is data driven
We must ensure that browsers, or proxies never
cache requests.
Therefore we generate the following headers:
Expires: -1
Cache-Control: no-cache, no-store, max-age=0, must-
revalidate
Pragma: no-cache
Close the Door
try {
in.close();
out.close();
client.close();
} catch (IOException e) {
sendPanic(e.toString());
System.exit(6);
}
• One client at a time
• Avoid locking using
single process database
inserts
• Let the TCP stack buffer
incoming requests
• Latent DoS problem
(if connection not properly closed)
Client Examples in practice
• Normal clients can be browsers like Firefox [8], Opera [8], or Internet
Explorer:
http://servername:2006/hub/2186/ivr?p1=1234&p2=5678
• With telnet:
telnet servername 2006
GET /hub/2186/ivr?p1=1234&p2=5678
(enter)
• With Lynx [8]:
lynx "http://servername:2006/hub/2186/ivr?p1=1234&p2=5678"
• With netcat:
echo "GET /hub/2186/ivr?p1=1234&p2=5678n" |nc servername 2006
Other Clients
• Perl
• PHP
• Unix Socket programs
• .Net clients
• Database systems like Oracle [6]
• Telecom servers like IVR
Any program
capable of
talking HTTP
Example: Perl
use IO::Socket::INET;
$Server = "servername:2006";
my $sock = IO::Socket::INET->new(PeerAddr => $Server, Proto => 'tcp');
$webStat = "";
if ($sock) {
$sock->write("GET /statusnn");
$sock->recv($webStat, 2000);
close($sock);
}
print "$webStatn";
Example: Oracle stored Procedure
CREATE FUNCTION web_request (
request in string)
return string is
c utl_tcp.connection; len PLS_INTEGER;
result string(2000);
BEGIN
c := utl_tcp.open_connection ('servername',
2006);
len := utl_tcp.write_line(c, 'GET ' || request);
len := utl_tcp.write_line(c);
result := utl_tcp.get_line(c, TRUE);
utl_tcp.close_connection(c);
return result;
END;
select
web_request(‘/status’)
from dual;
Caveat Firefox and Opera
[server loop]
if (urlPath.length >= 2 &&
urlPath[1].equals("favicon.ico")) {
runMode = "ignore";
}
[generating output]
if (runMode.equals("ignore")) {
writeBody(clientResult);
}
Backend Systems
• Oracle with permanent connection
• MySQL database gateway
• Middleware
• Other SOA systems
Server Management – Starting the Server
• On Unix systems:
su - apache -c "java -jar /html/java/bin/webtalk.jar 2006 normal log,sms
> /dev/null 2> /dev/null &"
• On Windows platforms you can start the server from a scheduled tasks (at system startup)
java -jar C:htmljavabinwebtalk.jar 2006 trace > nul 2> nul
• General format of startup command:
java -jar webtalk.jar [port] [runmode] [log,sms] [> nul] [2> nul]
Parameters:
port: TCP port number (default 2006; you could use 2005 for testing)
runmode: debug | test | trace | quiet | disable | enable | normal (default)
log: "log" to trigger the logging to disk, with nolog logging is disabled
sms: "nosms" to disable sending SMS messages
Stopping the Server
[Unix]
• ps –ef |grep webtalk
• kill $pid
[Windows]
• tasklist /fi “imagename eq java.exe”
• taskkill /pid ProcessID
Killing the
daemon
Temporarily Disable – Panic Button
• Uncouple the system to avoid downstream catastrophes
http://servername:2006/disable
http://servername:2006/enable
Internally the following code gets executed:
if (urlPath.length >= 2
&& (urlPath[1].equals("disable")
|| urlPath[1].equals("enable") ) ) {
runModeDefault = urlPath[1];
clientResult = "Server: " + runModeDefault;
sendPanic(clientResult);
}
Resume normal functioning
http://servername:2006/normal
http://servername:2006/[no]trace
http://servername:2006/[no]debug
Internally the following code gets executed:
if (urlPath.length >= 2 && urlPath[1].equals("normal")) {
runModeDefault = urlPath[1];
clientResult = "Enable " + urlPath[1];
}
Recycling the Log File
daynum=$(date +%u)
for port in 2005 2006 ;do
lynx -dump http://servername:$port/nolog > /dev/null
mv /tmp/webtalk-$port.log 
/scratch/webtalk/log/webtalk-$port-$daynum.log
lynx -dump http://servername:$port/log > /dev/null
done
Avoid
disk full
errors
Open and close the Log File
if (urlPath.length >= 2 && urlPath[1].equals("log") && log == null) {
clientResult = "Append logfile";
logFile = new FileOutputStream(new File(logDirName,
logFileName), true);
log = new PrintWriter (logFile, true);
}
else if (urlPath.length >= 2 && urlPath[1].equals("nolog")
&& log != null) {
clientResult = "Closing logfile";
logFile.close();
log = null;
}
Local Server Monitoring
To verify that the process runs:
[Unix]
ps -ef |grep "java -jar"
apache 114167 1 0.0 14:01:20 pts/0 0:01.63 java -jar /html/java/bin/webtalk.jar
2006
[Windows]
tasklist /fi "imagename eq java.exe" /v
Image Name PID Session Name Session# Mem Usage Status
java.exe 716 Console 0 12.676 K Unknown
Network monitoring
netstat [-a] [-n] |grep webtalk |wc
Remote Server Monitoring
for tn in 2005 2006 ;do
telnet servername $tn <<EOF 2>&1 >/dev/null |
grep -v 'Connection closed by foreign host.' |
/usr/local/bin/sendpanic "" "servername:$tn"
GET /status HTTP/1.0
EOF
done
Status of the Server
http://servername:2006/status
[server loop]
if (urlPath.length >= 2 && urlPath[1].equals("status")) {
runMode = "status";}
[generating output]
if (runMode.equals("status")) { runMode = runModeDefault;
clientResult = "Server: " + serverName + "(" + serverHost + ")" +
"nPort: " + tcpPort + "nClient: " + remoteHost + "nStart
Time: " + startTime + "nReset Time: " + resetTime + "nTime:
" + gmTime + "nVersion: " + versionString + "nMode: " +
runMode + "nLanguage: " + acceptLanguage + "nUser Agent:
" + userAgent + "nTop Client: " + allowAddress[1] + "(" +
clientCount[1] + ")" ; writeBody(clientResult);}
Usage Statistics
http://servername:2006/counters
Usage counters per IP address since Mon Jun 04 20:56:39 GMT 2007:
192.168.86.109 2110 client1.mshome.net
192.168.219.113 1198 client2.mshome.net
192.168.112.50 916 client3.mshome.net
• This is handled by the following code:
if (urlPath.length >= 2 && urlPath[1].equals("counters")) {
clientResult = "Usage counters per client since " + resetTime + ":n";
for (int i = 1; i < allowAddress.length; i++)
if (!allowAddress[i].equals(""))
clientResult += "n" + allowAddress[i]
+ "t" + clientCount[i]
+ "t" + rnslookup (allowAddress[i]);
Resetting Counters
http://servername:2006/reset
if (urlPath.length >= 2 && urlPath[1].equals("reset")) {
resetTime = gmTime;
clientResult = "Reset counters";
for (int i = 1; i < allowAddress.length; i++)
clientCount[i] = 0;
}
Building the Application
• java -version
java version "1.4.2"
Java(TM) 2 Runtime Environment, Standard Edition
Fast VM (build 1.4.2-6, native threads, mixed mode, 01/09/2007-23:17)
• Store the source code in a subdirectory, and
compile with javac.
javac webtalk/WebTalk.java
mkdir webtalk/RCS
ci –l webtalk/WebTalk.java
Run Interactively
• [Unix]
export CLASSPATH=lib1.jar:lib2.jar:.
• [Windows]
set CLASSPATH=lib1.jar;lib2.jar;.
• To start the program interactively, immediately after
compiling:
java webtalk/WebTalk 2005 debug nolog,nosms
Build the JAR File
• The manifest file can contain a list of all other required jar files and
the name of the main module to run. If your application is
complex, multiple jar files will be needed.
jar -cv0mf webtalk.jar webtalk/manifest.mf webtalk/*.class
vi webtalk/manifest.mf
Manifest-Version: 1.0
Created-By: 1.3.0_05 (Sun Microsystems Inc.)
Class-Path: lib1.jar lib2.jar
Main-Class: webtalk/WebTalk
Simulate a Server (1)
vi netcat.txt
HTTP/1.1 200 OK
Date: Tue, 29 May 2007 11:47:08 GMT
Server: WebTalk/20070528 (Java/HTTP)
Last-Modified: Tue, 29 May 2007 11:47:08 GMT
Expires: -1
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Content-Length: 18
Connection: close
Content-Type: text/html
P1:0003|P2:42108
Coding,
compiling,
testing takes a
lot of time
Simulate a Server (2)
while true ;do nc –lp 2005 < netcat.txt ;done
lynx –dump http://localhost:2005/test
lynx –source http://localhost:2005/test
nc –p 2005 localhost
telnet localhost 2005
Error Handling
Daemon process!Alarm notification
Allow for easy supportError logging
The program stopsFatal errors
Logged, status to clientUnexpected errors
Returned to client
application, not logged
Expected errors
How to handleError Type
General Exception Handling
try {
// put main program logic here
} catch (Exception e) {
e.printStackTrace ();
sendPanic(e.toString()); // could also use e.getMessage()
System.exit(7);
} finally {
sendPanic("Server closing down");
}
Enable Debugging
• To enable & disable debugging
http://servername:2006/[no]debug
if (urlPath.length >= 2 && urlPath[1].equals("debug")) {
runModeDefault = urlPath[1];
clientResult = "Enable debug";
}
Enable Tracing
• To enable & disable tracing:
http://servername:2006/[no]trace
if (urlPath.length >= 2 && urlPath[1].equals("trace")) {
runModeDefault = urlPath[1];
clientResult = "Enable trace";
}
Enable Quiet Mode
• To set quiet mode
(when you would have too many transactions)
http://servername:2006/quiet
if (urlPath.length >= 2 && urlPath[1].equals("quiet")) {
runModeDefault = urlPath[1];
clientResult = "Enable quiet mode";
}
Error Logging
static void sendPanic (String panicString) {
writeLog(panicString);
if (smsRelayAddress != null)
sendSMS (serverName + " " + serverHost + ":"
+ tcpPort + " " + panicString
+ " from " + remoteHost + " " + gmTime);
}
static void writeLog (String logText) {
system.err.println(logText);
if (log != null) {
log.println(logText);
}
}
Sending SMS Alarm
static void sendSMS (String smsString) { byte sms[];
DatagramPacket smsPacket;
DatagramSocket smsSocket;
try {
sms = smsString.getBytes();
smsPacket = new DatagramPacket (sms,
sms.length, smsRelayAddressDefault, 535);
smsSocket = new DatagramSocket();
smsSocket.send (smsPacket);
smsSocket.close();
} catch (Exception e) { // ignore
}
}
UDP
UDP to TCP Relay Server (1)
import java.io.*;
import java.net.*;
public class RelaySmsUdp {
public static void main (String args[]) {
DatagramSocket smsSocket;
DatagramPacket smsPacket;
InetAddress clientAddress;
String clientHost;
String panicString;
Socket outSocket;
PrintWriter out;
byte sms[] = new byte[165]; // byte array
String panicDest;
String smsHost;
panicDest = "0123456789"; // your Mobile alarm number
if (args.length >= 1 && !args[0].equals("")) panicDest = args[0];
smsHost = "sms-tcp-gateway";
if (args.length >= 2 && !args[1].equals("")) smsHost = args[1];
try {
smsSocket = new DatagramSocket(535); // listen to port
while (true) try {
smsPacket = new DatagramPacket (sms, sms.length);
smsSocket.receive (smsPacket);
clientAddress = smsPacket.getAddress();
clientHost = clientAddress.getHostAddress();
panicString = new String (smsPacket.getData(), 0, smsPacket.getLength());
if (!panicString.startsWith("0")) panicString = panicDest + " " + panicString;
// prefix with Mobile number if needed
if (clientHost.startsWith("192.168.")) { // Filter non-registered IP
addresses
outSocket = new Socket (smsHost, 535);
out = new PrintWriter(outSocket.getOutputStream());
if (panicString.length() > 165)
panicString = panicString.substring(0, 165);
out.println (panicString);
out.close();
outSocket.close();
}
} catch (UnknownHostException e) {
writeLog(e.toString());
} catch (IOException e) {
writeLog(e.toString());
}
} catch (SocketException e) {
writeLog(e.toString());
}
Relay to TCP
for message
transmission
Disable SMS at Startup
• Can be very handy while developing
private static InetAddress smsRelayAddressDefault;
private static InetAddress smsRelayAddress;
try {
smsRelayAddressDefault = InetAddress.getByName("sms-udp-gateway");
} catch (Exception e) {
smsRelayAddressDefault = null;
}
smsRelayAddress = smsRelayAddressDefault;
if (args.length >= 3 && args[2].indexOf("nosms") >= 0)
smsRelayAddress = null;
Disable or Enable SMS at runtime
lynx -dump http://servername:2006/[no]sms
• The following code gets executed:
if (urlPath.length >= 2 && urlPath[1].equals("sms")) {
smsRelayAddress = smsRelayAddressDefault;
clientResult = "Enable sms notification";
sendPanic(clientResult);
}
else if (urlPath.length >= 2 && urlPath[1].equals("nosms")) {
smsRelayAddress = null;
clientResult = "Disable sms notification";
sendPanic(clientResult);
}
If you are
drown with
alarms
Supported Server Platforms
Java virtually runs on all platforms:
– You only have to distribute the byte code in a Jar file!
• Any Intel Linux
• Any Intel Windows XP or Vista desktop
• Alpha Tru64 UNIX V5.1B-3 with Java 1.4.2-7, see [11]
• Itanium
• Windows server 2003 with Java 1.5.0_07
Implemented Standards
• DNS
• HTTP, see [2]
• Internet date & time
format
• Java, see [3] [4] [5]
• NFS during development
• NTP
• TCP/IP
• UDP for error relay
• Web Cache control
• ZIP for jar file
manipulation
What I have learned
• Think in objects
• Keep it simple (divide and conquer)
• Client /server programming is fun
... debugging and fault finding even more
• It boosts the team spirit
• It stimulates your own brains and
creativity
• Java (network) programming is not
really exotic, nor complicated
• Web applications must disable proxy
and browser cache
• TCP sockets do not have an end-of-
file...
– a server socket cannot know when the
client sends the last packet
– this is why a final empty line has been
built into the HTTP and SMTP protocol
• Java initialises all variables to 0, "", or
null
• Java always start with element 0
• In Java you do not use pointers and
linked lists to create dynamic data
structures
• Sparse matrices are natural in Java
• Error handling and planning for the
unforeseen is the most important –
try / catch should be a habit
• The HTTP protocol terminates every
line by CRLF. Originally I terminated
HTTP server output only by LF using
println. None of the Unix systems did
complain. .Net servers replied with an
error “The server committed a
protocol violation”.
• Keep the history of your changes. Use
e.g. RCS, Revision Control System.
Design Principles
• Separate code and data
• Flexible data structures
• Fully table & data driven
• Keep the code simple
• Code only once (no duplicate
instructions)
• Do normal things straight
• Use industry standards
• Use as much as possible standard
system functions
• Do care about and be
conservative with system
resources
• Give the power to the users
• Be flexible, and extensible
• Be future proof
• Be generous on what you accept,
be strict on what you return
• Do the unforeseen via exception
handling; be robust
• Be fool proof
• Avoid catastrophes at all times
• Use what you know
• Have maximum security and
access control
• Allow for logging, tracing, and
debugging
• Care about supportability
• Be platform independent
• Write a complete documentation
Future Extensions and Migration
• Advanced Java debugging
• Build and use appliances
• Application setup via config file
instead of hardcoded
• DNS caching
• IPv6, Internet protocol V6
• Java runtime optimisation
(tuning, memory usage)
• kill –HUP, to change debug level,
or restart the server
• Legacy applications
• Load balancing
• Multithreaded server
• UDP monitoring
• NTLM, Active Directory user
authentication
• POST or other HTTP protocols
• SIF, Service Invocation
Framework = SOA services for
Java
• Timeout, drop a bad behaving
client (currently we suffer from a
DoS problem)
• SNMP monitoring
• SOA, Service Oriented
Architecture
• System parameters
• XML, Input / Output in XML
format
References
[1] Full program source and additional material: http://www.hp-interex.be/pub/project/webtalk
[2] The HTTP protocol and related standards: http://www.w3.org
[3] Documentation and kits for Java: http://www.sun.com
[4] Java technical documentation: http://java.sun.com/j2se/1.4.2/docs
[5] Java search page: http://java.sun.com/j2se/1.4.2/search.html
[6] Oracle UTL_HTTP:
http://www.oracle.com/technology/sample_code/tech/pl_sql/htdocs/x/Utl_Http_Package_Enhan
cements/Cr_Using_Utl_Http.htm
[7] Iptables: http://www.netfilter.org
[8] Browsers:
The Lynx browser: http://lynx.browser.org
Firefox: http://www.firefox.com
Opera: http://www.opera.com
[9] Perl libraries: http://search.cpan.org
[10] NetBeans: http://www.netbeans.org
[11] HP Java implementations: http://www.hp.com/java
[12] Bruce Eckel, Thinking in Java, The definitive introduction to object-oriented programming in
the language of the World Wide Web, Fourth Edition, ISBN 0-13-187248-6, 2006, Prentice Hall,
Pearson Education, Code & Supplements at http://www.mindview.net
Thanks
• My colleague Danny De Thaye to encourage me in my
endeavors
• Connect Management for accepting my lecture
• To you, the audience
geert.van.pamel@belgacom.net
http://www.hp-interex.be/wiki/index.php/User:Geertivp
Questions
Abstract
• The speaker presents a Java application that behaves like a dedicated web server
and provides Web services. It runs as a standalone network daemon and only calls
system and network services that are directly executed by the JVM. No other
software is necessary. The URL path referenced by the clients is not referring to
files. Instead it is calling other middleware services and serves as a Web service
gateway. The application only requires TCP/IP and Java library programming.
Because it behaves as a web server, the browser can be any client capable of
interfacing with the standard HTTP protocol.
• Learn about Java network programming, specifically about security, reliability,
supportability, monitoring, browser cache disabling, kit building, starting the Java
server process, troubleshooting and monitoring, implementing on different
operating systems and platforms, and integrating with other services. Debugging
server processes is difficult because there is no direct user interface. Every
interaction goes via network sockets, or process signals. Remotely control the
server by using the HTTP protocol, enable and disable logging, tracing, and
debugging info.
About the Author
• The author works at Belgacom since 1997 in the Customer division as an
ICT Project Manager.
• He is Chairman of HP-Interex Belux since 2008.
• He is Board member of HP-Interex Belux since 2002. Member of DECUS
since 1984, and CUO since 1998.
• In his previous life he worked as Project manager at Digital Equipment
Corporation (DEC) [company merged with Compaq in 1998 and with HP,
Hewlett Packard in 2002].
• His early activities in the world of IT were biometrical statistics, teaching
computer courses, real time car manufacturing automation, and medical
image processing using nuclear isotopes computer tomography.
• He is an engineer in Agriculture and Applied Biological Sciences, and
postgraduate in Computer Science. He is eager to implement technology
as a business advantage.
Useful Utilities
• Any web browser [8] To test the server (Firefox, Internet Explorer, Opera, lynx)
• crontab To reset the daily log files, to monitor
• Google For help with Java, use e.g. Google: Java ServerSocket
• host To lookup the IP address of a machine
• init To start the Web Service
• jar Java archive builder
• java Java runtime interpreter
• javac Java Compiler
• nc, netcat To test, debug, or simulate
• NetBeans [10] Java IDE Integrated Development Environment Unix & Windows
• NFS The server is developed, build & tested cross-platform
• nmblookup, nbtstat To lookup the name of a Windows client, if not registered in DNS
• NTP, ntpq To synchronise system time
• Perl [9] Practical Extraction and Reporting Language
• ps, grep, kill To monitor or change run status
• telnet To monitor and test special requests (does not work in batch)
• vi Terminal based editor - on linux, you might have colours
• zip, Winzip To verify (or update) the contents of jar files.
Technical Terms
• API Application Programming Interface
• CRLF Carriage Return and Line Feed
• GMT Greenwich Mean Time, almost the same as UTS (Universal Time Coordinated)
• HW Hardware
• IVR Interactive Voice Response
• Jar Java archive
• JDK Java Development Kit
• JRE Java Runtime Environment
• JSP Java Server Pages
• JVM Java Virtual Machine
• OS Operating System
• Platform A combination of hardware and software
• SMS Mobile text message (do not confuse with Microsoft System Management Server)
• SW Software (operating system and layered products)
• Unix Any type of Unix, or Linux OS, in general
• UNIX™ UNIX Trademark
nslookup
static String nslookup (String hostName) {
String ipAddress;
try {
ipAddress =
InetAddress.getByName(hostName).getHostAddress();
} catch (UnknownHostException e) {
ipAddress = "";
}
return ipAddress;
}
rnslookup
static String rnslookup (String ipAddress) {
String hostName;
try {
hostName =
InetAddress.getByName(ipAddress).getHostName();
} catch (UnknownHostException e) {
hostName = "";
}
return hostName;
}

Weitere ähnliche Inhalte

Was ist angesagt?

Find the bottleneck of your system
Find the bottleneck of your systemFind the bottleneck of your system
Find the bottleneck of your systemJian-Hong Pan
 
Fluentd - CNCF Paris
Fluentd - CNCF ParisFluentd - CNCF Paris
Fluentd - CNCF ParisHorgix
 
From nothing to Prometheus : one year after
From nothing to Prometheus : one year afterFrom nothing to Prometheus : one year after
From nothing to Prometheus : one year afterAntoine Leroyer
 
Massively Scaled High Performance Web Services with PHP
Massively Scaled High Performance Web Services with PHPMassively Scaled High Performance Web Services with PHP
Massively Scaled High Performance Web Services with PHPDemin Yin
 
Using Grails to power your electric car
Using Grails to power your electric carUsing Grails to power your electric car
Using Grails to power your electric carMarco Pas
 
Iss letcure 7_8
Iss letcure 7_8Iss letcure 7_8
Iss letcure 7_8Ali Habeeb
 
NginX - good practices, tips and advanced techniques
NginX - good practices, tips and advanced techniquesNginX - good practices, tips and advanced techniques
NginX - good practices, tips and advanced techniquesClaudio Borges
 
JSON-RPC Proxy Generation with PHP 5
JSON-RPC Proxy Generation with PHP 5JSON-RPC Proxy Generation with PHP 5
JSON-RPC Proxy Generation with PHP 5Stephan Schmidt
 
Fluentd v1.0 in a nutshell
Fluentd v1.0 in a nutshellFluentd v1.0 in a nutshell
Fluentd v1.0 in a nutshellN Masahiro
 
Bulding a reactive game engine with Spring 5 & Couchbase
Bulding a reactive game engine with Spring 5 & CouchbaseBulding a reactive game engine with Spring 5 & Couchbase
Bulding a reactive game engine with Spring 5 & CouchbaseAlex Derkach
 
Customising Your Own Web Framework In Go
Customising Your Own Web Framework In GoCustomising Your Own Web Framework In Go
Customising Your Own Web Framework In GoJonathan Gomez
 
Fluentd v0.12 master guide
Fluentd v0.12 master guideFluentd v0.12 master guide
Fluentd v0.12 master guideN Masahiro
 
GOTO 2013: Why Zalando trusts in PostgreSQL
GOTO 2013: Why Zalando trusts in PostgreSQLGOTO 2013: Why Zalando trusts in PostgreSQL
GOTO 2013: Why Zalando trusts in PostgreSQLHenning Jacobs
 
Fluentd unified logging layer
Fluentd   unified logging layerFluentd   unified logging layer
Fluentd unified logging layerKiyoto Tamura
 

Was ist angesagt? (20)

Find the bottleneck of your system
Find the bottleneck of your systemFind the bottleneck of your system
Find the bottleneck of your system
 
Fluentd meetup
Fluentd meetupFluentd meetup
Fluentd meetup
 
Kommons
KommonsKommons
Kommons
 
Fluentd - CNCF Paris
Fluentd - CNCF ParisFluentd - CNCF Paris
Fluentd - CNCF Paris
 
From nothing to Prometheus : one year after
From nothing to Prometheus : one year afterFrom nothing to Prometheus : one year after
From nothing to Prometheus : one year after
 
Apache Beam de A à Z
 Apache Beam de A à Z Apache Beam de A à Z
Apache Beam de A à Z
 
gRPC in Go
gRPC in GogRPC in Go
gRPC in Go
 
Massively Scaled High Performance Web Services with PHP
Massively Scaled High Performance Web Services with PHPMassively Scaled High Performance Web Services with PHP
Massively Scaled High Performance Web Services with PHP
 
Using Grails to power your electric car
Using Grails to power your electric carUsing Grails to power your electric car
Using Grails to power your electric car
 
Iss letcure 7_8
Iss letcure 7_8Iss letcure 7_8
Iss letcure 7_8
 
NginX - good practices, tips and advanced techniques
NginX - good practices, tips and advanced techniquesNginX - good practices, tips and advanced techniques
NginX - good practices, tips and advanced techniques
 
JSON-RPC Proxy Generation with PHP 5
JSON-RPC Proxy Generation with PHP 5JSON-RPC Proxy Generation with PHP 5
JSON-RPC Proxy Generation with PHP 5
 
Fluentd v1.0 in a nutshell
Fluentd v1.0 in a nutshellFluentd v1.0 in a nutshell
Fluentd v1.0 in a nutshell
 
Bulding a reactive game engine with Spring 5 & Couchbase
Bulding a reactive game engine with Spring 5 & CouchbaseBulding a reactive game engine with Spring 5 & Couchbase
Bulding a reactive game engine with Spring 5 & Couchbase
 
Fluentd meetup #2
Fluentd meetup #2Fluentd meetup #2
Fluentd meetup #2
 
Customising Your Own Web Framework In Go
Customising Your Own Web Framework In GoCustomising Your Own Web Framework In Go
Customising Your Own Web Framework In Go
 
DevOps Odessa #TechTalks 21.01.2020
DevOps Odessa #TechTalks 21.01.2020DevOps Odessa #TechTalks 21.01.2020
DevOps Odessa #TechTalks 21.01.2020
 
Fluentd v0.12 master guide
Fluentd v0.12 master guideFluentd v0.12 master guide
Fluentd v0.12 master guide
 
GOTO 2013: Why Zalando trusts in PostgreSQL
GOTO 2013: Why Zalando trusts in PostgreSQLGOTO 2013: Why Zalando trusts in PostgreSQL
GOTO 2013: Why Zalando trusts in PostgreSQL
 
Fluentd unified logging layer
Fluentd   unified logging layerFluentd   unified logging layer
Fluentd unified logging layer
 

Ähnlich wie WebTalk - Implementing Web Services with a dedicated Java daemon

Networking and Data Access with Eqela
Networking and Data Access with EqelaNetworking and Data Access with Eqela
Networking and Data Access with Eqelajobandesther
 
Fast SOA with Apache Synapse
Fast SOA with Apache SynapseFast SOA with Apache Synapse
Fast SOA with Apache SynapsePaul Fremantle
 
Rapid java backend and api development for mobile devices
Rapid java backend and api development for mobile devicesRapid java backend and api development for mobile devices
Rapid java backend and api development for mobile devicesciklum_ods
 
.NET @ apache.org
 .NET @ apache.org .NET @ apache.org
.NET @ apache.orgTed Husted
 
Introduction to Node.js
Introduction to Node.jsIntroduction to Node.js
Introduction to Node.jsRichard Lee
 
16network Programming Servers
16network Programming Servers16network Programming Servers
16network Programming ServersAdil Jafri
 
Into The Box 2018 Going live with commandbox and docker
Into The Box 2018 Going live with commandbox and dockerInto The Box 2018 Going live with commandbox and docker
Into The Box 2018 Going live with commandbox and dockerOrtus Solutions, Corp
 
Going live with BommandBox and docker Into The Box 2018
Going live with BommandBox and docker Into The Box 2018Going live with BommandBox and docker Into The Box 2018
Going live with BommandBox and docker Into The Box 2018Ortus Solutions, Corp
 
Socket programming, and openresty
Socket programming, and openrestySocket programming, and openresty
Socket programming, and openrestyTavish Naruka
 
Networking & Socket Programming In Java
Networking & Socket Programming In JavaNetworking & Socket Programming In Java
Networking & Socket Programming In JavaAnkur Agrawal
 
IT Operations for Web Developers
IT Operations for Web DevelopersIT Operations for Web Developers
IT Operations for Web DevelopersMahmoud Said
 
swift-nio のアーキテクチャーと RxHttpClient
swift-nio のアーキテクチャーと RxHttpClientswift-nio のアーキテクチャーと RxHttpClient
swift-nio のアーキテクチャーと RxHttpClientShinya Mochida
 
Node.js System: The Approach
Node.js System: The ApproachNode.js System: The Approach
Node.js System: The ApproachHaci Murat Yaman
 
The Recording HTTP Proxy: Not Yet Another Messiah - Bulgaria PHP 2019
The Recording HTTP Proxy: Not Yet Another Messiah - Bulgaria PHP 2019The Recording HTTP Proxy: Not Yet Another Messiah - Bulgaria PHP 2019
The Recording HTTP Proxy: Not Yet Another Messiah - Bulgaria PHP 2019Viktor Todorov
 

Ähnlich wie WebTalk - Implementing Web Services with a dedicated Java daemon (20)

Networking and Data Access with Eqela
Networking and Data Access with EqelaNetworking and Data Access with Eqela
Networking and Data Access with Eqela
 
About Node.js
About Node.jsAbout Node.js
About Node.js
 
Fast SOA with Apache Synapse
Fast SOA with Apache SynapseFast SOA with Apache Synapse
Fast SOA with Apache Synapse
 
Rapid java backend and api development for mobile devices
Rapid java backend and api development for mobile devicesRapid java backend and api development for mobile devices
Rapid java backend and api development for mobile devices
 
Play Framework
Play FrameworkPlay Framework
Play Framework
 
.NET @ apache.org
 .NET @ apache.org .NET @ apache.org
.NET @ apache.org
 
Introduction to Node.js
Introduction to Node.jsIntroduction to Node.js
Introduction to Node.js
 
REST made simple with Java
REST made simple with JavaREST made simple with Java
REST made simple with Java
 
16network Programming Servers
16network Programming Servers16network Programming Servers
16network Programming Servers
 
Into The Box 2018 Going live with commandbox and docker
Into The Box 2018 Going live with commandbox and dockerInto The Box 2018 Going live with commandbox and docker
Into The Box 2018 Going live with commandbox and docker
 
Going live with BommandBox and docker Into The Box 2018
Going live with BommandBox and docker Into The Box 2018Going live with BommandBox and docker Into The Box 2018
Going live with BommandBox and docker Into The Box 2018
 
Ipc
IpcIpc
Ipc
 
Socket programming, and openresty
Socket programming, and openrestySocket programming, and openresty
Socket programming, and openresty
 
T2
T2T2
T2
 
Dart Workshop
Dart WorkshopDart Workshop
Dart Workshop
 
Networking & Socket Programming In Java
Networking & Socket Programming In JavaNetworking & Socket Programming In Java
Networking & Socket Programming In Java
 
IT Operations for Web Developers
IT Operations for Web DevelopersIT Operations for Web Developers
IT Operations for Web Developers
 
swift-nio のアーキテクチャーと RxHttpClient
swift-nio のアーキテクチャーと RxHttpClientswift-nio のアーキテクチャーと RxHttpClient
swift-nio のアーキテクチャーと RxHttpClient
 
Node.js System: The Approach
Node.js System: The ApproachNode.js System: The Approach
Node.js System: The Approach
 
The Recording HTTP Proxy: Not Yet Another Messiah - Bulgaria PHP 2019
The Recording HTTP Proxy: Not Yet Another Messiah - Bulgaria PHP 2019The Recording HTTP Proxy: Not Yet Another Messiah - Bulgaria PHP 2019
The Recording HTTP Proxy: Not Yet Another Messiah - Bulgaria PHP 2019
 

Kürzlich hochgeladen

WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneUiPathCommunity
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical InfrastructureVarsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructureitnewsafrica
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFMichael Gough
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkPixlogix Infotech
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...Nikki Chapple
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentMahmoud Rabie
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfROWELL MARQUINA
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...BookNet Canada
 
Kuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialKuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialJoão Esperancinha
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI AgeCprime
 
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...panagenda
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfpanagenda
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 

Kürzlich hochgeladen (20)

WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyone
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical InfrastructureVarsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDF
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App Framework
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career Development
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdf
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
 
How Tech Giants Cut Corners to Harvest Data for A.I.
How Tech Giants Cut Corners to Harvest Data for A.I.How Tech Giants Cut Corners to Harvest Data for A.I.
How Tech Giants Cut Corners to Harvest Data for A.I.
 
Kuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialKuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorial
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI Age
 
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 

WebTalk - Implementing Web Services with a dedicated Java daemon

  • 1. WebTalk Implementing Web Services with a dedicated Java daemon Geert Van Pamel – HP-Interex Belux Mannheim, Germany 10 November 2008 Talking to the web
  • 2. Introduction • A Java application behaving as a web server • Java & the HTTP protocol is the base • and IP TCP + UDP programming • KISS – Keep it simple, but fully functional • Virtual file system – Not about Applets, Servlets or JSP – Not about Jakarta or Tomcat – No SOAP, no BPEL, no WSDL
  • 3. Why this Project? • Interoperability of networked clients & servers • Using Open standards • HTTP protocol • Network socket programming (the network is the system) • Interfacing with middleware (building blocks) • Any client supporting the HTTP protocol • Reliable and performance • 24/7 daemon written in Java
  • 4. Why Java? • Using standard Java • Extensible • Can run on any platform – CORE (code once run everywhere) • Easy testing • Easy implementation (distribute byte code) • Can easily interface with other Java classes • Use with any backend supporting Java • Stable, resilient & very good exception handling • Well documented, tools available • Modern language – many programmers available
  • 6. Useful? Let me know! • Home use • Enterprise use • You manage everything • No dependencies with other packages • It is free, no licenses • Just invest your own few mandays • Download the template!
  • 7. The Basic Algorithm • Initialise the program • Verify that it runs on one of the registered servers • Verify the command parameters, overruling default parameters • Open the log file if requested • Create the main server listening socket • Perform the main server loop (endless) – Listen to the network port and wait for the next client request – Get the client identification – Check access authorisation – Get the client capabilities – Get the client input parameters – Parse input parameters – Validate input parameters – Perform a server transaction – Convert the status and results to the client capabilities – Log the request and the results in the error log – Return status and results to the client program – Close the client network connection A daemon never dies
  • 8. Program Header package webtalk; import java.io.*; import java.net.*; import java.util.*; public class WebTalk { // Creates a new instance of WebTalk public WebTalk() { } // main program }
  • 9. Program initialization (1) private static int tcpPort = 2006; // P1 = listen port: 2006 private static String logDirName = "/tmp"; private static FileOutputStream logFile; // P3 = logging mode: log,sms private static PrintWriter log = null; public static void main(String[] args) { String crlfStr = System.getProperty("line.separator"); String runModeDefault = "normal"; // P2 = run mode: normal if (crlfStr.equals("rn")) logDirName = "c:temp"; if (args.length >= 1 && !args[0].equals("")) tcpPort = Integer.parseInt(args[0]); // P1: listen port logFileName = "webtalk-" + tcpPort + ".log"; if (args.length < 3 || args[2].indexOf("nolog") < 0) try { // P3: Open the log file at startup logFile = new FileOutputStream(new File(logDirName, logFileName), true); log = new PrintWriter (logFile, true); } catch (FileNotFoundException e) { writeLog(e.toString()); // But we continue processing, since not fatal } if (args.length >= 2 && !args[1].equals("")) runModeDefault = args[1]; // P2: Set trace mode if (!runModeDefault.equals("normal")) writeLog("Enable " + runModeDefault); } First implemented in 2006
  • 10. Program initialization (2) // P3: Open the log file at startup if (args.length >= 3 && args[2].indexOf("log") >= 0) try { logFile = new FileOutputStream(new File(logDirName, logFileName), true); log = new PrintWriter (logFile, true); } catch (FileNotFoundException e) { writeLog(e.toString()); // But we continue processing, since not fatal } default nolog
  • 11. Blocking Unauthorized Server Implementation try { serverAddress = InetAddress.getLocalHost(); remoteHost = serverAddress.getHostAddress(); serverHost = remoteHost; serverName = serverAddress.getCanonicalHostName(); } catch (UnknownHostException e) { sendPanic(e.toString()); System.exit(8); } if (!allowRemoteHost (remoteHost)) { sendPanic("Reject host"); // Unauthorized server (address) System.exit(2); }
  • 12. Open main Server Socket try { server = new ServerSocket(tcpPort); } catch (IOException e) { sendPanic(e.toString()); System.exit(1); } private final static String versionString = "WebTalk/20081013 (Java/HTTP)"; sendPanic("Starting " + versionString); Server up and running
  • 13. Waiting the next Client while (true) { try { client = server.accept(); } catch (IOException e) { sendPanic(e.toString()); System.exit(4); } remoteAddress = client.getRemoteSocketAddress().toString(); remoteHost = remoteAddress.substring(1, remoteAddress.indexOf(':', 1)); try{ in = new BufferedReader( new InputStreamReader(client.getInputStream(), webCharset)); out = new PrintStream(client.getOutputStream(), false, webCharset); } catch (IOException e) { sendPanic(e.toString()); System.exit(5); } // main processing logic here } A daemon never dies
  • 14. Client Access Control private final static int MAXIP = 50; private static int clientCount[] = new int [1+MAXIP]; private static String allowAddress[] = new String [1+MAXIP]; … allowAddress[14] = nslookup("client1.mshome.net"); allowAddress[15] = "192.168.210.34"; … if (!allowRemoteHost (remoteHost)) { sendPanic("Reject host: " + remoteHost); } static boolean allowRemoteHost (String remoteAddress) { boolean allowIP; allowIP = false; for (int i = 1; i < allowAddress.length; i++) { if (remoteAddress.equals(allowAddress[i])) { allowIP = true; clientCount[i]++; break; } } return allowIP; }
  • 15. Getting Browser Input String webTalkIn; webTalkIn = "n-- Start request"; // Get the client request while (webTalkIn != null && !webTalkIn.equals("")) { // Here comes the parsing of the browser request // Get the next HTTP header until empty line encountered webTalkIn = in.readLine(); } TCP does not have EOF
  • 16. Parsing the Client URL String webRequest[]; // GET /urlPath/...?webParam=value&... HTTP/1.1 String mainPath[]; String urlPath[]; String webParam[]; String paramPair[]; writeLog(remoteHost + " " + gmTime + " " + webTalkIn); webRequest = webTalkIn.split(" "); mainPath = webRequest[1].split("[?]", 2); urlPath = mainPath[0].split("/"); webParam = mainPath[1].split("&"); /* Here we should map the key/value pairs into the application logic */ for (int c = 0; c < webParam.length; c++) { paramPair = webParam[c].split("=", 2); paramPair[0] = paramPair[0].toUpperCase(); //System.out.println(paramPair[0]); //System.out.println(paramPair[1]); }
  • 17. Parsing HTTP Headers • User-Agent • Via • Accept-Language • HTTP/1.x • Authentication • Connection: keep-alive
  • 18. Identifying the Browser /* set default userAgent */ if (userAgent == unkownUserAgent && webRequest.length >= 2 && webRequest[0].equals("User-Agent:")) { if (runMode.equals("trace")) writeLog(webTalkIn); if (webRequest[1].startsWith("Mozilla")) { // Firefox, Internet Explorer userAgent = mozillaUserAgent; } else { // Lynx, Opera userAgent = otherUserAgent; } }
  • 19. The main Server Logic Some examples we have implemented with success in production are: • Validating Web forms by calling WebTalk via TCP socket programming • .Net calling the WebTalk server to validate customer orders • Validating customer orders via a DTMF and a text to speech system • Oracle PL/SQL calling the WebTalk server to validate customer orders via the UTL_HTTP and UTL_TCP package [6] • Storing database transactions via HTTP protocol (avoid locking) • Apache Embedded Perl calling WebTalk for order validation
  • 20. Returning Server Results static void writeBody (String bodyText) { int bodyLength; String headerStr; if (userAgent > unknownUserAgent) { bodyLength = 2 + bodyText.length(); if (userAgent >= otherUserAgent) bodyLength += 6; // <PRE> tag headerStr = "HTTP/1.1 200 OKnDate: " + httpTime + "nServer: " + versionString + "nExpires: -1nCache-Control: no-cache, no-store, max-age=0, must-revalidatenPragma: no-cache" + "nContent-Length: " + bodyLength + "nConnection: closenContent-Type: text/htmln“; if (userAgent >= otherUserAgent) headerStr += "n<pre>"; if (runMode.equals("debug")) writeLog(headerStr); headerStr = headerStr.replaceAll("n","rn") + "r"; out.println(headerStr); } if (runMode.equals("debug")) writeLog(bodyText); if (userAgent > unknownUserAgent) bodyText = bodyText.replaceAll("n","rn") + "r"; // Avoid protocol error out.println(bodyText); } HTTP needs rn
  • 21. GMT Time in HTTP Format 01234567890123456789012345678 Fri Nov 03 13:51:56 GMT 2006 standard GMT format (gmTime) Fri, 03 Nov 2006 14:51:56 GMT internet GMT format (httpTime) private static String gmTime; private static String httpTime; Date now; TimeZone tz = TimeZone.getTimeZone("GMT"); tz.setDefault(tz); now = new java.util.Date(); gmTime = now.toString(); httpTime = gmTime.substring(0,3) + ',' + gmTime.substring(7,10) + gmTime.substring(3,7) + gmTime.substring(23,28) + gmTime.substring(10,23); HTTP = WWW thus GMT time
  • 22. Disabling Client Browser Cache Our application is data driven We must ensure that browsers, or proxies never cache requests. Therefore we generate the following headers: Expires: -1 Cache-Control: no-cache, no-store, max-age=0, must- revalidate Pragma: no-cache
  • 23. Close the Door try { in.close(); out.close(); client.close(); } catch (IOException e) { sendPanic(e.toString()); System.exit(6); } • One client at a time • Avoid locking using single process database inserts • Let the TCP stack buffer incoming requests • Latent DoS problem (if connection not properly closed)
  • 24. Client Examples in practice • Normal clients can be browsers like Firefox [8], Opera [8], or Internet Explorer: http://servername:2006/hub/2186/ivr?p1=1234&p2=5678 • With telnet: telnet servername 2006 GET /hub/2186/ivr?p1=1234&p2=5678 (enter) • With Lynx [8]: lynx "http://servername:2006/hub/2186/ivr?p1=1234&p2=5678" • With netcat: echo "GET /hub/2186/ivr?p1=1234&p2=5678n" |nc servername 2006
  • 25. Other Clients • Perl • PHP • Unix Socket programs • .Net clients • Database systems like Oracle [6] • Telecom servers like IVR Any program capable of talking HTTP
  • 26. Example: Perl use IO::Socket::INET; $Server = "servername:2006"; my $sock = IO::Socket::INET->new(PeerAddr => $Server, Proto => 'tcp'); $webStat = ""; if ($sock) { $sock->write("GET /statusnn"); $sock->recv($webStat, 2000); close($sock); } print "$webStatn";
  • 27. Example: Oracle stored Procedure CREATE FUNCTION web_request ( request in string) return string is c utl_tcp.connection; len PLS_INTEGER; result string(2000); BEGIN c := utl_tcp.open_connection ('servername', 2006); len := utl_tcp.write_line(c, 'GET ' || request); len := utl_tcp.write_line(c); result := utl_tcp.get_line(c, TRUE); utl_tcp.close_connection(c); return result; END; select web_request(‘/status’) from dual;
  • 28. Caveat Firefox and Opera [server loop] if (urlPath.length >= 2 && urlPath[1].equals("favicon.ico")) { runMode = "ignore"; } [generating output] if (runMode.equals("ignore")) { writeBody(clientResult); }
  • 29. Backend Systems • Oracle with permanent connection • MySQL database gateway • Middleware • Other SOA systems
  • 30. Server Management – Starting the Server • On Unix systems: su - apache -c "java -jar /html/java/bin/webtalk.jar 2006 normal log,sms > /dev/null 2> /dev/null &" • On Windows platforms you can start the server from a scheduled tasks (at system startup) java -jar C:htmljavabinwebtalk.jar 2006 trace > nul 2> nul • General format of startup command: java -jar webtalk.jar [port] [runmode] [log,sms] [> nul] [2> nul] Parameters: port: TCP port number (default 2006; you could use 2005 for testing) runmode: debug | test | trace | quiet | disable | enable | normal (default) log: "log" to trigger the logging to disk, with nolog logging is disabled sms: "nosms" to disable sending SMS messages
  • 31. Stopping the Server [Unix] • ps –ef |grep webtalk • kill $pid [Windows] • tasklist /fi “imagename eq java.exe” • taskkill /pid ProcessID Killing the daemon
  • 32. Temporarily Disable – Panic Button • Uncouple the system to avoid downstream catastrophes http://servername:2006/disable http://servername:2006/enable Internally the following code gets executed: if (urlPath.length >= 2 && (urlPath[1].equals("disable") || urlPath[1].equals("enable") ) ) { runModeDefault = urlPath[1]; clientResult = "Server: " + runModeDefault; sendPanic(clientResult); }
  • 33. Resume normal functioning http://servername:2006/normal http://servername:2006/[no]trace http://servername:2006/[no]debug Internally the following code gets executed: if (urlPath.length >= 2 && urlPath[1].equals("normal")) { runModeDefault = urlPath[1]; clientResult = "Enable " + urlPath[1]; }
  • 34. Recycling the Log File daynum=$(date +%u) for port in 2005 2006 ;do lynx -dump http://servername:$port/nolog > /dev/null mv /tmp/webtalk-$port.log /scratch/webtalk/log/webtalk-$port-$daynum.log lynx -dump http://servername:$port/log > /dev/null done Avoid disk full errors
  • 35. Open and close the Log File if (urlPath.length >= 2 && urlPath[1].equals("log") && log == null) { clientResult = "Append logfile"; logFile = new FileOutputStream(new File(logDirName, logFileName), true); log = new PrintWriter (logFile, true); } else if (urlPath.length >= 2 && urlPath[1].equals("nolog") && log != null) { clientResult = "Closing logfile"; logFile.close(); log = null; }
  • 36. Local Server Monitoring To verify that the process runs: [Unix] ps -ef |grep "java -jar" apache 114167 1 0.0 14:01:20 pts/0 0:01.63 java -jar /html/java/bin/webtalk.jar 2006 [Windows] tasklist /fi "imagename eq java.exe" /v Image Name PID Session Name Session# Mem Usage Status java.exe 716 Console 0 12.676 K Unknown Network monitoring netstat [-a] [-n] |grep webtalk |wc
  • 37. Remote Server Monitoring for tn in 2005 2006 ;do telnet servername $tn <<EOF 2>&1 >/dev/null | grep -v 'Connection closed by foreign host.' | /usr/local/bin/sendpanic "" "servername:$tn" GET /status HTTP/1.0 EOF done
  • 38. Status of the Server http://servername:2006/status [server loop] if (urlPath.length >= 2 && urlPath[1].equals("status")) { runMode = "status";} [generating output] if (runMode.equals("status")) { runMode = runModeDefault; clientResult = "Server: " + serverName + "(" + serverHost + ")" + "nPort: " + tcpPort + "nClient: " + remoteHost + "nStart Time: " + startTime + "nReset Time: " + resetTime + "nTime: " + gmTime + "nVersion: " + versionString + "nMode: " + runMode + "nLanguage: " + acceptLanguage + "nUser Agent: " + userAgent + "nTop Client: " + allowAddress[1] + "(" + clientCount[1] + ")" ; writeBody(clientResult);}
  • 39. Usage Statistics http://servername:2006/counters Usage counters per IP address since Mon Jun 04 20:56:39 GMT 2007: 192.168.86.109 2110 client1.mshome.net 192.168.219.113 1198 client2.mshome.net 192.168.112.50 916 client3.mshome.net • This is handled by the following code: if (urlPath.length >= 2 && urlPath[1].equals("counters")) { clientResult = "Usage counters per client since " + resetTime + ":n"; for (int i = 1; i < allowAddress.length; i++) if (!allowAddress[i].equals("")) clientResult += "n" + allowAddress[i] + "t" + clientCount[i] + "t" + rnslookup (allowAddress[i]);
  • 40. Resetting Counters http://servername:2006/reset if (urlPath.length >= 2 && urlPath[1].equals("reset")) { resetTime = gmTime; clientResult = "Reset counters"; for (int i = 1; i < allowAddress.length; i++) clientCount[i] = 0; }
  • 41. Building the Application • java -version java version "1.4.2" Java(TM) 2 Runtime Environment, Standard Edition Fast VM (build 1.4.2-6, native threads, mixed mode, 01/09/2007-23:17) • Store the source code in a subdirectory, and compile with javac. javac webtalk/WebTalk.java mkdir webtalk/RCS ci –l webtalk/WebTalk.java
  • 42. Run Interactively • [Unix] export CLASSPATH=lib1.jar:lib2.jar:. • [Windows] set CLASSPATH=lib1.jar;lib2.jar;. • To start the program interactively, immediately after compiling: java webtalk/WebTalk 2005 debug nolog,nosms
  • 43. Build the JAR File • The manifest file can contain a list of all other required jar files and the name of the main module to run. If your application is complex, multiple jar files will be needed. jar -cv0mf webtalk.jar webtalk/manifest.mf webtalk/*.class vi webtalk/manifest.mf Manifest-Version: 1.0 Created-By: 1.3.0_05 (Sun Microsystems Inc.) Class-Path: lib1.jar lib2.jar Main-Class: webtalk/WebTalk
  • 44. Simulate a Server (1) vi netcat.txt HTTP/1.1 200 OK Date: Tue, 29 May 2007 11:47:08 GMT Server: WebTalk/20070528 (Java/HTTP) Last-Modified: Tue, 29 May 2007 11:47:08 GMT Expires: -1 Cache-Control: no-cache, no-store, max-age=0, must-revalidate Pragma: no-cache Content-Length: 18 Connection: close Content-Type: text/html P1:0003|P2:42108 Coding, compiling, testing takes a lot of time
  • 45. Simulate a Server (2) while true ;do nc –lp 2005 < netcat.txt ;done lynx –dump http://localhost:2005/test lynx –source http://localhost:2005/test nc –p 2005 localhost telnet localhost 2005
  • 46. Error Handling Daemon process!Alarm notification Allow for easy supportError logging The program stopsFatal errors Logged, status to clientUnexpected errors Returned to client application, not logged Expected errors How to handleError Type
  • 47. General Exception Handling try { // put main program logic here } catch (Exception e) { e.printStackTrace (); sendPanic(e.toString()); // could also use e.getMessage() System.exit(7); } finally { sendPanic("Server closing down"); }
  • 48. Enable Debugging • To enable & disable debugging http://servername:2006/[no]debug if (urlPath.length >= 2 && urlPath[1].equals("debug")) { runModeDefault = urlPath[1]; clientResult = "Enable debug"; }
  • 49. Enable Tracing • To enable & disable tracing: http://servername:2006/[no]trace if (urlPath.length >= 2 && urlPath[1].equals("trace")) { runModeDefault = urlPath[1]; clientResult = "Enable trace"; }
  • 50. Enable Quiet Mode • To set quiet mode (when you would have too many transactions) http://servername:2006/quiet if (urlPath.length >= 2 && urlPath[1].equals("quiet")) { runModeDefault = urlPath[1]; clientResult = "Enable quiet mode"; }
  • 51. Error Logging static void sendPanic (String panicString) { writeLog(panicString); if (smsRelayAddress != null) sendSMS (serverName + " " + serverHost + ":" + tcpPort + " " + panicString + " from " + remoteHost + " " + gmTime); } static void writeLog (String logText) { system.err.println(logText); if (log != null) { log.println(logText); } }
  • 52. Sending SMS Alarm static void sendSMS (String smsString) { byte sms[]; DatagramPacket smsPacket; DatagramSocket smsSocket; try { sms = smsString.getBytes(); smsPacket = new DatagramPacket (sms, sms.length, smsRelayAddressDefault, 535); smsSocket = new DatagramSocket(); smsSocket.send (smsPacket); smsSocket.close(); } catch (Exception e) { // ignore } } UDP
  • 53. UDP to TCP Relay Server (1) import java.io.*; import java.net.*; public class RelaySmsUdp { public static void main (String args[]) { DatagramSocket smsSocket; DatagramPacket smsPacket; InetAddress clientAddress; String clientHost; String panicString; Socket outSocket; PrintWriter out; byte sms[] = new byte[165]; // byte array String panicDest; String smsHost; panicDest = "0123456789"; // your Mobile alarm number if (args.length >= 1 && !args[0].equals("")) panicDest = args[0]; smsHost = "sms-tcp-gateway"; if (args.length >= 2 && !args[1].equals("")) smsHost = args[1];
  • 54. try { smsSocket = new DatagramSocket(535); // listen to port while (true) try { smsPacket = new DatagramPacket (sms, sms.length); smsSocket.receive (smsPacket); clientAddress = smsPacket.getAddress(); clientHost = clientAddress.getHostAddress(); panicString = new String (smsPacket.getData(), 0, smsPacket.getLength()); if (!panicString.startsWith("0")) panicString = panicDest + " " + panicString; // prefix with Mobile number if needed if (clientHost.startsWith("192.168.")) { // Filter non-registered IP addresses outSocket = new Socket (smsHost, 535); out = new PrintWriter(outSocket.getOutputStream()); if (panicString.length() > 165) panicString = panicString.substring(0, 165); out.println (panicString); out.close(); outSocket.close(); } } catch (UnknownHostException e) { writeLog(e.toString()); } catch (IOException e) { writeLog(e.toString()); } } catch (SocketException e) { writeLog(e.toString()); } Relay to TCP for message transmission
  • 55. Disable SMS at Startup • Can be very handy while developing private static InetAddress smsRelayAddressDefault; private static InetAddress smsRelayAddress; try { smsRelayAddressDefault = InetAddress.getByName("sms-udp-gateway"); } catch (Exception e) { smsRelayAddressDefault = null; } smsRelayAddress = smsRelayAddressDefault; if (args.length >= 3 && args[2].indexOf("nosms") >= 0) smsRelayAddress = null;
  • 56. Disable or Enable SMS at runtime lynx -dump http://servername:2006/[no]sms • The following code gets executed: if (urlPath.length >= 2 && urlPath[1].equals("sms")) { smsRelayAddress = smsRelayAddressDefault; clientResult = "Enable sms notification"; sendPanic(clientResult); } else if (urlPath.length >= 2 && urlPath[1].equals("nosms")) { smsRelayAddress = null; clientResult = "Disable sms notification"; sendPanic(clientResult); } If you are drown with alarms
  • 57. Supported Server Platforms Java virtually runs on all platforms: – You only have to distribute the byte code in a Jar file! • Any Intel Linux • Any Intel Windows XP or Vista desktop • Alpha Tru64 UNIX V5.1B-3 with Java 1.4.2-7, see [11] • Itanium • Windows server 2003 with Java 1.5.0_07
  • 58. Implemented Standards • DNS • HTTP, see [2] • Internet date & time format • Java, see [3] [4] [5] • NFS during development • NTP • TCP/IP • UDP for error relay • Web Cache control • ZIP for jar file manipulation
  • 59. What I have learned • Think in objects • Keep it simple (divide and conquer) • Client /server programming is fun ... debugging and fault finding even more • It boosts the team spirit • It stimulates your own brains and creativity • Java (network) programming is not really exotic, nor complicated • Web applications must disable proxy and browser cache • TCP sockets do not have an end-of- file... – a server socket cannot know when the client sends the last packet – this is why a final empty line has been built into the HTTP and SMTP protocol • Java initialises all variables to 0, "", or null • Java always start with element 0 • In Java you do not use pointers and linked lists to create dynamic data structures • Sparse matrices are natural in Java • Error handling and planning for the unforeseen is the most important – try / catch should be a habit • The HTTP protocol terminates every line by CRLF. Originally I terminated HTTP server output only by LF using println. None of the Unix systems did complain. .Net servers replied with an error “The server committed a protocol violation”. • Keep the history of your changes. Use e.g. RCS, Revision Control System.
  • 60. Design Principles • Separate code and data • Flexible data structures • Fully table & data driven • Keep the code simple • Code only once (no duplicate instructions) • Do normal things straight • Use industry standards • Use as much as possible standard system functions • Do care about and be conservative with system resources • Give the power to the users • Be flexible, and extensible • Be future proof • Be generous on what you accept, be strict on what you return • Do the unforeseen via exception handling; be robust • Be fool proof • Avoid catastrophes at all times • Use what you know • Have maximum security and access control • Allow for logging, tracing, and debugging • Care about supportability • Be platform independent • Write a complete documentation
  • 61. Future Extensions and Migration • Advanced Java debugging • Build and use appliances • Application setup via config file instead of hardcoded • DNS caching • IPv6, Internet protocol V6 • Java runtime optimisation (tuning, memory usage) • kill –HUP, to change debug level, or restart the server • Legacy applications • Load balancing • Multithreaded server • UDP monitoring • NTLM, Active Directory user authentication • POST or other HTTP protocols • SIF, Service Invocation Framework = SOA services for Java • Timeout, drop a bad behaving client (currently we suffer from a DoS problem) • SNMP monitoring • SOA, Service Oriented Architecture • System parameters • XML, Input / Output in XML format
  • 62. References [1] Full program source and additional material: http://www.hp-interex.be/pub/project/webtalk [2] The HTTP protocol and related standards: http://www.w3.org [3] Documentation and kits for Java: http://www.sun.com [4] Java technical documentation: http://java.sun.com/j2se/1.4.2/docs [5] Java search page: http://java.sun.com/j2se/1.4.2/search.html [6] Oracle UTL_HTTP: http://www.oracle.com/technology/sample_code/tech/pl_sql/htdocs/x/Utl_Http_Package_Enhan cements/Cr_Using_Utl_Http.htm [7] Iptables: http://www.netfilter.org [8] Browsers: The Lynx browser: http://lynx.browser.org Firefox: http://www.firefox.com Opera: http://www.opera.com [9] Perl libraries: http://search.cpan.org [10] NetBeans: http://www.netbeans.org [11] HP Java implementations: http://www.hp.com/java [12] Bruce Eckel, Thinking in Java, The definitive introduction to object-oriented programming in the language of the World Wide Web, Fourth Edition, ISBN 0-13-187248-6, 2006, Prentice Hall, Pearson Education, Code & Supplements at http://www.mindview.net
  • 63. Thanks • My colleague Danny De Thaye to encourage me in my endeavors • Connect Management for accepting my lecture • To you, the audience geert.van.pamel@belgacom.net http://www.hp-interex.be/wiki/index.php/User:Geertivp
  • 65. Abstract • The speaker presents a Java application that behaves like a dedicated web server and provides Web services. It runs as a standalone network daemon and only calls system and network services that are directly executed by the JVM. No other software is necessary. The URL path referenced by the clients is not referring to files. Instead it is calling other middleware services and serves as a Web service gateway. The application only requires TCP/IP and Java library programming. Because it behaves as a web server, the browser can be any client capable of interfacing with the standard HTTP protocol. • Learn about Java network programming, specifically about security, reliability, supportability, monitoring, browser cache disabling, kit building, starting the Java server process, troubleshooting and monitoring, implementing on different operating systems and platforms, and integrating with other services. Debugging server processes is difficult because there is no direct user interface. Every interaction goes via network sockets, or process signals. Remotely control the server by using the HTTP protocol, enable and disable logging, tracing, and debugging info.
  • 66. About the Author • The author works at Belgacom since 1997 in the Customer division as an ICT Project Manager. • He is Chairman of HP-Interex Belux since 2008. • He is Board member of HP-Interex Belux since 2002. Member of DECUS since 1984, and CUO since 1998. • In his previous life he worked as Project manager at Digital Equipment Corporation (DEC) [company merged with Compaq in 1998 and with HP, Hewlett Packard in 2002]. • His early activities in the world of IT were biometrical statistics, teaching computer courses, real time car manufacturing automation, and medical image processing using nuclear isotopes computer tomography. • He is an engineer in Agriculture and Applied Biological Sciences, and postgraduate in Computer Science. He is eager to implement technology as a business advantage.
  • 67. Useful Utilities • Any web browser [8] To test the server (Firefox, Internet Explorer, Opera, lynx) • crontab To reset the daily log files, to monitor • Google For help with Java, use e.g. Google: Java ServerSocket • host To lookup the IP address of a machine • init To start the Web Service • jar Java archive builder • java Java runtime interpreter • javac Java Compiler • nc, netcat To test, debug, or simulate • NetBeans [10] Java IDE Integrated Development Environment Unix & Windows • NFS The server is developed, build & tested cross-platform • nmblookup, nbtstat To lookup the name of a Windows client, if not registered in DNS • NTP, ntpq To synchronise system time • Perl [9] Practical Extraction and Reporting Language • ps, grep, kill To monitor or change run status • telnet To monitor and test special requests (does not work in batch) • vi Terminal based editor - on linux, you might have colours • zip, Winzip To verify (or update) the contents of jar files.
  • 68. Technical Terms • API Application Programming Interface • CRLF Carriage Return and Line Feed • GMT Greenwich Mean Time, almost the same as UTS (Universal Time Coordinated) • HW Hardware • IVR Interactive Voice Response • Jar Java archive • JDK Java Development Kit • JRE Java Runtime Environment • JSP Java Server Pages • JVM Java Virtual Machine • OS Operating System • Platform A combination of hardware and software • SMS Mobile text message (do not confuse with Microsoft System Management Server) • SW Software (operating system and layered products) • Unix Any type of Unix, or Linux OS, in general • UNIX™ UNIX Trademark
  • 69. nslookup static String nslookup (String hostName) { String ipAddress; try { ipAddress = InetAddress.getByName(hostName).getHostAddress(); } catch (UnknownHostException e) { ipAddress = ""; } return ipAddress; }
  • 70. rnslookup static String rnslookup (String ipAddress) { String hostName; try { hostName = InetAddress.getByName(ipAddress).getHostName(); } catch (UnknownHostException e) { hostName = ""; } return hostName; }