This presentation aims at introducing the interested reader to the very important topics of (i) Data Compression and (ii) Lightweight Network Communication in SoSE (Systems of Systems Engineering). These very interesting topics in the SOA context, in particular for data-centric application integrations. A working implementation is provided in Python, a very powerful main stream programming language. Check the project out from GitHub at https://github.com/hailpam/pyclientserver.

1. PYClientServer
Python powered Client-Server architecture on
compressed JSON channel

2. Outline
❖ Intro
❖ Requirements
❖ Architecture
❖ Implementation
❖ Tests
❖ Conclusion

3. Introduction - 1 of 2
❏ A need: seamless integration using a lightweight data
exchange format
❏ JSON is emerging as the modern de-facto standard data format for
services integration
❏ TCP is a must, it is reliable and can be secure
❏ JSON over TCP is a trustable solution
❏ Why compression? JSON strings are naturally verbose,
but less than XML, so
❏ compression can help in reducing the footprint
❏ bandwidth utilisation can be effectively improved
❏ RTT latency benefits from (much more responsiveness)

4. Introduction - 2 of 2
❏ And, Python? Well, it is an agile main stream
programming language
❏ very well suited for network communication
❏ very well supported
❏ platform-independent (so, portable solutions)
❏ powerful in string/data processing
❏ ...
❏ PYClientServer represents an attempt to implement a
working example of the previously outlined needs

5. Requirements
❏ Let’s go through the needs introduced
a. services integration over the network
b. seamless lightweight communication
c. effective bandwidth utilisation
d. easy extensibility
e. rapid development
f. working code to extend for enhancements
❏ Let’s summarize as follows
a. A Server offering a data-centric service
b. A Client asking for the service
c. A flexible Data Model offering
❏ marshalling/unmarshalling facility
❏ compression/decompression facility

6. Architecture
Client Server
TCP
Channel
Decoder Encoder
Process#1
Thread#1
Process#2
Thread#1
File System
1. JSON request
2. File Reading
3. Text
compression
4. Server
response
5. Decompressed
Text
0101010101
{ GET, 100 }

7. Implementation - 1 of 5
❏ Key implementation aspects concerning the Server
a. it takes advantage of Python’s powerful network library: 1 server
handler in few LoC
b. it receives text-based request and replies providing compressed data
❏ Let’s have a look at the code
class TCPRequestHandler(SocketServer.BaseRequestHandler):
…
try:
print "[TCPRequestHandler][handle] Connection accepted... processing"
# Reading request (assuming a small amount of bytes)
data = self.request.recv(1024).strip()
# Unmarshall the request
request = Request('', 0)
data = request.from_json(data)
…

8. Implementation - 2 of 5
# Read lines from a text file
resource = []
resource.append(self.__resourcepath)
resource.append(self.__filename)
testfile = open(''.join(resource), 'r')
# Prepare the response data
response = Data(True, [], 0)
list = testfile.readlines()
response.vector = list
response.nrbytes = len(list)
# Marshall JSON representation
json_str = response.to_json()
c_response = self.__compression.compress(json_str)
start = time.time()
self.request.sendall(c_response)
print "[TCPRequestHandler][handle] Bunch of compressed data sent back!"
end = time.time()
if (self.__isdebug):
print "Delivery::Time Elapsed::", str(end - start)

9. Implementation - 3 of 5
❏ As already seen in the Server code, system Data Model
is very important, because
a. it allows to seamlessly marshall/unmarshall the data
b. it allows to perform compression/decompression, according to the
communication protocol
class Request(object):
def __init__(self, name, code):
self.name = name
self.code = code
def to_json(self):
self.__dict__['name'] = self.name
self.__dict__['code'] = self.code
return json.dumps(self.__dict__, indent=4)
def from_json(self, sting):
dict = json.loads(sting)
self.__dict__ = dict

10. Implementation - 4 of 5
class Utility(object):
def compress(self, data):
string = str(data)
return zlib.compress(string, zlib.Z_BEST_COMPRESSION)
def decompress(self, data):
string = str(zlib.decompress(data))
return string
❏ Till now, Server and Data Model have been briefly
discussed, but those are only simple snippets of code
❏ To have a look at the full code of Server and Data Model, check the
project out on GitHub.

11. Implementation - 5 of 5
❏ Client thread is the most simple entity involved in the
scenario sketched on the architecture, in fact, it acts as
follows
❏ send a pure JSON request (Data Model aids in doing that)
❏ upon a reception of server data, it performs the decompression as
dictated by the defined communication protocol
# Request creation
data = Request('GET', 100)
# Client socket binding
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((self.__serverhost, self.__serverport))
# Sending JSON data over the socket
sock.send(data.to_json())
response = self.__receive_data(sock)
# Treating compressed data
result = self.__compression.decompress(response)
data = Data(False, [], 0)
data.from_json(result)

12. Tests
❏ Hereafter some screenshots captured during a session
test are proposed
Server-side: it is important to consider how the compression reduces
drastically the dimensions (form 4.2MB to 1MB)
Client-Side: in less than 1 second, the data are received. Afterwards, data
are decompressed and the first line of the document is disaplyed

13. Conclusion - 1 of 2
❏ Architecting the communication protocol for the service
integration as shown in the architectural sketch, brings
several advantages
a. effective bandwidth utilization: the compression reduces the verbosity
b. much more responsiveness: the entire data frame is trasferred in less
than a second
❏ Python network programming is really powerful: with
few lines of code, the application programmer can
define and deploy a working multithreaded service
❏ JSON format is really amazing to work with in Python:
as shown, with few lines of code, the library makes the
serialization and deserialization a pretty easy job

14. ❏ Despite the simplified snippets presented, in the
previous slides
❏ Client, Server and Data Model have been presented as the three main
entities involved by design, but some other interesting features have to
be discovered
❏ System Architecture has been depicted and, as seen, it is simple but
powerful at the same time: there is the reason to believe that that
Architecture might scale very well
❏ If you want to contribute or to have a deeper look at the
code, do not hesitate to check the project out from
GitHub.
Conclusion - 2 of 2