WebOfThings: Paper for my presentation on DesignWest Silicon Valley in San Jose, where I presented an Embedded Web Server for HTML5, using SSE - Server Sent Events, in a very efficient and fast implementation on ARM Cortex-M.

1. Web of Things: HTML5 for resource constrained Embedded
Systems
Jonny Doin, Principal Engineer
Sergio Eduardo Alves de Paula, R&D Engineer
Several current embedded systems, like industrial instrumentation, sensor
networks and control oriented systems have embedded HTML servers. Often
these HTML servers need to have dynamic updating webpages, with realtime
data. Using the new features of HTML5, rich and lightweight embedded web
servers with dynamic content and graphs can be designed. This white paper
will show techniques combining these new HTML5 features with managed
HTTP frames, which can be applied to any microcontroller to achieve
embedded servers using a minimum of resources. The interaction of page
script and server side processing will be shown for implementing SSE and
HTML5 dynamic content. The systems targeted are small bare-metal
embedded systems with hardware-based TCP/IP implemented on separate
chips, but the techniques presented can be used in any system.
Embedded Systems
The class of “Embedded Systems” range from dedicated application controllers that run
full-featured operating systems such as Linux, down to microcontrollers running very small
RTOS or bare-metal applications.
We are targeting the low-end systems, often used in control-intensive applications.
These systems usually are based on microcontrollers with limited resources, like small on-
chip Flash program memory ranging from 128KB to 2MB and on-chip RAM ranging from
32KB to 256KB.
Such systems are usually found in Industrial instrumentation and control devices, often
connected to fieldbus and other dedicated networks. In recent years, these Industrial
control systems are being connected to TCP/IP networks, driven by the need to integrate
their process data to the factory IT backend.
Besides normal M2M connectivity, that includes traditional socket-level TCP/IP and higher-
level fieldbus protocols, these systems are also required to serve web-based human
interfaces.
Embedded Web Servers
The demands of an embedded instrumentation web server today go far beyond showing a
collection of simple pages with fixed content. Current embedded web servers are dynamic
monitors for the realtime process data, interact with the control process to send realtime
commands, and are also complex configuration user interfaces, often with dynamic
graphics content. Add the need to be compatible with several different browsers and you
have a formidable set of requirements.
Traditionally, these requirements demanded a powerful web server, comparable to a
LAMP stack (Linux, Apache, MySql, PHP/Perl/Python). The dynamic content coupled with

2. a responsive user interface usually requires large RAM buffers and backend support for
the dynamic content.
In a dedicated embedded system with constrained resources, the toll of such a fast
dynamic web server can be excessive, moreover when the web server functionality is not
the main purpose of the system. This toll takes on RAM and processing resources, and
may introduce processing latency and significantly reduce available RAM for core data
processing.
Many embedded web server implementations allow allocation of system resources, by
limiting the RAM footprint taken by network buffers and reducing processor bandwidth for
the web server task. Not surprisingly, there is a clear tradeoff involved in reducing these
system resources, and the most evident is a reduced performance in dynamic web page
update and responsiveness.
The TCP/IP Stack
We based our implementation on a hardware-based TCP/IP implementation.
This decision suits small embedded systems very well, because the main system
resources needed on a web server application are located in the TCP/IP stack.
From the point of view of system resources allocation, the TCP/IP core demands more
RAM usage and CPU load for the protocol state machine processing than the socket-layer
state machines needed to support a web server functionality.
Keeping the network access on a separate processing environment also brings higher
system reliability to low-end embedded systems.
The core app is insulated from the toxic environment of the network. Stack overflow
attacks directed to the network protocol layers are contained and can be dealt with by the
core application by resetting the network chip. Payload buffer overflow and CGI abuse still
need to be addressed by the application.
Socket Layer backend
The server must implement the HTTP state machine, which consists of a HTTP text parser
and an HTTP frame composer. The server also needs to implement the handlers for
Server Sent Events (SSE - more on that later), and the Common Gateway Interface (CGI)
command parser and dispatcher.
Some Non-Functional Requirements are very important to keep the system reliability.
Special consideration must be dispensed to buffer overflow in the parsers, and memory
allocation must be strictly controlled, following the rules of the core application. Another
NFR is that all state machines must be safe, i.e., must not lock up in a frozen state.
The main NFR on this implementation is that functional failures on the web server should
not affect the main application core.
This is all it takes for a simple web server.
Dynamic Content Handling
Web pages with rich data sets tend to be large. Frequently, large Javascript frameworks
and libraries are used to support platform agnostic behavior.

3. Like other encapsulated frames, HTTP frames need a header descriptor that specify the
length of the payload. This imposes that HTTP frames must have payload sizes
determined prior to be transmitted. That is not a problem for fixed frames, such as fixed
content pages, because their length can be predetermined and saved with a page
descriptor, and access to the length can be made without reading the full payload at once,
which would demand excessively large RAM buffers.
One method used in the past by simple embedded servers for dynamic HTML pages was
to printf() the dynamic data directly into the page text. Besides being an extremely slow
method for content update, dependent on page refresh, this usually means that the entire
frame must reside in RAM, have its length determined, and only then be sent to the socket
layer. This also imposes large RAM buffers to hold the page rendering.
Instead of rendering the data onto the page text, a much better approach is to use a fixed
page with placeholders for event sources of dynamic data. With the resulting fixed frames,
the server can apply the same approach to fixed and dynamic pages, because they are
essentially fixed text frames.
One aspect of this approach is that it improves concern separation for the server and
client, and delegates to the page script and description full control over location and
appearance of the dynamic data.
Another important aspect for the implementation is that by applying a “window” buffer to
the data stream to the socket layer, we can serve multiple sockets with very little RAM. A
typical responsive system demands less than 200bytes per socket.
HTML5 streaming data: SSE
With HTML5, several aspects of dynamic web page content got addressed by new
constructs. Especially, SSE (Server Sent Events) allow streaming data to be sent to a
client and displayed by the page javascript, without changing the page text. Although this
was possible before HTML5, SSE allow asynchronous streams to be sent to the client, at
the server timing, without a formal request. That feature simplifies the collection of realtime
data, reduces latency and cuts bandwidth for embedded data traffic.
The main contribution of the SSE for dynamic data streams is that a single request is
needed by the page script to start the event generator on the server side, and from that
point on, the event source will supply a stream of data to the client, simplifying client script
and reducing network traffic by half.
These features allow a lightweight server engine, using less resources and less network
bandwidth for streaming data, making SSE very attractive to embedded web server
applications.
JSON Data formatting
JSON (Javascript Object Notation) is a text data format used for data exchange. It is
based in simple comma-separated name:value pair descriptors, making it very simple to
parse and compose.
When used with javascript, JSON formatted data are converted to javascript objects,
readily accessible to the page script.

4. JSON data can easily be applied to streaming text data, making it perfect for use with SSE
streams.
Its simplicity and {name:value} also helps to generalize the server side processing, by
using data descriptors on the server side.
XML HTTP Requests
XML HTTP Requests is an API to send HTTP requests from the page to the server directly
under javascript control. It is used in the AJAX framework to implement asynchronous
dynamic pages, with the webpage requesting data and displaying in formatted fields.
We use XML HTTP Requests to send data and commands to the server under user
interaction.
Putting it all together
When using SSE to send streaming server data, the page script must open a data session
and keep it open for the whole duration of the stream. Usually, for a embedded web server
that is monitoring realtime data, this means that the socket is dedicated to the connection.
The client cannot send data to the server while the SSE data stream is active.
To do this, the page script detects the need to send commands or data to the server,
closes the SSE session, sends the data as a XML HTTP Request, then re-opens the SSE
session for the streaming data. This process is very fast, and is almost imperceptible to the
user.
The page script must detect errors in all stages of the process, including browser support
for the features used. The advantage of this simple process is that the server can be
implemented as a very lightweight and robust backend.
Some Performance Data
A simple web server using the techniques shown is realizable with very little code and
SRAM footprint, and yet shows remarkable user responsiveness and data update rate.
A server with a few pages and more than 10 streams of sensor data takes less than 5KB
of code footprint, and as low as 2KB for up to 4 sessions of client browsing, with very
responsive user interfaces and page update rates.
The full web server layer, implemented in a ARM7 or ARM Cortex-M3 takes less than 500
lines of code.