A talk about mobile apps that rely on data from the Internet, and some of the decisions and choices facing mobile app developers in writing them
SlideShare kinda screws with the speaker's notes, so if you'd like the notes it's probably best to download the presentation file.
Overview of the talk is written up at http://dalelane.co.uk/blog/?p=1009
55. “ Micro Broker” Time of day Calendar based Message based Watermark Low watermark Number of messages Message priority Tariff based High watermark Low watermark High watermark Always connected Manual correction Connection Frequency Duration
Welcome In this session, I want to talk about mobile apps that rely on data from the Internet, and some of the decisions and choices facing mobile app developers in writing them
Who am I? Hobbyist mobile developer for ages As a day-job, I worked as a networking and messaging developer for many years, originally for Motorola in GSM networks then for IBM in enterprise messaging systems More recently, I joined IBM’s Emerging Technology team so I get to do some mobile stuff as my day-job which is great
Warning disclaimer bit As mentioned, I work for IBM I do plan to talk for a bit about an IBM technology – to illustrate a point I want to make about mobile apps
This is not a sales pitch I'm not on commission. I'm not trying to sell you anything. I want to talk about an area of technology and a challenge as a developer that I find interesting. That said, if you’re not comfortable with the idea of an IBM employee talking to you about an IBM technology at a barcamp at an IBM location, then fair enough – I entirely understand, and wont be offended if you want to leave now and catch one of the other talks.
Mobile apps are changing Cheaper data plans mean mobile developers are writing more apps that make more use of the Internet and web services The more we do this, the more we hammer our user's batteries
Poor battery life is one of the biggest pain points for smartphone users A challenge for mobile app developers is how to help. Or at least, how not to make things too much worse!
An example of the problem How many of you have a twitter client on your phone? It probably relies upon (or at least offers) polling. You probably have a setting where you specify how often the app should check twitter.com for new replies or DMs This can rinse your battery
This is massively inefficient A polling app is like a kid saying “Are we there yet?” “Is there any new data yet?” No. “ Is there any new data yet?” No. “ Is there any new data yet?” No. “ Is there any new data yet?” No. And each wasted request sent and reply received uses up precious battery life
Push can be better than pull Instead of making your phone – with it’s limited battery life – keep asking for information, better for it to sit idle and let the server tell it when new data is available
Your phone doesn’t poll for text messages SMS messages are received by having a data channel that the network operator can use to push notifications to the phone
BlackBerry became famous by extending this to email They have dedicated data channel support for pushing notifications of new email to the smartphone And more recently, this year RIM opened this up by providing a Push API for third party developers So if you are a mobile developer, and want to write an app that reacts to data online without hammering the battery, you can write it for BlackBerry
Apple Push Notification service is a similar idea They provide a push API where Apple servers can push a notification to the mobile From what I’ve read, the Apple approach is implemented differently to BlackBerry, though – relying on keeping an IP connection open between the mobile and server But as a developer, it’s the same kind of thing
Vodafone’s developer program betavine offer Push APIs They offer WAP and SMS flavours of a Push API for developers to experiment with. As a beta program, betavine use a credit based system for the SMS messages, with credits awarded for involvement in the community.
Ericsson offer a Java ME Push API This is for Sony Ericsson and some other J2ME-compatible phones They use an SMS-notification approach to signal to a Java client library on the phone that something is ready to download It is also a beta program, with developers provided with a limited number of credits
Or you can do it yourself Many mobile platforms allow the developer to intercept incoming SMS messages before the user sees them Any text messages received by the phone that meet certain criteria (e.g. sent from a certain number, or containing a certain keyphrase, or starting with a certain word, etc.) are sent to your app instead of the normal messaging application I’ve done this with Windows Mobile before and it’s pretty straightforward. Your app doesn’t even need to be running when messages are received – you add the criteria to the Windows Registry, and the OS launches your app when a SMS message matching the criteria is received The problem with all of these SMS-based approaches, though, is that it’s very expensive per message – you wouldn’t want to write an Instant Messaging app that relied on SMS for every message
Holding an IP connection open is the other main approach An app can connect to the server, then leave the connection open.
MQTT is a protocol and toolkit from IBM for doing this This is the technology I referred to earlier – that I want to quickly introduce you to
MQTT isn’t new MQTT was created over ten years ago – originally to solve the problem of satellite telemetry comms It was a similar challenge to mobile apps today – satellite applications couldn’t poll when comms cost dollars per byte So MQTT was created to address the need for communications with an absolute minimum of network usage – no polling and minimal overhead per message
MQTT starts with a CONNECT An MQTT client on a phone starts by connecting to an MQTT message broker
This is followed by a SUBSCRIBE The app can inform the server of what data it wants pushed to it by subscribing to particular topics
The connection is left open Although idle, the connection is maintained between the mobile and server
This makes push notification possible The server can use this connection to push any data to the client running on the phone
So, instead of this… For apps that need to access data in the cloud that you can’t control, you can compromise. Instead of the typical app which polls something online or in the cloud…
… you can do this With MQTT you can move that polling to a server, letting something on the server (where you don’t care about battery life, etc.) poll the web service. And use MQTT to push it to the mobile when it gets something of interest
A very quick “Hello World” example of how to do it What we have is a free set of tools for doing this easily I want to quickly demo how this can be done
First you need the server This is the message broker software to run on a server. You can download one for free from alphaWorks
Unzip it What you get is a zip file with versions of the server software for a variety of platforms Including the obvious – Windows, Red Hat, Ubuntu, Mac, etc.
Run it It is very configurable, but for now, you can just run the executable – the server is running
An aside for the really geeky You might notice from previous slide that the server was listening on port 1883 This is the assigned port number for the MQTT protocol, which is why it is used by default But you can easily change this if you want to use a different port number
Next, get a client We need an MQTT client – something to send and receive (to publish and subscribe) messages to the broker Java client is an easy place to start. You can download it from an IBM site
Unzip it The zip file you download includes a jar with a sample Java client
Run it Just run the sample jar …
Run it … and you get a little GUI utility app
A first code-free test I want two clients – so that I can send and receive messages First thing to do is to give each client a unique name – I’ve imaginatively used “client1” and “client2”
Get them both to connect to the server Once that’s done, click “Connect” on each GUI You can see the connection in the server log Note you need to specify the address and port number, but I can just leave the localhost defaults for now
Send a message from one. Get it on the other Subscribe to a topic in client 1. Publish a message to that topic from client 2 Client 1 gets the message
Doing the same from code is almost as easy The client zip we downloaded earlier includes a smaller client library This is a sample Java app I wrote using that client library The same three steps apply – first I need to connect to the message broker
Send a message To publish a message, you just call the publish method, giving it the topic and the actual data to send
Receive a message To receive messages, we need to implement one of the MQTT callback interfaces, and register to receive call backs from the MQTT client library
Receive a message Once we’ve done this, we can subscribe to receive messages for topics When a message is pushed from the server, the publishArrived method is called
You don’t have to use Java I’ve demonstrated this using Java because it’s a nice, easy start. But you can do the same from other languages if you prefer Note: not all of these are produced by IBM. The MQTT spec is published, and is relatively simple, so some developers have implemented client libraries in their language-of-choice and shared it with the community
Getting back to mobile The Java client library, for example, runs on many mobile platforms. For example – I created a HelloWorld project for doing this from an Android phone using the J2SE jar
MQTT on Android And here is it running on an emulator
Receiving messages from the sample GUI I can use the sample Java GUI on my desktop as before to publish messages to and from my Android phone
How have we used MQTT? A few examples of how we’ve found push notification useful
Choosing information to be updated about We have this running on mobiles of most platforms – I’ve got it running on my Windows Mobile phone now For demonstration purposes we have some random sources of information from around the office, and if any of these change – such as the light in the office being switched on – this is pushed to the phone Messages pushed to the phone also contain a description of what instructions the users can send in reply – e.g. turn the light back off. It’s not just notification – it’s a control mechanism too The point is this isn’t that we need a mobile app for controlling the office lights, but that we have a generic framework for pushing info to people’s phones, and letting them reply in some way
Other demonstrations A location-based example – pushing information to people’s phone in musems or art galleries, that is relevant to where they are A banking example – securely pushing bank statement updates or payment alerts to people’s phones
Home energy monitoring Pushing updates on how much electricity your home is using
Location-based updates for office and campus environments For each area of the site, a list of topics is broadcast – these are the sources of information that are available in this part of the site A user can choose which ones of these they want to subscribe to, and receive them as notifications pushed to their phone Particularly useful as an accessibility tool – example here shows that fire alarms can be pushed to people’s phones, which is ideal for those with hearing impairments who might otherwise not hear a fire alarm, but could feel a mobile vibrate in their pocket
But… it’s not a silver bullet For all that MQTT is a useful tool in building data-centric mobile apps, it doesn’t mean developers don’t still have to think about battery life.
Push is good It’s true that pushing data to the phone rather than making the phone repeatedly poll does reduce the mobile’s network usage – which by itself is a battery saving But holding the connection open does introduce other factors
Things to consider when holding a data connection open… For example, you generally can’t just open a connection and leave it there. The client needs to periodically send keep-alive messages to keep the connection active You specify the interval to use when you connect – but how do you choose a sensible interval? Difficult to know. Anecdotal evidence, for example, suggests that Vodafone will drop an idle connection after 10 minutes, so if you want your app to run on Vodafone without the mobile having to repeatedly reconnect (expensive in battery terms) then you probably want to specify a keepalive of under 10 minutes Another example might be power management routines in the mobile OS. Instead of thinking “nothing is using the network, so I can dial down to a low-power mode" your open connection might make the mobile think "I've got a connection open, so I must keep the radio in a high-power mode". You could argue that apps that poll allow the mobile OS to make intelligent choices about it's power management - because when the app disconnects, the idle times give the phone an opportunity to switch the radio off or limit it to a low-power mode. For some mobile OS’, an MQTT app that holds a connection open all the time might screw with that, resulting in making the mobile burn through it's battery to maintain a (battery-) expensive data connection that isn't being used anyway.
Developers still need a strategy They might decide that their app doesn’t need to stay connected the whole time. It could disconnect for periods of time under-the-covers to help with power management, then reconnect periodically. Sounds like polling in a roundabout way It is more efficient, because the broker server can remember what topics of information you subscribed to so when you reconnect you don’t need to specify all of the stuff you want to be sent again. It also can buffer any messages that would have been sent to you while you were disconnected, so you they are safely stored until you reconnect.
There are different sizes of message broker From free, to cheap, to expensive – there are different implementations of the server side software – each of which introduces additional capabilities With the free “Really Small Message Broker” I showed earlier, the developer would have to implement any logic of periodically connecting and disconnecting, any client-side buffering needed, etc. in their app In the larger brokers, some of this is done out-of-the-box
MicroBroker offers a transmission control policy You can configure when you want your client to connect and disconnect: At specific times Based on messages buffered up ready to send – e.g. connect after a high watermark of at least 10 buffered messages is passed, and stay connected for as long as a low watermark of 3 buffered messages is exceeded. You can specify that a connection is made if any messages over a certain priority are sent You can specify limits on how often a connection should be made, or how long a connection should be kept open Note: This is using JMS not MQTT – another protocol supported by Micro Broker. You could even factor in information from your mobile device – reducing your connection rate when the battery is running low, for example The point is (whether they code the logic themselves, or configure existing features) the developer needs to decide what is appropriate for the data their app handles For example, pushing out fire alarm notifications needs a connection to be kept open – you can’t poll every 30 minutes with something that urgent! For an app that is getting weather updates, though, always connected is overkill For apps that will be regularly receiving data, without long periods of idleness, keeping the connection open would likely save battery life For apps that will receive data infrequently, polling (or pseudo-polling like this) may be more efficient
The mobile developer has several choices To conclude, if they decide not to poll, the mobile developer writing a data-centric app has a few options open to them: Develop for a platform that provides dedicated support from the network operator – such as BlackBerry and iPhone. The support is baked in to both the network and the handset, and the use of battery life is as efficient as you can hope for SMS Pretty much as efficient, but introduces a cost element that makes it unsuitable for some situations For pushing urgent data on platforms other than BlackBerry/iPhone, however, it’s hard to beat And you have a bunch of APIs to choose from – from network supported ones that include the server side element of how you send messages, to just writing the message handling code yourself using mobile OS APIs Make a data connection to a server and keep it open Again, either doing it yourself, or using a protocol and toolkit such as MQTT How long you hold the connection open for is a decision you have to make, based on the nature of your app – toolkits such as MQTT at least provide you with the building blocks to make it easier to implement, if not any easier to decide what to implement!
Links to everything I’ve talked about at this address