Everybody knows that iOS is a shiny, modern operating system with a sleek object-oriented framework, Cocoa Touch, that makes development uncluttered and easy. Everybody is wrong. As a successor to both Unix and the Classic Mac OS and OS X, iOS has a wide-ranging mass of frameworks and libraries, employing different design patterns and conventions and sometimes employing different programming languages. The developer who's new to iOS can go only so far with Objective-C and the UIKit frameworks and their modern friends before he or she discovers the need to go deeper. But what's down there? This session digs down into the iOS stack to show the lower levels of the platform's APIs: the Media Layer, Core Services, and the Core OS Layer. As we go, we'll have to abandon Objective-C in favor of plain ol' C, which is used for the Core Foundation framework that does the heavy lifting for Cocoa Touch's strings, collections, memory management, I/O and more. We'll also look at specialized low-level frameworks for security (including certificate management and the confounding but useful Keychain), CPU-accelerated math and DSP functions, high performance graphics and sound, and more. At the lowest level, we hit Unix, and we'll see how conventional Unix-style programming practices are often appropriate (and sometimes necessary) on iOS, including pthreads and BSD sockets.
ICT role in 21st century education and its challenges
The Dark Depths of iOS [CodeMash 2011]
1. The Dark Depths of iOS
Chris Adamson — @invalidname
CodeMash 2011
Thursday, January 13, 2011
2. What We'll Dig Up
• Architectural layers of iOS and their contents
• How to find stuff
• Ask me about:
• "When would I ever use this?"
• "How would I…"
Thursday, January 13, 2011
3. iOS Architectural Layers
Cocoa Touch
Media Layer
Core Services
Core OS
Thursday, January 13, 2011
4. Cocoa Touch
• Application infrastructure
• Touch UI Media Layer
• High-level features Services
Core
• Likely the initial focus of new iOS developers
Core OS
Thursday, January 13, 2011
5. Media Layer
• Graphics, Audio, and Video technologies
Cocoa Touch
• 2D graphics, 3D graphics, animation, text
• Audio capture, streaming, effects, iPod Library
access
Core Services
• Video capture, editing, effects, playback
Corelibrary
OS
• Access to photo/video
Thursday, January 13, 2011
6. Core Services
• Foundation of Apple-specific APIs
Cocoa Touch
• Initially meant for use by Cocoa (Obj-C) and
Carbon (C++)Media OS X
on Mac Layer
• Data objects (strings, URLs)
• Threading
Corein-app purchase, etc.
OS
• SQL, XML, networking,
Thursday, January 13, 2011
7. Core OS
Cocoa Touch
• Lowest-level Apple frameworks
Media Layer
• Security, hardware-accelerated math
• System Core Services
• Low-level kernel and UNIX APIs
Thursday, January 13, 2011
11. Frameworks
• Apple’s definition: “A framework is a directory that
contains a dynamic shared library and the resources
(such as header files, images, helper applications, and
so on) needed to support that library.”
• Most Apple APIs are packaged as .framework
• Most third-party APIs are packaged as .dylib or .o
Thursday, January 13, 2011
15. Deep default dig
#import <UIKit/UIKit.h>
Thursday, January 13, 2011
16. Deep default dig
//
// UIKit.h
// UIKit
//
// Copyright 2005-2010 Apple Inc. All rights reserved.
//
#import <UIKit/UIKitDefines.h>
#import <UIKit/UIAccelerometer.h>
#import <UIKit/UIAccessibility.h>
Thursday, January 13, 2011
17. Deep default dig
//
// UIAccelerometer.h
// UIKit
//
// Copyright 2007-2010 Apple Inc. All rights reserved.
//
#import <Foundation/Foundation.h>
#import <UIKit/UIKitDefines.h>
Thursday, January 13, 2011
18. Deep default dig
/*!Foundation.h
! Copyright (c) 1994-2010, Apple Inc. All rights reserved.
*/
#include <CoreFoundation/CoreFoundation.h>
#import <Foundation/NSObjCRuntime.h>
#import <Foundation/NSArray.h>
#import <Foundation/NSAutoreleasePool.h>
#import <Foundation/NSBundle.h>
#import <Foundation/NSByteOrder.h>
#import <Foundation/NSCalendar.h>
#import <Foundation/NSCharacterSet.h>
#import <Foundation/NSCoder.h>
Thursday, January 13, 2011
19. Deep default dig
/*!CoreFoundation.h
! Copyright (c) 1998-2010, Apple Inc. All rights reserved.
*/
#if !defined(__COREFOUNDATION_COREFOUNDATION__)
#define __COREFOUNDATION_COREFOUNDATION__ 1
#define __COREFOUNDATION__ 1
#if !defined(CF_EXCLUDE_CSTD_HEADERS)
#include <sys/types.h>
#include <stdarg.h>
#include <assert.h>
#include <ctype.h>
#include <errno.h>
#include <float.h>
#include <limits.h>
#include <locale.h>
#include <math.h>
#include <setjmp.h>
#include <signal.h>
#include <stddef.h>
#include <stdio.h>
Thursday, January 13, 2011
21. Your App
Cocoa Touch
Media Layer
Core Services
Core OS
Thursday, January 13, 2011
22. Your App
<UIKit.h> Cocoa Touch
Media Layer
<Foundation.h>
<CoreFoundation.h> Core Services
<stdio.h> Core OS
Thursday, January 13, 2011
23. #include, #import, @class
• #include literally includes all of a header file
• #import, defined by Obj-C, includes a given file only
once, intended to help with circular dependencies
• @class just promises that a given Obj-C class will be
#import’ed at some point
• Useful for circular header dependencies; one can
use @class in .h, then #import in .m
Thursday, January 13, 2011
24. Cocoa Touch
Media Layer
Core Services
Core OS
Thursday, January 13, 2011
25. Core Graphics (Quartz)
• C-based API for 2D graphics
• Structs and functions start with CG…
• Some UIKit methods work with struct CGRect
• Most drawing functions (stroke, fill, set path, etc.)
start with with CGContext… and take a
CGContextRef as first parameter
Thursday, January 13, 2011
26. Core Animation
• Obj-C API for hardware-accelerated compositing,
rendering, and animation
• Don’t think of it as “just” animation
• CALayer is a presentable surface; every UIView is
backed by a layer (see UIView.layer property),
which it uses for rendering
• You could write a book on this stuff
Thursday, January 13, 2011
27. Core Text
• C-based API for advanced text layout and font
handling
• Most apps can just use Cocoa UITextView and
UIFont
• Can render text into CGContext
Thursday, January 13, 2011
28. Image I/O
• C-based API split out of Core Graphics
• Allows high-performance reading and writing of
many image formats
• JPEG, JPEG2000, RAW, TIFF, BMP, PNG, etc.
• Provides access to image metadata and color
management
Thursday, January 13, 2011
29. Assets Library
• Obj-C API to access photos and videos in users library
(c.f., “Photos” app)
• Introduced in iOS 4.0
• ALAssetsGroup provides ALAssets, which provide
ALAssetRepresentations, which provide
metadata, CGImageRefs, etc.
• Save new images to photo library via
ALAssetLibrary
Thursday, January 13, 2011
30. OpenGL ES
• C-based industry-standard API for 2D and 3D
rendering
• Not an Apple API, so code conventions are different
• All drawing is done to an EAGLContext object,
which you set up with a EAGLDrawable (the only
implementation of which is CAEAGLLayer).
• You could write a book on this too
Thursday, January 13, 2011
31. Media Player
• Obj-C API to access user’s iPod Library
• Audio only: music, audio podcasts, audiobooks
• Use MPMediaQuery to get MPMediaItems, then play
with MPMediaPlayer, or inspect metadata
properties
• MPMoviePlayerController offers a simple movie
player; AV Foundation is better.
Thursday, January 13, 2011
32. AV Foundation
• Obj-C API for audio and video capture, editing, and
playback, introduced in iOS 4.0
• iMovie for iPhone is apparently written with this
• Huge framework, comparable to QuickTime
• Can open audio URLs from Media Library
Thursday, January 13, 2011
33. Core Media / Core Video
• Core Media: C API to describe byte buffers, formats,
and time for AV Foundation
• CMTime struct includes units and timescale, so it is
always exact for a media-appropriate scale
• Core Video: C API to provide pixel buffers and image
buffers for AV Foundation
• Neither is directly applicable to app developers
outside of AV Foundation
Thursday, January 13, 2011
34. OpenAL
• C-based API for 3D spatialized sound
• Third-party API, designed to resemble OpenGL
• Create buffers of single-channel PCM, connect these
to sources, set properties on the sources (location,
orientation, etc.), then configure properties of a
listener
Thursday, January 13, 2011
35. Core Audio
• C API for low-latency audio processing (capture and
play-out to speaker or headphones)
• Provided utility classes are in C++
• Consists of two “engines” - Audio Units and Audio
Queues - along with convenience APIs for audio file I/
O, network I/O, format conversion, etc.
• OpenAL is implemented atop CA as an Audio Unit
• Legendarily hard to use, needs a book…
Thursday, January 13, 2011
36. Core MIDI
• Obj-C and C APIs for communicating with external
MIDI devices (musical instruments) over network or
via dock connector
• Introduced in iOS 4.2
• Does not provide a software synthesizer
Thursday, January 13, 2011
37. Cocoa Touch
Media Layer
Core Services
Core OS
Thursday, January 13, 2011
38. Foundation
• Obj-C API with essential collection of data types
(100+ classes, 20+ protocols) used throughout Cocoa
• Collections: NSArray, NSSet, NSDictionary
• Data objects: NSString, NSDate, NSCalendar
• Primitive wrappers: NSNumber, NSValue, NSData
• Many of these are immutable, have mutable
subclasses
Thursday, January 13, 2011
39. Foundation: Notifications
• NSNotification represents broadcastable data –
typically a “change” – as a name, source object, and
optional info dictionary
• Each app has an NSNotificationCenter to broadcast
notifications
• Mac OS X has an NSDistributedNotificationCenter
• Interested parties add themselves as observers of the
NSNotificationCenter for specific notification names
(and, optionally, sources)
Thursday, January 13, 2011
40. Foundation: URL Loading System
• Create an NSURL
• From this, create an NSURLRequest
• From this, create NSURLConnection, providing
delegate to receive NSURLResponse
• Classes also provided for caching, authentication
• Only works with file:, http:, https:, and ftp:
URLs.
Thursday, January 13, 2011
41. Foundation: Handy Stuff
• XML parsing: NSXMLParser (event-driven)
• Bonjour: NSNetService, NSNetServiceBrowser
• NSScanner: substring matching and extraction
• NSUndoManager
• NSKeyedArchiver, NSKeyedUnarchiver
• NSSortDescriptor: Used by sort methods in
NSArray and NSMutableArray
Thursday, January 13, 2011
42. Foundation’s C Stuff
• NSAssert…()
• NSLocalizedString…(): gets localized string from
app bundle
• NSLog()
• NSRange: Struct used for substrings in NSStrings.
Comes with helper functions like NSMakeRange(),
NSEqualRanges(), NSIntersectionRange()…
Thursday, January 13, 2011
43. Foundation: NSObject
• Polymorphism: isKindOfClass:,
respondsToSelector:, conformsToProtocol:
• Reference-counting: retain, release,
autorelease
• Fundamental rule: you own any object you create
with alloc, new, or copy, and must eventually
release (or autorelease) it.
Thursday, January 13, 2011
44. Core Foundation
• C API originally intended to provide common services
to Cocoa (Obj-C) and Carbon (C++) on Mac OS X
• Defines class-like “opaque types”; instances of these
are still called “objects”.
typedef const struct __CFString * CFStringRef;
• CF objects use same reference-counting scheme as
Foundation: CFRetain(), CFRelease(), but no
autorelease.
Thursday, January 13, 2011
45. Toll Free Bridging
• Many CF opaque types are effectively identical to
Foundation classes, and can be cast with zero cost
NSString *myString = @"My Foundation string";
CFStringRef myCFString = CFSTR ("My Core Foundation string");
CFStringRef hisString = (CFStringRef) myString;
NSString *hisCFString = (NSString*) myCFString;
Thursday, January 13, 2011
47. Not Toll-Free Bridged
From cocoadev.com:
NSBundle != CFBundle
NSHost != CFHost
NSRunLoop != CFRunLoop
NSNotificationCenter != CFNotificationCenter
NSSocket != CFSocket
Note: NSHost does not exist on iOS, and NSSocket doesn’t exist (or
isn’t public) on iOS or OS X.
Thursday, January 13, 2011
48. Core Foundation Conventions
• Core Foundation functions typically name the opaque type
they take as a first (or second) argument.
• Notice similarity between Foundation (Obj-C) and Core
Foundation calls:
! int i = [myString length];
! int j = CFStringGetLength(myCFString);
! NSArray *components = [myString componentsSeparatedByString:@" "];
! CFArrayRef cfComponents =
! ! CFStringCreateArrayBySeparatingStrings (kCFAllocatorDefault,
! ! ! ! ! ! ! ! ! ! ! ! ! ! myCFString,
! ! ! ! ! ! ! ! ! ! ! ! ! ! CFSTR(" "));
• Important difference here: you own and must
CFRelease() the cfComponents variable.
Thursday, January 13, 2011
49. CF Exclusives
• Some Core Foundation APIs have no equivalent classes
or similar functionality in Foundation
• CFBagRef: unordered set that allows duplicates
• CFStringTokenizerRef (compare to NSScanner)
• CFTreeRef: tree-structured data
• CFUUIDRef: Universally Unique Identifier (RFC
4122)
Thursday, January 13, 2011
50. CFNetwork
• Much deeper networking API than Foundation’s URL
Loading System
• Provides CF-friendly abstraction over BSD sockets
(client or server)
• APIs for DNS host name lookup, read and write
streams
• Also has a CF version of Bonjour
Thursday, January 13, 2011
51. Threads
• NSThread provides Obj-C semantics for executing code
in a separate thread of execution
• Subclass NSThread and override main, or use
initWithTarget:selector:object:
• All UIKit code should be running on the “main” thread.
• See +[NSThread mainThread], +[NSThread
isMainThread], +[NSObject
performSelectorOnMainThread:withObject:w
aitUntilDone:]
Thursday, January 13, 2011
52. Run Loops
• An event processing loop attached to a single thread
• Inputs to the loop come from NSPorts (distributed
messaging), NSTimers, custom sources, and
performSelector:onThread:… calls.
• Main thread’s run loop is set up by UIApplication.
You need to create your own for custom threads.
• If a method asks for a run loop “mode”, you almost
always want NSDefaultRunLoopMode (aka
kCFRunLoopDefaultMode)
Thursday, January 13, 2011
53. NSOperation
• Abstract class that defines a single task that may be
run concurrently, or as a consequence of one or more
other operations completing
• Generally used with an NSOperationQueue, which
will determine what thread to call your operation on
Thursday, January 13, 2011
54. Blocks
• Closures for C — code plus stack and heap variables.
• Allows you to pass state and the code to work with it
• New Apple frameworks prefer blocks to delegates and
other asynchronous design patterns (e.g., take a
completionHandler block argument, to be executed
when the asynchronous task completes)
• Can/should create an NSOperation with a block
• Hope you saw Daniel Steinberg’s session earlier
Thursday, January 13, 2011
55. Grand Central Dispatch
• Multicore-savvy programming for Mac and iOS, in C
• Based on a “queue of blocks” metaphor
• Can dispatch your blocks on main queue (main thread),
concurrently, or serially
• Can group blocks and get notification when they complete,
even if they run on different threads
• GCD is defined in Core Services, but typically used directly
only by lower-level programmers (consider NSOperation
instead)
Thursday, January 13, 2011
56. System Configuration
• Another C API. Only iOS functionality:
SCNetworkReachability
• Synchronous check:
SCNetworkReachabilityGetFlags()
• Asynch: SCNetworkReachabilitySetCallback()
• Apple always tests what your app does when the
network goes away (e.g., Airplane Mode). You will be
rejected if your app bricks without network.
Thursday, January 13, 2011
57. Other Apple Core Services
• Store Kit: Obj-C API for In-App Purchasing
• Core Location: Obj-C API for determining current
location from device features (GPS, cellular radio),
reckoning course, distances, etc.
• Quick Look: Obj-C API allows apps to provide
thumbnails of documents for previewing
• Address Book: C API to access names, addresses, …
• Core Data: Obj-C API for object-relational mapping
Thursday, January 13, 2011
58. 3rd Party Core Services
• SQLite3: C API for simple file-based relational
database
• Libxml2: C API for XML parsing, DOM-style or SAX-
style
• Documentation for these are on their respective
websites, not in Xcode
Thursday, January 13, 2011
59. Cocoa Touch
Media Layer
Core Services
Core OS
Thursday, January 13, 2011
60. Depth Check
You Are
Here
Thursday, January 13, 2011
61. Accelerate
• Hardware-accelerated math, big number, and DSP
functions, in C
• LAPACK (Linear Algebra Package) – high-level LA,
solve linear systems, Eigenvalues and Eigenvectors
• BLAS (Basic Linear Algebra Subroutines) – low-level
LA, matrix, matrix-vector, matrix-matrix
• vDSP – basic math on arrays, convolution and
correlation, Fast Fourier Transform
• float faster than double, esp. on ARMv7
Thursday, January 13, 2011
62. External Accessory
• Requires paid membership in “Made for iPod /
Works with iPhone” program
• If you thought App Store was picky…
• Three Obj-C classes and one delegate protocol:
Iterate over available wired and Bluetooth
EAAccessory objects with EAAccessoryManager,
create an EASession and access its inputStream
and outputStream.
Thursday, January 13, 2011
63. Keychain Services
• API to store user passwords, purchase history, other
sensitive data.
• C functions to query, add, update, delete CFDictionary
objects containing key-value pairs
• Each dictionary is one user item. Pairs are things like an
account name, the item type, the item’s ID, the item’s
value, etc.
• Keychain survives app deletion and re-install.
• “Friendly” apps can share Keychain data
Thursday, January 13, 2011
64. Other Security Services
• C functions for managing certificates, public and private
keys, representing trust policies
• Match certificates to private keys
• Create and request certificate objects
• Import certificates, keys, and identies
• Create public-private key pairs
• Randomization Services provides crypto-secure random
numbers
Thursday, January 13, 2011
65. System
• Low-level UNIX interfaces
• No third-party access to kernel or drivers
• No Xcode documentation (other than code
completion)
• Investigate man pages, headers in <iOS_SDK>/usr/
include, or web-based documentation
Thursday, January 13, 2011
66. man page
Yuna:~ cadamson$ man memset
MEMSET(3) BSD Library Functions Manual MEMSET(3)
NAME
memset -- fill a byte string with a byte value
LIBRARY
Standard C Library (libc, -lc)
SYNOPSIS
#include <string.h>
void *
memset(void *b, int c, size_t len);
DESCRIPTION
The memset() function writes len bytes of value c (converted to an
unsigned char) to the byte string b.
RETURN VALUES
The memset() function returns its first argument.
SEE ALSO
bzero(3), memset_pattern(3), swab(3)
:
Thursday, January 13, 2011
67. man page
Yuna:~ cadamson$ man memset
Thursday, January 13, 2011
68. System - C Standard Library
• stdio.h – Standard I/O: open, close, get data from / put
data to FILEs (not necessarily a flat file)
• stdlib.h – General-purpose functions: malloc()/
free(), type conversions, qsort(), etc.
• string.h & wchar.h — C string utilities
• math.h – Basic math constants and functions
• And more… http://en.wikipedia.org/wiki/
C_standard_library or just read K&R.
Thursday, January 13, 2011
69. System - BSD Sockets
• sys/socket.h, netinet/in.h, netinet6/in6.h, etc.
• Create socket with socket()
• Client-side: connect(), then send()/recv() or
read()/write()
• Server-side: bind(), accept(), then send()/recv
() or read()/write()
Thursday, January 13, 2011
70. System - POSIX Threads
• pthread.h – functions for creating and using threads,
thread-safety via mutexes, conditions, and
synchronization
• Create thread with pthread_create(), passing a
pthread_t* to receive the created thread, thread
attributes (NULL for default), function pointer to run,
and void* to pass as argument to the function
• Can use sys/semaphore.h as an alternate thread-
safety mechanism
Thursday, January 13, 2011
71. Other neat stuff in /usr/include
• 150 files and directories, including
• zlib.h, tar.h – Data compression
• CommonCrypto/ – MD5, SHA, other crypto
• regex.h – Regular expressions (Standard C library)
• asl/ – Apple System Log access
• However, not everything in /usr/lib has a public
header (e.g., libbz2.dylib, libtidy.dylib)
Thursday, January 13, 2011
72. Where to?
Cocoa Touch
Media Layer
Core Services
Core OS
Thursday, January 13, 2011
73. Portability considerations
• POSIX code is easy to port to/from iOS
• Con: C strings are ASCII, stdio & inet calls block
• Core Foundation doesn’t port
• But data types are much richer, network and I/O
APIs are asynchronous by design
• How far does POSIX get you on mobile anyways
(Android, PalmOS, Windows Phone 7)?
Thursday, January 13, 2011
74. Developer considerations
• Most iOS developers did not start as C programmers
• Came from Java, JavaScript, Windows, etc.
• Can be tricky to context-switch between Obj-C and C
• For switchers, Foundation is easier than Core
Foundation, and Core Foundation is easier than
standard C libraries
Thursday, January 13, 2011
75. What’s Next
Cocoa Touch
Media Layer
Core Services
Core OS
Thursday, January 13, 2011
76. Future directions
• Much of Mac OS X has now been ported to iOS.
Interesting stuff that hasn’t been:
• Speech synthesis/recognition (c.f., App Kit)
• Already available to Apple’s apps
• Ink services (handwriting recognition)
• WebServicesCore (XML-RPC, SOAP)
• Non-hacky resolution independence
Thursday, January 13, 2011
77. Q&A
• invalidname [at] gmail [dot] com
• @invalidname
• http://www.subfurther.com/blog
Thursday, January 13, 2011