The document discusses the advantages of using binary formats like FlatBuffers and FlexBuffers over JSON for serialization. It notes that binary formats have smaller payload sizes and faster encoding/decoding speeds which can significantly improve performance for machine to machine communication and state persistence. The document demonstrates how FlatBuffers and FlexBuffers can be used to efficiently store and access structured data like user profiles and city data. It concludes that developers should evaluate their use cases and consider binary formats instead of only using JSON because others do.

1. Talk Binary to Me
Maxim Zaks
@iceX33

2. What’s going to happen
here???
• Rant about JSON
• Explain why binary formats are better
• Save the world

3. I love JSON

4. Is this JSON Valid?

5. Is this JSON Valid?

7. Is this JSON Valid?

8. 0
22,5
45
67,5
90
JSON JSON minified FlexBuffers FlatBuffers colfer
22
40
70
66
81
Size in bytes

9. WTF is colfer?

10. Colfer is binary serialization format
optimized for speed and size
https://github.com/pascaldekloe/colfer

11. https://github.com/eishay/jvm-serializers/wiki
Ser Time+Deser Time (ns)

12. {
"name":"Maxim Zaks",
"birthday":{
"year":1981,
"month":6,
"day":12
}
}

13. {
"name":"Maxim Zaks",
"birthday":{
"year":1981,
"month":6,
"day":12
}
}
package demo
type Person struct {
name text
birthday Date
}
type Date struct {
year uint16
month uint8
day uint8
}

14. 0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>

15. 0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>

16. 0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>

17. 0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>

18. 0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>

19. 0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>

20. 0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>

21. 0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>

22. 0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>

23. 0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>

24. 0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>

25. 0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>

26. 0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>

27. GENERATED CODE

28. colfer all the things?

29. Data serialisation use cases
• Machine to Machine communication
• State persistence
• Representation of configurations or data dumps

30. Machine to Machine
Requirements
• Small payload
• Efficient encoding & decoding
• Formalised / Typed API
• Versioning & compatibility

31. colfer
• Small payload 👍
• Efficient encoding & decoding 👍
• Formalised / Typed API 👍
• Versioning & compatibility 👎

32. can we do better?

33. FlatBuffers is an efficient cross platform
serialization library for games and other
memory constrained apps.
https://github.com/google/flatbuffers

34. Swift port from yours truly
https://github.com/mzaks/
FlatBuffersSwift

35. 0
1500
3000
4500
6000
JSONSerialization FlexBuffersSwift FlatBuffersSwift
17335
4.771
559
848
5.794
encode in ms decode in ms
https://github.com/mzaks/FlexBuffersSwift

36. 0
125
250
375
500
JSONSerialization FlexBuffersSwift FlatBuffersSwift
0,0111,563
235,762
77
102
436
encode in MB decode in MB
https://github.com/mzaks/FlexBuffersSwift

37. package demo
type Person struct {
name text
birthday Date
}
type Date struct {
year uint16
month uint8
day uint8
}
table Person {
name : string;
birthday : Date;
}
struct Date {
year : short;
month : byte;
day : byte;
}
root_type Person;

38. root table: 12
vTable size: 8
table size: 12
property offsets:
1: 4
2: 8
vTable: 8
name offset: 8
date inline:
1981
6
12
name:[10]Maxim Zaks[0]

39. root table: 12
vTable size: 8
table size: 12
property offsets:
1: 4
2: 8
vTable: 8
name offset: 8
date inline:
1981
6
12
name:[10]Maxim Zaks[0]

40. root table: 12
vTable size: 8
table size: 12
property offsets:
1: 4
2: 8
vTable: 8
name offset: 8
date inline:
1981
6
12
name:[10]Maxim Zaks[0]

41. root table: 12
vTable size: 8
table size: 12
property offsets:
1: 4
2: 8
vTable: 8
name offset: 8
date inline:
1981
6
12
name:[10]Maxim Zaks[0]

42. root table: 12
vTable size: 8
table size: 12
property offsets:
1: 4
2: 8
vTable: 8
name offset: 8
date inline:
1981
6
12
name:[10]Maxim Zaks[0]

43. root table: 12
vTable size: 8
table size: 12
property offsets:
1: 4
2: 8
vTable: 8
name offset: 8
date inline:
1981
6
12
name:[10]Maxim Zaks[0]

44. FlatBuffers
• Small payload 👍*
• Efficient encoding & decoding 👍*
• Formalised / Typed API 👍
• Versioning & compatibility 👍
* not as small/fast as colfer

45. Let’s top that!

46. Data serialisation use cases
• Machine to Machine communication
• State persistence
• Representation of configurations or data dumps

47. Application state is interconnected

48. table Message {
...
}
table MessageLink {
message : Message;
nextMessages : [MessageLink];
}
table History {
seenMessages : [Message];
nextMessageLink : MessageLink;
timeStamp : double;
}
root_type History;
http://resiapp.io

49. Data serialisation use cases
• Machine to Machine communication
• State persistence
• Representation of configurations or data
dumps

50. Random value access is KING!

51. Show case:
find city by name or country code
https://github.com/mzaks/FindCityFB

52. Data dump of all cities in the world with
country code, name, normalised name and
map coordinates
150 MB as CSV
177 MB as FlatBuffers binary

53. table CityList {
cityByCountryCode : [City];
cityByName : [City];
}
table City {
countryCode : string;
searchName : string;
name : string;
latitude : float;
longitude : float;
}
root_type CityList;

54. https://youtu.be/P5tRwY39_JU

55. 45MB214MB

56. table CityList {
cityByCountryCode : [City];
cityByName : [City];
}
table City {
countryCode : string;
searchName : string;
name : string;
latitude : float;
longitude : float;
}
root_type CityList;
by Name
by Country Code

57. Are we done?

58. There was one more format

59. 0
1500
3000
4500
6000
JSONSerialization FlexBuffersSwift FlatBuffersSwift
17335
4.771
559
848
5.794
encode in ms decode in ms

60. Schema-less binary encoding
http://google.github.io/flatbuffers/
flexbuffers.html
https://github.com/mzaks/FlexBuffersSwift

61. Benefits of FlexBuffers
• No schema, no code generation needed
• Random value access
• Can easily transform JSON to FlexBuffers
• Supports every type JSON does and a bit more
• Keeps type information in the binary

62. let data = FlexBuffer.dataFrom(jsonData:
"{name:"Maxim", birthday:{"year": 1981, month: 6, day: 12}}"
.data(using: .utf8)!
)
let accessor = FlexBuffer.decode(data:data)
let name = accessor?["name"]?.asString
let day = accessor?["birthday"]?["day"]?.asInt

63. Outlook
• Don’t use JSON just because other people do, think
about your use case
• Imagine how much more responses your server could
deliver if it took 200x less time to process a request
• Start thinking in Graphs not Trees
• please ask questions after the session
• … and btw. I like to collaborate 😉 @iceX33

64. References
• All Gifs are powered by giphy.com
• JSON number grammar diagrams taken from json.org
• Binary Visualiser „HEX Fiend“ by ridiculous_fish http://ridiculousfish.com/hexfiend/
• https://github.com/eishay/jvm-serializers/wiki
• https://github.com/pascaldekloe/colfer
• https://github.com/google/flatbuffers
• https://github.com/mzaks/FlatBuffersSwift
• https://github.com/mzaks/FlexBuffersSwift
• https://github.com/mzaks/FindCityFB
• http://resiapp.io