Protobuf overview. For python and other

1. Protobuf it!
Александр Мокров

2. Serialization
/
Deserialization

3. Python
Go
C++
C#
JavaScript

4. W3C

5. Annotea
CC/PP
Compound Document
Formats
CSS
DOM
HTML
HTTP
InkML
MathML
OWL
PICS
PNG
P3P
RDF
SMIL
SPARQL
Style
SVG
TAG
Timed Text
URI/URL
Voice Browser
WAI
WAI-ARIA
WebCGM
Web Services

6. XML

7. <?xml version="1.0"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="note">
<xs:complexType>
<xs:sequence>
<xs:element name="to" type="xs:string"/>
<xs:element name="from" type="xs:string"/>
<xs:element name="heading" type="xs:string"/>
<xs:element name="body" type="xs:string"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>

8. XML
XPath XSLT XQuery
XInclude
XSDXLink
XML Schema
DTD
XKMS
XML
Encryption
XPointer
XHTML
XForms

9. Google

10. XML but smaller,
faster and simpler

11. Protocol Buffers

12. C++
Java
Python
Objective-C
C#
JavaNano
JavaScript
Ruby
Go
PHP
Dart

13. Twitter

14. Protobuf vs XML
are simpler
are 3 to 10 times smaller
are 20 to 100 times faster
are less ambiguous
generate data access classes that are easier to
use programmatically

15. message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
enum PhoneType {
MOBILE = 0;
HOME = 1;
WORK = 2;
}
message PhoneNumber {
required string number = 1;
optional PhoneType type = 2 [default = HOME];
}
repeated PhoneNumber phone = 4;
}

16. message Person {
string name = 1;
int32 id = 2;
string email = 3;
enum PhoneType {
MOBILE = 0;
HOME = 1;
WORK = 2;
}
message PhoneNumber {
string number = 1;
PhoneType type = 2;
}
repeated PhoneNumber phone = 4;
}

17. protoc -I=$SRC_DIR --python_out=$DST_DIR $SRC_DIR/person.proto
↓
person_pb2.py
>>> from person_pb2 import Person
Compiling

18. Person = _reflection.GeneratedProtocolMessageType('Person',
(_message.Message,), dict(
PhoneNumber =
_reflection.GeneratedProtocolMessageType('PhoneNumber',
(_message.Message,), dict(
DESCRIPTOR = _PERSON_PHONENUMBER,
__module__ = 'person_pb2'
# @@protoc_insertion_point(class_scope:Person.PhoneNumber)
)),
DESCRIPTOR = _PERSON,
__module__ = 'person_pb2'
# @@protoc_insertion_point(class_scope:Person)
))

19. _PERSON = _descriptor.Descriptor(
name='Person',
full_name='Person',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='name', full_name='Person.name', index=0,
number=1, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None, options=None),

20. from person_pb2 import Person
person = Person()
person.id = 1
person.name = "Vasya Pupkin"
person.email = "vasya@example.com"
phone = person.phones.add()
phone.number = "+7 499 270-27-27"
phone.type = Person.HOME

21. >>> person
name: "Vasya Pupkin"
id: 1234
email: "vasya@example.com"
phones {
number: "+7 499 270-27-27"
type: HOME
}
phones {
number: "2702727"
type: WORK
}

22. >>> data = person.SerializeToString()
>>> data
b'nx0cVasya
Pupkinx10xd2tx1ax11vasya@example.com"x14nx10
+7 499 270-27-27x10x01"x0bnx072702727x10x02'
>>> person = Person.FromString(data)
>>> person = Person()
>>> person.ParseFromString(data)

23. Any
import "google/protobuf/any.proto";
message ErrorStatus {
string message = 1;
repeated google.protobuf.Any details = 2;
}

24. OneOf
message SampleMessage {
oneof test_oneof {
string name = 4;
SubMessage sub_message = 9;
}
}

25. Map
map<string, uint32> projects = 5;
>>> person.projects['one'] = 1
>>> person
projects {
key: "one"
value: 1
}
>>> person.projects['one']
1

26. Library
import "google/protobuf/timestamp.proto";
import "google/protobuf/empty.proto";
message MyMessage {
google.protobuf.Timestamp my_field = 1;
}
service Foo {
rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty);
}

27. JSON Mapping
>>> from google.protobuf.json_format import MessageToJson
>>> json_string = MessageToJson(person)
>>> json_string
'{n "name": "Vasya Pupkin",n "email":
"vasya@example.com",n "phones": [n {n "number":
"+7 499 270-27-27",n "type": "HOME"n },n {n
"number": "2702727",n "type": "WORK"n }n ],n "id":
1234n}'

28. >>> from google.protobuf.json_format import Parse
>>> person = Person()
>>> Parse(json_string, person)
>>> person
name: "Vasya Pupkin"
id: 1234
email: "vasya@example.com"
...

29. Unknown Fields
message Foo {
reserved 2, 15, 9 to 11;
reserved "foo", "bar";
}

30. RPC
service SearchService {
rpc Search (SearchRequest) returns (SearchResponse);
}

31. gRPC

32. ● Быстрый
● Компактный
● Позволяет формализовать данные
● Бинарный
● Лаконичный
● Поддерживается всеми распространенными
языками
● Позволяет обеспечить совместимость со

33. Ссылки
https://github.com/google/protobuf
https://developers.google.com/protocol-buffers
http://www.computerworld.com/article/2513433/app-development/twitter-solves-
its-data-formatting-challenge.html

34. Спасибо за внимание!

35. Вопросы?