Everybody knows that it is hard to scale old highload monolithic projects that use pythonic templates for frontend. I am gonna tell how we transformed our product using trending and proper technologies like GraphQL, Apollo, Node.js with limited developer resources in a short period of time.

1. How to separate frontend
from a highload python
project with no problems
Oleksandr Tarasenko
EVO.company / prom.ua

2. Some prom.ua numbers
● RPS 2000-3000
● users over 2.5 million per day
● paid sites ~ 50 000
● total sites over 300 000
● products ~ 100 million
● pages ~ 300 million
2

3. What we had
● monolithic WSGI app (10 years)
● mako templates
● monolithic database
● monolithic Nginx
● poor REST API
● slow Delivery and Deployment
● new features were hard to develop
3

4. 4
1. Resolve url
2. Collect base context
3. Call render_mako
4. Collect template context
5. Return html

5. Our goals
● SEO
● Independent frontend
● Good user experience
● Documentation
● Data validation
● Microservices way
● Independent CI/CD
5

6. Tools and expertise
● ReactJS / Redux
● GraphQL server (hiku)
● Container system (vagga/lithos)
● Metrics (grafana)
● Logs (kibana)
6

7. GraphQL + React + Apollo + Node.js =
success story
7

8. 8
GraphQL

9. http://graphql.org/users/
9

10. Why we choose GraphQL
● good for readonly requests and data
● mutations to REST
● every site has unique user settings
● different templates
10

11. Some prom.ua sites code facts
● a lot of views and url endpoints
● almost no documentation
● poor versioning
● a lot of template context
● duplicating queries
11

12. We built two-level graph
● low-level graph for database mapping
(data loading)
● high-level graph for business logic
● auto documentation with graphiql tool
● data validation
12

13. product_query = sa.FieldsQuery('db.session', Product.__table__)
low_level_graph = Graph([
Node('Product', [
Field('id', Integer, product_query),
Field('name', String, product_query),
Field('price', None, product_query),
])
13

14. """
SELECT id, name, price FROM product
WHERE id IN (:id1, :id2, ..., :idN)
"""
product(ids: [1234, 1235]) {
id
name
price
}
14

15. low_level_graph = Graph([
Node('Product', [
Field('id', Integer, product_query),
...
Link(
'discount',
Optional[TypeRef['Discount']],
product_to_discount_query,
requires='id',
),
]),
Node('Discount', [
Field('id', Integer, discount_query),
Field('amount', Integer, discount_query),
])
])
15

16. 16
discount_query = sa.FieldsQuery('db.session', Discount.__table__)
"""
SELECT id, name, price, price_currency_id FROM product
WHERE id IN (:id1, :id2, ..., :idN)
"""
product_to_discount_query = sa.LinkQuery(
'db.session',
from_column=Discount.product_id,
to_column=Discount.product_id,
)
"""
SELECT product_id FROM discount
WHERE product_id IN (:id1, :id2, ..., :idN)
"""

17. 17
product_sg = SubGraph(low_level_graph, 'Product')
high_level_graph = Graph([
Node('Product', [
Field('id', String, product_sg),
Field('name', String, product_sg.compile(S.this.name)),
Field('priceText', String, product_sg.compile(
get_price_text(S.this)
)),
Field('hasDiscount', Boolean, product_sg.compile(
is_discount_available(S.this, S.this.discount)
)),
Field('discountedPriceText', String, product_sg.compile(
get_discounted_price_text(S.this, S.this.discount)
)),
]),
])

18. 18
@define(Record[{
'price': Float,
'price_currency_id': Integer,
'currency_settings': Any
}])
def get_price_text(product):
product_price_text = format_currency_data(
product_price,
product['price_currency_id'],
)
return product_price_text.formatted_number

19. Some facts of the new graph
● number of fields: ~ 500
● number of links: ~ 125
● number of nodes: ~ 75
● single entry point /graphql
● refactoring and new vision for code
● better monitoring
● easy to test API with graphiql
19
Hiku

20. 20
Frontend

21. Stateless ReactJS
● React Apollo
● React Router
● React Helmet
● No Redux/Flux/State management lib
21

22. Apollo client benefits
● great GraphQL implementation
● no sugar in React components
● query batching
● state cache
● community
● combine | join graphs (Apollo server)
22

23. 23
export const Article = ({ data }) => {
if (data.loading) return <Spinner />;
if (!data.article.id) return <NotFound />;
return (
<BasePage>
<ArticleViewMeta id={data.article.id} />
<BaseViewComponent
{...data.article}
/>
<Footer />
</BasePage>
);
};

24. 24
const articleViewQuery = gql`
query getArticleViewData($id: String) {
article(id: $id){
id
title
text
}
}`;
export const ArticleWithData =
graphql(articleViewQuery, {
options: ({ id }) => {
return { variables: { id } };
},
})(Article);

25. Other javascript implementations
● Relay modern / Relay classic
● Apollo client / Apollo server / Apollo engine
● Lokka
● Prisma
● GraphQL.js
● express-graphql
25

26. 26
Node.js

27. Node.js implementation
● Only for the first request (SSR)
● Good for React prerender
● Routing
● Two-step implementation
27

28. 28

29. 29

30. 30

31. 31
Nginx

32. 32

33. 33

34. Success product metrics
34
from 4.8 to 7.1

35. Success product metrics
35
from 57.5% to 49%

36. Some numbers of the new scheme
● node.js workers 8
● React render 5 ms
● prom.ua workers over 750
● number of requests split to node/python
36

37. 37
Problems

38. 38
Node.js leaks

39. 39
Node.js leaks
:)

40. 40
Node.js leaks
;(

41. Sometimes it’s hard to analyze js
errors from graph and Apollo
41

42. 42
Hard to split routing
in monolithic app

43. 43
Some resources and
logic need duplication

44. Plans
● GraphQL everywhere
● Backend as API
● Progressive web apps
● More frontends
44

45. 45
GraphQL
REST API
Questions/Answers