The Japanese government has adopted Drupal8 as a portal site for huge numbers of statistical information. The site was renewed on January 4, 2018 Portal Site of Official Statistics of Japan  https://www.e-stat.go.jp/en https://www.e-stat.go.jp/ e-Stat is a resource with huge amounts of data.  It is one of the world's leading statistical portals boasting terabytes of data.  There was challenge as to how to make the huge preexisting back-end system built with Java＋Oracle compatible with the front end system which can be updated by the administrator. Drupal was used to solve this issue.  Drupal uses the new Decoupled as a front end controller. We used its architecture, developing a user friendly interface and augmentability, effectively applying Drupal's merits to the fullest to create a seamless integration.  These case studies demonstrate that Drupal will have huge impacts with its role in integrating large systems and web systems. We hope to convey this to the Drupal Community around the world.

1. The Japan Government has adopted
Drupal8 for its official statistics system.
April 11th 2018 DrupalCon NASHVILLE
ANNAI Inc. Co-founder & CEO Satoshi Kino

2. e-Stat: What is it
2

3. 3
is a Drupal development agency
● Founded in 2007
● Started with Drupal 5
● HQ in Kyoto, Japan
● Team of 6 people working remotely

4. A different angle - an unconventional approach to
decoupled Drupal
Hello Kitty ©1976 SANRIO CO., LTD. BABY METAL 4

5. e-Stat : What is it?
5

6. v Intended purpose: become a one-stop portal
for Japanese government statistics
v A huge system containing all statistics
for all government ministries
v A repository of all national census data.
(currently 625 statistics, 1.3 million files published)
e-Stat : What is it?
6

7. e-Stat : First, the previous site
7

8. v Create a smooth user experience for users
to search and display data in an easy to understand format
v Reduce costs for site updates, maintenance,
and adding new functionality
e-Stat : Goals of the Site Rebuild
8

9. Challenges
9

10. v Massive JAVA + Oracle back end.
• Front and back ends not decoupled
10
Challenges
• High cost for small/simple changes
• Very very slow

11. • Menu based site UI/Navigation
• A huge number of statistical data
(currently 625 statistics, 1.3 million files published)
v Outdated UX
Difficult to find what one was looking for
11
Challenges

12. Challenges
v Development timeline: max 18 months
12

13. 13
Challenges
v We considered the DKAN distribution,
but determined it didn't meet our requirements.

14. 14
Challenges
v The government requires large agencies
v Large agencies in Japan have little or no Drupal
experience

15. v Features and Configuration Management were not yet
ready for production sites in Drupal 8.
15
Challenges

16. 16
Challenges
v Considered Java back-end with decoupled JavaScript
front-end for search and display
Various issues arose・・・

17. v Considered Java back-end with decoupled JavaScript
front-end for search and display
• Requests were slow
17
Challenges

18. v Considered Java back-end with decoupled JavaScript
front-end for search and display
• JavaScript front-end lost Drupal admin functionality
18
Challenges
- Multilingualism
- User authentication

19. v Considered Java back-end with decoupled JavaScript
front-end for search and display
• Difficult to utilize Drupal’s cache
19
Challenges

20. v Considered Java back-end with decoupled JavaScript
front-end for search and display
• Additional layer increased maintenance overhead
20
Challenges

21. v Government required 8 active servers spread out
over two data centers (in case of natural disasters)
21
Challenges

22. v All the previous issues meant a full replacement of the
existing Java + Oracle system could not be achieved
with Drupal alone
22
Challenges

23. Solution
23

24. Solution
v We went with a reverse decoupling
• Standard decoupled system integrate JavaScript
front end with Drupal back end
24

25. v We went with a reverse decoupling
• Drupal front end UI
25
Solution
• Java + Oracle for the back-end

26. MENU
NEWS
CONTENTS AREA
SEARCH
26
Custom menu
module
Custom news
module
Custom search
module
Drupal contents
module
twig
twig
twig
API Bridge pattern / Drupal Design Pattern
Java
Java
Java
Java
twig
● Business logic is handled in back end.

27. MENU
NEWS
CONTENTS AREA
SEARCH
27
Custom menu
module
Custom news
module
Custom search
module
Drupal contents
module
twig
twig
twig
API Bridge pattern / Drupal Design Pattern
Java
Java
Java
Java
twig
● Each module uses REST API to pull data from the data storage.

28. MENU
NEWS
CONTENTS AREA
SEARCH
28
Custom menu
module
Custom news
module
Custom search
module
Drupal contents
module
twig
twig
twig
API Bridge pattern / Drupal Design Pattern
Java
Java
Java
Java
twig
● Modules were created for each block and URL.

29. MENU
NEWS
CONTENTS AREA
SEARCH
29
Custom menu
module
Custom news
module
Custom search
module
Drupal contents
module
twig
twig
twig
API Bridge pattern / Drupal Design Pattern
Java
Java
Java
Java
twig
● Twig templates render HTML.

30. MENU
NEWS
CONTENTS AREA
SEARCH
30
Custom menu
module
Custom news
module
Custom search
module
Drupal contents
module
twig
twig
twig
API Bridge pattern / Drupal Design Pattern
Java
Java
Java
Java
twig
● Changes to layout etc. done through the Drupal UI

31. 31
System Architecture
/search?keyboard=abc&tag=1
Custom
module
Drupal Java API Solr
API Call
Result
Solr query
Solr result
class JavaSearchApiBridgeController extends ControllerBase {
① Query parameter normalization
② Searches Drupal cash. If a hit is found, a Twig template is used for display
③ If no hit is found, the JAVA API makes a call to Solr
④ The response is cached. The parameters are used to create a cache key for step 1
⑤ Twig is used to display the response
}

32. 32
DNS
Load Balancer
1st data center
IP based
routing
2nd data center
IP based
routing
Galera cluster(database)
instance
CMS server
instance
Internet
Virtual Network
Network system
DB Clustering

33. Merits
33

34. Merits
v Rapid Development
• Drupal modules
• Twig templates
• Result = Developers without Drupal experience
could work on the project
34

35. v Java integration with Solr overcame limits of
Drupal Search API
35
Merits

36. v Drupal's theming layer allows changes
independent of Java backend
36
Merits

37. v Drupal caching and BigPipe module created
fast front-end experience
37
Merits

38. v Drupal permissions and block visibility,
allow administrators to independently maintain the site
38
Merits

39. v Drupal pulls news and other contents
from 3rd party APIs
39
Merits

40. v Drupal CMS used for:
40
• Creating help pages
• Contact forms
• News
• Multilingualism
Merits

41. ANNAI and Drupal
41

42. ANNAI and Drupal
v We don't look at Drupal purely as a CMS,
but as a road on which to move forward
42

43. v Drupal as an enterprise level CMS is great
v More than a CMS, it’s a framework
v For us, Drupal is a Data Management Platform
43
ANNAI and Drupal

44. v Drupal has matured into a full platform, much like Intel,
Linux, PHP and Javascript
44
ANNAI and Drupal

45. v We liked the atmosphere of "Do it with Drupal"
in Drupal 5 and 6
• Drupal is almost like stem-cells – a tool that can be
adapted to whatever your needs may be
45
ANNAI and Drupal

46. v We liked the atmosphere of "Do it with Drupal"
in Drupal 5 and 6
• Open source software is great because
it's fun and easy to step in to
46
ANNAI and Drupal

47. v We liked the atmosphere of "Do it with Drupal"
in Drupal 5 and 6
• In addition to being a fun hobby, it's also important
to keep new people coming in who can imagine
new ways to use Drupal, and further its
development
47
ANNAI and Drupal

48. v We need to be careful to not become complacent
48
ANNAI and Drupal

49. v Improvement is important, but we also need to foster
opportunities for innovation
49
ANNAI and Drupal

50. v For e-Stat, we used a reverse decoupled system
v Let’s find more ways to use Drupal in a fun,
innovative manner
50
ANNAI and Drupal

51. Moving Forward
51

52. Moving Forward
v We're experimenting with using Drupal as the
backend of an application system for grants and
subsidies.
52
v The creation of an API for government data
v Pulling data into a SPARQL server using RDF

53. Working Together
v ANNAI is a small regional company,
with staff working remotely
53
• We do large-scale development
• We benefit from the Drupal community

54. v Giving back to the community
54
Working Together
• Putting together Japan meetups
• Drupal camps for Asia
• Creating contents for new developers in Japanese
• Wrote a book for site builders in Japanese

55. And Finally...
55

56. Work With Us!
v We are always looking for Drupal developers,
so if you have an interest in working in Japan,
please drop us a line!
56
Software Engineer DevOps EngineerFrontend Engineer

57. Q&A
57

58. Thank you!
ANNAI Inc.
https://annai.co.jp/en
support@annai.co.jp