Weitere ähnliche Inhalte
Ähnlich wie FOSDEM19 MySQL Component Infrastructure (20)
Mehr von Georgi Kodinov (20)
Kürzlich hochgeladen (20)
FOSDEM19 MySQL Component Infrastructure
- 1. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
MySQL 8.0
Component Infrastructure
Georgi “Joro” Kodinov
Team Lead, MySQL SrvGen Team
Why ? What's in it ? What's next ? How to use it ?
- 2. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Georgi “Joro” Kodinov, MySQL @ Oracle
Server General Team Lead
Works on MySQL since 2006
Specializes in:
Security
Client/server protocol
Performance monitoring
Component infrastructure
Loves history, diverse world cultures, gardening
A devoted Formula 1 fan (Go, Vettel !)
- 3. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Agenda
3
Why ?
Architecture
The inventory
How to write my own …
Tips & tricks
What’s next ?
Homework !
- 4. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Components: Why ?
• Too much technical debt accumulated in plugins
• Simpler and more extensible infrastructure
• Better code isolation and encapsulation
• Explicit dependencies
• All components are equal: can call and can be called
4
- 5. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Component Infrastructure Architecture
5
- 6. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Server Component
Server Binary
Server Functionality
Persisted Loader
File Scheme
Implementation
MySQL Server Modularization: The Big Picture
The Minimal Chassis
Implementation 1
Implementation 2
…
Registry
Component 1
Component 2
…
Dynamic Loader
Component 1
Component 2
Component 3
Implementation 1
Implementation 2
Plugins
Plugin
Services
Plugin
APIs
- 7. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Components: Terminology
7
Component
Code
Service
Implementations
Contains
Services
Implement
Registry
Register
Service
References
Dynamic
Loader
Load
Unload
Persisted Dynamic
Loader
Encapsulate
INSTALL COMPONENT
UNINSTALL COMPONENT
UDFs
Performance Schema
System Variables, etc.
- 8. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Basic Concepts: A Component
• A container for code.
– Can be an executable or a shared library
– As a compromise it can be a static library too, but that’s making the lines fuzzy, so NO
• Has init() and deinit() methods (called by the dyloader)
– Macros always add a registry reference
• Can consume services from the registry
– Should not access code in other components in any other way (linker) !
– Can have the dyloader fill these references in at load time
• Can provide service implementations to the registry
– Can have the dyloader register these at load time
8
- 9. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Basic Concepts: A Service
• An ABSTRACT NAMED interface definition !
– Although it can’t be in the registry without at least one implementation
• Can have multiple implementations:
– One implementation is default at any time
• It’s STATELESS !
– If you want state (a.k.a. data instances) you need to provide factories for these and
use handles
• Is an IMMUTABLE definition
– I.e. if you need to add a method or change any definition of any parameter you need
a new name for the service.
9
- 10. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Basic Concepts: A Service Contd.
• Has a public header file to describe the pointer returned
– The header must be self-sufficient (include all of the headers it depends on)
• It better (controversial, but explained later):
– use “simple” data types. A pointer to a 200+ member structure is a RED flag !
– use inout parameters and reserve return value for state.
10
- 11. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Basic Concepts: A Service Implementation
• A NAMED implementation of an abstract service interface registered in the
registry
– The name is prefixed with the service name and delimited with a dot. E.g. foo.bar is
bar’s implementation of the foo service.
– The implementation name is usually the component name !
• Resides in a component
• Pointer to it is stored in and returned by the registry
– Reference counted !
• Usually a C structure with a bunch of function pointers
– Defined in the service header, registry is agnostic to that. Macros provided to do it
this way
11
- 12. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Basic Concepts: A Related Service Implementation
• A normal service implementation
• Compatible with another service implementation
– Can operate on the same objects as the other service implementation
• Allows implementing sets of services
– And breaking large APIs in small logically independent sub-parts
• Usually residing in the same component as the other service
implementation.
• Example: registry query and registry registration services
12
- 13. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Basic concepts: The Registry
• A set of services to manipulate a global map of services.
• Does reference counting on service implementations
• Allows one to:
– Register service implementations
– Query for implementations by name and get a counted reference
– Release a counted reference to a service implementation
– Enumerate the registered service implementations and their metadata (name/value)
– Search for related implementations to a service already acquired
• Usually the part that’s bootstrapped first as it depends on no services
13
- 14. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Basic Concepts: The Dynamic Loader
• Requires the registry
• A single global ordered list of component generations.
• Loads and unloads component generations
– Handles registration/unregistration for the service implementations provided by
component(s)
• Can load multiple components in a single “generation” to handle circular
dependencies
• No persistent state
• Loading is done via calling “scheme” service implementations
– Currently only “file://”
14
- 15. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Basic Concepts: The Persisted Dynamic Loader
• A separate implementation (usually in a different component)
• Requires the Dynamic loader
• Ensures persistency for the Dynamic loader:
– remembers the sequence of component generations loaded on a persistent storage
– re-loads the stored sequence on startup into the dynamic loader
• The current implementation inside the server component uses a system
table and provides SQL commands.
15
- 16. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Concepts: The Minimal Chassis
• A static library
• The basic component infrastructure
– The registry
– The dynamic loader
• Enough to bootstrap a functional registry and a dynamic loader
• Can load any compliant component, given its service dependencies are met
by service implementations registered in the registry
• Great for (unit) testing !
• Allows using the component infrastructure outside of the server binary
16
- 17. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Component Services Inventory
17
- 18. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Component Services: The Inventory
• Service Registry
• Dynamic Loader
• Error logging
• System variables
• Status variables
• User defined functions
• Performance Schema
– New tables and Instrumentation
• Security Context
• Password validation
• Runtime errors
• Collated Strings
• Etc.
– 92 service related headers !
18
- 19. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
How To Write Your Own Components And
Services
19
- 20. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
How To Write Your Own Component/Service
• Use the utility macros
• Copy some of the existing components
– plenty of examples, specially in the tests
• Copy some of the existing services
– The services are done a bit differently in the server component
• Compile using a server source distribution
• Be careful !
– No crash protection, No security checks !
20
The Executive Summary
- 21. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Relevant Source Tree Directories
21
Root
Components
Library_mysys
Mysql_server
C1
C2
Include MySQL Components
Services
Library_mysys
sql Server_component
Agenda:
3d party Component Code
Minimal Chassis
Server Component Specific
Implementation macros
- 22. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Tutorial: How To Write A Component
#include <mysql/components/component_implementation.h>
#include <mysql/components/service_implementation.h>
#include <mysql/components/services/udf_registration.h>
REQUIRES_SERVICE_PLACEHOLDER(udf_registration);
// my code
static mysql_service_status_t deinit() { return false; }
static mysql_service_status_t init() { return false; }
22
Step 1: Add your component code: components/comp1/comp1.cc
- 23. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Tutorial: How To Write A Component
BEGIN_COMPONENT_PROVIDES(comp1) END_COMPONENT_PROVIDES();
BEGIN_COMPONENT_REQUIRES(comp1)
REQUIRES_SERVICE(udf_registration),
END_COMPONENT_REQUIRES();
BEGIN_COMPONENT_METADATA(comp1)
METADATA("mysql.author", "Oracle Corporation"),
METADATA("mysql.license", "GPL"), END_COMPONENT_METADATA();
DECLARE_COMPONENT(comp1, “comp1")
init, deinit END_DECLARE_COMPONENT();
DECLARE_LIBRARY_COMPONENTS &COMPONENT_REF(comp1)
END_DECLARE_LIBRARY_COMPONENTS
23
Step 2: Add the component boilerplate: components/comp1/comp1.cc
- 24. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Tutorial: How To Write A Component
MYSQL_ADD_COMPONENT(comp1
comp1.cc
TEST MODULE)
24
Step 3: Add a make file: components/comp1/CMakeFile.txt
- 25. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Tutorial: How To Create a Service
#include <mysql/components/service.h>
/**
@ingroup group_components_services_inventory
*/
BEGIN_SERVICE_DEFINITION(host_application_signal)
/**
Send a signal.
*/
DECLARE_BOOL_METHOD(signal, (int signal_no, void *arg));
END_SERVICE_DEFINITION(host_application_signal)
25
Step 1: Add a Service definition in include/mysql/components/services/
- 26. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Tutorial: How To Create a Service
#include <mysql/components/service_implementation.h>
#include <mysql/components/services/host_application_signal.h>
/**
An implementation of host application signal service for the mysql server as a host application.
*/
class mysql_component_host_application_signal_imp {
public:
static DEFINE_BOOL_METHOD(signal, (int signal_no, void *arg));
};
26
Step 2: OPTIONAL: Add an implementation header (server: sql/server_component/)
- 27. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Tutorial: How To Create a Service
#include <mysql/components/service_implementation.h>
#include <sql/server_component/host_application_signal_imp.h>
#include <components/mysql_server/server_component.h>
/**
An implementation of host application signal service for the mysql server as a host application.
*/
class mysql_component_host_application_signal_imp {
public:
static DEFINE_BOOL_METHOD(signal, (int signal_no, void *arg));
};
27
Step 3: Add an implementation file (server: sql/server_component/, your component
dir)
- 28. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Tutorial: How To Create a Service
#include <sql/server_component/host_application_signal_imp.h>
…
BEGIN_SERVICE_IMPLEMENTATION(comp1, host_application_signal)
mysql_component_host_application_signal_imp::signal
END_SERVICE_IMPLEMENTATION();
…
BEGIN_COMPONENT_PROVIDES(comp1)
PROVIDES_SERVICE(comp1, host_application_signal)
END_COMPONENT_PROVIDES();
28
Step 4: Add it to a component. (components/mysql_server/server_component.cc if
server)
- 29. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Component Services Tips And Tricks
29
- 30. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
C++ Way
• Obj *inst;
• inst = new Obj(12);
• int ret;
• ret = Inst->m1(1);
• delete inst;
The Handle Way
• HOBJ hobj;
• obj_svc->create(&hobj);
– obj_svc->init(hobj, 12);
• int ret;
• obj_svc->m1(hobj, 1, &ret);
• obj_svc->dispose(hobj);
30
Working with State: Handles
- 31. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Single Service
• HOBJ hobj;
• obj_svc->create(&hobj);
– obj_svc->init(hobj, 12);
• int ret;
• obj_svc->m1(hobj, 1, &ret);
• obj_svc->dispose(hobj);
Related Services
• HOBJ hobj;
• obj_factory_svc->create(&hobj);
– obj_integer_init_svc->init(hobj, 12);
• int ret;
• obj_svc->m1(hobj, 1, &ret);
• obj_factory_svc->dispose(hobj);
31
Working with State: Related Services
- 32. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 32
Related Service Sets: Why ?
C1
obj_svc.C1 obj_factory_svc.C1
obj_svc.C1 obj_svc.C1
obj_svc obj_svc.C1
obj_factory_svc.C1 obj_factory_svc.C1
obj_factory_svc obj_factory_svc.C2
obj_svc.C2 obj_svc.C2
obj_factory_svc.C2 obj_factory_svc.C2
C2
obj_svc.C2 obj_factory_svc.C2
- 33. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 33
Overloading
C1
obj_svc.C1 obj_factory_svc.C1
my_service<SERVICE_TYPE(obj_svc)>
obj_svc(“obj_svc”, registry);
C2
obj_svc.C2 obj_factory_svc.C2
my_service<SERVICE_TYPE(obj_svc)>
obj_svc2(“obj_svc”, registry);
obj_svc
- 34. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Random Guidelines
• Try to use the component infrastructure macros
• Use basic parameter types. Anything more complex goes into an
ANONYMOUS(!) handle and has service(s) to drive it
• Taking references is “expensive” (global structure). Reuse references.
• Do not make high volume calls services. Price to call a function pointer may
be too much !
• Make sure to avoid reference or handle leaks !
• Always use the designated destructor for handles !
• Keep the service reference with the consumer.
34
- 35. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
DEFINE_SERVICE_HANDLE(HOBJ);
BEGIN_SERVICE_DEFINITION(obj_svc)
DECLARE_BOOL_METHOD(create, (HOBJ *outHandle));
DECLARE_BOOL_METHOD(dispose, (HOBJ outHandle));
DECLARE_BOOL_METHOD(init, (HOBJ handle, int arg));
DECLARE_BOOL_METHOD(m1, (HOBJ handle, int arg, int
*outRet));
END_SERVICE_DEFINITION(obj_svc)
DEFINE_SERVICE_HANDLE(HOBJ);
BEGIN_SERVICE_DEFINITION(obj_factory_svc)
DECLARE_BOOL_METHOD(create, (HOBJ *outHandle));
DECLARE_BOOL_METHOD(dispose, (HOBJ outHandle));
END_SERVICE_DEFINITION(obj_factory_svc)
BEGIN_SERVICE_DEFINITION(obj_integer_init_svc)
DECLARE_BOOL_METHOD(init, (HOBJ handle, int arg));
END_SERVICE_DEFINITION(obj_integer_init_svc)
BEGIN_SERVICE_DEFINITION(obj_svc)
DECLARE_BOOL_METHOD(m1, (HOBJ handle, int arg, int
*outRet));
END_SERVICE_DEFINITION(obj_svc)
35
Working with State: Related Services Definitions
- 36. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
my_service<SERVICE_TYPE(obj_svc)>
obj_svc(“obj_svc”, registry);
my_service<SERVICE_TYPE(obj_svc)>
obj_svc(“obj_svc”, registry);
my_service<SERVICE_TYPE(obj_factory_svc)>
obj_factory_svc(“obj_factory_svc”, registry);
my_service<SERVICE_TYPE(obj_integer_init_svc)>
obj_int_init_svc(“obj_integer_init_svc”, registry);
36
Working with State: Related Services Use
- 37. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
my_service<SERVICE_TYPE(obj_svc)>
obj_svc(“obj_svc.c1”, registry);
my_service<SERVICE_TYPE(obj_svc)>
obj_svc(“obj_svc.c1”, registry);
my_service<SERVICE_TYPE(obj_factory_svc)>
obj_factory_svc(“obj_factory_svc.c1”, registry);
my_service<SERVICE_TYPE(obj_integer_init_svc)>
obj_int_init_svc(“obj_integer_init_svc.c1”, registry);
37
Working with State: 2nd Try
- 38. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
my_service<SERVICE_TYPE(obj_svc)>
obj_svc(“obj_svc”, registry);
my_service<SERVICE_TYPE(obj_svc)>
obj_svc(“obj_svc”, registry);
my_service<SERVICE_TYPE(obj_factory_svc)>
obj_factory_svc(“obj_factory_svc”, obj_svc, registry);
my_service<SERVICE_TYPE(obj_integer_init_svc)>
obj_int_init_svc(“obj_integer_init_svc”, obj_svc, registry);
38
Working with State: Final !
- 39. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
What’s Next ?
39
- 40. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
What’s Ahead ?
• Migration of major plugin APIs and plugin services into components
– Current State
• No new plugin APIs
• No new plugin service APIs
– Goal
• Deprecate the plugins !
• Further sub-division of the (now extremely chunky) server component
40
- 41. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The preceding is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
41