SlideShare ist ein Scribd-Unternehmen logo
1 von 35
Downloaden Sie, um offline zu lesen
Extending MEM,
Custom Advisors,
Graphs and Data
Collections
Mark Leith
Sun Microsystems

(leith@sun.com)
Agenda


   Quick MEM Agent Architecture
   Creating Data Collections
   Viewing Data Collection Information
   Creating Custom Graphs
   Creating and Modifying Advisors
   Questions / Input
Quick MEM Agent Architecture
Architecture – MEM Agent
Creating Data Collections
Data Collection Overview


 All Data Collections defined on each
  Agent installation locally
 Defaults are within the
  “./agent/share/mysql-proxy” directory
 SQL and Lua based collections
  supported
 SQL statements defined in XML format
 QUAN based collections are all Lua via
  the Proxy
Internal Data Collection Format

 Defined in XML files in the Advisor bundles
 Here's our definition of the
  innodb_buffer_pool_size variable:

 <itemList>
     <attribName>innodb_buffer_pool_size</attribName>
     <nameSpace>mysql</nameSpace>
     <attribType>INTEGER</attribType>
     <className>variables</className>
     <categoryName>Memory</categoryName>
     <subCategoryName>Buffers</subCategoryName>
     <isCounter>false</isCounter>
 </itemList>
Internal Data Collection Format

 Types: nameSpace::className::attribName
  i.e: mysql::variables::innodb_buffer_pool_size or
  os::cpu::cpu_user
 attribType – INTEGER | VARCHAR | DECIMAL
 isCounter – Whether the variable is a counter
  (true) or point in time value (false)
 Counter based variables are always evaluated
  using the delta between “now” and “now – 1”
  collections
 Multiple instances of attributes supported
  internally (for things such as CPUs, disks, etc.)
 Single instance collections use a “local” instance
SQL Based Data Collections
 Defaults in:
  ./agent/share/mysql-proxy/items/items-mysql-monitor.xml

   <class>
        <namespace>mysql</namespace>
        <classname>classTitle</classname>
        <query><![CDATA[SELECT ...]]></query>
   </class>

 SELECT must return one row (others ignored
  with warnings in the log file)
 Can return multiple columns in 2.x
        ● Assign <classname> in column order


 Note: <isCounter> is not available until 2.1 here
SQL Based Data Collections
   Collections can be added to the default, or a new file
   New files should be added to the agent-item-files
    variable in the mysql-monitor-agent.ini file
   Let's try an example with the new
    PERFORMANCE_SCHEMA!

<class>
       <namespace>mysql</namespace>
       <classname>wait_sum_open_mutex</classname>

       <query><![CDATA[

  SELECT sum_timer_wait
   FROM PERFORMANCE_SCHEMA.EVENTS_WAITS_SUMMARY_BY_EVENT_NAME
  WHERE EVENT_NAME = 'wait/synch/mutex/sql/LOCK_open'

       ]]></query>

</class>
Lua Based Data Collections

 Lua is a powerful embedded language – many
  possibilities exist to collect data (running
  external commands, gathering external data by
  reading files, etc.)
 Lua collections also go to:
  ./agent/share/mysql-proxy/items/
 They should also be added to the agent-items-
  files variable in the mysql-monitor-agent.ini file
 We build up the items, attributes and instances
  as associative arrays within Lua
Basic Lua Collections
if not items then
    items = { }
end


if not items.oldag then
    items[quot;oldagquot;] = { }
end


items[quot;oldagquot;][quot;childrenquot;] = {
    instances = { { name = quot;ethanquot; } },
    attributes = {
         [quot;couch-divesquot;] = {
             returns = quot;intquot;
         }
    },
    collector = function() return { [quot;couch-divesquot;] = math.random(100) + 120 } end
}
Matching Lua Arrays to Types

 The namespace is the top level of the items array
         ●   “oldag”
 The classname / type name comes next
         ●   “children”
 We then define the instances of the
  namespace::classname by name
         ●   “ethan”
 And the attributes and their return types
         ●   “couchdives” returning an int
 Finally we define a collector function that pushes the
  values we want to collect by attribute
Data Collections in the Dashboard

 Data collection metadata is stored in the
  “inventory_*” tables

 inventory_namespaces (namespace info)
 inventory_types (maps to “classname”
  information)
 inventory_attributes (attributes for
  namespace::type)
 inventory_instances (the instance names of
  each attribute)
 inventory_instance_attributes (pivot like table for
  attributes to instances mappings)
Data Collections in the Dashboard
 A useful SQL statement to get DC information:

SELECT DISTINCT
       namespace, type_name, attribute_name, instance_name
 FROM inventory_namespaces iin,
       inventory_types iit,
       inventory_instance_attributes iia,
       inventory_instances ii,
       inventory_attributes ia
WHERE iia.instance_id = ii.instance_id
   AND ii.type_id = iit.type_id
   AND iin.namespace_id = iit.namespace_id
   AND iia.attribute_id = ia.attribute_id;


*************************** 1. row ***************************
    namespace: os
    type_name: fs
attribute_name: fs_avail
 instance_name: mac:{0025003d17370000}./
Data Collections in the Dashboard
 Actual data is stored within the “dc_ng_*” tables
 The collected data is stored in it's raw form, whether it
  is a counter based variable or not
 Currently data is only purged, the infrastructure is
  ready for aggregation and aging for historical data in
  2.x (we want to test more!)


 dc_ng_long_now (integer based data)
 dc_ng_double_now (doubles)
 dc_ng_string_now (strings)
 dc_ng_(long|double)_age(0|1|2) (the rollup tables)


 String data obviously can not be aggregated
Data Collections in the Dashboard
 A useful SQL statement to get the collected data:

SELECT namespace, type_name, attribute_name, instance_name,
       value, from_unixtime(end_time/1000)
 FROM inventory_namespaces iin,
       inventory_types iit,
       dc_ng_long_now dcn,
       inventory_instance_attributes iia,
       inventory_instances ii,
       inventory_attributes ia
WHERE dcn.instance_attribute_id = iia.instance_attribute_id
   AND iia.instance_id = ii.instance_id
   AND ii.type_id = iit.type_id
   AND iin.namespace_id = iit.namespace_id
   AND iia.attribute_id = ia.attribute_id
   AND namespace = '...'
   AND type_name = '...'
   AND attribute_name = '...';
Creating Custom Graphs
Creating Custom Graphs
 Graphs also use XML for their structure
 Graphs can deal with both counter and non-counter
  variables
 2.0 now allows you to import your own graphs:
Graph XML Template
 <com_mysql_merlin_server_graph_Design>
   <version>...</version>         /* Internal graph version numbering */
   <uuid>...</uuid>               /* Internal graph unique identifier */
   <tag>...</tag>                 /* Tags for the graph (will be exposed soon) */
   <name>...</name>                /* Visible graph name */
   <rangeLabel>...</rangeLabel>   /* Visible Y axis range label */
   <series>
     <label>...</label>            /* Visible Series name / label */
     <expression>...</expression> /* The expression used for the series */
   </series>
   <variables>
     <name>...</name>              /* Arbitrary name for variable in expression */
     <dcItem>
        <nameSpace>...</nameSpace>
        <className>...</className>
        <attribName>...</attribName>
     </dcItem>
     <instance>...</instance>      /* The instance of the dc item to be used */
   </variables>
 </com_mysql_merlin_server_graph_Design>
Building a Graph - Metadata
 So let's build one up! Who wants to monitor disks?
 Start with the basic info

 <com_mysql_merlin_server_graph_Design>
      <version>1.0</version>
      <uuid>a57c2bba-ea9b-102b-b396-94aca32bee28</uuid>


      <name>filesystem usage - /</name>
      <rangeLabel>MB</rangeLabel>
      <frequency>00:05:00</frequency>


 Note <frequency>, this is not required, and defaults to
  1 minute
 We use 5 minutes here to share with current collection
Building a Graph - Variables
 Next, skip to the end, and define the variables
    <variables>
      <name>used_fs</name>
      <dcItem>
               <nameSpace>os</nameSpace>
               <className>fs</className>
               <attribName>fs_used</attribName>
      </dcItem>
      <instance>/</instance>
    </variables>
    <variables>
      <name>total_fs</name>
      <dcItem>
               <nameSpace>os</nameSpace>
               <className>fs</className>
               <attribName>fs_total</attribName>
      </dcItem>
      <instance>/</instance>
    </variables>
    </com_mysql_merlin_server_graph_Design>
Building a Graph - Series
 Now finally define each series


 <series>
   <label>used</label>
   <expression>used_fs/1024/1024</expression>
 </series>
 <series>
   <label>total size</label>
   <expression>total_fs/1024/1024</expression>
 </series>
View the Graph!
Graph PERFORMANCE_SCHEMA
 Viewing Global Mutex Information
Graph PERFORMANCE_SCHEMA
 Tracking Temporary File IO!
Graphs – Current Caveats

 Currently only line graphs are supported
 Graphs are shared across all servers – so
  they will show up even if the
  instances/collections are not available on a
  specific server
 Graphs sort alphabetically – if you want
  custom graphs to sort together, keep that in
  mind
 There is no “Delete” for graphs
Creating and Modifying Advisors
Modifying Current Rules

 All of the current rules are conservative – they
  should certainly be modified to your environment
 You can edit current rues on the “Manage Rules”
  page within the Advisors tab
 Editing only allows you to modify the
  THRESHOLD and frequency values for the rule
 Copying allows you to alter the expression, and
  advice or problem descriptions etc., as well as
  add new variables
Modifying Current Rules
Creating New Rules

 Also done on the Manage Rules page
 Any data collection can be used in a rule, with no limit
  on the number of data collections that can be used
 We still only support the default 3 THRESHOLD values
  – Info / Warning / Critical
 Wrapping the variable names in %percents% allows us
  to do variable value substitution within the Advice and
  Recommended Action sections
 We can also format numerical output nicely with
  '{'formatNumber:%variable_name%'}'
 You can make parts bold with two underscores on
  either side of a section of the advice etc. too
Creating New Rules
Creating New Rules
Creating New Rules
 Once saved, you will be able to schedule the rule
  within the Advisors->Add to Schedule page
Questions?


 Ask me now!
 Join us at the “Stump the DBA” BoF
  tonight
 Ask a MySQL Support Engineer
    http://support.mysql.com

Weitere ähnliche Inhalte

Was ist angesagt?

Getting to Know MySQL Enterprise Monitor
Getting to Know MySQL Enterprise MonitorGetting to Know MySQL Enterprise Monitor
Getting to Know MySQL Enterprise MonitorMark Leith
 
MySQL Monitoring Mechanisms
MySQL Monitoring MechanismsMySQL Monitoring Mechanisms
MySQL Monitoring MechanismsMark Leith
 
Performance schema and sys schema
Performance schema and sys schemaPerformance schema and sys schema
Performance schema and sys schemaMark Leith
 
Performance Schema for MySQL troubleshooting
Performance Schema for MySQL troubleshootingPerformance Schema for MySQL troubleshooting
Performance Schema for MySQL troubleshootingSveta Smirnova
 
The MySQL SYS Schema
The MySQL SYS SchemaThe MySQL SYS Schema
The MySQL SYS SchemaMark Leith
 
Capturing, Analyzing, and Optimizing your SQL
Capturing, Analyzing, and Optimizing your SQLCapturing, Analyzing, and Optimizing your SQL
Capturing, Analyzing, and Optimizing your SQLPadraig O'Sullivan
 
MySQL's Performance Schema, SYS Schema and Workbench Integration
MySQL's Performance Schema, SYS Schema and Workbench IntegrationMySQL's Performance Schema, SYS Schema and Workbench Integration
MySQL's Performance Schema, SYS Schema and Workbench IntegrationMario Beck
 
MySQL sys schema deep dive
MySQL sys schema deep diveMySQL sys schema deep dive
MySQL sys schema deep diveMark Leith
 
Oracle Data redaction - GUOB - OTN TOUR LA - 2015
Oracle Data redaction - GUOB - OTN TOUR LA - 2015Oracle Data redaction - GUOB - OTN TOUR LA - 2015
Oracle Data redaction - GUOB - OTN TOUR LA - 2015Alex Zaballa
 
OTN TOUR 2016 - DBA Commands and Concepts That Every Developer Should Know
OTN TOUR 2016 - DBA Commands and Concepts That Every Developer Should KnowOTN TOUR 2016 - DBA Commands and Concepts That Every Developer Should Know
OTN TOUR 2016 - DBA Commands and Concepts That Every Developer Should KnowAlex Zaballa
 
MySQL Troubleshooting with the Performance Schema
MySQL Troubleshooting with the Performance SchemaMySQL Troubleshooting with the Performance Schema
MySQL Troubleshooting with the Performance SchemaSveta Smirnova
 
Oracle Database 11g Product Family
Oracle Database 11g Product FamilyOracle Database 11g Product Family
Oracle Database 11g Product FamilyN/A
 
OOW16 - Oracle Database 12c - The Best Oracle Database 12c New Features for D...
OOW16 - Oracle Database 12c - The Best Oracle Database 12c New Features for D...OOW16 - Oracle Database 12c - The Best Oracle Database 12c New Features for D...
OOW16 - Oracle Database 12c - The Best Oracle Database 12c New Features for D...Alex Zaballa
 
common_schema, DBA's framework for MySQL
common_schema, DBA's framework for MySQLcommon_schema, DBA's framework for MySQL
common_schema, DBA's framework for MySQLShlomi Noach
 
REST in Piece - Administration of an Oracle Cluster/Database using REST
REST in Piece - Administration of an Oracle Cluster/Database using RESTREST in Piece - Administration of an Oracle Cluster/Database using REST
REST in Piece - Administration of an Oracle Cluster/Database using RESTChristian Gohmann
 
Oracle Database 12c - Data Redaction
Oracle Database 12c - Data RedactionOracle Database 12c - Data Redaction
Oracle Database 12c - Data RedactionAlex Zaballa
 
Oracle Database 12c Release 2 - New Features On Oracle Database Exadata Expre...
Oracle Database 12c Release 2 - New Features On Oracle Database Exadata Expre...Oracle Database 12c Release 2 - New Features On Oracle Database Exadata Expre...
Oracle Database 12c Release 2 - New Features On Oracle Database Exadata Expre...Alex Zaballa
 
Oracle Data Redaction
Oracle Data RedactionOracle Data Redaction
Oracle Data RedactionAlex Zaballa
 
MySQL Best Practices - OTN LAD Tour
MySQL Best Practices - OTN LAD TourMySQL Best Practices - OTN LAD Tour
MySQL Best Practices - OTN LAD TourRonald Bradford
 

Was ist angesagt? (20)

Getting to Know MySQL Enterprise Monitor
Getting to Know MySQL Enterprise MonitorGetting to Know MySQL Enterprise Monitor
Getting to Know MySQL Enterprise Monitor
 
MySQL Monitoring Mechanisms
MySQL Monitoring MechanismsMySQL Monitoring Mechanisms
MySQL Monitoring Mechanisms
 
Performance schema and sys schema
Performance schema and sys schemaPerformance schema and sys schema
Performance schema and sys schema
 
Performance Schema for MySQL troubleshooting
Performance Schema for MySQL troubleshootingPerformance Schema for MySQL troubleshooting
Performance Schema for MySQL troubleshooting
 
The MySQL SYS Schema
The MySQL SYS SchemaThe MySQL SYS Schema
The MySQL SYS Schema
 
Capturing, Analyzing, and Optimizing your SQL
Capturing, Analyzing, and Optimizing your SQLCapturing, Analyzing, and Optimizing your SQL
Capturing, Analyzing, and Optimizing your SQL
 
MySQL's Performance Schema, SYS Schema and Workbench Integration
MySQL's Performance Schema, SYS Schema and Workbench IntegrationMySQL's Performance Schema, SYS Schema and Workbench Integration
MySQL's Performance Schema, SYS Schema and Workbench Integration
 
MySQL sys schema deep dive
MySQL sys schema deep diveMySQL sys schema deep dive
MySQL sys schema deep dive
 
Oracle Data redaction - GUOB - OTN TOUR LA - 2015
Oracle Data redaction - GUOB - OTN TOUR LA - 2015Oracle Data redaction - GUOB - OTN TOUR LA - 2015
Oracle Data redaction - GUOB - OTN TOUR LA - 2015
 
OTN TOUR 2016 - DBA Commands and Concepts That Every Developer Should Know
OTN TOUR 2016 - DBA Commands and Concepts That Every Developer Should KnowOTN TOUR 2016 - DBA Commands and Concepts That Every Developer Should Know
OTN TOUR 2016 - DBA Commands and Concepts That Every Developer Should Know
 
Php with MYSQL Database
Php with MYSQL DatabasePhp with MYSQL Database
Php with MYSQL Database
 
MySQL Troubleshooting with the Performance Schema
MySQL Troubleshooting with the Performance SchemaMySQL Troubleshooting with the Performance Schema
MySQL Troubleshooting with the Performance Schema
 
Oracle Database 11g Product Family
Oracle Database 11g Product FamilyOracle Database 11g Product Family
Oracle Database 11g Product Family
 
OOW16 - Oracle Database 12c - The Best Oracle Database 12c New Features for D...
OOW16 - Oracle Database 12c - The Best Oracle Database 12c New Features for D...OOW16 - Oracle Database 12c - The Best Oracle Database 12c New Features for D...
OOW16 - Oracle Database 12c - The Best Oracle Database 12c New Features for D...
 
common_schema, DBA's framework for MySQL
common_schema, DBA's framework for MySQLcommon_schema, DBA's framework for MySQL
common_schema, DBA's framework for MySQL
 
REST in Piece - Administration of an Oracle Cluster/Database using REST
REST in Piece - Administration of an Oracle Cluster/Database using RESTREST in Piece - Administration of an Oracle Cluster/Database using REST
REST in Piece - Administration of an Oracle Cluster/Database using REST
 
Oracle Database 12c - Data Redaction
Oracle Database 12c - Data RedactionOracle Database 12c - Data Redaction
Oracle Database 12c - Data Redaction
 
Oracle Database 12c Release 2 - New Features On Oracle Database Exadata Expre...
Oracle Database 12c Release 2 - New Features On Oracle Database Exadata Expre...Oracle Database 12c Release 2 - New Features On Oracle Database Exadata Expre...
Oracle Database 12c Release 2 - New Features On Oracle Database Exadata Expre...
 
Oracle Data Redaction
Oracle Data RedactionOracle Data Redaction
Oracle Data Redaction
 
MySQL Best Practices - OTN LAD Tour
MySQL Best Practices - OTN LAD TourMySQL Best Practices - OTN LAD Tour
MySQL Best Practices - OTN LAD Tour
 

Ähnlich wie Extending MySQL Enterprise Monitor

[DSBW Spring 2009] Unit 07: WebApp Design Patterns & Frameworks (3/3)
[DSBW Spring 2009] Unit 07: WebApp Design Patterns & Frameworks (3/3)[DSBW Spring 2009] Unit 07: WebApp Design Patterns & Frameworks (3/3)
[DSBW Spring 2009] Unit 07: WebApp Design Patterns & Frameworks (3/3)Carles Farré
 
Intro to Rails ActiveRecord
Intro to Rails ActiveRecordIntro to Rails ActiveRecord
Intro to Rails ActiveRecordMark Menard
 
Debugging and Error handling
Debugging and Error handlingDebugging and Error handling
Debugging and Error handlingSuite Solutions
 
NHibernate (The ORM For .NET Platform)
NHibernate (The ORM For .NET Platform)NHibernate (The ORM For .NET Platform)
NHibernate (The ORM For .NET Platform)Samnang Chhun
 
XQuery Triggers in Native XML Database Sedna
XQuery Triggers in Native XML Database SednaXQuery Triggers in Native XML Database Sedna
XQuery Triggers in Native XML Database Sednamaria.grineva
 
Nhibernatethe Orm For Net Platform 1226744632929962 8
Nhibernatethe Orm For Net Platform 1226744632929962 8Nhibernatethe Orm For Net Platform 1226744632929962 8
Nhibernatethe Orm For Net Platform 1226744632929962 8Nicolas Thon
 
Intro Open Social and Dashboards
Intro Open Social and DashboardsIntro Open Social and Dashboards
Intro Open Social and DashboardsAtlassian
 
Zero One Or Many Namespaces
Zero One Or Many NamespacesZero One Or Many Namespaces
Zero One Or Many NamespacesLiquidHub
 
Struts Tags Speakernoted
Struts Tags SpeakernotedStruts Tags Speakernoted
Struts Tags SpeakernotedHarjinder Singh
 
Introducing Struts 2
Introducing Struts 2Introducing Struts 2
Introducing Struts 2wiradikusuma
 
Система рендеринга в Magento
Система рендеринга в MagentoСистема рендеринга в Magento
Система рендеринга в MagentoMagecom Ukraine
 
12-security.ppt - PHP and Arabic Language - Index
12-security.ppt - PHP and Arabic Language - Index12-security.ppt - PHP and Arabic Language - Index
12-security.ppt - PHP and Arabic Language - Indexwebhostingguy
 

Ähnlich wie Extending MySQL Enterprise Monitor (20)

Seam Glassfish Slidecast
Seam Glassfish SlidecastSeam Glassfish Slidecast
Seam Glassfish Slidecast
 
[DSBW Spring 2009] Unit 07: WebApp Design Patterns & Frameworks (3/3)
[DSBW Spring 2009] Unit 07: WebApp Design Patterns & Frameworks (3/3)[DSBW Spring 2009] Unit 07: WebApp Design Patterns & Frameworks (3/3)
[DSBW Spring 2009] Unit 07: WebApp Design Patterns & Frameworks (3/3)
 
Intro to Rails ActiveRecord
Intro to Rails ActiveRecordIntro to Rails ActiveRecord
Intro to Rails ActiveRecord
 
Debugging and Error handling
Debugging and Error handlingDebugging and Error handling
Debugging and Error handling
 
Struts2
Struts2Struts2
Struts2
 
Os Leonard
Os LeonardOs Leonard
Os Leonard
 
NHibernate (The ORM For .NET Platform)
NHibernate (The ORM For .NET Platform)NHibernate (The ORM For .NET Platform)
NHibernate (The ORM For .NET Platform)
 
XQuery Triggers in Native XML Database Sedna
XQuery Triggers in Native XML Database SednaXQuery Triggers in Native XML Database Sedna
XQuery Triggers in Native XML Database Sedna
 
Nhibernatethe Orm For Net Platform 1226744632929962 8
Nhibernatethe Orm For Net Platform 1226744632929962 8Nhibernatethe Orm For Net Platform 1226744632929962 8
Nhibernatethe Orm For Net Platform 1226744632929962 8
 
Intro Open Social and Dashboards
Intro Open Social and DashboardsIntro Open Social and Dashboards
Intro Open Social and Dashboards
 
Broadleaf Presents Thymeleaf
Broadleaf Presents ThymeleafBroadleaf Presents Thymeleaf
Broadleaf Presents Thymeleaf
 
Schema201 webinar
Schema201 webinarSchema201 webinar
Schema201 webinar
 
Zero One Or Many Namespaces
Zero One Or Many NamespacesZero One Or Many Namespaces
Zero One Or Many Namespaces
 
Download It
Download ItDownload It
Download It
 
Framework
FrameworkFramework
Framework
 
Struts Tags Speakernoted
Struts Tags SpeakernotedStruts Tags Speakernoted
Struts Tags Speakernoted
 
Introducing Struts 2
Introducing Struts 2Introducing Struts 2
Introducing Struts 2
 
Система рендеринга в Magento
Система рендеринга в MagentoСистема рендеринга в Magento
Система рендеринга в Magento
 
12-security.ppt - PHP and Arabic Language - Index
12-security.ppt - PHP and Arabic Language - Index12-security.ppt - PHP and Arabic Language - Index
12-security.ppt - PHP and Arabic Language - Index
 
Security.ppt
Security.pptSecurity.ppt
Security.ppt
 

Kürzlich hochgeladen

Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 

Kürzlich hochgeladen (20)

Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 

Extending MySQL Enterprise Monitor

  • 1. Extending MEM, Custom Advisors, Graphs and Data Collections Mark Leith Sun Microsystems (leith@sun.com)
  • 2. Agenda  Quick MEM Agent Architecture  Creating Data Collections  Viewing Data Collection Information  Creating Custom Graphs  Creating and Modifying Advisors  Questions / Input
  • 3. Quick MEM Agent Architecture
  • 6. Data Collection Overview  All Data Collections defined on each Agent installation locally  Defaults are within the “./agent/share/mysql-proxy” directory  SQL and Lua based collections supported  SQL statements defined in XML format  QUAN based collections are all Lua via the Proxy
  • 7. Internal Data Collection Format  Defined in XML files in the Advisor bundles  Here's our definition of the innodb_buffer_pool_size variable: <itemList> <attribName>innodb_buffer_pool_size</attribName> <nameSpace>mysql</nameSpace> <attribType>INTEGER</attribType> <className>variables</className> <categoryName>Memory</categoryName> <subCategoryName>Buffers</subCategoryName> <isCounter>false</isCounter> </itemList>
  • 8. Internal Data Collection Format  Types: nameSpace::className::attribName i.e: mysql::variables::innodb_buffer_pool_size or os::cpu::cpu_user  attribType – INTEGER | VARCHAR | DECIMAL  isCounter – Whether the variable is a counter (true) or point in time value (false)  Counter based variables are always evaluated using the delta between “now” and “now – 1” collections  Multiple instances of attributes supported internally (for things such as CPUs, disks, etc.)  Single instance collections use a “local” instance
  • 9. SQL Based Data Collections  Defaults in: ./agent/share/mysql-proxy/items/items-mysql-monitor.xml <class> <namespace>mysql</namespace> <classname>classTitle</classname> <query><![CDATA[SELECT ...]]></query> </class>  SELECT must return one row (others ignored with warnings in the log file)  Can return multiple columns in 2.x ● Assign <classname> in column order  Note: <isCounter> is not available until 2.1 here
  • 10. SQL Based Data Collections  Collections can be added to the default, or a new file  New files should be added to the agent-item-files variable in the mysql-monitor-agent.ini file  Let's try an example with the new PERFORMANCE_SCHEMA! <class> <namespace>mysql</namespace> <classname>wait_sum_open_mutex</classname> <query><![CDATA[ SELECT sum_timer_wait FROM PERFORMANCE_SCHEMA.EVENTS_WAITS_SUMMARY_BY_EVENT_NAME WHERE EVENT_NAME = 'wait/synch/mutex/sql/LOCK_open' ]]></query> </class>
  • 11. Lua Based Data Collections  Lua is a powerful embedded language – many possibilities exist to collect data (running external commands, gathering external data by reading files, etc.)  Lua collections also go to: ./agent/share/mysql-proxy/items/  They should also be added to the agent-items- files variable in the mysql-monitor-agent.ini file  We build up the items, attributes and instances as associative arrays within Lua
  • 12. Basic Lua Collections if not items then items = { } end if not items.oldag then items[quot;oldagquot;] = { } end items[quot;oldagquot;][quot;childrenquot;] = { instances = { { name = quot;ethanquot; } }, attributes = { [quot;couch-divesquot;] = { returns = quot;intquot; } }, collector = function() return { [quot;couch-divesquot;] = math.random(100) + 120 } end }
  • 13. Matching Lua Arrays to Types  The namespace is the top level of the items array ● “oldag”  The classname / type name comes next ● “children”  We then define the instances of the namespace::classname by name ● “ethan”  And the attributes and their return types ● “couchdives” returning an int  Finally we define a collector function that pushes the values we want to collect by attribute
  • 14. Data Collections in the Dashboard  Data collection metadata is stored in the “inventory_*” tables  inventory_namespaces (namespace info)  inventory_types (maps to “classname” information)  inventory_attributes (attributes for namespace::type)  inventory_instances (the instance names of each attribute)  inventory_instance_attributes (pivot like table for attributes to instances mappings)
  • 15. Data Collections in the Dashboard  A useful SQL statement to get DC information: SELECT DISTINCT namespace, type_name, attribute_name, instance_name FROM inventory_namespaces iin, inventory_types iit, inventory_instance_attributes iia, inventory_instances ii, inventory_attributes ia WHERE iia.instance_id = ii.instance_id AND ii.type_id = iit.type_id AND iin.namespace_id = iit.namespace_id AND iia.attribute_id = ia.attribute_id; *************************** 1. row *************************** namespace: os type_name: fs attribute_name: fs_avail instance_name: mac:{0025003d17370000}./
  • 16. Data Collections in the Dashboard  Actual data is stored within the “dc_ng_*” tables  The collected data is stored in it's raw form, whether it is a counter based variable or not  Currently data is only purged, the infrastructure is ready for aggregation and aging for historical data in 2.x (we want to test more!)  dc_ng_long_now (integer based data)  dc_ng_double_now (doubles)  dc_ng_string_now (strings)  dc_ng_(long|double)_age(0|1|2) (the rollup tables)  String data obviously can not be aggregated
  • 17. Data Collections in the Dashboard  A useful SQL statement to get the collected data: SELECT namespace, type_name, attribute_name, instance_name, value, from_unixtime(end_time/1000) FROM inventory_namespaces iin, inventory_types iit, dc_ng_long_now dcn, inventory_instance_attributes iia, inventory_instances ii, inventory_attributes ia WHERE dcn.instance_attribute_id = iia.instance_attribute_id AND iia.instance_id = ii.instance_id AND ii.type_id = iit.type_id AND iin.namespace_id = iit.namespace_id AND iia.attribute_id = ia.attribute_id AND namespace = '...' AND type_name = '...' AND attribute_name = '...';
  • 19. Creating Custom Graphs  Graphs also use XML for their structure  Graphs can deal with both counter and non-counter variables  2.0 now allows you to import your own graphs:
  • 20. Graph XML Template <com_mysql_merlin_server_graph_Design> <version>...</version> /* Internal graph version numbering */ <uuid>...</uuid> /* Internal graph unique identifier */ <tag>...</tag> /* Tags for the graph (will be exposed soon) */ <name>...</name> /* Visible graph name */ <rangeLabel>...</rangeLabel> /* Visible Y axis range label */ <series> <label>...</label> /* Visible Series name / label */ <expression>...</expression> /* The expression used for the series */ </series> <variables> <name>...</name> /* Arbitrary name for variable in expression */ <dcItem> <nameSpace>...</nameSpace> <className>...</className> <attribName>...</attribName> </dcItem> <instance>...</instance> /* The instance of the dc item to be used */ </variables> </com_mysql_merlin_server_graph_Design>
  • 21. Building a Graph - Metadata  So let's build one up! Who wants to monitor disks?  Start with the basic info <com_mysql_merlin_server_graph_Design> <version>1.0</version> <uuid>a57c2bba-ea9b-102b-b396-94aca32bee28</uuid> <name>filesystem usage - /</name> <rangeLabel>MB</rangeLabel> <frequency>00:05:00</frequency>  Note <frequency>, this is not required, and defaults to 1 minute  We use 5 minutes here to share with current collection
  • 22. Building a Graph - Variables  Next, skip to the end, and define the variables <variables> <name>used_fs</name> <dcItem> <nameSpace>os</nameSpace> <className>fs</className> <attribName>fs_used</attribName> </dcItem> <instance>/</instance> </variables> <variables> <name>total_fs</name> <dcItem> <nameSpace>os</nameSpace> <className>fs</className> <attribName>fs_total</attribName> </dcItem> <instance>/</instance> </variables> </com_mysql_merlin_server_graph_Design>
  • 23. Building a Graph - Series  Now finally define each series <series> <label>used</label> <expression>used_fs/1024/1024</expression> </series> <series> <label>total size</label> <expression>total_fs/1024/1024</expression> </series>
  • 25. Graph PERFORMANCE_SCHEMA  Viewing Global Mutex Information
  • 27. Graphs – Current Caveats  Currently only line graphs are supported  Graphs are shared across all servers – so they will show up even if the instances/collections are not available on a specific server  Graphs sort alphabetically – if you want custom graphs to sort together, keep that in mind  There is no “Delete” for graphs
  • 29. Modifying Current Rules  All of the current rules are conservative – they should certainly be modified to your environment  You can edit current rues on the “Manage Rules” page within the Advisors tab  Editing only allows you to modify the THRESHOLD and frequency values for the rule  Copying allows you to alter the expression, and advice or problem descriptions etc., as well as add new variables
  • 31. Creating New Rules  Also done on the Manage Rules page  Any data collection can be used in a rule, with no limit on the number of data collections that can be used  We still only support the default 3 THRESHOLD values – Info / Warning / Critical  Wrapping the variable names in %percents% allows us to do variable value substitution within the Advice and Recommended Action sections  We can also format numerical output nicely with '{'formatNumber:%variable_name%'}'  You can make parts bold with two underscores on either side of a section of the advice etc. too
  • 34. Creating New Rules  Once saved, you will be able to schedule the rule within the Advisors->Add to Schedule page
  • 35. Questions?  Ask me now!  Join us at the “Stump the DBA” BoF tonight  Ask a MySQL Support Engineer http://support.mysql.com