SlideShare ist ein Scribd-Unternehmen logo
1 von 90
Downloaden Sie, um offline zu lesen
Control system based on a
  Highly
  Abstracted and
  Open
  Structure




                              seminar 2012



   http://chaos.infn.it                      C.
Thursday, January 10, 13
What is
       !CHAOS (Control system based on Highly Abstracted and Open Structure), is
       an experiment of CSN 5(technological research experiments) at the LNF and
       Roma-TV section.


       The project has been appreciated by educational world and the University of




                                                       Università
                                                       TorVergata   LNF - Roma2




                           Università
                           di Cagliari




Thursday, January 10, 13
Objective



    !CHAOS has been designed to widely scale and for manage
    large apparatus with different needs in term of storage
    technology and speed for data acquisition




Thursday, January 10, 13
Objective


   new topology in the CS that permit to integrate the triggered
    DAQ.

   define a new way to access realtime data

   data acquisition, storage and indexing fully scalable




Thursday, January 10, 13
Objective

              A CPP Framework and services that permit to have:


 • 99% of adaptation time of instrument to !CHOAS done by
    framework

 • simplify the creation of the user interfaces
 • simplify the creation of the control algorithms basing it on an
    abstraction of the I/O channels or full dataset not on physical
    device




Thursday, January 10, 13
!CHAOS Simplify Vision




Thursday, January 10, 13
Simplify Vision
                                            User specialized
                                               Boundary
                                                                         Implementation

                                            CHAOS Boundary                     Control's
                                                                               Algorithm


                           Implementation




                                                                                                             Implementation
                                              Instrument's
                                                                                network     User Interface
                                                 Driver




                                                                 Live Data

                                                                                           Directory
                                                                Data Cloud                  Server
                                                                                            Cloud

                                                                History Data




                                                               Implementation




Thursday, January 10, 13
Simplify Vision
                                            User specialized
                                               Boundary
                                                                         Implementation

                                            CHAOS Boundary                     Control's
                                                                               Algorithm
                                                                                             Abstract RPC
                                                                                                Driver
                           Implementation




                                                                                                             Implementation
                                              Instrument's                                     Abstract
                                                                                network     User Interface
                                                 Driver                                      Event Driver



                                                                                               Abstract
                                                                                               Direct I/O

                                                                 Live Data

                                                                                           Directory
                                                                Data Cloud                  Server
                                                                                            Cloud

                                                                History Data




                                                               Implementation




Thursday, January 10, 13
Simplify Vision
                                            User specialized
                                               Boundary
                                                                         Implementation

                                            CHAOS Boundary                     Control's
                                                                               Algorithm
                                                                                              Abstract RPC
                                                                                                 Driver
                           Implementation




                                                                                                             Implementation
                                              Instrument's                                     Abstract
                                                                                network     User Interface
                                                 Driver                                      Event Driver



                                                                                                Abstract
                                                                                                Direct I/O

                                                                 Live Data
                                                Live Data                                    Live Data
                                                                                           Directory
                                                Data Cloud      Data Cloud                  Server
                                                                                            Data Cloud
                                                                                            Cloud
                                                 History                                      History
                                                  Data                                          Data
                                                                History Data

                                              Implementation                               Implementation

                                                               Implementation




Thursday, January 10, 13
!CHAOS Real Model




Thursday, January 10, 13
User                                                                         User Interface for monitoring and controls
                   Customization
                   Abstraction
                   Boundary                                                                                      User Interface Toolkit
                                                                                                                    Common Toolkit
                                                                                                                   RPC - Event - Direct I/O

                                                                                         Data Boundary    CQL Proxy Nodes           Informatio
                    Execution Unit Toolkit




                                                                                                                                    n
                                                              RPC - Event - Direct I/O
                                                                                                             Indexer Nodes
                                                                                                                                                    Node
                                             Common Toolkit



                                                                                                         Management Nodes                        Information




                                                                                                                                                               HTTP - API
                                                                                                Live / History                                    Security
                                                                                                   domain
Controls                                                                                                                                           Node
Algorithm                                                                                                                                         Activity
                                                                                                                                      Chaos
                                                                                                         Live / History
                                                                                                                                     Directory
                                                                                                            domain
                                                                                                                                      Nodes



                                                                                                                   RPC - Event - Direct I/O

                                                                                                                   Common Toolkit
                                                                                                                  Control Unit Toolkit



                                                                                              Driver for sensor and instruments




Thursday, January 10, 13
Toolkit and Services



           !CHAOS is a set of toolkit and services that will
                  permit the realization of an control system




Thursday, January 10, 13
!CHAOS Toolkit and Services

             Common Toolkit                        Directory
                                                    Services
  CU Toolkit
                                              Live and History Data
                           UI Toolkit
                                                   CQL Proxy
                                 EU Toolkit
                                                    Indexer

                                                  Managment



Thursday, January 10, 13
!CHAOS Toolkit




Thursday, January 10, 13
!CHAOS Toolkit
                Common       implements the fundamental software's
                             layer to RPC, I/O, Event and other’s utility
                   Toolkit   code and for logic


                             abstracts the !CHAOS resources to the
             CU Toolkit      device drivers developers.


                             abstracts the CHAOS resources to the
               UI Toolkit    developers that need to develop control
                             and monitoring user interface


                             abstracts the CHAOS resources to the
               EU Toolkit    developers that need to develop control
                             and computing algorithms




Thursday, January 10, 13
!CHAOS Services




Thursday, January 10, 13
!CHAOS Services

          Directory        gives services via RPC (or with other
                           higher transport layers) for get device
            Services       structured information, system topology



                           give the access to insert and query
        CQL Proxy          operation on live and history data to
                           other CHOAS node using CHOAS Query


                           performs the    indexing   of   the   new
             Indexer       inserted data



                           performs the management of the datas
    Management             chunk saved on the storage (erase, move,




Thursday, January 10, 13
!CHOAS Scalability and Fault-



       Some node needs to be scaled for performance reason or for
        fault-tolerant:
            • Directory Nodes
            • Live and History nodes




Thursday, January 10, 13
!CHOAS Scalability and Fault-
                           Server P1
                           Server P2
                                                   Framework

                                                         Device 1
                                                          Driver       X
                                                                                                X
                                                                                        Server P1


                           Server PN


                                                                                    X               Server P1
                                                                                                    Server P2


                                                                    Client Reader
                                       Server P2
                                                                        Driver
                                                                                                    Server PN
                                                                           Framework




         Clients that need to talk to scaled services, has more than one
                                  ip address for every service




Thursday, January 10, 13
!CHOAS Scalability and Fault-
                           Server P1
                           Server P2
                                                   Framework

                                                         Device 1
                                                          Driver       X
                                                                                                X
                                                                                        Server P1


                           Server PN


                                                                                    X               Server P1
                                                                                                    Server P2


                                                                    Client Reader
                                       Server P2
                                                                        Driver
                                                                                                    Server PN
                                                                           Framework




        Every ip address has a priority, the client, as first address, uses
                                 the one with highest priority




Thursday, January 10, 13
!CHOAS Scalability and Fault-
                                   Server P1
                                   Server P2
                                                           Framework

                                                                 Device 1
                                                                  Driver       X
                                                                                                        X
                                                                                                Server P1


                                   Server PN


                                                                                            X               Server P1
                                                                                                            Server P2


                                                                            Client Reader
                                               Server P2
                                                                                Driver
                                                                                                            Server PN
                                                                                   Framework




                           when a server goes down, another ip will be used




Thursday, January 10, 13
!CHOAS Scalability and Fault-

                           Server P1
                           Server P2
                                                   Framework

                                                                    use at
                                                                                                  ok
                                                         Device 1
                                                                                      Server P1
                                                          Driver
                                                                     TS1
                           Server PN
                                                                      use at
                                                                                                  Server P1
                                                                       TS1                        Server P2


                                                                    Client Reader
                                       Server P2
                                                                        Driver
                                                                                                  Server PN
                                                                          Framework




            the re-balance occurs when a server with a highest priority,
             previously goes down or stopped for maintenance, becomes
                                                     up and running



Thursday, January 10, 13
!CHOAS Scalability and Fault-

                           Server P1
                           Server P2
                                                   Framework

                                                                    use at
                                                                                                  ok
                                                         Device 1
                                                                                      Server P1
                                                          Driver
                                                                     TS1
                           Server PN
                                                                      use at
                                                                                                  Server P1
                                                                       TS1                        Server P2


                                                                    Client Reader
                                       Server P2
                                                                        Driver
                                                                                                  Server PN
                                                                          Framework




          the new server will be used starting from a pre-determinated
                                                   time stamp (TS1)




Thursday, January 10, 13
!CHOAS Scalability and Fault-

                           Server P1
                           Server P2
                                                   Framework

                                                                    use at
                                                                                                  ok
                                                         Device 1
                                                                                      Server P1
                                                          Driver
                                                                     TS1
                           Server PN
                                                                      use at
                                                                                                  Server P1
                                                                       TS1                        Server P2


                                                                    Client Reader
                                       Server P2
                                                                        Driver
                                                                                                  Server PN
                                                                          Framework




       all the operations and the calculation of TS1 are coordinated by
                                   one of the directory nodes




Thursday, January 10, 13
!CHOAS Communication



                           11/dic/2011   1
Thursday, January 10, 13
Communication Systems

  !CHOAS has three different communication system:


  RPC, used for command (messages that need to be securely
  received and recorded and for message that need a response
  [ asynchronous ])

  Events, used for inform to a wide range of nodes, that
  something has happened (device heartbeat, alert on device
  channel, received command, etc..)




Thursday, January 10, 13
Communication Systems



                           Every !CHAOS node has at last
                           one client and server instance
                            of any of the three system




Thursday, January 10, 13
RPC System


•RPC layer is implemented as a plug-ins System, it's is
   abstracted to internal CHAOS framework so we con change it
• used to send important messages that need to be sure of it's
   delivering status
• can be used to receive asynchronous responses
• the set command for input channel on control unit is done
   with RPC




                             09/dic/2011
Thursday, January 10, 13
RPC System




                           Message                        Answer [optional]
                           Messagge forwarding                      Response forwarding


                Node A                           Node B    Node A                         Node B
                                ack [sync]                               ack [sync]




Thursday, January 10, 13
RPC System
    RPC Message Flow
    • request message is constructed with BSON, serialized and
     sent to RPC end point
    • if the requester want a response it put answer-code and
     sender-address in BSON pack,and automatically RPC System
     wait for the answer
    • the request and response involves the forwarding of the
     ack message, from the receiver to the sender in


                           Message                        Answer [optional]
                           Messagge forwarding                      Response forwarding


                Node A                           Node B    Node A                         Node B
                                ack [sync]                               ack [sync]




Thursday, January 10, 13
Event System
  • Event is implemented as a plug-ins System it's is
     abstracted to internal CHAOS framework so we con change it
  • there are four type of event:
      • Alert (highest priority)
      • Instrument
      • Command
      • Custom
  • all information are packed into 1024 byte and transported
     by protocols that support multicast (UDP, PGM[http://
     en.wikipedia.org/wiki/Pragmatic_General_Multicast])



                               09/dic/2011
Thursday, January 10, 13
Event System


                           Some numbers about event data


  • Every event type has 2^16 sub type
  • Every sub type can have up to 2^16 priority level
  • Every subtype can have up to 255 byte of information




                                      09/dic/2011
Thursday, January 10, 13
Direct I/O System


  • Direct I/O is implemented as a plug-ins system
  • used to make insert or select into the live and the history
     data
  • can be used as stream for receive result data of the query
     or read the stored raw data file of the devices




                                09/dic/2011
Thursday, January 10, 13
!CHAOS Abstraction of the



                           11/dic/2011   2
Thursday, January 10, 13
Abstraction
     !CHAOS use a structured serialization for data description
      and serialization


     • BSON (http://bsonspec.org/) has been chosen, very
      similar to JSON but it is binary
     • serialization is wrapped into a CPP class so it can be
      changed infuture
     • it is used in:
          •Hardware dataset description's and acquired data
           packaging (“channel desc” = value)
          •RPC message
          •Chaos Query Language


Thursday, January 10, 13
Hardware Dataset

    •Hardware attributes and I/O channels are described into the
     DATASET
    •Each attribute is defined by
       •Name
       •Description
       •Type (kind+basic type ex: VOLT32, CUSTOM_STR, etc...)
       •Cardinality (single or array)
       •Flow (Input, Output, Bidirectional)
       •Range



                              11/dic/2011
Thursday, January 10, 13
Hardware Dataset

                                                                 DATASET
                                                         name:VOLT_CHANNEL_1
                                                         type:VOLT32
                                                         flow:output
                                                         range:1-20
                                                         card: 1
                           VOLT_CHANNEL_
                                                         name:VOLT_CHANNEL_2
             RESET                                       type:VOLT32
                                                         flow:output
                                                         range:1-20
                      VOLT_CHANNEL_
                                                         card: 1

                                                         name: RESET
                                                         type: BYTE
                                                         flow: input
                                                         card: 1



                                           11/dic/2011
Thursday, January 10, 13
Hardware DATASET

                                                               The JSON visualization
                                                             of BSON DS representation

                                                         {“device_id”: DEVICE_ID
                           VOLT_CHANNEL_
                                                           “device_ds”: {
                                                            "name"          :   "VOLT_CHANNEL_1",
             RESET
                                                            "desc"          :   "Output volt...",
                                                            "attr_dir"      :   1,
                                                            "type"          :   4,
                                                            "cardinality"   :   1
                      VOLT_CHANNEL_
                                                         !    }

                                                         ....

                                                         }




                                           11/dic/2011                                              2
Thursday, January 10, 13
!CHAOS Unit toolkit



                                   11/dic/2011   3
Thursday, January 10, 13
CU Toolkit


       CUToolkit help the Control Unit development

       Developer need to extend only one class to create a
       CHAOS driver, the “AbstractControlUnit”

       AbstractControlUnit expose all the APIs needed for
       interacting with !CHOAS




                                11/dic/2011
Thursday, January 10, 13
Control Unit

     AbstractControlUnit is an abstract cpp class, that force
     developer to implement some method



                           Control Unit Toolkit
                            Common Toolkit




                                                  11/dic/2011
Thursday, January 10, 13
Control Unit

     AbstractControlUnit is an abstract cpp class, that force
     developer to implement some method


                                                    AbstractControlUnit
                           Control Unit Toolkit       •defineActionAndData
                            Common Toolkit             set
                                                      •init
                                                      •run
                                                      •stop
                                                      •deinit




                                                  11/dic/2011
Thursday, January 10, 13
CUToolkit feature
           multi-threading messaging dispatcher; every CU has it’s own
           thread for dispatching messages after it has been received by
           RPC System

                                                       cpp method:
                                                        setDatasetAttribute
                           CommonToolk
                                                                 Control Unit
                                                                   Toolkit
                                                                   Common
         BSON                                                       Toolkit
            PACK




          sent
          from




Thursday, January 10, 13
CUToolkit feature
           multi-threading messaging dispatcher; every CU has it’s own
           thread for dispatching messages after it has been received by
           RPC System

                                                       cpp method:
                                                        setDatasetAttribute
                           CommonToolk
                                                                 Control Unit
                                                                   Toolkit
                                                                   Common
         BSON                                                       Toolkit
            PACK
                            the RPC dispatcher
                              find destination


          sent
          from




Thursday, January 10, 13
CUToolkit feature
           multi-threading messaging dispatcher; every CU has it’s own
           thread for dispatching messages after it has been received by
           RPC System

                                                               cpp method:
                                                                setDatasetAttribute
                           CommonToolk
                                       CU Message Queue is filled         Control Unit
                                                                           Toolkit
                                                                           Common
         BSON                                                               Toolkit
            PACK
                            the RPC dispatcher
                              find destination


          sent
          from




Thursday, January 10, 13
CUToolkit feature
           multi-threading messaging dispatcher; every CU has it’s own
           thread for dispatching messages after it has been received by
           RPC System

                                                               cpp method:
                                                                setDatasetAttribute
                           CommonToolk                                 AbstractControlU

                                       CU Message Queue is filled        Control Unit
                                                                          Toolkit
                                                 BSON                     Common
         BSON                                     PACK                     Toolkit
            PACK
                            the RPC dispatcher
                              find destination
                                                          Pack is sent
                                                          to CUToolkit
                                                 BSON       scheduler
          sent                                    PACK

          from
                                                 BSON
                                                  PACK




Thursday, January 10, 13
!CHAOS UI Toollkit



                                  11/dic/2011   3
Thursday, January 10, 13
CU Toolkit
      UIToolkit helps the user interface development

      Gives API that help to develop: monitoring processes,
      device controller etc...

      Gives an high performance queue using "locking free"
      design. The queue can manage quantizations , over time,
      for the dataset data acquisition
                                                    UIToolkit

                           DeviceControl                                  DeviceControl


                                            InterThread Cache Logic


                                           Live Acquisition Thread Pool
                            AbsractIO




                                                                                 AbsractIO
                             Driver




                                                                                  Driver


                                                 Network



Thursday, January 10, 13
!CHAOS EU Toolkit



                                  11/dic/2011   3
Thursday, January 10, 13
EU Toolkit


        helps the development of distributed algorithms
        (Execution Unit) that can be used to compute or automate
        the devices control

        The EU can have, in input the device channels or all the
        datasset description, so it work on a "Class" of device not
        directly on one device




Thursday, January 10, 13
EU Toolkit

                           scheme of flow of input or output of an EU

                                                          Specialized Execution Unit

                                   Execution                Abstract Execution Unit                 Result
                                    Queue                                                           Queue

                                                   Input Parameter          Output Parameter




                                Input Param Pack                                               Output Result Pack




Thursday, January 10, 13
EU Toolkit
  exception could be concatenated creating a sequence of any
  combination of:
             • Computing algorithm
             • Control algorithm

                                                     Specialized Execution Unit

                              Execution                Abstract Execution Unit                 Result
                               Queue                                                           Queue

                                              Input Parameter          Output Parameter




                           Input Param Pack                                               Output Result Pack




                                                                                          Input Param Pack

                                                                                                                      Specialized Execution Unit

                                                                                               Execution                Abstract Execution Unit            Result
                                                                                                Queue                                                      Queue

                                                                                                               Input Parameter          Output Parameter




Thursday, January 10, 13
!CHAOS Data Management



                           11/dic/2011   4
Thursday, January 10, 13
Data Management



       !CHOAS has two different management system for the data
       apparatus controlled environment data



                            Live Data
                           History Data



Thursday, January 10, 13
!CHAOS Live Data                                ui_1




                                                       push data
                                                                    live
                                               dev_1                       ui_2
                                                                   cache




                                                                           eu_1




                                 11/dic/2011                                      4
Thursday, January 10, 13
Live Data



          The system of the Live Data, is ideated for the sharing of
           the real time state of the instruments (CU) to other nodes




Thursday, January 10, 13
Live Data

     Every instrument pushes the data of his I/O channels into
      the shared cached through the controller CU



                                                       ui_1




                                   push data
                                                live
                           dev_1                       ui_2
                                               cache




                                                       eu_1




Thursday, January 10, 13
Live Data
   1. every node can read data from the central cache at any
    time[polling]
   2. the nodes can register to central cache for receive
    update[push]

                                                       ui_1




                                   push data
                                                live
                           dev_1                       ui_2
                                               cache




                                                       eu_1




Thursday, January 10, 13
Live Data

                                               scalability & reliability



                                                     ui_1
                           push data
                                        live
             dev_1

                                                            All data of all instruments are
                                       cache



                                                     ui_2
                                                            distributed across many cache
                                                                         server
                           push data
                                        live
             dev_2
                                       cache

                                                     eu_1




Thursday, January 10, 13
Live Data

                                                 scalability & reliability



                                         X             ui_1   with the algorithm previously
             dev_1
                                          live
                                         cache                shown, when one server goes
                             push data
                                                       ui_2
                                                              down, all client that push's, or
                           push data                          read, data on that server, use
                                          live

                                                               another server, with highest
             dev_2
                                         cache

                                                       eu_1

                                                                          priority




Thursday, January 10, 13
Live Data

   The usage of the two modes permits to manage, more
   efficiently, the bandwidth of the flows of the real time data
   (read and write)


                not all nodes need realtime data at the same speed:
      • the mode (1) is for monitoring and control
      • the mode (2) is for control algorithm that can’t lose
      packages




Thursday, January 10, 13
!CHAOS History Data



                             11/dic/2011   4
Thursday, January 10, 13
Live & History Data

                                          ChaosQL
                                          Proxy CQL Interpreter


                           Abstract History Logic                           Cache Logic




                           Common Layer




                                                           Storage Driver




                                                                                Cache Driver
                                            Index Driver




   the live data management is merged into history data
   management into the ChaosQL node with the Cache Driver




Thursday, January 10, 13
History Data



             The System for History Data has been designed to
             efficiently store the serialized data pack, manage
             different type of index on demand, and managing data
             file




Thursday, January 10, 13
History Data
     It is realized by three different services and two


                      ChaosQL                                                                            Indexer                                                Management




                                                                                                                                                                 Storage Management Logic
                                                                                           Management Index Logic




                                                                                                                                                                                            Abstract History Logic
                                                                                                                    Abstract History Logic
                                         Abstract History Logic




                                                                  Storage Driver                                                             Storage Driver                                                          Storage Driver
                       CQL Interpreter




                                                                   Index Driver                                                               Index Driver                                                            Index Driver



                                                                  Common Layer                                                               Common Layer                                                            Common Layer




                                                    Storage Driver                                                                                            Index Driver
                                                                          Storage Driver                                                                                  Index Driver




Thursday, January 10, 13
Chaos Query Language


         ChaosQL implement the following operation on data
            • push history/live data per device
                  • create index on device::attribute::{rule}
                  • delete index on device::attribute
                  • retrieve data with logic operation on attribute and index




Thursday, January 10, 13
History Data

                                                                                 Manages      the insert and select
                                                                                 operations   by parsing the Chaos
                     ChaosQL                                                     Query Language
                                       Abstract History Logic




                                                                Storage Driver
                     CQL Interpreter




                                                                 Index Driver
                                                                                 Insert - as fast as possible stores

                                                                Common Layer
                                                                                 the data into the storage system


                                                                                 select - decodes the query, asks at
                                                                                 the indexer the position in the
                                                                                 storage of the data that match the


Thursday, January 10, 13
History Data

                                UI or CU or EU

                                UI-CU-EU Toolkit

                                Chaos Common
                                                                                  All nodes, using the ChaosQL Driver
                                 CQL I/O Driver

                                Direct I/O Driver                                 (that uses direct I/O) can query the
                                                                                  history and live data.
                                 Query




                                                        Result




                       Proxy CQL Interpreter                                      the   results    are     received   in
                       Abstract History Logic
                                                                                  asynchronous mode
                 Common Layer




                                                                 Storage Driver
                                         Index Driver




Thursday, January 10, 13
Storage




                                  oxy
                            Q L Pr
                           C
                                                                           Managemen
                                                                               t
                                                                 Indexer
                 CUTK
                                                  xy             Cluster
                                             L Pro
               CommonTK                 CQ




                                                            xy
                                                          ro
                                                        LP
                                                       CQ




                                                                           Index




                                              11/dic/2011
Thursday, January 10, 13
Storage




                                         oxy
                                   Q L Pr
                                  C
                                                                                  Managemen
                                                                                      t
                                                                        Indexer
                 CUTK
                                                         xy             Cluster
                                                    L Pro
               CommonTK                        CQ




                                                                   xy
                                                                 ro
                                                               LP

                           Push
                                                              CQ




                           Data
                                                                                  Index




                                                     11/dic/2011
Thursday, January 10, 13
Storage
                                                              Store
                                                              Data




                                         oxy
                                   Q L Pr
                                  C
                                                                                   Managemen
                                                                                       t
                                                                         Indexer
                 CUTK
                                                         xy              Cluster
                                                    L Pro
               CommonTK                        CQ




                                                                    xy
                                                                  ro
                                                                LP

                           Push
                                                               CQ




                           Data
                                                                                   Index




                                                     11/dic/2011
Thursday, January 10, 13
Storage
                                                              Store
                                                              Data




                                         oxy
                                   Q L Pr
                                  C
                                                              Launch               Managemen
                                                               Index                   t
                                                                         Indexer
                 CUTK
                                                         xy              Cluster
                                                    L Pro
               CommonTK                        CQ




                                                                    xy
                                                                  ro
                                                                LP

                           Push
                                                               CQ




                           Data
                                                                                   Index




                                                     11/dic/2011
Thursday, January 10, 13
Storage
                                                              Store
                                                              Data

                                                                                   get data
                                                                                    pack



                                         oxy
                                   Q L Pr
                                  C
                                                              Launch                          Managemen
                                                               Index                              t
                                                                         Indexer
                 CUTK
                                                         xy              Cluster
                                                    L Pro
               CommonTK                        CQ




                                                                    xy
                                                                  ro
                                                                LP

                           Push
                                                               CQ




                           Data
                                                                                              Index




                                                     11/dic/2011
Thursday, January 10, 13
Storage
                                                              Store
                                                              Data

                                                                                   get data
                                                                                    pack



                                         oxy
                                   Q L Pr
                                  C
                                                              Launch                           Managemen
                                                               Index                     Store     t
                                                                                         Index
                                                                         Indexer
                 CUTK
                                                         xy              Cluster
                                                    L Pro
               CommonTK                        CQ




                                                                    xy
                                                                  ro
                                                                LP

                           Push
                                                               CQ




                           Data
                                                                                              Index




                                                     11/dic/2011
Thursday, January 10, 13
Managemen
                                                            t
                                                                               Storage




                    CUTK                                            Indexer
                 CommonTK                                           Cluster
                                   oxy
                             Q L Pr
                            C




                                                   xy
                                              L Pro
                                         CQ




                                                             xy
                                                           ro
                                                         LP
                                                        CQ




                                                                              Index




                                               11/dic/2011
Thursday, January 10, 13
Managemen
                                                                       t
                                                                                          Storage




                    CUTK                                                       Indexer
                 CommonTK                                                      Cluster
                                              oxy
                                        Q L Pr
                                       C




                                                              xy
                                                         L Pro
                                                    CQ



                 EUTK


                                                                        xy
               CommonTK
                                                                      ro
                                                                    LP
                                                                   CQ




                                                                                         Index
                              UITK

                            CommonTK




                                                          11/dic/2011
Thursday, January 10, 13
Managemen
                                                                             t
                                                                                                Storage




                    CUTK                                                             Indexer
                 CommonTK                                                            Cluster
                                                    oxy
                                              Q L Pr
                                             C




                                                                    xy
                                                                 Pro
                            Query Live                    CQ
                                                               L

                              Data
                 EUTK


                                                                              xy
               CommonTK
                                                                            ro
                                                                          LP
                                                                         CQ




                                                                                               Index
                                    UITK

                                  CommonTK




                                                                11/dic/2011
Thursday, January 10, 13
Managemen
                                                                             t
                                                                                                Storage




                    CUTK                                                             Indexer
                 CommonTK                                                            Cluster
                                                    oxy
                                              Q L Pr
                                             C




                                                                    xy
                                                                 Pro
                            Query Live                    CQ
                                                               L

                              Data
                 EUTK


                                                                              xy
               CommonTK
                                                                            ro
                                                                          LP
                                                                         CQ




                                                                                               Index
                                    UITK
                                                           Query
                                  CommonTK
                                                           History
                                                            Data


                                                                11/dic/2011
Thursday, January 10, 13
Managemen
                                                                             t
                                                                                                                 Storage




                    CUTK                                                             Indexer
                 CommonTK                                                            Cluster
                                                    oxy
                                              Q L Pr
                                             C
                                                                                                    execute
                                                                                                 search on idx
                                                                                                      DB
                                                                    xy
                                                                 Pro
                            Query Live                    CQ
                                                               L
                                                                                       send
                              Data                                                   search on
                 EUTK                                                                   idx

                                                                              xy
               CommonTK
                                                                            ro
                                                                          LP
                                                                         CQ




                                                                                                             Index
                                    UITK
                                                           Query
                                  CommonTK
                                                           History
                                                            Data


                                                                11/dic/2011
Thursday, January 10, 13
Managemen
                                                                               t
                                                                                                                   Storage




                                                                          get pack
                    CUTK                                                 at location   Indexer
                 CommonTK                                                              Cluster
                                                    oxy
                                              Q L Pr
                                             C
                                                                                                      execute
                                                                                                   search on idx
                                                                                                        DB
                                                                    xy
                                                                 Pro
                            Query Live                    CQ
                                                               L
                                                                                         send
                              Data                                                     search on
                 EUTK                                                                     idx

                                                                                xy
               CommonTK
                                                                              ro
                                                                            LP
                                                                           CQ




                                                                                                               Index
                                    UITK
                                                           Query
                                  CommonTK
                                                           History
                                                            Data


                                                                11/dic/2011
Thursday, January 10, 13
Managemen
                                                                               t
                                                                                                                   Storage




                                                                          get pack
                    CUTK                                                 at location   Indexer
                 CommonTK                                                              Cluster
                                                    oxy
                                              Q L Pr
                                             C
                                                                                                      execute
                                                                                                   search on idx
                                                                                                        DB
                                                                    xy
                                                                 Pro
                            Query Live                    CQ
                                                               L
                                                                                         send
                              Data                                                     search on
                 EUTK                                                                     idx
                                                 Receive
                                                 History                        xy
               CommonTK
                                                                              ro
                                                                            LP

                                                  Data
                                                                           CQ




                                                                                                               Index
                                    UITK
                                                           Query
                                  CommonTK
                                                           History
                                                            Data


                                                                11/dic/2011
Thursday, January 10, 13
History Data

                                      UI or CU or EU
                                                                                       the query can be splitted in more than
                                      UI-CU-EU Toolkit
                                   QueryChannel
                                                                                       one time period from the receiving proxy.
                                                         Chaos
                                                        Common


                                                                                       The splitted query are sent to other proxy
                                       CQL I/O Driver

                                      Direct I/O Driver


            CQLPack
                                                                                       that will process the sub query
             Original
                                                                        Re
                              ry




                                             lt
                                                          Result




                                                                             sul
                          ue




                                          su                                       t
                                     Re
                         Q




 Proxy CQL Interpreter                                                                 Proxy CQL Interpreter
                                            Proxy CQL Interpreter
 Abstract History Logic                                                                Abstract History Logic
                                            Abstract History Logic
  Common




                                                                                       Common
                    Storage




                                                                                                          Storage
                                              Common
           Driver


                     Driver




                                                                                                 Driver


                                                                                                           Driver
   Layer




                                                                                        Layer
           Index




                                                                                                 Index
                                                                   Storage
                                                        Driver


                                                                    Driver
                                               Layer



                                                        Index




              CQLPack                                  CQLPack                                  CQLPack
                 1                                        2                                        3




                                                       Sub Query




Thursday, January 10, 13
History Data

                                                                                        Executes the necessary operations
                     Indexer                                                            to index the data saved on the
                                                                                        storage system
                     Management Index Logic

                                              Abstract History Logic




                                                                       Storage Driver



                                                                        Index Driver

                                                                                        two type of index:
                                                                                        • TMI - Time Machine index
                                                                       Common Layer




                                                                                        • DSI - Dataset Index




Thursday, January 10, 13
History Data


             TMI - Time Machine index
           •This index permits to catalog every data pack for a
               device for the timestamp.
           • all the data pack are recorded and associated to their
               timestamp
           • this index is managed by system




Thursday, January 10, 13
History Data

             DSI - Dataset Index
           •This index permits to catalog attributes of the dataset
               for a determinated instrument
           • there will be possibility of customize the index
               algorithm
           • this index is created on demand and on specific time
               periods




Thursday, January 10, 13
History Data


                                                                                          Executes management operations
               Management
                                                                                          on files that store the data packs
                     Storage Management Logic

                                                Abstract History Logic




                                                                         Storage Driver



                                                                          Index Driver
                                                                                          some data pack could be subject to
                                                                         Common Layer
                                                                                          ageing


                                                                                          the ageing state will determine the




Thursday, January 10, 13
Project Status
                                 C User Interface                                                                                  LV User Interface                                                                                                         MatLab UI

                                                                                                                 User Interface Toolkit 20%

                                                                                                                                                 Common Toolkit


                                                 distributed                                                                                                  distributed                                                                                              Directory
                                                  Live Data                                                                                                     History                                                                                                 server
          infrastructure                           service                                                                                                    Data service

                                                                                                                                                 Common Toolkit

                                                                                                                                                        CU Toolkit

                                                Control Unit                                                                                            Control Unit                                                                         I/O Unit                                 Exec Unit

                                                                                                                                         CU-LV interface
                           CU modules (config)

                                                 CU modules (init)

                                                                     CU modules (set)

                                                                                        CU modules (de-init)

                                                                                                               CU modules (stop)




                                                                                                                                                                                                                                           I/O modules

                                                                                                                                                                                                                                                         I/O modules

                                                                                                                                                                                                                                                                       I/O modules

                                                                                                                                                                                                                                                                                     EU modules

                                                                                                                                                                                                                                                                                                  EU modules

                                                                                                                                                                                                                                                                                                               EU modules
                                                                                                                                   CU modules (config)


                                                                                                                                                         CU modules (init)

                                                                                                                                                                             CU modules (set)


                                                                                                                                                                                                CU modules (de-init)


                                                                                                                                                                                                                       CU modules (stop)


                                                INFN-Vitrociset meeting                                                                                                                                                                                                                                        23/02/12
Thursday, January 10, 13
developments’ details
                               QT User Interface:	

        prototype ready (Qt)
                               UI Toolkit:	

 	

   	

     prototype ready
                               C wrapper UI Toolkit:	

     prototype ready
                               LV User Interface:	

	

     prototype in preparation (shlib for LV)

                               Live Data:	

    	

   	

   single-instance (no-distributed) prototype ready
                               RPC client/server:	

	

     prototype ready (msgpack and zmq impl)
                               Event client/server:	

  prototype ready (boost asio impl)
             infrastructure    Direct I/O c/s:          todo
                               History Data service:	

 performance test (mongodb) ongoing (UniTV)
                               Directory services:	

        DS simulator ready; DS under development
                               scalability & availability:	

test facility in preparation

                               Common Toolkit:	

 	

       prototype ready
                               CU Toolkit:	

 	

     	

   prototype ready
                               CU modules:	

 	

     	

   prototype ready (C++)
                               CU-LV Interface	

 	

       prototypes
                               & modules:	

 	

  	

       almost ready

                                         INFN-Vitrociset meeting                                       23/02/12
Thursday, January 10, 13
...closing



   in the next year all development force will be focused to
   realize the system for the history data and for the directory
   service, a prototype for each system will be available for 2013
   2Q




Thursday, January 10, 13
CHOAS Group
                                          Claudio Bisegni, Luciano Catani,
                                         Giampiero Di Pirro, Luca Foggetta,
                                         Mazziteli Giovanni, Matteo Mara,
                                             Antonello Paoletti, Stecchi
                                             Alessandro, Federico Zani


                                                      thesis's
                                          Flaminio Antonucci (tor Vergata
                                         Ingengeria), Andrea Capozzi (Tor
                                          Vergata Ingengeria), Francesco
                                            Iesu (Cagliari Infromatica),




                           thanks for the time
Thursday, January 10, 13

Weitere ähnliche Inhalte

Andere mochten auch

2063 p3-in v-teknik komputer dan jaringan 2014-2015
2063 p3-in v-teknik komputer dan jaringan 2014-20152063 p3-in v-teknik komputer dan jaringan 2014-2015
2063 p3-in v-teknik komputer dan jaringan 2014-2015Ygrex Thebygdanns
 
Managing the Chargemaster
Managing the ChargemasterManaging the Chargemaster
Managing the ChargemasterCZNANIEC
 
Preparing the Physician Practice for ICD10
Preparing the Physician Practice for ICD10Preparing the Physician Practice for ICD10
Preparing the Physician Practice for ICD10CZNANIEC
 
Outpatient Auditing Techniques
Outpatient Auditing TechniquesOutpatient Auditing Techniques
Outpatient Auditing TechniquesCZNANIEC
 
E/M Physician And Hospital
E/M Physician And HospitalE/M Physician And Hospital
E/M Physician And HospitalCZNANIEC
 
Soal uts smp 9 terpadu lampang tahun ajaran 2013 2014
Soal uts smp 9 terpadu lampang tahun ajaran 2013 2014Soal uts smp 9 terpadu lampang tahun ajaran 2013 2014
Soal uts smp 9 terpadu lampang tahun ajaran 2013 2014Ygrex Thebygdanns
 
2063 p1-in v-teknik-komputer_jaringan 2014-2015
2063 p1-in v-teknik-komputer_jaringan 2014-20152063 p1-in v-teknik-komputer_jaringan 2014-2015
2063 p1-in v-teknik-komputer_jaringan 2014-2015Ygrex Thebygdanns
 
ACDIS OP-CDI 2011
ACDIS OP-CDI 2011ACDIS OP-CDI 2011
ACDIS OP-CDI 2011CZNANIEC
 
Horizon rptpresentationdraft (3)
Horizon rptpresentationdraft (3)Horizon rptpresentationdraft (3)
Horizon rptpresentationdraft (3)Julie Giuliani
 
Breast Surgery Coding 2010
Breast Surgery Coding 2010Breast Surgery Coding 2010
Breast Surgery Coding 2010CZNANIEC
 
Cấu hình modem alcatel speed touch để xem camera
Cấu hình modem alcatel speed touch để xem cameraCấu hình modem alcatel speed touch để xem camera
Cấu hình modem alcatel speed touch để xem cameraThien Binh
 
2011 CDM Updates Day 2
2011 CDM Updates Day 22011 CDM Updates Day 2
2011 CDM Updates Day 2CZNANIEC
 
2011 CDM Updates Day 1
2011 CDM Updates Day 12011 CDM Updates Day 1
2011 CDM Updates Day 1CZNANIEC
 

Andere mochten auch (17)

2063 p3-in v-teknik komputer dan jaringan 2014-2015
2063 p3-in v-teknik komputer dan jaringan 2014-20152063 p3-in v-teknik komputer dan jaringan 2014-2015
2063 p3-in v-teknik komputer dan jaringan 2014-2015
 
Managing the Chargemaster
Managing the ChargemasterManaging the Chargemaster
Managing the Chargemaster
 
Preparing the Physician Practice for ICD10
Preparing the Physician Practice for ICD10Preparing the Physician Practice for ICD10
Preparing the Physician Practice for ICD10
 
Enhance engage and
Enhance engage andEnhance engage and
Enhance engage and
 
Outpatient Auditing Techniques
Outpatient Auditing TechniquesOutpatient Auditing Techniques
Outpatient Auditing Techniques
 
Iii technology
Iii technologyIii technology
Iii technology
 
E/M Physician And Hospital
E/M Physician And HospitalE/M Physician And Hospital
E/M Physician And Hospital
 
!Chaos Control System
!Chaos Control System!Chaos Control System
!Chaos Control System
 
Beer photos
Beer photosBeer photos
Beer photos
 
Soal uts smp 9 terpadu lampang tahun ajaran 2013 2014
Soal uts smp 9 terpadu lampang tahun ajaran 2013 2014Soal uts smp 9 terpadu lampang tahun ajaran 2013 2014
Soal uts smp 9 terpadu lampang tahun ajaran 2013 2014
 
2063 p1-in v-teknik-komputer_jaringan 2014-2015
2063 p1-in v-teknik-komputer_jaringan 2014-20152063 p1-in v-teknik-komputer_jaringan 2014-2015
2063 p1-in v-teknik-komputer_jaringan 2014-2015
 
ACDIS OP-CDI 2011
ACDIS OP-CDI 2011ACDIS OP-CDI 2011
ACDIS OP-CDI 2011
 
Horizon rptpresentationdraft (3)
Horizon rptpresentationdraft (3)Horizon rptpresentationdraft (3)
Horizon rptpresentationdraft (3)
 
Breast Surgery Coding 2010
Breast Surgery Coding 2010Breast Surgery Coding 2010
Breast Surgery Coding 2010
 
Cấu hình modem alcatel speed touch để xem camera
Cấu hình modem alcatel speed touch để xem cameraCấu hình modem alcatel speed touch để xem camera
Cấu hình modem alcatel speed touch để xem camera
 
2011 CDM Updates Day 2
2011 CDM Updates Day 22011 CDM Updates Day 2
2011 CDM Updates Day 2
 
2011 CDM Updates Day 1
2011 CDM Updates Day 12011 CDM Updates Day 1
2011 CDM Updates Day 1
 

Ähnlich wie !Chaos seminary 12_2012

Dell open stack powered cloud solution introduce & crowbar demo cosug-2012
Dell open stack powered cloud solution introduce & crowbar demo cosug-2012Dell open stack powered cloud solution introduce & crowbar demo cosug-2012
Dell open stack powered cloud solution introduce & crowbar demo cosug-2012OpenCity Community
 
Enabling the next generation airport at DXB
Enabling the next generation airport at DXBEnabling the next generation airport at DXB
Enabling the next generation airport at DXBAmor Group
 
Cloud foundry elastic architecture and deploy based on openstack
Cloud foundry elastic architecture and deploy based on openstackCloud foundry elastic architecture and deploy based on openstack
Cloud foundry elastic architecture and deploy based on openstackOpenCity Community
 
OpenStack Quantum Network Service
OpenStack Quantum Network ServiceOpenStack Quantum Network Service
OpenStack Quantum Network ServiceLew Tucker
 
Compatibleone @ OpenStack In Action
Compatibleone @ OpenStack In Action Compatibleone @ OpenStack In Action
Compatibleone @ OpenStack In Action CompatibleOne
 
Open stackinaction compatibleone 09212011
Open stackinaction compatibleone  09212011Open stackinaction compatibleone  09212011
Open stackinaction compatibleone 09212011CompatibleOne
 
HP Service Delivery Platform 3.0 Launch
HP Service Delivery Platform 3.0 LaunchHP Service Delivery Platform 3.0 Launch
HP Service Delivery Platform 3.0 Launchgrahamwright
 
MPLS/SDN Intersections Next Generation Access Networks at MPLS & Ethernet Wor...
MPLS/SDN Intersections Next Generation Access Networks at MPLS & Ethernet Wor...MPLS/SDN Intersections Next Generation Access Networks at MPLS & Ethernet Wor...
MPLS/SDN Intersections Next Generation Access Networks at MPLS & Ethernet Wor...ADVA
 
Venus-c: Using open source clouds in eScience
Venus-c: Using open source clouds in eScienceVenus-c: Using open source clouds in eScience
Venus-c: Using open source clouds in eScienceOW2
 
The CORA Model Explained
The CORA Model ExplainedThe CORA Model Explained
The CORA Model Explainedtelzinga
 
SAP technology roadmap- 2012 Update
SAP technology roadmap- 2012 UpdateSAP technology roadmap- 2012 Update
SAP technology roadmap- 2012 UpdateA J
 
SDN and Photonics for Dynamic Cloud Connectivity
SDN and Photonics for Dynamic Cloud Connectivity SDN and Photonics for Dynamic Cloud Connectivity
SDN and Photonics for Dynamic Cloud Connectivity ADVA
 
Play with cloud foundry
Play with cloud foundryPlay with cloud foundry
Play with cloud foundryPeng Wan
 
SVG in Data Acquisition and Control Systems
SVG in Data Acquisition and Control SystemsSVG in Data Acquisition and Control Systems
SVG in Data Acquisition and Control SystemsTao Jiang
 
Sdf Panel Nice 2009 V1.0
Sdf Panel Nice 2009 V1.0Sdf Panel Nice 2009 V1.0
Sdf Panel Nice 2009 V1.0lgradina
 
02 Ms Online Identity Session 1
02 Ms Online Identity   Session 102 Ms Online Identity   Session 1
02 Ms Online Identity Session 1Sivadon Chaisiri
 
Riverbed Cascade and VXLAN Monitoring
Riverbed Cascade and VXLAN MonitoringRiverbed Cascade and VXLAN Monitoring
Riverbed Cascade and VXLAN MonitoringRiverbed Technology
 
Riverbed's presentation at SFSE Meetup 2/22
Riverbed's presentation at SFSE Meetup 2/22Riverbed's presentation at SFSE Meetup 2/22
Riverbed's presentation at SFSE Meetup 2/22Sauce Labs
 
Fujitsu Cloud Computing Professional Services
Fujitsu Cloud Computing Professional ServicesFujitsu Cloud Computing Professional Services
Fujitsu Cloud Computing Professional ServicesWilliam Ho (何添福)
 

Ähnlich wie !Chaos seminary 12_2012 (20)

Dell open stack powered cloud solution introduce & crowbar demo cosug-2012
Dell open stack powered cloud solution introduce & crowbar demo cosug-2012Dell open stack powered cloud solution introduce & crowbar demo cosug-2012
Dell open stack powered cloud solution introduce & crowbar demo cosug-2012
 
Enabling the next generation airport at DXB
Enabling the next generation airport at DXBEnabling the next generation airport at DXB
Enabling the next generation airport at DXB
 
Cloud foundry elastic architecture and deploy based on openstack
Cloud foundry elastic architecture and deploy based on openstackCloud foundry elastic architecture and deploy based on openstack
Cloud foundry elastic architecture and deploy based on openstack
 
OpenStack Quantum Network Service
OpenStack Quantum Network ServiceOpenStack Quantum Network Service
OpenStack Quantum Network Service
 
Compatibleone @ OpenStack In Action
Compatibleone @ OpenStack In Action Compatibleone @ OpenStack In Action
Compatibleone @ OpenStack In Action
 
Open stackinaction compatibleone 09212011
Open stackinaction compatibleone  09212011Open stackinaction compatibleone  09212011
Open stackinaction compatibleone 09212011
 
HP Service Delivery Platform 3.0 Launch
HP Service Delivery Platform 3.0 LaunchHP Service Delivery Platform 3.0 Launch
HP Service Delivery Platform 3.0 Launch
 
MPLS/SDN Intersections Next Generation Access Networks at MPLS & Ethernet Wor...
MPLS/SDN Intersections Next Generation Access Networks at MPLS & Ethernet Wor...MPLS/SDN Intersections Next Generation Access Networks at MPLS & Ethernet Wor...
MPLS/SDN Intersections Next Generation Access Networks at MPLS & Ethernet Wor...
 
Venus-c: Using open source clouds in eScience
Venus-c: Using open source clouds in eScienceVenus-c: Using open source clouds in eScience
Venus-c: Using open source clouds in eScience
 
The CORA Model Explained
The CORA Model ExplainedThe CORA Model Explained
The CORA Model Explained
 
SAP technology roadmap- 2012 Update
SAP technology roadmap- 2012 UpdateSAP technology roadmap- 2012 Update
SAP technology roadmap- 2012 Update
 
SDN and Photonics for Dynamic Cloud Connectivity
SDN and Photonics for Dynamic Cloud Connectivity SDN and Photonics for Dynamic Cloud Connectivity
SDN and Photonics for Dynamic Cloud Connectivity
 
Play with cloud foundry
Play with cloud foundryPlay with cloud foundry
Play with cloud foundry
 
SVG in Data Acquisition and Control Systems
SVG in Data Acquisition and Control SystemsSVG in Data Acquisition and Control Systems
SVG in Data Acquisition and Control Systems
 
Sdf Panel Nice 2009 V1.0
Sdf Panel Nice 2009 V1.0Sdf Panel Nice 2009 V1.0
Sdf Panel Nice 2009 V1.0
 
02 Ms Online Identity Session 1
02 Ms Online Identity   Session 102 Ms Online Identity   Session 1
02 Ms Online Identity Session 1
 
Riverbed Cascade and VXLAN Monitoring
Riverbed Cascade and VXLAN MonitoringRiverbed Cascade and VXLAN Monitoring
Riverbed Cascade and VXLAN Monitoring
 
Riverbed's presentation at SFSE Meetup 2/22
Riverbed's presentation at SFSE Meetup 2/22Riverbed's presentation at SFSE Meetup 2/22
Riverbed's presentation at SFSE Meetup 2/22
 
Fujitsu Cloud Computing Professional Services
Fujitsu Cloud Computing Professional ServicesFujitsu Cloud Computing Professional Services
Fujitsu Cloud Computing Professional Services
 
Quantum Networks
Quantum NetworksQuantum Networks
Quantum Networks
 

Kürzlich hochgeladen

unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxBkGupta21
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 

Kürzlich hochgeladen (20)

unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptx
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 

!Chaos seminary 12_2012

  • 1. Control system based on a Highly Abstracted and Open Structure seminar 2012 http://chaos.infn.it C. Thursday, January 10, 13
  • 2. What is !CHAOS (Control system based on Highly Abstracted and Open Structure), is an experiment of CSN 5(technological research experiments) at the LNF and Roma-TV section. The project has been appreciated by educational world and the University of Università TorVergata LNF - Roma2 Università di Cagliari Thursday, January 10, 13
  • 3. Objective !CHAOS has been designed to widely scale and for manage large apparatus with different needs in term of storage technology and speed for data acquisition Thursday, January 10, 13
  • 4. Objective new topology in the CS that permit to integrate the triggered DAQ. define a new way to access realtime data data acquisition, storage and indexing fully scalable Thursday, January 10, 13
  • 5. Objective A CPP Framework and services that permit to have: • 99% of adaptation time of instrument to !CHOAS done by framework • simplify the creation of the user interfaces • simplify the creation of the control algorithms basing it on an abstraction of the I/O channels or full dataset not on physical device Thursday, January 10, 13
  • 7. Simplify Vision User specialized Boundary Implementation CHAOS Boundary Control's Algorithm Implementation Implementation Instrument's network User Interface Driver Live Data Directory Data Cloud Server Cloud History Data Implementation Thursday, January 10, 13
  • 8. Simplify Vision User specialized Boundary Implementation CHAOS Boundary Control's Algorithm Abstract RPC Driver Implementation Implementation Instrument's Abstract network User Interface Driver Event Driver Abstract Direct I/O Live Data Directory Data Cloud Server Cloud History Data Implementation Thursday, January 10, 13
  • 9. Simplify Vision User specialized Boundary Implementation CHAOS Boundary Control's Algorithm Abstract RPC Driver Implementation Implementation Instrument's Abstract network User Interface Driver Event Driver Abstract Direct I/O Live Data Live Data Live Data Directory Data Cloud Data Cloud Server Data Cloud Cloud History History Data Data History Data Implementation Implementation Implementation Thursday, January 10, 13
  • 10. !CHAOS Real Model Thursday, January 10, 13
  • 11. User User Interface for monitoring and controls Customization Abstraction Boundary User Interface Toolkit Common Toolkit RPC - Event - Direct I/O Data Boundary CQL Proxy Nodes Informatio Execution Unit Toolkit n RPC - Event - Direct I/O Indexer Nodes Node Common Toolkit Management Nodes Information HTTP - API Live / History Security domain Controls Node Algorithm Activity Chaos Live / History Directory domain Nodes RPC - Event - Direct I/O Common Toolkit Control Unit Toolkit Driver for sensor and instruments Thursday, January 10, 13
  • 12. Toolkit and Services !CHAOS is a set of toolkit and services that will permit the realization of an control system Thursday, January 10, 13
  • 13. !CHAOS Toolkit and Services Common Toolkit Directory Services CU Toolkit Live and History Data UI Toolkit CQL Proxy EU Toolkit Indexer Managment Thursday, January 10, 13
  • 15. !CHAOS Toolkit Common implements the fundamental software's layer to RPC, I/O, Event and other’s utility Toolkit code and for logic abstracts the !CHAOS resources to the CU Toolkit device drivers developers. abstracts the CHAOS resources to the UI Toolkit developers that need to develop control and monitoring user interface abstracts the CHAOS resources to the EU Toolkit developers that need to develop control and computing algorithms Thursday, January 10, 13
  • 17. !CHAOS Services Directory gives services via RPC (or with other higher transport layers) for get device Services structured information, system topology give the access to insert and query CQL Proxy operation on live and history data to other CHOAS node using CHOAS Query performs the indexing of the new Indexer inserted data performs the management of the datas Management chunk saved on the storage (erase, move, Thursday, January 10, 13
  • 18. !CHOAS Scalability and Fault- Some node needs to be scaled for performance reason or for fault-tolerant: • Directory Nodes • Live and History nodes Thursday, January 10, 13
  • 19. !CHOAS Scalability and Fault- Server P1 Server P2 Framework Device 1 Driver X X Server P1 Server PN X Server P1 Server P2 Client Reader Server P2 Driver Server PN Framework Clients that need to talk to scaled services, has more than one ip address for every service Thursday, January 10, 13
  • 20. !CHOAS Scalability and Fault- Server P1 Server P2 Framework Device 1 Driver X X Server P1 Server PN X Server P1 Server P2 Client Reader Server P2 Driver Server PN Framework Every ip address has a priority, the client, as first address, uses the one with highest priority Thursday, January 10, 13
  • 21. !CHOAS Scalability and Fault- Server P1 Server P2 Framework Device 1 Driver X X Server P1 Server PN X Server P1 Server P2 Client Reader Server P2 Driver Server PN Framework when a server goes down, another ip will be used Thursday, January 10, 13
  • 22. !CHOAS Scalability and Fault- Server P1 Server P2 Framework use at ok Device 1 Server P1 Driver TS1 Server PN use at Server P1 TS1 Server P2 Client Reader Server P2 Driver Server PN Framework the re-balance occurs when a server with a highest priority, previously goes down or stopped for maintenance, becomes up and running Thursday, January 10, 13
  • 23. !CHOAS Scalability and Fault- Server P1 Server P2 Framework use at ok Device 1 Server P1 Driver TS1 Server PN use at Server P1 TS1 Server P2 Client Reader Server P2 Driver Server PN Framework the new server will be used starting from a pre-determinated time stamp (TS1) Thursday, January 10, 13
  • 24. !CHOAS Scalability and Fault- Server P1 Server P2 Framework use at ok Device 1 Server P1 Driver TS1 Server PN use at Server P1 TS1 Server P2 Client Reader Server P2 Driver Server PN Framework all the operations and the calculation of TS1 are coordinated by one of the directory nodes Thursday, January 10, 13
  • 25. !CHOAS Communication 11/dic/2011 1 Thursday, January 10, 13
  • 26. Communication Systems !CHOAS has three different communication system: RPC, used for command (messages that need to be securely received and recorded and for message that need a response [ asynchronous ]) Events, used for inform to a wide range of nodes, that something has happened (device heartbeat, alert on device channel, received command, etc..) Thursday, January 10, 13
  • 27. Communication Systems Every !CHAOS node has at last one client and server instance of any of the three system Thursday, January 10, 13
  • 28. RPC System •RPC layer is implemented as a plug-ins System, it's is abstracted to internal CHAOS framework so we con change it • used to send important messages that need to be sure of it's delivering status • can be used to receive asynchronous responses • the set command for input channel on control unit is done with RPC 09/dic/2011 Thursday, January 10, 13
  • 29. RPC System Message Answer [optional] Messagge forwarding Response forwarding Node A Node B Node A Node B ack [sync] ack [sync] Thursday, January 10, 13
  • 30. RPC System RPC Message Flow • request message is constructed with BSON, serialized and sent to RPC end point • if the requester want a response it put answer-code and sender-address in BSON pack,and automatically RPC System wait for the answer • the request and response involves the forwarding of the ack message, from the receiver to the sender in Message Answer [optional] Messagge forwarding Response forwarding Node A Node B Node A Node B ack [sync] ack [sync] Thursday, January 10, 13
  • 31. Event System • Event is implemented as a plug-ins System it's is abstracted to internal CHAOS framework so we con change it • there are four type of event: • Alert (highest priority) • Instrument • Command • Custom • all information are packed into 1024 byte and transported by protocols that support multicast (UDP, PGM[http:// en.wikipedia.org/wiki/Pragmatic_General_Multicast]) 09/dic/2011 Thursday, January 10, 13
  • 32. Event System Some numbers about event data • Every event type has 2^16 sub type • Every sub type can have up to 2^16 priority level • Every subtype can have up to 255 byte of information 09/dic/2011 Thursday, January 10, 13
  • 33. Direct I/O System • Direct I/O is implemented as a plug-ins system • used to make insert or select into the live and the history data • can be used as stream for receive result data of the query or read the stored raw data file of the devices 09/dic/2011 Thursday, January 10, 13
  • 34. !CHAOS Abstraction of the 11/dic/2011 2 Thursday, January 10, 13
  • 35. Abstraction !CHAOS use a structured serialization for data description and serialization • BSON (http://bsonspec.org/) has been chosen, very similar to JSON but it is binary • serialization is wrapped into a CPP class so it can be changed infuture • it is used in: •Hardware dataset description's and acquired data packaging (“channel desc” = value) •RPC message •Chaos Query Language Thursday, January 10, 13
  • 36. Hardware Dataset •Hardware attributes and I/O channels are described into the DATASET •Each attribute is defined by •Name •Description •Type (kind+basic type ex: VOLT32, CUSTOM_STR, etc...) •Cardinality (single or array) •Flow (Input, Output, Bidirectional) •Range 11/dic/2011 Thursday, January 10, 13
  • 37. Hardware Dataset DATASET name:VOLT_CHANNEL_1 type:VOLT32 flow:output range:1-20 card: 1 VOLT_CHANNEL_ name:VOLT_CHANNEL_2 RESET type:VOLT32 flow:output range:1-20 VOLT_CHANNEL_ card: 1 name: RESET type: BYTE flow: input card: 1 11/dic/2011 Thursday, January 10, 13
  • 38. Hardware DATASET The JSON visualization of BSON DS representation {“device_id”: DEVICE_ID VOLT_CHANNEL_ “device_ds”: { "name" : "VOLT_CHANNEL_1", RESET "desc" : "Output volt...", "attr_dir" : 1, "type" : 4, "cardinality" : 1 VOLT_CHANNEL_ ! } .... } 11/dic/2011 2 Thursday, January 10, 13
  • 39. !CHAOS Unit toolkit 11/dic/2011 3 Thursday, January 10, 13
  • 40. CU Toolkit CUToolkit help the Control Unit development Developer need to extend only one class to create a CHAOS driver, the “AbstractControlUnit” AbstractControlUnit expose all the APIs needed for interacting with !CHOAS 11/dic/2011 Thursday, January 10, 13
  • 41. Control Unit AbstractControlUnit is an abstract cpp class, that force developer to implement some method Control Unit Toolkit Common Toolkit 11/dic/2011 Thursday, January 10, 13
  • 42. Control Unit AbstractControlUnit is an abstract cpp class, that force developer to implement some method AbstractControlUnit Control Unit Toolkit •defineActionAndData Common Toolkit set •init •run •stop •deinit 11/dic/2011 Thursday, January 10, 13
  • 43. CUToolkit feature multi-threading messaging dispatcher; every CU has it’s own thread for dispatching messages after it has been received by RPC System cpp method: setDatasetAttribute CommonToolk Control Unit Toolkit Common BSON Toolkit PACK sent from Thursday, January 10, 13
  • 44. CUToolkit feature multi-threading messaging dispatcher; every CU has it’s own thread for dispatching messages after it has been received by RPC System cpp method: setDatasetAttribute CommonToolk Control Unit Toolkit Common BSON Toolkit PACK the RPC dispatcher find destination sent from Thursday, January 10, 13
  • 45. CUToolkit feature multi-threading messaging dispatcher; every CU has it’s own thread for dispatching messages after it has been received by RPC System cpp method: setDatasetAttribute CommonToolk CU Message Queue is filled Control Unit Toolkit Common BSON Toolkit PACK the RPC dispatcher find destination sent from Thursday, January 10, 13
  • 46. CUToolkit feature multi-threading messaging dispatcher; every CU has it’s own thread for dispatching messages after it has been received by RPC System cpp method: setDatasetAttribute CommonToolk AbstractControlU CU Message Queue is filled Control Unit Toolkit BSON Common BSON PACK Toolkit PACK the RPC dispatcher find destination Pack is sent to CUToolkit BSON scheduler sent PACK from BSON PACK Thursday, January 10, 13
  • 47. !CHAOS UI Toollkit 11/dic/2011 3 Thursday, January 10, 13
  • 48. CU Toolkit UIToolkit helps the user interface development Gives API that help to develop: monitoring processes, device controller etc... Gives an high performance queue using "locking free" design. The queue can manage quantizations , over time, for the dataset data acquisition UIToolkit DeviceControl DeviceControl InterThread Cache Logic Live Acquisition Thread Pool AbsractIO AbsractIO Driver Driver Network Thursday, January 10, 13
  • 49. !CHAOS EU Toolkit 11/dic/2011 3 Thursday, January 10, 13
  • 50. EU Toolkit helps the development of distributed algorithms (Execution Unit) that can be used to compute or automate the devices control The EU can have, in input the device channels or all the datasset description, so it work on a "Class" of device not directly on one device Thursday, January 10, 13
  • 51. EU Toolkit scheme of flow of input or output of an EU Specialized Execution Unit Execution Abstract Execution Unit Result Queue Queue Input Parameter Output Parameter Input Param Pack Output Result Pack Thursday, January 10, 13
  • 52. EU Toolkit exception could be concatenated creating a sequence of any combination of: • Computing algorithm • Control algorithm Specialized Execution Unit Execution Abstract Execution Unit Result Queue Queue Input Parameter Output Parameter Input Param Pack Output Result Pack Input Param Pack Specialized Execution Unit Execution Abstract Execution Unit Result Queue Queue Input Parameter Output Parameter Thursday, January 10, 13
  • 53. !CHAOS Data Management 11/dic/2011 4 Thursday, January 10, 13
  • 54. Data Management !CHOAS has two different management system for the data apparatus controlled environment data Live Data History Data Thursday, January 10, 13
  • 55. !CHAOS Live Data ui_1 push data live dev_1 ui_2 cache eu_1 11/dic/2011 4 Thursday, January 10, 13
  • 56. Live Data The system of the Live Data, is ideated for the sharing of the real time state of the instruments (CU) to other nodes Thursday, January 10, 13
  • 57. Live Data Every instrument pushes the data of his I/O channels into the shared cached through the controller CU ui_1 push data live dev_1 ui_2 cache eu_1 Thursday, January 10, 13
  • 58. Live Data 1. every node can read data from the central cache at any time[polling] 2. the nodes can register to central cache for receive update[push] ui_1 push data live dev_1 ui_2 cache eu_1 Thursday, January 10, 13
  • 59. Live Data scalability & reliability ui_1 push data live dev_1 All data of all instruments are cache ui_2 distributed across many cache server push data live dev_2 cache eu_1 Thursday, January 10, 13
  • 60. Live Data scalability & reliability X ui_1 with the algorithm previously dev_1 live cache shown, when one server goes push data ui_2 down, all client that push's, or push data read, data on that server, use live another server, with highest dev_2 cache eu_1 priority Thursday, January 10, 13
  • 61. Live Data The usage of the two modes permits to manage, more efficiently, the bandwidth of the flows of the real time data (read and write) not all nodes need realtime data at the same speed: • the mode (1) is for monitoring and control • the mode (2) is for control algorithm that can’t lose packages Thursday, January 10, 13
  • 62. !CHAOS History Data 11/dic/2011 4 Thursday, January 10, 13
  • 63. Live & History Data ChaosQL Proxy CQL Interpreter Abstract History Logic Cache Logic Common Layer Storage Driver Cache Driver Index Driver the live data management is merged into history data management into the ChaosQL node with the Cache Driver Thursday, January 10, 13
  • 64. History Data The System for History Data has been designed to efficiently store the serialized data pack, manage different type of index on demand, and managing data file Thursday, January 10, 13
  • 65. History Data It is realized by three different services and two ChaosQL Indexer Management Storage Management Logic Management Index Logic Abstract History Logic Abstract History Logic Abstract History Logic Storage Driver Storage Driver Storage Driver CQL Interpreter Index Driver Index Driver Index Driver Common Layer Common Layer Common Layer Storage Driver Index Driver Storage Driver Index Driver Thursday, January 10, 13
  • 66. Chaos Query Language ChaosQL implement the following operation on data • push history/live data per device • create index on device::attribute::{rule} • delete index on device::attribute • retrieve data with logic operation on attribute and index Thursday, January 10, 13
  • 67. History Data Manages the insert and select operations by parsing the Chaos ChaosQL Query Language Abstract History Logic Storage Driver CQL Interpreter Index Driver Insert - as fast as possible stores Common Layer the data into the storage system select - decodes the query, asks at the indexer the position in the storage of the data that match the Thursday, January 10, 13
  • 68. History Data UI or CU or EU UI-CU-EU Toolkit Chaos Common All nodes, using the ChaosQL Driver CQL I/O Driver Direct I/O Driver (that uses direct I/O) can query the history and live data. Query Result Proxy CQL Interpreter the results are received in Abstract History Logic asynchronous mode Common Layer Storage Driver Index Driver Thursday, January 10, 13
  • 69. Storage oxy Q L Pr C Managemen t Indexer CUTK xy Cluster L Pro CommonTK CQ xy ro LP CQ Index 11/dic/2011 Thursday, January 10, 13
  • 70. Storage oxy Q L Pr C Managemen t Indexer CUTK xy Cluster L Pro CommonTK CQ xy ro LP Push CQ Data Index 11/dic/2011 Thursday, January 10, 13
  • 71. Storage Store Data oxy Q L Pr C Managemen t Indexer CUTK xy Cluster L Pro CommonTK CQ xy ro LP Push CQ Data Index 11/dic/2011 Thursday, January 10, 13
  • 72. Storage Store Data oxy Q L Pr C Launch Managemen Index t Indexer CUTK xy Cluster L Pro CommonTK CQ xy ro LP Push CQ Data Index 11/dic/2011 Thursday, January 10, 13
  • 73. Storage Store Data get data pack oxy Q L Pr C Launch Managemen Index t Indexer CUTK xy Cluster L Pro CommonTK CQ xy ro LP Push CQ Data Index 11/dic/2011 Thursday, January 10, 13
  • 74. Storage Store Data get data pack oxy Q L Pr C Launch Managemen Index Store t Index Indexer CUTK xy Cluster L Pro CommonTK CQ xy ro LP Push CQ Data Index 11/dic/2011 Thursday, January 10, 13
  • 75. Managemen t Storage CUTK Indexer CommonTK Cluster oxy Q L Pr C xy L Pro CQ xy ro LP CQ Index 11/dic/2011 Thursday, January 10, 13
  • 76. Managemen t Storage CUTK Indexer CommonTK Cluster oxy Q L Pr C xy L Pro CQ EUTK xy CommonTK ro LP CQ Index UITK CommonTK 11/dic/2011 Thursday, January 10, 13
  • 77. Managemen t Storage CUTK Indexer CommonTK Cluster oxy Q L Pr C xy Pro Query Live CQ L Data EUTK xy CommonTK ro LP CQ Index UITK CommonTK 11/dic/2011 Thursday, January 10, 13
  • 78. Managemen t Storage CUTK Indexer CommonTK Cluster oxy Q L Pr C xy Pro Query Live CQ L Data EUTK xy CommonTK ro LP CQ Index UITK Query CommonTK History Data 11/dic/2011 Thursday, January 10, 13
  • 79. Managemen t Storage CUTK Indexer CommonTK Cluster oxy Q L Pr C execute search on idx DB xy Pro Query Live CQ L send Data search on EUTK idx xy CommonTK ro LP CQ Index UITK Query CommonTK History Data 11/dic/2011 Thursday, January 10, 13
  • 80. Managemen t Storage get pack CUTK at location Indexer CommonTK Cluster oxy Q L Pr C execute search on idx DB xy Pro Query Live CQ L send Data search on EUTK idx xy CommonTK ro LP CQ Index UITK Query CommonTK History Data 11/dic/2011 Thursday, January 10, 13
  • 81. Managemen t Storage get pack CUTK at location Indexer CommonTK Cluster oxy Q L Pr C execute search on idx DB xy Pro Query Live CQ L send Data search on EUTK idx Receive History xy CommonTK ro LP Data CQ Index UITK Query CommonTK History Data 11/dic/2011 Thursday, January 10, 13
  • 82. History Data UI or CU or EU the query can be splitted in more than UI-CU-EU Toolkit QueryChannel one time period from the receiving proxy. Chaos Common The splitted query are sent to other proxy CQL I/O Driver Direct I/O Driver CQLPack that will process the sub query Original Re ry lt Result sul ue su t Re Q Proxy CQL Interpreter Proxy CQL Interpreter Proxy CQL Interpreter Abstract History Logic Abstract History Logic Abstract History Logic Common Common Storage Storage Common Driver Driver Driver Driver Layer Layer Index Index Storage Driver Driver Layer Index CQLPack CQLPack CQLPack 1 2 3 Sub Query Thursday, January 10, 13
  • 83. History Data Executes the necessary operations Indexer to index the data saved on the storage system Management Index Logic Abstract History Logic Storage Driver Index Driver two type of index: • TMI - Time Machine index Common Layer • DSI - Dataset Index Thursday, January 10, 13
  • 84. History Data TMI - Time Machine index •This index permits to catalog every data pack for a device for the timestamp. • all the data pack are recorded and associated to their timestamp • this index is managed by system Thursday, January 10, 13
  • 85. History Data DSI - Dataset Index •This index permits to catalog attributes of the dataset for a determinated instrument • there will be possibility of customize the index algorithm • this index is created on demand and on specific time periods Thursday, January 10, 13
  • 86. History Data Executes management operations Management on files that store the data packs Storage Management Logic Abstract History Logic Storage Driver Index Driver some data pack could be subject to Common Layer ageing the ageing state will determine the Thursday, January 10, 13
  • 87. Project Status C User Interface LV User Interface MatLab UI User Interface Toolkit 20% Common Toolkit distributed distributed Directory Live Data History server infrastructure service Data service Common Toolkit CU Toolkit Control Unit Control Unit I/O Unit Exec Unit CU-LV interface CU modules (config) CU modules (init) CU modules (set) CU modules (de-init) CU modules (stop) I/O modules I/O modules I/O modules EU modules EU modules EU modules CU modules (config) CU modules (init) CU modules (set) CU modules (de-init) CU modules (stop) INFN-Vitrociset meeting 23/02/12 Thursday, January 10, 13
  • 88. developments’ details QT User Interface: prototype ready (Qt) UI Toolkit: prototype ready C wrapper UI Toolkit: prototype ready LV User Interface: prototype in preparation (shlib for LV) Live Data: single-instance (no-distributed) prototype ready RPC client/server: prototype ready (msgpack and zmq impl) Event client/server: prototype ready (boost asio impl) infrastructure Direct I/O c/s: todo History Data service: performance test (mongodb) ongoing (UniTV) Directory services: DS simulator ready; DS under development scalability & availability: test facility in preparation Common Toolkit: prototype ready CU Toolkit: prototype ready CU modules: prototype ready (C++) CU-LV Interface prototypes & modules: almost ready INFN-Vitrociset meeting 23/02/12 Thursday, January 10, 13
  • 89. ...closing in the next year all development force will be focused to realize the system for the history data and for the directory service, a prototype for each system will be available for 2013 2Q Thursday, January 10, 13
  • 90. CHOAS Group Claudio Bisegni, Luciano Catani, Giampiero Di Pirro, Luca Foggetta, Mazziteli Giovanni, Matteo Mara, Antonello Paoletti, Stecchi Alessandro, Federico Zani thesis's Flaminio Antonucci (tor Vergata Ingengeria), Andrea Capozzi (Tor Vergata Ingengeria), Francesco Iesu (Cagliari Infromatica), thanks for the time Thursday, January 10, 13