SlideShare ist ein Scribd-Unternehmen logo
1 von 9
Q-Aggregate Based
                                   Gene Expression Programming



                                                          2006-03-16

                                                          Magdi Mohamed (PRR CoE, Labs)
                                                          Weimin Xiao   (PRR CoE, Labs)
                                                          Chi Zhou      (PRR CoE, Labs)




              Method for Constructing Compact Infinite-Valued Logical Forms
             Using Gene Expression Programming with Q-Aggregate Operators

2006:03:16                             Magdi A. Mohamed                             1/9
Q-Aggregate Operator Impacts
                   on soft computing and computational intelligence paradigms

1.    Neural Networks
      •  avoids “sigmoid-like” function limitations
      •  simplifies implementations using per-unit values
2.    Genetic Computing
      •  reduces complexity of Gene Expression Programming (GEP)
      •  provides natural interpretations of rules
3.    Crisp and Soft Rule-Based Systems
      •   opens a new paradigm shift in rule-based systems (Beyond Fuzzy Logic)
      •   provides higher degrees of automation




2006:03:16                               Magdi A. Mohamed                       2/9
Closest Prior Art on GEP
        IEEE-TEC 12/2003, “Evolving Accurate & Compact Classification Rules with GEP”, by Chi Zhou et al.




                                   7−S
• A genetic-algorithm-based
                                                                                              /
  symbolic regression algorithm
                                F ( P + 3)
  (Ferreira 2001, Zhou 2003).
                                                                                      -               *
• Simpler and faster than
                                                                               7          S       +           F
  conventional Genetic
  Programming approaches.
                                                                                              3
• An expression tree is flattened
  to form a chromosome.
                                                                                                          P

                       /      -      *      7      S       +        F   3             P
  2006:03:16                                     Magdi A. Mohamed                                             3/9
The GEP Algorithm
                           Probabilistic Search



             Create Chromosomes of Initial Population
             Create Chromosomes of Initial Population

                  Express Chromosomes as ETs
                  Express Chromosomes as ETs

               Evaluate Fitness of Expression Trees
               Evaluate Fitness of Expression Trees

              Select New Population Probabilistically
              Select New Population Probabilistically

                Crossover, Mutation, and Rotation
                Crossover, Mutation, and Rotation

                Chromosomes for New Generation
                Chromosomes for New Generation

                                                        Yes
                      Termination Criterion                   Terminate
                          Satisfied?
                                    No


2006:03:16                    Magdi A. Mohamed                            4/9
Innovation
                            Gene Expression Programming with Q-Aggregate Operators



•        A binary operation expressed by a unique Q-Aggregate operator
         characterizing disjunction, compensation, and conjunction operations:

                 A(x,y) = (x + y + lambda * x * y) / (2 + lambda)

•        A unary operation expressed by a formula which generates value in the unit
         interval characterizing Negation operations:

                 N(x) = (1 – x) / (1 + gamma * x)

•        The training task is performed as an optimization process where flexible
         model parameters (lambda/gamma values) are adjusted for each internal
         node in a general non-fixed tree structure to minimize an overall risk criterion
         using Gene Expression Programming and Differential Evolution techniques.




    2006:03:16                                  Magdi A. Mohamed                       5/9
Innovation
                            Gene Expression Programming with Q-Aggregate Operators


                           y1                                                    y2

                            A                                                    A


                  A                A                                   A              N

             x1       x2         A      0.6                        N        x2            x3


                           0.3         N                           A


                                       x3                    0.7       N


                                                                       x1

2006:03:16                                      Magdi A. Mohamed                               6/9
Experimental Results
                              Nonlinear Regression Case


//XOR problem

 a=0; b=0; Y = 0;
 a=1; b=0; Y = 1;
 a=0; b=1; Y = 1;
 a=1; b=1; Y = 0;

//Best Tree
 QA       0
 |QA      0
 ||a      1.0
 ||N      0
 |||b     1.0
 |QA      0
 ||b      1.0
 ||N      0
 |||a     1.0
;PGEP Formula:
;QA(QA(a, N(b, gamma = 0), lambda= 164015206785.06702), QA(b, N(a, gamma = 0),
lambda = 231379355057.6994 ), lambda = -0.99998 )

 2006:03:16                        Magdi A. Mohamed                              7/9
Experimental Results
                                     Nonlinear Classification Case



                           Test Accuracy on Monk's Problems
                                           M1             M2          M3

                           C4.5         75.70%         65.00%        97.20%

                         C4.5Rules       100%          66.20%        96.30%

                           GEP           100%          99.07%        100%

                         QAB-GEP         100%           …..%         100%


  QAB-GEP is able to generate accurate, compact, and noise tolerant rules
             Rule 1: QA(a5_1, N(a2_3, gamma= 0.050061), lambda= 0.548952)
             Rule 2: N(QA(a2_3, a5_4, lambda= 0.221941), gamma= 0.535057)
             Rule 3: QA(a4_1, a5_3, lambda= 0.602964)
                 The average size of Monks-3 rules is 4, while GEP has 14

  The set of operations {QA, N} can replace the list of pre-selected functions
or operators, e.g., {IF, AND, OR, +, -, *, /, …}
2006:03:16                                  Magdi A. Mohamed                     8/9
Advantages of QAB-GEP
                                    Characteristics and Promises



1.      compact set of operations { Aggregate (x,y), Negate (x) }
2.      numerical stability
3.      per unit calculations simplify implementations (software and hardware)
4.      suitability for massive parallel implementations
5.      automatic discovery of complex logical expressions
6.      consistent handling of unary, binary, and n-ary operations
7.      improvement over conventional GEP techniques in terms of average rule size
8.      usability for both classification and regression applications
9.      ease of use and interpretability




     2006:03:16                            Magdi A. Mohamed                      9/9

Weitere ähnliche Inhalte

Andere mochten auch

4901 4905.output
4901 4905.output4901 4905.output
4901 4905.outputj1075017
 
Afla ce tip de angajat esti
Afla ce tip de angajat estiAfla ce tip de angajat esti
Afla ce tip de angajat estiEdenred Romania
 
UMAlliesTrainingStudent2015
UMAlliesTrainingStudent2015UMAlliesTrainingStudent2015
UMAlliesTrainingStudent2015Timothy Kay
 
Het gebruik van mobiele toeristische apps
Het gebruik van mobiele toeristische appsHet gebruik van mobiele toeristische apps
Het gebruik van mobiele toeristische appsMichelle Lenaerts
 

Andere mochten auch (6)

4901 4905.output
4901 4905.output4901 4905.output
4901 4905.output
 
Afla ce tip de angajat esti
Afla ce tip de angajat estiAfla ce tip de angajat esti
Afla ce tip de angajat esti
 
Mg 8 gwst geo park
Mg 8 gwst geo parkMg 8 gwst geo park
Mg 8 gwst geo park
 
UMAlliesTrainingStudent2015
UMAlliesTrainingStudent2015UMAlliesTrainingStudent2015
UMAlliesTrainingStudent2015
 
Het gebruik van mobiele toeristische apps
Het gebruik van mobiele toeristische appsHet gebruik van mobiele toeristische apps
Het gebruik van mobiele toeristische apps
 
CVForm-1
CVForm-1CVForm-1
CVForm-1
 

Mehr von Magdi Mohamed

Gradient Direction Transform
Gradient Direction TransformGradient Direction Transform
Gradient Direction TransformMagdi Mohamed
 
Smart Power Amplifier
Smart Power AmplifierSmart Power Amplifier
Smart Power AmplifierMagdi Mohamed
 
Relaxation Methods and Means for Optical Tracking of Deformable Objects
Relaxation Methods and Means for Optical Tracking of Deformable ObjectsRelaxation Methods and Means for Optical Tracking of Deformable Objects
Relaxation Methods and Means for Optical Tracking of Deformable ObjectsMagdi Mohamed
 
Q-Metrics in Theory and Practice
Q-Metrics in Theory and PracticeQ-Metrics in Theory and Practice
Q-Metrics in Theory and PracticeMagdi Mohamed
 
Q-Metric Based Support Vector Machines
Q-Metric Based Support Vector MachinesQ-Metric Based Support Vector Machines
Q-Metric Based Support Vector MachinesMagdi Mohamed
 
Q-filter Structures for Advancing Pattern Recognition Systems
Q-filter Structures for Advancing Pattern Recognition SystemsQ-filter Structures for Advancing Pattern Recognition Systems
Q-filter Structures for Advancing Pattern Recognition SystemsMagdi Mohamed
 

Mehr von Magdi Mohamed (7)

Gradient Direction Transform
Gradient Direction TransformGradient Direction Transform
Gradient Direction Transform
 
The Broker Phone
The Broker PhoneThe Broker Phone
The Broker Phone
 
Smart Power Amplifier
Smart Power AmplifierSmart Power Amplifier
Smart Power Amplifier
 
Relaxation Methods and Means for Optical Tracking of Deformable Objects
Relaxation Methods and Means for Optical Tracking of Deformable ObjectsRelaxation Methods and Means for Optical Tracking of Deformable Objects
Relaxation Methods and Means for Optical Tracking of Deformable Objects
 
Q-Metrics in Theory and Practice
Q-Metrics in Theory and PracticeQ-Metrics in Theory and Practice
Q-Metrics in Theory and Practice
 
Q-Metric Based Support Vector Machines
Q-Metric Based Support Vector MachinesQ-Metric Based Support Vector Machines
Q-Metric Based Support Vector Machines
 
Q-filter Structures for Advancing Pattern Recognition Systems
Q-filter Structures for Advancing Pattern Recognition SystemsQ-filter Structures for Advancing Pattern Recognition Systems
Q-filter Structures for Advancing Pattern Recognition Systems
 

Q-Aggregate Based Gene Expression Programming

  • 1. Q-Aggregate Based Gene Expression Programming 2006-03-16 Magdi Mohamed (PRR CoE, Labs) Weimin Xiao (PRR CoE, Labs) Chi Zhou (PRR CoE, Labs) Method for Constructing Compact Infinite-Valued Logical Forms Using Gene Expression Programming with Q-Aggregate Operators 2006:03:16 Magdi A. Mohamed 1/9
  • 2. Q-Aggregate Operator Impacts on soft computing and computational intelligence paradigms 1. Neural Networks • avoids “sigmoid-like” function limitations • simplifies implementations using per-unit values 2. Genetic Computing • reduces complexity of Gene Expression Programming (GEP) • provides natural interpretations of rules 3. Crisp and Soft Rule-Based Systems • opens a new paradigm shift in rule-based systems (Beyond Fuzzy Logic) • provides higher degrees of automation 2006:03:16 Magdi A. Mohamed 2/9
  • 3. Closest Prior Art on GEP IEEE-TEC 12/2003, “Evolving Accurate & Compact Classification Rules with GEP”, by Chi Zhou et al. 7−S • A genetic-algorithm-based / symbolic regression algorithm F ( P + 3) (Ferreira 2001, Zhou 2003). - * • Simpler and faster than 7 S + F conventional Genetic Programming approaches. 3 • An expression tree is flattened to form a chromosome. P / - * 7 S + F 3 P 2006:03:16 Magdi A. Mohamed 3/9
  • 4. The GEP Algorithm Probabilistic Search Create Chromosomes of Initial Population Create Chromosomes of Initial Population Express Chromosomes as ETs Express Chromosomes as ETs Evaluate Fitness of Expression Trees Evaluate Fitness of Expression Trees Select New Population Probabilistically Select New Population Probabilistically Crossover, Mutation, and Rotation Crossover, Mutation, and Rotation Chromosomes for New Generation Chromosomes for New Generation Yes Termination Criterion Terminate Satisfied? No 2006:03:16 Magdi A. Mohamed 4/9
  • 5. Innovation Gene Expression Programming with Q-Aggregate Operators • A binary operation expressed by a unique Q-Aggregate operator characterizing disjunction, compensation, and conjunction operations: A(x,y) = (x + y + lambda * x * y) / (2 + lambda) • A unary operation expressed by a formula which generates value in the unit interval characterizing Negation operations: N(x) = (1 – x) / (1 + gamma * x) • The training task is performed as an optimization process where flexible model parameters (lambda/gamma values) are adjusted for each internal node in a general non-fixed tree structure to minimize an overall risk criterion using Gene Expression Programming and Differential Evolution techniques. 2006:03:16 Magdi A. Mohamed 5/9
  • 6. Innovation Gene Expression Programming with Q-Aggregate Operators y1 y2 A A A A A N x1 x2 A 0.6 N x2 x3 0.3 N A x3 0.7 N x1 2006:03:16 Magdi A. Mohamed 6/9
  • 7. Experimental Results Nonlinear Regression Case //XOR problem a=0; b=0; Y = 0; a=1; b=0; Y = 1; a=0; b=1; Y = 1; a=1; b=1; Y = 0; //Best Tree QA 0 |QA 0 ||a 1.0 ||N 0 |||b 1.0 |QA 0 ||b 1.0 ||N 0 |||a 1.0 ;PGEP Formula: ;QA(QA(a, N(b, gamma = 0), lambda= 164015206785.06702), QA(b, N(a, gamma = 0), lambda = 231379355057.6994 ), lambda = -0.99998 ) 2006:03:16 Magdi A. Mohamed 7/9
  • 8. Experimental Results Nonlinear Classification Case Test Accuracy on Monk's Problems M1 M2 M3 C4.5 75.70% 65.00% 97.20% C4.5Rules 100% 66.20% 96.30% GEP 100% 99.07% 100% QAB-GEP 100% …..% 100% QAB-GEP is able to generate accurate, compact, and noise tolerant rules Rule 1: QA(a5_1, N(a2_3, gamma= 0.050061), lambda= 0.548952) Rule 2: N(QA(a2_3, a5_4, lambda= 0.221941), gamma= 0.535057) Rule 3: QA(a4_1, a5_3, lambda= 0.602964) The average size of Monks-3 rules is 4, while GEP has 14 The set of operations {QA, N} can replace the list of pre-selected functions or operators, e.g., {IF, AND, OR, +, -, *, /, …} 2006:03:16 Magdi A. Mohamed 8/9
  • 9. Advantages of QAB-GEP Characteristics and Promises 1. compact set of operations { Aggregate (x,y), Negate (x) } 2. numerical stability 3. per unit calculations simplify implementations (software and hardware) 4. suitability for massive parallel implementations 5. automatic discovery of complex logical expressions 6. consistent handling of unary, binary, and n-ary operations 7. improvement over conventional GEP techniques in terms of average rule size 8. usability for both classification and regression applications 9. ease of use and interpretability 2006:03:16 Magdi A. Mohamed 9/9