Meta-Classes in Python

Dynamic OOP and Meta-Classes in Python

Guy Wiener

Dept. of Applied Math and Computer Science
Weizmann Institute for Science

24/5/2011

Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 1 / 57

Introduction

Outline

1 Introduction

2 OOP Hacking

3 Meta-Classes

4 Examples

5 Summary


Introduction

What is OOP?

The OOP Manifest (abridged)
Alan Kay, 1993
Everything is an object
Computation is performed by objects communicating with each
other by sending and receiving messages
Each object has its own memory (state), which consists of other
objects.
Every object is an instance of a class.
The class is the repository for behavior. All instances of the
same class can perform the same actions.
Classes are organized into a singly rooted tree, called the
inheritance hierarchy.

Introduction

What is an object?

An object, according to the OOP manifest:
State Memory
Behavior Receiving and sending messages
Identity Each object is unique


Introduction

Objects Can Change!

Changing objects
State Read, write, add and remove ﬁeld
Behavior Create, add and replace methods
Identity Remains the same


Introduction

Static Objects and Classes

Objects and Classes Example
Class A
Field descriptors x: int
Method desc. and code y: String
Hierarchy information foo() {. . . }
goo() {. . . }
Object
Run-time
Fields Instance of A
type infor-
content 5
mation
“hello”

Introduction

Dynamic Objects and Classes

Objects and Classes Example
Class A
A dictionary “foo” ⇒ {. . . }
of methods
“goo” ⇒ {. . . }
and static
ﬁelds
Instance of A
Object “x” ⇒ 5
A dictionary of “y” ⇒ “hello”
Class
ﬁelds and bound
pointer
methods


OOP Hacking

Outline

1 Introduction

2 OOP Hacking

3 Meta-Classes

4 Examples

5 Summary


OOP Hacking

Everything is a Dictionary

Python’s modules, classes and objects = Dictionaries.
The dictionary holds:
Classes Methods and class variables
Objects Bound methods and ﬁelds

The “.” operator = Access the inner dictionary
The dir command = Lists keys of the inner dictionary


OOP Hacking

Manipulating the Inner Dictionary

The inner dictionary is accessible and manipulable by string keys!
Inner dictionary manipulation
getattr(o, ’a’) Get the attribute a from the dictionary of o
setattr(o, ’a’, v) Set the attribute a of o to be v
obj.foo is same as getattr(obj, ’foo’)
obj.foo = val is same as setattr(obj, ’foo’, val)


OOP Hacking

Manipulating the Inner Dictionary (cont’d)

Example
>>> a = A()
>>> a.x = 5
>>> dir(a)
[..., ’x’]
>>> getattr(a, ’x’)
5
>>> setattr(a, ’x’, 6)
>>> a.x
6


OOP Hacking

Adding Fields

Adding ﬁelds to an object
class A: pass
>>> a = A()
>>> a.x = 5
>>> a.x
5
>>> setattr(a, ’y’, 6)
>>> a.y
6


OOP Hacking

Adding Fields

Adding ﬁelds to an object Gain & Loss
class A: pass Gained Flexible object content
>>> a = A() Lost Type safety
>>> a.x = 5
>>> a.x
5
>>> setattr(a, ’y’, 6)
>>> a.y
6


OOP Hacking

Replacing Methods

Swapping methods
class A:
def f(self):
print "f"
def g(self):
print "g"
>>> a = A()
>>> a.f, a.g = a.g, a.f
>>> a.f()
g
>>> a.g()
f


OOP Hacking

Replacing Methods

Swapping methods Gain & Loss
class A: Gained Change an object
def f(self): behavior at runtime
print "f" Lost Control ﬂow analysis
def g(self):
print "g"
>>> a = A()
>>> a.f, a.g = a.g, a.f
>>> a.f()
g
>>> a.g()
f


OOP Hacking

Replacing Methods for a Class

Swapping class methods
class A:
def f(self):
print "f"
def g(self):
print "g"
>>> a = A()
>>> A.f, A.g = A.g, A.f
>>> a.f()
g
>>> a.g()
f


OOP Hacking

Replacing Methods for a Class

Swapping class methods Gain & Loss
class A: Gained Change a class at
def f(self): runtime
print "f" Lost Sanity
def g(self):
print "g"
>>> a = A()
>>> A.f, A.g = A.g, A.f
>>> a.f()
g
>>> a.g()
f


OOP Hacking

Classes are Return Values Too

A function that returns a class
def PersonClass(salut):
class Person(object):
def __init__(self, name):
self.name = name
def greet(self):
print salut, self.name
return Person


OOP Hacking

Classes are Return Values Too (cont’d)

Making classes Output
Man = PersonClass("Mr.") for c in a: c.greet()
Woman = PersonClass("Ms.")
Doctor = PersonClass("Dr.") >>> Ms. Rose Tylar
Ms. Amy Pond
a = [Woman("Rose Tylar"), Mr. Mickey Smith
Woman("Amy Pond"), Mr. Rory Williams
Man("Mickey Smith"), Dr. Who
Man("Rory Williams"),
Doctor("Who")]


Meta-Classes

Outline

1 Introduction

2 OOP Hacking

3 Meta-Classes

4 Examples

5 Summary


Meta-Classes

What is Meta-Programming?
Meta-Program
A program that:
One of its inputs is a program (possibly itself)
Its output is a program


Meta-Classes

What is Meta-OOP?

Meta-Class
A class that creates classes
Objects are instances of classes
Classes are objects
Classes are instances of meta-classes


Meta-Classes

Object Systems

3-levels 4-levels
Object Object

A Class Type A Class Type
super: Type[]
super: Type[]
An Object
An Object
A MetaClass
Legend
instance of
extends


Meta-Classes Meta-Classes in Python

Outline

1 Introduction
2 OOP Hacking
3 Meta-Classes
Meta-Classes in Python
4 Examples
Logging
Counting
Delegation
Associations
5 Summary




The type class
Base class for meta-classes
Creates new classes
Dynamic members become static members of the instance class

type. init (cls, name, bases, dict)
cls The class object itself (i.e., “self”)
name The name of the class
bases A list of the base classes
dict A dictionary of the methods and static ﬁelds of the class



Creating Classes from Meta-Classes

Declaring a new class programatically
# Meta-Class
class Printable(type):
def whoami(self): print "I am ", self.__name__

>>> Foo = Printable(’Foo’,(),{}) # Empty new class
>>> Foo.whoami()
I am Foo
>>> Foo.__class__
<class ’Printable’>
>>> f = Foo() # Object
>>> f.__class__
<class ’Foo’>



The metaclass ﬁeld

Class C is an instance of a meta-class M if:
1 C has a static ﬁeld metaclass
2 One of the ancestors classes of C is an instance of M
3 There is a global variable metaclass
4 Otherwise, the default meta-class type is used



metaclass Example

Declaring a meta-class with metaclass
class Bar:
__metaclass__ = Printable
def foo(self): print ’foo’

>>> Bar.whoami()
I am a Bar
>>> b = Bar()
>>> b.foo()
foo


Examples

Outline

1 Introduction

2 OOP Hacking

3 Meta-Classes

4 Examples

5 Summary


Examples Logging

Outline

1 Introduction
2 OOP Hacking
3 Meta-Classes
4 Examples
Logging
Counting
Delegation
Associations
5 Summary


Examples Logging

Logging with Meta-Classes

A logger decorator
def log(name, f):
def ret(*args):
print "enter", name
f(*args)
print "exit", name
return ret


Examples Logging

Logging with Meta-Classes (cont’d)

A logger meta-class
class Logged(type):
def init (cls, name, bases, dict):
type. init (cls, name, bases, dict)
p = re.compile(cls. logmatch )
for attr, item in dict.items():
if callable(item) and p.match(attr):
setattr(cls, attr, log(attr, item))


Examples Logging

Logging with Meta-Classes (cont’d)

A logged class
class Test:
metaclass = Logged
logmatch = ’.*’
def foo(self):
print ’foo’

>>> t = Test()
>>> t.foo()
enter foo
foo
exit foo


Examples Counting

Outline

1 Introduction
2 OOP Hacking
3 Meta-Classes
4 Examples
Logging
Counting
Delegation
Associations
5 Summary


Examples Counting

Counting Instances

Countable meta-class
class Countable(type):
def new_init(cls, init):
def ret(self, *args):
init(self, *args)
cls.count = cls.count + 1
return ret

def __init__(cls, name, bases, dct):
type.__init__(cls, name, bases, dct)
cls.count = 0
cls.__init__ = cls.new_init(cls.__init__)


Examples Counting

Counting Example

Some classes
class Person:
__metaclass__ = Countable
def __init__(self, name):
self.name = name

class Book:
__metaclass__ = Countable
def __init__(self, name, author):
self.name = name
self.author = author


Examples Counting

Counting Example (cont’d)

Testing
rah = Person("Robert Anson Heinlin")
b1 = Book("Stranger in a Strange Land", rah)
b2 = Book("Starship Troopers", rah)

print Person.count, Book.count
>>> 1 2


Examples Delegation

Outline

1 Introduction
2 OOP Hacking
3 Meta-Classes
4 Examples
Logging
Counting
Delegation
Associations
5 Summary


Examples Delegation

Automatic Delegation

Delegate

An object of class A that
dispatches all message of
class B to a target ﬁeld of
type B.

Writing a delegate class is a monotonous work
But it can be done automatically


Examples Delegation

Delegation Using Meta-Classes

A delegation function decorator
def dlgt(cls, method):
method(self.__tgt__, *args)
return instancemethod(ret, None, cls)
instancemethod takes a function, an object/None and a class and
returns a method of the class
Auxiliary – Print class name
def clsname(self):
return self.__class__.__name__


Examples Delegation

Delegation Using Meta-Classes (cont’d)

The Delegate meta-class
class Delegate(type):
def __init__(cls, name, bases, dict):
type.__init__(cls, name, bases, dict)
tgtclass = cls.__tgtclass__
for name in dir(tgtclass):
val = getattr(tgtclass, name)
if callable(val):
setattr(cls, name, dlgt(cls, val))


Examples Delegation


The delegated class
class A:
def bar(self):
print clsname(self), ’bar’
def baz(self):
print clsname(self), ’baz’


Examples Delegation


The delegating class
class B:
metaclass = Delegate
tgtclass = A
def init (self, tgt):
self. tgt = tgt
def boo(self):
print clsname(self), ’boo’


Examples Delegation


Delegation test
>>> b = B(A())
>>> b.bar()
A bar
>>> b.baz()
A baz
>>> b.boo()
B boo


Examples Delegation

Simpliﬁed Syntax for with Meta-Classes with Args.
Thanks to Yoav Goldberg

Use a function to create the meta-class
def delegate_of(tgtclass):
class Delegate(type):
def __init__(cls, name, bases, dict):
type.__init__(cls, name, bases, dict)
for name in dir(tgtclass):
val = getattr(tgtclass, name)
if callable(val):
setattr(cls, name, dlgt(cls,val))


Examples Delegation

Simpliﬁed Syntax for Meta-Classes with Args.
Thanks to Yoav Goldberg

Use the function to generate a meta-class
class B:
metaclass = delegate_of(A)
def init (self, tgt):
self. tgt = tgt
def boo(self):
print clsname(self), ’boo’


Examples Associations

Outline

1 Introduction
2 OOP Hacking
3 Meta-Classes
4 Examples
Logging
Counting
Delegation
Associations
5 Summary



Associations

0..m ab 0..n
A myA myB
B

Goals
Aggregation operations, like
a.getMyB().foreach().g()
r = b.getMyA().filter().f().map().h()
Optimizations: Multithreading, caching
No textual code generation



Basic Aggregation
Replace all public methods with an aggregator wrapper
class Aggr(type):
def __init__(cls, name, bases, dct):
p = re.compile(’__.*__’)
type.__init__(cls, name, bases, {})
for key, item in dct.iteritems():
if callable(item) and not p.match(key):
m = instancemethod(cls.wrap(item), None, self)
setattr(self, key, m)

def wrap(cls, func):
return self.__aggr__(func, *args)
return ret


Basic Aggregation (cont’d)
Make aggrs. that extend impl and wrap ifce
class ListAggr(Aggr):
@staticmethod
def make(name, impl, ifce):
dct = {}
for key in dir(ifce):
dct[key] = getattr(ifce, key)
return ListAggr(name, (impl,), dct)

def __init__(self, name, bases, dct):
Aggr.__init__(self, name, bases, dct)
base = bases[0]
ctr = base.__init__ % Use impl’s constructor
self.__init__ = instancemethod(ctr, None, self)


List-Based Sequential Aggrs.

Store list in private ﬁeld
class ListFunc(object):
def __init__(self, data):
self.__data__ = data
def onAdd(self, other): pass
def onRemove(self, other): pass



List-Based Sequential Aggrs. (cont’d)

Sequential Foreach
class ListSeqForeach(ListFunc):
ListFunc.__init__(self, data)
def __aggr__(self, func, *args):
for inst in self.__data__:
func(inst, *args)



List-Based Sequential Aggrs. (cont’d)

Sequential Map
class ListSeqMap(ListFunc):
ret = []
for inst in self.__data__:
ret.append(func(inst, *args))
return ret



Binding It Together

Making an aggr. that extends seq. foreach
def makeForeachClass(cls):
return ListAggr.make(’...’, ListSeqForeach, cls)

ListFunc Aggr

ListSeqForeach ListAggr

cls ForeachClass



Optimizations

Multithreaded Foreach
class ListSpawnForeach(ListFunc):
n = max(len(self.__data__)/K, 1)
ts = []
for c in chunks(self.__data__, n):
t = ForeachThread(c, func, args)
ts.append(t)
t.start()
for tr in ts: tr.join()



Binding It Together, Multithreaded Version

Making an aggr. that extends parallel foreach
def makeForeachClass(cls):
return ListAggr.make(’...’, ListSpawnForeach, cls)

ListFunc Aggr

ListSpawnForeach ListAggr

cls ForeachClass



Links

Download complete code
http://www.cs.bgu.ac.il/~gwiener/software/associations/

Read full paper
Proceedings of ENASE’2010 / By request


Summary

Outline

1 Introduction

2 OOP Hacking

3 Meta-Classes

4 Examples

5 Summary


Summary

Advantages of Meta-Classes

Turn this. . . Into this

P haarazu-Urh
os a
cr

-Y
eku Uru-Ne

tah
i Ia Dai-Uk
-W
ka

a-

Ha
kal -U
Tru-Saka


Summary

Thank You!

wiener.guy@gmail.com


Meta-Classes in Python

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Meta-Classes in Python

Ähnlich wie Meta-Classes in Python (20)

Meta-Classes in Python