http://gr8conf.eu/Presentations/The-next-generation-MOP

1. The next generation MOP
Jochen "blackdrag" Theodorou

2. About blackdrag
● Working on Groovy Core since about 2005
● Almost as long as that, Tech Lead of Groovy
● Currently Employed at Pivotal
● Responsible for most of the technical side of
Groovy
Email: blackdrag@gmx.org

3. About a new MOP
● discussions since 2005
● good-for-all solution always missing
● some ideas open to discussion
● and some cleanup duties
● we insist on you helping us

4. Why change

5. Why change
● has many inconsistencies
● makes many optimizations impossible
● in theory powerful
● have to know implementation details
● Impossible to extend and spec resistent
● the API just plain sucks

6. Some Basics and History

7. The Meta Class
● Each class has a meta class
● Saves all the dynamic and static
properties/methods for Groovy
● Internal control flow involves exceptions
● Designed for invoking methods through
the meta class

8. The Meta Class
● early Groovy meta programming:
● MOP methods invokeMethod and get/setProperty
● A custom meta class (foo.metaClass = x)
● later class based automated meta class lookup
● transformed into the meta class creation handle
(used by ExpandoMetaClass#enableGlobally())

9. The Meta Class
● Basic principle: Each instance has a meta class
● More specified: Only every GroovyObject instance
(later we changed that with a global map)
● Global registry specifying
initial meta class on first use

10. The Meta Class
// myMetaClass some custom metaclass
// meta class in registry different
def x1 = new X()
assert x1.metaClass != myMetaClass
x1.metaClass = myMetaClass
assert x1.metaClass == myMetaClass
def x2 = new X()
assert x2.metaClass != x1.metaClass
X.metaClass = myMetaClass
def phantom = new X()
def x3 = new X()
assert x3.metaClass == x1.metaClass
assert x3.metaClass != x2.metaClass

11. The Meta Class
X.metaClass = x2.metaClass
assert phantom.metaClass == ???

12. Adding Methods/Properties
● Standard meta class: MetaClassImpl
● does not support modifications
● New meta class for this: ExpandoMetaClass
● enabled with ExpandoMetaClass.enableGlobally()
● not always equally behaving to MetaClassImpl

13. More MetaClasses
● ProxyMetaClass (intercepting, decorating)
● MixinMetaClass (mixins)
● ClosureMetaClass (GeneratedClosure)
● DelegatingMetaClass (base class)
● OwnedMetaClass (related to mixins)
● HandleMetaClass (related to mixins)
Plus your own custom meta class

14. DSL not consistent
Foo.metaClass.bar = 1 //defines property
Foo.metaClass.bar = {1} //defines method
to use a closure as property:
foo.metaClass.bar = null
foo.bar = {1}
● only for the instance
● get metaproperty and set initial value creator

15. Overriding Super Methods
class A {
def doIt(){two() + ' done.'}
def two(){'two'}
}
class B extends A {}
B.metaClass.two = {'my new two!'}
def b = new B()
assert b.two() == 'my new two!'
assert b.doIt() == 'two done.'

16. Overriding Super Methods
To make it work:
class A implements GroovyInterceptable {
def doIt(){two() + ' done.'}
def two(){'two'}
}
class B extends A {}
B.metaClass.two = {'my new two!'}
def b = new B()
assert b.two() == 'my new two!'
assert b.doIt() == 'my new two! done.'

17. Adding Super Methods
class A {
def doIt(){two() + ' done.'}
def methodMissing(String name, args){'two'}
}
class B extends A {}
def b = new B()
assert b.two() == 'two'
assert b.doIt() == 'two done.'
A.metaClass.two = {'my new two!'}
assert b.two() == 'my new two!'
assert b.doIt() == 'my new two! done.'

18. Super Methods Overload
Does not:
class A {
def doIt(x){two(x) + ' done.'}
def two(x) {'two'}
}
class B extends A {}
def b = new B()
assert b.two('1') == 'two'
assert b.doIt('1') == 'two done.'
A.metaClass.two = {String s->'my new two!'}
assert b.two('1') == 'my new two!'
assert b.doIt('1') == 'my new two! done.'

19. Private Multi Methods
class A {
def doIt(){two() + ' done.'}
}
class B extends A {
private two(){1}
}
def b = new B()
assert b.two() == 1
assert b.doIt() == '1 done.'

20. Speaking of private
class A {
private foo=1
def b={foo}
}
class B extends A {}
def b = new B()
assert b.b() == 1 //fails
● Information loss thorugh Closure#getProperty

21. get/setMetaClass
● persistency framework needs to be aware
● transient works for Serialization
● what about other frameworks?
● seamless integration anyone?

22. Properties
class X extends
org.xml.sax.helpers.XMLFilterImpl {
def foo
}
● XMLFilterImpl has a get/setProperty
● cannot do new X().foo = bar
● cannot do println new X().foo

23. invokeMethod
No such conflict known.... but!
● dynamic entry point from Java
● as methodMissing
● with GroovyInterceptable (EMC too) as upfront
method
conflicting concepts

24. What to make better?

25. … besides fixing those problems

26. Optimization efforts
Lesson:
Java7 with invokedynamic is much better suited for
Groovy's dynamic method calls
Reaction:
make Java7 the default (backport); rewrite
DefaultGroovyMethods to use indy; throw out a lot of
old code

27. Optimization efforts
Lesson:
Hotspot is not happy about invoking target methods in
the meta class (mega morphic call sites)
Reaction:
The meta class only gives back something you can call
and does not do the call itself.

28. Optimization efforts
Lesson:
Synchronization, Locks, volatiles usages on each
method call destroy multithread performance as well
as hotspot optimizations. Most applications set up mc
changes on startup.
Reaction:
metaclass goes immutable; adding methods creates
new meta class; lazy thread update (user level
synchronization required)

29. Hot Swapping
Lesson:
Keeping internal state in the class is bad (see
timestamp_xxx, timestamp, $callSiteArray)
Reaction:
Removal. CallSiteArray not needed anymore, the
timestamps are kept track off by the loader, not in the
class

30. Optimization efforts
Lesson:
Garbage collecting and recreating meta classes is very
bad.
Reaction:
Keep the base meta class around and reuse everything
possible to make new meta classes as lean as possible

31. API Design
Lesson:
Conventions are good, forcing them is bad
(GroovyObject)
Reaction:
Don't implement GroovyObject by default anymore.

32. General Design
Lesson:
Too many ways of doing the same thing is no good
Reaction:
Most probably only methodMissing/propertyMissing
anymore but easy way to „register“ a method to do
invokeMethod.

33. API Design
Lesson:
Having multiple, not equivalent entry points is bad.
(MetaClassImpl has 3 for methods, multiusage of
invokeMethod, information loss through
get/setProperty)
Reaction:
Clean up the API to have only one entry point
(removal of MetaClass#invokeMethod)

34. Possibilities

35. Internal vs. External
Internal usage:
class X {
def methodMissing(String name, args) {1}
}
External usage:
class X {}
X.metaClass.methodMissing = {1}
Combined:
class X {
static {this.metaClass.methodMissing = {1}}
}

36. Dynamic Invoke from Java
Before:
GroovyObject foo = ...;
String s = (String)
foo.invokeMethod(“bar“, new Object[]{});
After:
Object foo = ...;
String s = MopInterface.invoke(foo, “bar“);
● helper class for dynamic interactions with Groovy
● similiar for properties

37. Adding a method from Java
Before:
GroovyObject foo = ...;
foo.getMetaClass().registerInstanceMethod(“foo“,
new Closure(null){
public Object doCall(){1}};
After:
Object foo = ...;
MopInterface.addInstanceMethod(foo, “foo“,
new T() {
public Object myFancyMethod(){1}});
● Doesn't have to be a Closure or MetaMethod
● All declared methods used (lambdas too)

38. Limited Meta Class Changes
Use Case:
unrolling all changes a GroovyShell or Eval.me did
Before:
● tracking meta class mutations impossible
● „unmutate“ the metaclass certainly is
● can only to track newly set meta classes

39. Limited Meta Class Changes
Use Case:
I am a library developer and I don't want intrusive
changes from user code to my library classes,
changing the way my library is calling methods.
Before:
If the change is not effecting the library, then most
probably because of a bug.

40. Realms

41. Realms
A realm here defines how the class in the realm
„sees“ the meta classes. Different classes can have
different realms.
MyClass
(y)
MyOtherClass
(x)
x.foo()
using metaClass(x)

42. Realms
MyClass
(y)
MyOtherClass
(x)
x.foo()
using metaClass(x)
from realm(MyClass)
<Realm defined by MyClass>

43. Realms
MyClass
(y)
MyOtherClass
(x)
y.bar()
using metaClass(y)
from realm(MyOtherClass)
<Realm defined by MyOtherClass>

44. Realms
INIT
DEFAULT
realm(A_n)
realm(C_n)realm(A_1)
........
realm(B_1)
realm(C_1)
........
realm(D_1)

45. Realms
INIT does not allow changing the meta class
INIT
DEFAULT
Non isolated
isolated

46. Realms
Default gets all the unspecified changes
INIT
DEFAULT
Non isolated
isolated

47. Realms
Isolated realms don't get unspecific changes
INIT
DEFAULT
Non isolated
isolated

48. Realms
Non isolated realms can override meta classes in
default without impossing them
INIT
DEFAULT
Non isolated
isolated

49. Isolated Realm
@Realm(
marker=SomeMarkerClass,
parentRealm=Realm.INIT)
class MyLibraryClass {
def foo(){bar()}
def bar(){1}
}
MyLibraryClass.metaClass.bar = {2}
assert new MyLibraryClass().foo() == 1

50. Non-Isolated Realm
@Realm(marker=SomeMarkerClass)
class MyLibraryClass {
def foo(){bar()}
def bar(){1}
}
MyLibraryClass.metaClass.bar = {2}
assert new MyLibraryClass().foo() == 2

51. Testing Realm
class X {private foo(){1}}
class MyTest extends GroovyTestCase {
@Realm(marker=SomeMarkerClass,
allowPrivate=true)
void testPrivateWorking() {
def x = new X()
assert x.foo() == 1
}
void testPrivateNotWorking() {
def x = new X()
shouldFail(MissingMethodException) {
x.foo()
}
}
}

52. Limited Meta Class Changes
Before:
Change intrusive, visible to everyone
After:
def foo(){1}
assert foo() == 1
realm.withSubRealm {
this.metaClass.foo = {2}
assert foo() == 2
}
assert foo() == 1

53. Q/A?