This presentation is about a tech demo for EclipseCon Europe 2012. We connect Microsoft Kinect with Eclipse technologies using: - the Jnect plugin for processing Kinect Data - the Eclipse Modeling Framework as the underlying live model representation format - EMF-IncQuery for realtime pattern matching to recognize static gestures - the Esper complex event processor to recognize gesture sequences in the event stream generated by IncQuery. More details at http://viatra.inf.mit.bme.hu/incquery/demos/jnect

1. “STANDING
QUERIES”
Real2me
gesture
recogni2on
with
Eclipse
technologies
István
DÁVID,
István
RÁTH
hDp://viatra.inf.mit.bme.hu/incquery/demos/jnect
Budapest
University
of
Technology
and
Economics
Fault
Tolerant
Systems
Research
Group
Budapest
University
of
Technology
and
Economics
Department
of
Measurement
and
Informa<on
Systems

2. Overview
User
moving
in
front
of
the
sensor
PaDern
match
EMF
setX(…) deltas
no2fica2on
EMF-­‐ Esper
Live
EMF
IncQuery
complex
model paDern
event
Kinect matcher processor
sensor
data
Recognized
gesture
sequences

3. Overview
User
moving
in
front
of
the
sensor
PaDern
match
EMF
setX(…) deltas
no2fica2on
EMF-­‐ Esper
Live
EMF
IncQuery
complex
model paDern
event
Kinect matcher processor
sensor
data
Full
model
update
at
~20
Recognized
FPS gesture
sequences

4. Overview
User
moving
in
front
of
the
sensor
PaDern
match
EMF
setX(…) deltas
no2fica2on
EMF-­‐ Esper
Live
EMF
IncQuery
complex
model PaBern
Y()
{
paDern
event
Kinect matcher processor
sensor
data
Full
model
update
at
~20
Recognized
FPS gesture
sequences
}

5. Overview
User
moving
in
front
of
the
sensor
PaDern
match
EMF
setX(…) deltas
no2fica2on
EMF-­‐ Esper
Live
EMF
IncQuery
complex
model paDern
event
Kinect matcher processor
sensor
data
Full
model
update
at
~20
Recognized
FPS gesture
sequences

6. Overview
Event
PaBern
YMCA
User
moving
in
front
of
the
sensor
SELECT
*
FROM
paDern[
every(('Y')
-­‐>
('M')
-­‐>
EMF
PaDern
match
deltas
setX(…) no2fica2on
('C')
-­‐>
EMF-­‐ Esper
('A')
Live
EMF
IncQuery
complex
WHERE
2mer:within(10
sec))]
model paDern
event
Kinect matcher processor
sensor
data
Full
model
update
at
~20
Recognized
FPS gesture
sequences

7. Overview
User
moving
in
front
of
the
sensor
PaDern
match
EMF
setX(…) deltas
no2fica2on
EMF-­‐ Esper
Live
EMF
IncQuery
complex
model paDern
event
Kinect matcher processor
sensor
data
Full
model
update
at
~20
Recognized
FPS gesture
sequences

8. Overview
User
moving
in
front
of
the
sensor
PaDern
match
EMF
setX(…) deltas
no2fica2on
EMF-­‐ Esper
Live
EMF
IncQuery
complex
model paDern
event
Kinect matcher processor
sensor
data
Full
model
update
at
~20
Recognized
FPS gesture
sequences

9. Overview
User
moving
in
front
of
the
sensor
CEP
output
PaDern
match
EMF
setX(…) no2fica2on
PaDern
matcher
output
deltas
EMF-­‐ Esper
Live
EMF
IncQuery
complex
model paDern
event
Kinect matcher processor
sensor
data
GEF3D-­‐based
visualiza2on
Full
model
update
at
~20
Recognized
FPS gesture
sequences

10. EMF-­‐IncQuery
§ High
performance
incremental
paDern
matcher
o Provides
on-­‐the-­‐fly
model
query
evalua2on
performance
for
complex
queries
over
very
large
EMF
instance
models
o Typical
use
cases:
on-­‐the-­‐fly
valida2on,
view
maintenance/
abstract
databinding,
derived
edges/aDributes,
…
§ Coming
in
the
following
weeks
to
Eclipse
Modeling
§ Version
0.6.7
available
from
o The
Eclipse
Marketplace
o hDp://viatra.inf.mit.bme.hu/incquery
§ Featured
at
EclipseCon
Europe
2012:
Fast,
Faster
and
Super-­‐fast
Model
Queries
(Ed
Willink
and
István
Ráth)