The slides describe some of the aspects of developing a new Epsilon EMC driver. They cover the basics required to implement the IModel interface, follow with some additional details that can be added to the implementation and then provide a small introduction to providing optimized execution of first-order operations on collections.

1. Developing a new Epsilon
EMC Driver:
CSV Pro
Horacio Hoyos
@HoracioHoyosR

2. Epsilon Architecture

3. CSV Pro Driver
• A EMC Driver for Comma Separated Value (CSV)
with added functionality
• Each row is an element
• Each column is an attribute
• NEW:
• One column to identify type
• One column used as index – optimized search

4. Overview
• Two Eclipse plugins
• One for the EMC driver implementation
• org.eclipse.epsilon.emc.csvpro
• Implement IModel interface
• Provide Property Getter and Setter
• Provide optimized select operations
• Can run outside eclipse
• One for the driver’s Eclipse-based development tools
• org.eclipse.epsilon.emc.csvpro.dt
• Source code available under
https://git.eclipse.org/c/epsilon/org.eclipse.epsilon
.git/tree/examples

5. EMC Driver Overview
General architecture and implementation

6. Epsilon EOL Engine Overview
for (e in m!Row.all()) {
e.name.println();
e.name = “Epsilon”;
}
Model identifier All of Type
Get property ‘name’
of the element
Set property ‘name’
of the element

7. Activities
• Provide an implementation of the IModel interface
• Retrieve the model
• Get all elements of the model
• Get all elements of the model by type
• Provide an implementation of IPropertyGetter
• Retrieve the value of a property
• Provide an implementation of IPropertySetter
• Set the value of a property

8. IModel Overview

9. Alternatives
• Provide a custom implementation of the data
model - e.g. custom file parser
• CSV, Bibtex, FileSystem, etc…
• Use the data model engine/framework – e.g. wrap
existing file parser
• Databases, EMF, XML, etc…

10. CSV Pro EMC Driver
org.eclipse.epsilon.emc.csvpro

11. Implementing IModel
• Easy: extend CachedModel
(org.eclipse.epsilon.eol.models.Cached
Model<ModelElementType>)
• Cached model improves performance
• Basic implementation of most methods
• Hard: Implement all the interface

12. Implementing IModel
• How to represent the data?
• Existing: Each Row is a Map <property: value>, the
model is a list of maps
• Alternative: Each Row is an Integer (representing line
number), the model is a Map <property: List<values>>,
list of values indexed by row
• Implement IPropertyGetter and IPropertyGetter
• Advice: Extend the existing abstract implementations

13. Implementing IModel
public class CsvProModel extends CachedModel<Integer> {
private Map<String, List<String>> data;
private List<Integer> rows; // Keep track of rows add/delete
@Override
protected Collection<? extends Integer> allContentsFromModel() {
return rows;
}
@Override
protected Object getCacheKeyForType(String type) throws
EolModelElementTypeNotFoundException {
return type;
}
…

14. Implementing IModel
@Override
protected Collection<Integer> getAllOfTypeFromModel(String type)
throws EolModelElementTypeNotFoundException {
if (!"Row".equals(type)) {
throw new EolModelElementTypeNotFoundException(
this.name, type);
}
return allContents();
}
@Override
protected Collection<Integer> getAllOfKindFromModel(String kind)
throws EolModelElementTypeNotFoundException {
return getAllOfTypeFromModel(kind);
}

15. Implementing IModel
• Property Getter
private class CsvPropertyGetter extends
AbstractPropertyGetter {
@Override
public Object invoke(Object object,
String property) throws EolRuntimeException {
assert object instanceof Integer;
return data.get(property).get((int) object);
}
}
private Map<String, List<String>> data;

16. Implementing IModel
• Property Setter
private class CsvPropertySetter extends
AbstractPropertySetter {
@Override
public void invoke(Object value) throws
EolRuntimeException {
data.get(property).set((int) object,
value.toString());
}
}
private Map<String, List<String>> data;

17. CSV Pro Eclipse-based
Development Tools
org.eclipse.epsilon.emc.csvpro.dt

18. In a nutshell
• Provide a dialog
to configure the
model

19. Overview
• CSV Pro Configuration Dialog
• Easy (50 Lines of Code)
• Some SWT knowledge
• Heavily reuse EOL’s DT facilities
• Advise: extend
AbstractCachedModelConfigurationDialog

20. Activities
Already in place
Add file browser
Add CSV options

21. Implement Configuration Dialog
public class CsvModelConfigurationDialogue
extends AbstractCachedModelConfigurationDialog {
private Text fileText;
private Text fieldSeparatorText;
private Button knownHeadersBtn;
private Button varargsHeadersBtn;
@Override
protected String getModelName() {
return "CSV Pro model";
}
@Override
protected String getModelType() {
return "CSV Pro";
}
…

22. Implement Configuration Dialog
@Override
protected void createGroups(Composite control) {
super.createGroups(control);
createFileGroup(control);
createLoadStoreOptionsGroup(control);
createCsvGroup(control);
}

23. Implement Configuration Dialog
private void createFileGroup(Composite parent) {
final Composite groupContent =
createGroupContainer(parent, "Files", 3);
final Label modelFileLabel = new Label(groupContent,
SWT.NONE);
modelFileLabel.setText("Model file: ");
fileText = new Text(groupContent, SWT.BORDER);
fileText.setLayoutData(
new GridData(GridData.FILL_HORIZONTAL));
final Button browseFile = new Button(groupContent,
SWT.NONE);
browseFile.setText("Browse Workspace...");
browseFile.addListener(SWT.Selection,
new BrowseWorkspaceForModelsListener(fileText,
"CSV files in the workspace",
"Select a CSV file"));
}

24. Implement Configuration Dialog
protected void createCsvGroup(Composite parent) {
final Composite groupContent = createGroupContainer(parent,
"CSV", 4);
final Label modelFieldSeparatorLabel = new
Label(groupContent, SWT.NONE);
modelFieldSeparatorLabel.setText("Field Separator: ");
fieldSeparatorText = new Text(groupContent, SWT.BORDER);
fieldSeparatorText.setLayoutData(new
GridData(GridData.HORIZONTAL_ALIGN_BEGINNING));
knownHeadersBtn = new Button(groupContent, SWT.CHECK);
knownHeadersBtn.setText("Known Headers");
…
varargsHeadersBtn = new Button(groupContent, SWT.CHECK);
varargsHeadersBtn.setText("Varargs Headers");
}

25. Hook Model to Epsilon
• We need to tell Epsilon there is a new model type
available
• The name of the model type
• The class that implements the model
• The configuration dialog
• An icon for the launch dialog
• Use org.eclipse.epsilon.common.dt.modelType
Extension Point
• Added using the MANIFEST.MF (Extension tab)

26. Hook Model to Epsilon
Must match getModelName() Must match getModelType()
@Override
protected String getModelName() {
return "CSV model";
}
@Override
protected String getModelType() {
return "CSV";
}

27. Hook Model to Epsilon
• Save the user options in a properties object
@Override
protected void storeProperties() {
super.storeProperties();
properties.setProperty(CsvProModel.PROPERTY_FILE,
fileText.getText());
properties.setProperty(
CsvProModel.PROPERTY_FIELD_SEPARATOR,
fieldSeparatorText.getText());
properties.setProperty(
CsvProModel.PROPERTY_HAS_KNOWN_HEADERS,
String.valueOf(knownHeadersBtn.getSelection()));
…

28. Implementing IModel
• With a configuration in place, we can finish the model
implementation
• Implement load (read additional properties), loadModel
(parse text file) and owns
@Override
public void load(StringProperties properties,
IRelativePathResolver resolver)
throws EolModelLoadingException {
super.load(properties, resolver);
this.file = resolver.resolve(
properties.getProperty(PROPERTY_FILE));
this.fieldSeparator =
properties.getProperty(PROPERTY_FIELD_SEPARATOR);

29. Cont…
this.knownHeaders =
properties.getBooleanProperty(
PROPERTY_HAS_KNOWN_HEADERS,true);
this.varargsHeaders =
properties.getBooleanProperty(
PROPERTY_HAS_VARARGS_HEADERS,false);
load();
}
@Override
public boolean owns(Object instance) {
return rows.contains(instance);
}
private List<Integer> rows;

30. Cont…
@Override
protected void loadModel() throws EolModelLoadingException {
…
List<String> keys = Arrays.asList(lines.get(0)
.split(this.fieldSeparator));
for (String k : keys) {
data.put(k, new ArrayList<String>());
}
List<String> values;
for (int i=1; i < lines.size(); i++) {
int index = i-1;
rows.add(index);
values = Arrays.asList(lines.get(i)
.split(this.fieldSeparator));
for (int f=0; f<keys.size(); f++) {
data.get(keys.get(f)).add(values.get(f));
}
}
private Map<String, List<String>> data;

31. Demo Time
See all is working!

32. Adding support for types

33. Support Types
• Modify the DT plugin to pick a field to define the
type
• Add a new label+text to specify field name
• Optional: Parse the CSV file to present headers as
dropdown for selection
• Modify the CSV Pro Model to support types
• Identify types during model loading
• Modify getAllOfTypeFromModel() to support types
• Modify getAllOfKindFromModel() to support types
• Row is a super type of all types

34. Support Types
• Modify the DT plugin to pick a field to define the
type
Add this

35. Support Types
• Modify the CSV Pro Model to support types
for (int f=0; f<keys.size(); f++) {
List<String> datavals = data.get(keys.get(f));
if (useTypeColum) {
if (f == typeColum) {
List<Integer> typed =
typedElements.get(values.get(f));
if (typed == null) {
typed = new ArrayList<Integer>();
typedElements.put(values.get(f), typed);
}
typed.add(index);
}
}
datavals.add(values.get(f));
}

36. Demo Time
See all is working!

37. Execution optimization
Select operation optimization

38. Select Operation Overview
m!Row.all().select(r | r.id == "531-52-7468");
for (Object element : list) {
if (element.id = "531-52-7468") {
return element;
}
}
SELECT * FROM ROWS WHERE id = "531-52-7468"
optimize

39. Epsilon optimization overview
• To provide optimized execution:
• Inform what operations can be optimized
• Provide an implementation of these operations
• Operations that can be optimized
• Collections returned by the model (e.g. from
selectAllOfTypeFromModel) should implement
IAbstractOperationContributor
• Or, the model implements
IAbstractOperationContributor and keep track of
“owned” collections
• Operation implementation
• Extend the operations to be optimized

40. Optimize select by “id”
• Provide an optimization of the select operation if
the filter field is the id
• Allow the user to specify what column to use for
optimized searches
• Add a data structure to the model that allows fast search
by id (e.g. map)
• Provide a wrapping collection to implement
IAbstractOperationContributor
• Provide a SelectOperation that identifies a select by id
and uses the data model to optimize execution

41. Optimize select by “id”
• Allow the user to specify what column to use for
optimized searches

42. Optimize select by “id”
• Add a data structure to the model that allows fast
search by id
private TreeMap<String, Integer> rows;
…
if (useIndexColum && (f == indexColum)) {
rows.put(values.get(f), index);
}
…

43. Optimize select by “id”
• Provide a wrapping collection to implement
IAbstractOperationContributor
• One possibility is to use the delegate pattern
public class CsvProCollection implements List<Integer>,
IAbstractOperationContributor {
List<Integer> delegate;
…
public boolean contains(Object o) {
return delegate.contains(o);
}
…
@Override
public AbstractOperation getAbstractOperation(String name) {
if ("select".equals(name)) {
return new CsvProCollectionSelectOperation();
}
…

44. Optimize select by “id”
m!Row.all().select(r | r.id == "531-52-7468");
public Object execute(Object target,
Variable iterator,
Expression ast, IEolContext context,
boolean returnOnFirstMatch)
throws EolRuntimeException {}
asttarget iterator
• SelectOperation that identifies a select by id

45. Optimize select by “id”
m!Row.all().select(r | r.id == "531-52-7468");
iterator
Assuming SSN has some ordering,
we want to support ==, >, >=, <=, <,
<>. So ast must be:
• EqualsOperatorExpression
• GreaterThanOperatorExpression
• Etc.
FirstOperand
PropertyCallExpression
TargetExpression
r.id
PropertyNameExpression
iterator
matches
“id “ column name
matches
ast

46. Optimize select by “id”
if (!(ast instanceof EqualsOperatorExpression ||
ast instanceof GreaterThanOperatorExpression ||
ast instanceof GreaterEqualOperatorExpression ||
ast instanceof LessEqualOperatorExpression ||
ast instanceof LessThanOperatorExpression ||
ast instanceof NotEqualsOperatorExpression)) {
return false;
}
Assuming SSN has some ordering, we want to
support ==, >, >=, <=, <, <>. So ast must be:
• EqualsOperatorExpression
• GreaterThanOperatorExpression
• Etc.

47. final OperatorExpression opExp =
(OperatorExpression) ast;
final Expression rawLOperand =
opExp.getFirstOperand();
if (!(rawLOperand instanceof PropertyCallExpression))
{
return false;
}
Optimize select by “id”
PropertyCallExpression
r.id

48. final PropertyCallExpression lOperand =
(PropertyCallExpression) rawLOperand;
final Expression rawTargetExpression =
lOperand.getTargetExpression();
if (!(lOperand.getTargetExpression()
instanceof NameExpression)) {
return false;
}
final NameExpression nameExpression =
(NameExpression) rawTargetExpression;
if (!iterator.getName()
.equals(nameExpression.getName())) {
return false;
}
Optimize select by “id”
TargetExpression
r.id
iterator
matches

49. final NameExpression propertyNameExpression =
lOperand.getPropertyNameExpression();
if (!index.equals(propertyNameExpression.getName())) {
return false;
}
Optimize select by “id”
r.id
PropertyNameExpression
“id “ column name
matches

50. Demo Time
See all is working!