My presentation for Scala Days Amsterdam. How to make a compile time string interpolator for a language you have? Use case and step by step code examples.

1. N
Embedding a language into string
interpolator
Mikhail Limanskiy
June 10, 2015

2. String interpolation is easy
val language = "English"

3. String interpolation is easy
val language = "English"
val embedded = s"Embedded $language"

4. MongoDB
MongoDB is a document oriented database, storing BSON documents.
db.people.insert({ name: "John Doe", age: 42 })

5. MongoDB
MongoDB is a document oriented database, storing BSON documents.
db.people.insert({ name: "John Doe", age: 42 })
db.people.insert({
name: "William Smith",
age: 28,
phone: [ "1234567", "7654321" ]
})

6. MongoDB
MongoDB is a document oriented database, storing BSON documents.
db.people.insert({ name: "John Doe", age: 42 })
db.people.insert({
name: "William Smith",
age: 28,
phone: [ "1234567", "7654321" ]
})
db.people.insert({
name: "Alice White",
age: 29,
address: {
country: "UK",
city: "London"
}
})

7. MongoDB
MongoDB is a document oriented database, storing BSON documents.
db.people.insert({ name: "John Doe", age: 42 })
db.people.insert({
name: "William Smith",
age: 28,
phone: [ "1234567", "7654321" ]
})
db.people.insert({
name: "Alice White",
age: 29,
address: {
country: "UK",
city: "London"
}
})
db.people.insert({ name : "Ivan Petrov", age : 28 })

8. Search and update
Quering:
db.people.find({ name: "John Doe"})
db.people.find({ age: { $lt : 30 }})
db.people.find({ phone: { $not: { $size : 0 }}})

9. Search and update
Quering:
db.people.find({ name: "John Doe"})
db.people.find({ age: { $lt : 30 }})
db.people.find({ phone: { $not: { $size : 0 }}})
db.people.update({ age : 42},
{ $set : { name : "Ford Prefect" } })

10. Search and update
Quering:
db.people.find({ name: "John Doe"})
db.people.find({ age: { $lt : 30 }})
db.people.find({ phone: { $not: { $size : 0 }}})
db.people.update({ age : 42},
{ $set : { name : "Ford Prefect" } })
db.people.aggregate(
[ { $group : { _id : "$age", count : {$sum : 1} } },
{ $sort : { count : -1 } },
{ $limit : 5 }
])

11. MongoDB in Scala
There are three main drivers for MongoDB:
Casbah – synchronous, on top of Java driver.

12. MongoDB in Scala
There are three main drivers for MongoDB:
Casbah – synchronous, on top of Java driver.
ReactiveMongo – asynchronous, built on Akka actors.

13. MongoDB in Scala
There are three main drivers for MongoDB:
Casbah – synchronous, on top of Java driver.
ReactiveMongo – asynchronous, built on Akka actors.
Tepkin – reactive, on top of Akka IO and Akka Streams.

14. How Casbah API looks like
val name = "John Doe"
people.insert(MongoDBObject(
"name" -> "James Bond",
"age" -> 80,
"phone" -> List("007007"),
"address" -> MongoDBObject("country" -> "UK")))

15. How Casbah API looks like
val name = "John Doe"
people.insert(MongoDBObject(
"name" -> "James Bond",
"age" -> 80,
"phone" -> List("007007"),
"address" -> MongoDBObject("country" -> "UK")))
val a = people.findOne(MongoDBObject("name" -> name))
val b = people.find(MongoDBObject("age" ->
MongoDBObject("$lt" -> 30)))

16. How Casbah API looks like
val name = "John Doe"
people.insert(MongoDBObject(
"name" -> "James Bond",
"age" -> 80,
"phone" -> List("007007"),
"address" -> MongoDBObject("country" -> "UK")))
val a = people.findOne(MongoDBObject("name" -> name))
val b = people.find(MongoDBObject("age" ->
MongoDBObject("$lt" -> 30)))
// Using Casbah DSL
val c = people.find("age" $lt 30)
val d = people.find("phone" -> $not(_ $size 0))

17. How Casbah API looks like
val name = "John Doe"
people.insert(MongoDBObject(
"name" -> "James Bond",
"age" -> 80,
"phone" -> List("007007"),
"address" -> MongoDBObject("country" -> "UK")))
val a = people.findOne(MongoDBObject("name" -> name))
val b = people.find(MongoDBObject("age" ->
MongoDBObject("$lt" -> 30)))
// Using Casbah DSL
val c = people.find("age" $lt 30)
val d = people.find("phone" -> $not(_ $size 0))
people.update(MongoDBObject("age" -> 42),
$set("name" -> "Ford Prefect"))

18. How Casbah API looks like
val name = "John Doe"
people.insert(MongoDBObject(
"name" -> "James Bond",
"age" -> 80,
"phone" -> List("007007"),
"address" -> MongoDBObject("country" -> "UK")))
val a = people.findOne(MongoDBObject("name" -> name))
val b = people.find(MongoDBObject("age" ->
MongoDBObject("$lt" -> 30)))
// Using Casbah DSL
val c = people.find("age" $lt 30)
val d = people.find("phone" -> $not(_ $size 0))
people.update(MongoDBObject("age" -> 42),
$set("name" -> "Ford Prefect"))
val e = people.aggregate(List(
MongoDBObject("$group" ->
MongoDBObject("_id" -> "$age", "count" ->
MongoDBObject("$sum" -> 1))),
MongoDBObject("$sort" -> MongoDBObject("count" -> -1)),
MongoDBObject("$limit" -> 5)))

19. How Casbah API looks like

20. ReactiveMongo
// Future[BSONDocument]
val a = people.find(BSONDocument("name" -> "John Doe"))
.one[BSONDocument]
// Future[List[Person]]
val b = people.find(BSONDocument("age" ->
BSONDocument("$lt" -> 30))
).cursor[Person].collect[List]()
val futureUpdate = people.update(
BSONDocument("age" -> 42),
BSONDocument("$set" -> BSONDocument("name" -> "Ford Prefect")))
// Future
val e = db.command(RawCommand(BSONDocument(
"aggregate" -> "people",
"pipeline" -> BSONArray(
BSONDocument("$group" ->
BSONDocument("_id" -> "$age",
"count" -> BSONDocument("$sum" -> 1))),
BSONDocument("$sort" -> BSONDocument("count" -> -1)),
BSONDocument("$limit" -> 5)))))

21. Why?

22. Meet MongoQuery
Using MongoQuery with Casbah:
import com.github.limansky.mongoquery.casbah._
val name = "John Doe"
val a = people.findOne(mq"{ name : $name }")

23. Meet MongoQuery
Using MongoQuery with Casbah:
import com.github.limansky.mongoquery.casbah._
val name = "John Doe"
val a = people.findOne(mq"{ name : $name }")
val b = people.find(mq"{age : { $$lt : 30 }}")

24. Meet MongoQuery
Using MongoQuery with Casbah:
import com.github.limansky.mongoquery.casbah._
val name = "John Doe"
val a = people.findOne(mq"{ name : $name }")
val b = people.find(mq"{age : { $$lt : 30 }}")
val d = people.find(
mq"{ phone : { $$not : { $$size : 0 }}}")
people.update(mq"{ age : 42 }",
mq"{ $$set { name : 'Ford Prefect' }}")
val e = people.aggregate(List(
mq"""{ $$group :
{ _id : "$$age", count : { $$sum : 1 }}}""",
mq"{ $$sort : { count : -1 }}",
mq"{ $$limit : 5}"))

25. String interpolation
implicit class MongoHelper(val sc: StringContext)
extends AnyVal {
def mq(args: Any*): DBObject = {
Parser.parseQuery(sc.parts, args) match {
case Success(v, _) =>
createObject(v)
case NoSuccess(msg, _) =>
throw new MqException(s"Invalid object: $msg")
}
}
}

26. String interpolation
implicit class MongoHelper(val sc: StringContext)
extends AnyVal {
def mq(args: Any*): DBObject = {
Parser.parseQuery(sc.parts, args) match {
case Success(v, _) =>
createObject(v)
case NoSuccess(msg, _) =>
throw new MqException(s"Invalid object: $msg")
}
}
}
mq"{ name : $name }"
sc.parts == List("{ name: ", " }")
args = List(name)

27. String interpolation

28. Wrapping it into macro
implicit class MongoHelper(val sc: StringContext) extends AnyVal {
def mq(args: Any*): DBObject = macro MongoHelper.mq_impl
}
object MongoHelper {
def mq_impl(c: Context)(args: c.Expr[Any]*):
c.Expr[DBObject] = ???
}

29. Wrapping it into macro
implicit class MongoHelper(val sc: StringContext) extends AnyVal {
def mq(args: Any*): DBObject = macro MongoHelper.mq_impl
}
object MongoHelper {
def mq_impl(c: Context)(args: c.Expr[Any]*):
c.Expr[DBObject] = {
import c.universe._
val q"$cn(scala.StringContext.apply(..$pTrees))"
= c.prefix.tree
val parsed = parse(c)(pTrees)
wrapObject(c)(parsed, args.map(_.tree).iterator)
}
}

30. Wrapping it into macro
object MongoHelper {
def parse(c: Context)(pTrees: List[c.Tree]) = {
import c.universe._
val parts = pTrees map {
case Literal(Constant(s: String)) => s
}
parser.parse(parts) match {
case Success(v, _) => v
case NoSuccess(msg, reader) =>
val partIndex = reader.asInstanceOf[PartReader].part
val pos = pTrees(partIndex).pos
c.abort(pos.withPoint(pos.point + reader.offset)),
s"Invalid BSON object: $msg")
}
}
}

31. Parsing BSON
mq"{ name : $name, age : { $$gte : 18, $$lte : $max }}"

32. Parsing BSON
mq"{ name : $name, age : { $$gte : 18, $$lte : $max }}"
Lexical
List("{", Field("name"), ":", Placeholder , ",", Field("age"),
":", "{", Keyword("$gte"), ":", NumericLit(18), ",",
Keyword("$lte"), ":", Placeholder , ",", "}", "}")

33. Parsing BSON
mq"{ name : $name, age : { $$gte : 18, $$lte : $max }}"
Lexical
List("{", Field("name"), ":", Placeholder , ",", Field("age"),
":", "{", Keyword("$gte"), ":", NumericLit(18), ",",
Keyword("$lte"), ":", Placeholder , ",", "}", "}")
Syntactical
Object(List(
(Member("name"), Placeholder),
(Member("age"), Object(List(
(Keyword("$gte"), 18),
(Keyword("$lte"), Placeholder))
))
))

34. Create objects
protected def wrapObject(c: Context)(obj: Object,
args: Iterator[c.Tree]): c.Expr[DBType] = {
val dbparts = obj.members.map {
case (lv, v) => (lv.asString , wrapValue(c)(v, args))
}
c.Expr(q"com.mongodb.casbah.commons.MongoDBObject(..$dbparts)")
}

35. Wrapping values
protected def wrapValue(c: Context) (value: Any,
args: Iterator[c.Tree]): c.Expr[Any] = {
import c.universe._
value match {
case BSON.Placeholder =>
c.Expr(args.next())

36. Wrapping values
protected def wrapValue(c: Context) (value: Any,
args: Iterator[c.Tree]): c.Expr[Any] = {
import c.universe._
value match {
case BSON.Placeholder =>
c.Expr(args.next())
case o: BSON.Object =>
wrapObject(c)(o, args)

37. Wrapping values
protected def wrapValue(c: Context) (value: Any,
args: Iterator[c.Tree]): c.Expr[Any] = {
import c.universe._
value match {
case BSON.Placeholder =>
c.Expr(args.next())
case o: BSON.Object =>
wrapObject(c)(o, args)
case BSON.Id(id) =>
c.Expr(q"new org.bson.types.ObjectId($id)")

38. Wrapping values
protected def wrapValue(c: Context) (value: Any,
args: Iterator[c.Tree]): c.Expr[Any] = {
import c.universe._
value match {
case BSON.Placeholder =>
c.Expr(args.next())
case o: BSON.Object =>
wrapObject(c)(o, args)
case BSON.Id(id) =>
c.Expr(q"new org.bson.types.ObjectId($id)")
case a: List[_] =>
val wrapped = a.map(i => wrapValue(c)(i, args))
c.Expr[List[Any]](q"List(..$wrapped)")

39. Wrapping values
protected def wrapValue(c: Context) (value: Any,
args: Iterator[c.Tree]): c.Expr[Any] = {
import c.universe._
value match {
case BSON.Placeholder =>
c.Expr(args.next())
case o: BSON.Object =>
wrapObject(c)(o, args)
case BSON.Id(id) =>
c.Expr(q"new org.bson.types.ObjectId($id)")
case a: List[_] =>
val wrapped = a.map(i => wrapValue(c)(i, args))
c.Expr[List[Any]](q"List(..$wrapped)")
case v =>
c.Expr[Any](Literal(Constant(v)))
}
}

40. Type safety

41. mqt – typechecking interpolator
case class Phone(kind: String, number: String)
case class Person(name: String, age: Int, phone: List[Phone])

42. mqt – typechecking interpolator
case class Phone(kind: String, number: String)
case class Person(name: String, age: Int, phone: List[Phone])
// OK
persons.update(mq"{}", mqt"{ $$inc : { age : 1 } }"[Person])
persons.find(mqt"{ phone.number : '223322' }"[Person])

43. mqt – typechecking interpolator
case class Phone(kind: String, number: String)
case class Person(name: String, age: Int, phone: List[Phone])
// OK
persons.update(mq"{}", mqt"{ $$inc : { age : 1 } }"[Person])
persons.find(mqt"{ phone.number : '223322' }"[Person])
// COMPILE ERROR
persons.update(mq"{}", mqt"""{$$set : { nme : "Joe" }}"""[Person])
persons.find(mqt"{ name.1 : 'Joe' }"[Person])
persons.find(mqt"{ phone.num : '223322' }"[Person])

44. Passing type into intepolator
implicit class MongoHelper(val sc: StringContext) extends AnyVal {
def mq(args: Any*): DBObject = macro MongoHelper.mq_impl
def mqt(args: Any*) = new QueryWrapper
}
class QueryWrapper {
def apply[T]: DBObject = macro MongoHelper.mqt_impl[T]
}
object MongoHelper {
def mqt_impl[T: c.WeakTypeTag](c: Context):
c.Expr[DBObject] = ???
}

45. Inside mqt_impl
def mqt_impl[T: c.WeakTypeTag](c: Context): c.Expr[DBObject] = {
val q"$cn(scala.StringContext.apply(..$pTrees)).mqt(..$aTrees)"
= c.prefix.tree
val args = aTrees.map(c.Expr(_))
val parsed = parse(c)(pTrees)
checkObject(c)(c.weakTypeOf[T], parsed)
wrapObject(c)(parsed, args.iterator)
}

46. Verifing the type
def checkType(c: Context)(tpe: c.Type, obj: Object) = {
import c.universe._
val ctor = tpe.decl(termNames.CONSTRUCTOR).asMethod
val params = ctor.paramLists.head
val className = t.typeSymbol.name.toString
val fields = params.map(s => s.name.toString -> s).toMap
obj.members.foreach { case (m, _) =>
if (!fields.contains(m.name)) {
c.abort(c.enclosingPosition ,
s"Class $className doesn't contain field '${m.name}'")
}
}
}

47. Testing interpolator
it should "support nested objects" in {
val q = mq"""{ user : "Joe", age : {$$gt : 25}}"""
q should equal(MongoDBObject("user" -> "Joe",
"age" -> MongoDBObject("$gt" -> 25)))
}

48. Testing error scenarios
import scala.reflect.runtime.{ universe => ru }
class CompileTest extends FlatSpec {
val cl = getClass.getClassLoader.asInstanceOf[URLClassLoader]
val cp = cl.getURLs.map(_.getFile).mkString(File.pathSeparator)
val mirror = ru.runtimeMirror(cl)
val tb = mirror.mkToolBox(options = s"-cp $cp")

49. Testing error scenarios
import scala.reflect.runtime.{ universe => ru }
class CompileTest extends FlatSpec {
val cl = getClass.getClassLoader.asInstanceOf[URLClassLoader]
val cp = cl.getURLs.map(_.getFile).mkString(File.pathSeparator)
val mirror = ru.runtimeMirror(cl)
val tb = mirror.mkToolBox(options = s"-cp $cp")
def getError(q: String): String = {
val e = intercept[ToolBoxError] {
tb.eval(tb.parse(q))
}
e.message
}

50. Testing error scenarios
import scala.reflect.runtime.{ universe => ru }
class CompileTest extends FlatSpec {
val cl = getClass.getClassLoader.asInstanceOf[URLClassLoader]
val cp = cl.getURLs.map(_.getFile).mkString(File.pathSeparator)
val mirror = ru.runtimeMirror(cl)
val tb = mirror.mkToolBox(options = s"-cp $cp")
def getError(q: String): String = {
val e = intercept[ToolBoxError] {
tb.eval(tb.parse(q))
}
e.message
}
it should "fail on malformed BSON objects" in {
val e = getError("""mq"{ test 5 }" """)
e should include("`:' expected , but 5 found")
}
}

51. Summary
Cons
Not easy to implement

52. Summary
Cons
Not easy to implement
Not highlighted in IDE

53. Summary
Cons
Not easy to implement
Not highlighted in IDE
Pros

54. Summary
Cons
Not easy to implement
Not highlighted in IDE
Pros
Less limitations on language structure

55. Summary
Cons
Not easy to implement
Not highlighted in IDE
Pros
Less limitations on language structure
Can preserve existing language

56. Summary
Cons
Not easy to implement
Not highlighted in IDE
Pros
Less limitations on language structure
Can preserve existing language
Martin said that string interpolation is cool

57. k
Thanks. Questions?