SlideShare ist ein Scribd-Unternehmen logo
1 von 105
Downloaden Sie, um offline zu lesen
Chicago, October 19 - 22, 2010
Dr Paul King, @paulk_asert
paulk at asert.com.au
Concurrency with GPars
"Andy giveth and Bill taketh away"
GPars - 2
Source:HerbSutter:http://www.gotw.ca/publications/concurrency-ddj.htm
Why is it hard?
• Many issues to deal with:
– Doing things in parallel, concurrently,
asynchronously
• Processes, Threads, Co-routines, Events, Scheduling
– Sharing/Synchronization Mechanisms
• shared memory, locks, transactions, wait/notify, STM,
message passing, actors, serializability, persistence,
immutability
– Abstractions
• Shared memory on top of messaging passing
• Message passing on top of shared memory
• Dataflow, Selective Communication, Continuations
– Data Structures and Algorithms
• Queues, Heaps, Trees
• Sorting, Graph Algorithms
GPars - 3
GPars - 4
Java Concurrency Features
• The early years
– Threads, synchronised and non-synchronised
collections, synchronisation at the language level,
Monitors (wait/notify), Locks, ThreadLocal, final, ...
• More recent enhancements
– java.util.concurrent: Executors, Thread Pools,
Optimistic updates, Blocking queues, Synchronizers,
Callables, Futures, Atomic operations, Deques, ...
• Emerging
– Fork/Join & others, Kilim, Phasers, PicoThreads ...
• Leverage related APIs/technologies
– Networking, real-time, GUIs, simulation, database,
multimedia, operating systems, parallel processing,
distribution, mobile agents, nio, ...
Java Concurrency Best Practice?
• Java Concurrency in Practice:
–“If mutable threads access the
same mutable state variable
without appropriate
synchronization,
your program is broken”
–“When designing thread-safe classes,
good object-oriented techniques –
encapsulation, immutability, and clear
specification of invariants – are your
best friends”
GPars - 5
Topics
Groovy Intro
• Useful Groovy features for Concurrency
• Related Concurrency Libraries & Tools
• GPars
• More Info
GPars - 6
©ASERT2006-2010
What is Groovy?
GPars - 7
©ASERT2006-2010
• “Groovy is like a super version
of Java. It can leverage Java's
enterprise capabilities but also
has cool productivity features like closures,
DSL support, builders and dynamic typing.”
Groovy = Java – boiler plate code
+ optional dynamic typing
+ closures
+ domain specific languages
+ builders
+ metaprogramming
Groovy Starter
GPars - 8
©ASERT2006-2010
System.out.println("Hello, World!"); // supports Java syntax
println 'Hello, World!' // but can remove some syntax
String name = 'Guillaume' // Explicit typing/awareness
println "$name, I'll get the car." // Gstring (interpolation)
def longer = """${name}, the car
is in the next row.""" // multi-line, implicit type
assert 0.5 == 1/2 // BigDecimal equals()
assert 0.1 + 0.2 == 0.3 // and arithmetic
def printSize(obj) { // implicit/duck typing
print obj?.size() // safe dereferencing
}
def pets = ['ant', 'bee', 'cat'] // native list syntax
pets.each { pet -> // closure support
assert pet < 'dog' // overloading '<' on String
} // or: for (pet in pets)...
A Better Java...
GPars - 9
©ASERT2006-2010
import java.util.List;
import java.util.ArrayList;
class Erase {
private List removeLongerThan(List strings, int length) {
List result = new ArrayList();
for (int i = 0; i < strings.size(); i++) {
String s = (String) strings.get(i);
if (s.length() <= length) {
result.add(s);
}
}
return result;
}
public static void main(String[] args) {
List names = new ArrayList();
names.add("Ted"); names.add("Fred");
names.add("Jed"); names.add("Ned");
System.out.println(names);
Erase e = new Erase();
List shortNames = e.removeLongerThan(names, 3);
System.out.println(shortNames.size());
for (int i = 0; i < shortNames.size(); i++) {
String s = (String) shortNames.get(i);
System.out.println(s);
}
}
}
This code
is valid
Java and
valid Groovy
Based on an
example by
Jim Weirich
& Ted Leung
...A Better Java...
GPars - 10
©ASERT2006-2010
import java.util.List;
import java.util.ArrayList;
class Erase {
private List removeLongerThan(List strings, int length) {
List result = new ArrayList();
for (int i = 0; i < strings.size(); i++) {
String s = (String) strings.get(i);
if (s.length() <= length) {
result.add(s);
}
}
return result;
}
public static void main(String[] args) {
List names = new ArrayList();
names.add("Ted"); names.add("Fred");
names.add("Jed"); names.add("Ned");
System.out.println(names);
Erase e = new Erase();
List shortNames = e.removeLongerThan(names, 3);
System.out.println(shortNames.size());
for (int i = 0; i < shortNames.size(); i++) {
String s = (String) shortNames.get(i);
System.out.println(s);
}
}
}
Do the
semicolons
add anything?
And shouldn‟t
we us more
modern list
notation?
Why not
import common
libraries?
...A Better Java...
GPars - 11
©ASERT2006-2010
class Erase {
private List removeLongerThan(List strings, int length) {
List result = new ArrayList()
for (String s in strings) {
if (s.length() <= length) {
result.add(s)
}
}
return result
}
public static void main(String[] args) {
List names = new ArrayList()
names.add("Ted"); names.add("Fred")
names.add("Jed"); names.add("Ned")
System.out.println(names)
Erase e = new Erase()
List shortNames = e.removeLongerThan(names, 3)
System.out.println(shortNames.size())
for (String s in shortNames) {
System.out.println(s)
}
}
}
...A Better Java...
GPars - 12
©ASERT2006-2010
class Erase {
private List removeLongerThan(List strings, int length) {
List result = new ArrayList()
for (String s in strings) {
if (s.length() <= length) {
result.add(s)
}
}
return result
}
public static void main(String[] args) {
List names = new ArrayList()
names.add("Ted"); names.add("Fred")
names.add("Jed"); names.add("Ned")
System.out.println(names)
Erase e = new Erase()
List shortNames = e.removeLongerThan(names, 3)
System.out.println(shortNames.size())
for (String s in shortNames) {
System.out.println(s)
}
}
}
Do we need
the static types?
Must we always
have a main
method and
class definition?
How about
improved
consistency?
...A Better Java...
GPars - 13
©ASERT2006-2010
def removeLongerThan(strings, length) {
def result = new ArrayList()
for (s in strings) {
if (s.size() <= length) {
result.add(s)
}
}
return result
}
names = new ArrayList()
names.add("Ted")
names.add("Fred")
names.add("Jed")
names.add("Ned")
System.out.println(names)
shortNames = removeLongerThan(names, 3)
System.out.println(shortNames.size())
for (s in shortNames) {
System.out.println(s)
}
...A Better Java...
GPars - 14
©ASERT2006-2010
def removeLongerThan(strings, length) {
def result = new ArrayList()
for (s in strings) {
if (s.size() <= length) {
result.add(s)
}
}
return result
}
names = new ArrayList()
names.add("Ted")
names.add("Fred")
names.add("Jed")
names.add("Ned")
System.out.println(names)
shortNames = removeLongerThan(names, 3)
System.out.println(shortNames.size())
for (s in shortNames) {
System.out.println(s)
}
Shouldn‟t we
have special
notation for lists?
And special
facilities for
list processing?
Is „return‟
needed at end?
...A Better Java...
GPars - 15
©ASERT2006-2010
def removeLongerThan(strings, length) {
strings.findAll{ it.size() <= length }
}
names = ["Ted", "Fred", "Jed", "Ned"]
System.out.println(names)
shortNames = removeLongerThan(names, 3)
System.out.println(shortNames.size())
shortNames.each{ System.out.println(s) }
...A Better Java...
GPars - 16
©ASERT2006-2010
def removeLongerThan(strings, length) {
strings.findAll{ it.size() <= length }
}
names = ["Ted", "Fred", "Jed", "Ned"]
System.out.println(names)
shortNames = removeLongerThan(names, 3)
System.out.println(shortNames.size())
shortNames.each{ System.out.println(s) }
Is the method
now needed?
Easier ways to
use common
methods?
Are brackets
required here?
...A Better Java
GPars - 17
©ASERT2006-2010
names = ["Ted", "Fred", "Jed", "Ned"]
println names
shortNames = names.findAll{ it.size() <= 3 }
println shortNames.size()
shortNames.each{ println it }
["Ted", "Fred", "Jed", "Ned"]
3
Ted
Jed
Ned
Output:
Grapes / Grab: Google collections
GPars - 18
©ASERT2006-2010
@Grab('com.google.collections:google-collections:1.0')
import com.google.common.collect.HashBiMap
HashBiMap fruit =
[grape:'purple', lemon:'yellow', lime:'green']
assert fruit.lemon == 'yellow'
assert fruit.inverse().yellow == 'lemon'
GPars - 19
©ASERT2006-2010
Better Design Patterns: Delegate…
import java.util.Date;
public class Event {
private String title;
private String url;
private Date when;
public String getUrl() {
return url;
}
public void setUrl(String url) {
this.url = url;
}
public String getTitle() {
return title;
}
public void setTitle(String title) {
this.title = title;
}
// ...
public Date getWhen() {
return when;
}
public void setWhen(Date when) {
this.when = when;
}
public boolean before(Date other) {
return when.before(other);
}
public void setTime(long time) {
when.setTime(time);
}
public long getTime() {
return when.getTime();
}
public boolean after(Date other) {
return when.after(other);
}
// ...
GPars - 20
©ASERT2006-2010
…Better Design Patterns: Delegate…
import java.util.Date;
public class Event {
private String title;
private String url;
private Date when;
public String getUrl() {
return url;
}
public void setUrl(String url) {
this.url = url;
}
public String getTitle() {
return title;
}
public void setTitle(String title) {
this.title = title;
}
// ...
public Date getWhen() {
return when;
}
public void setWhen(Date when) {
this.when = when;
}
public boolean before(Date other) {
return when.before(other);
}
public void setTime(long time) {
when.setTime(time);
}
public long getTime() {
return when.getTime();
}
public boolean after(Date other) {
return when.after(other);
}
// ...
boilerplate
GPars - 21
©ASERT2006-2010
…Better Design Patterns: Delegate
class Event {
String title, url
@Delegate Date when
}
def gr8conf = new Event(title: "GR8 Conference",
url: "http://www.gr8conf.org",
when: Date.parse("yyyy/MM/dd", "2009/05/18"))
def javaOne = new Event(title: "JavaOne",
url: "http://java.sun.com/javaone/",
when: Date.parse("yyyy/MM/dd", "2009/06/02"))
assert gr8conf.before(javaOne.when)
Why Groovy? Technical Answer
• Minimal learning curve
• Compiles to bytecode
• Java object model & integration
• Annotations
• "Optional" static typing
• Both run-time and compile-time
metaprogramming
GPars - 22
Why Groovy? Adoption Assessment
• Innovators/Thought leaders
– Ideas, power, flexibility, novelty, thinking community
• Early adopters
– Productivity benefits and collegiate community
– Leverage JVM and potential for mainstream
• Mainstream
– Leverage existing Java skills, low learning curve
– Leverage JVM and production infrastructure
– Professional community
– Tools, tools, tools
Topics
• Groovy Intro
Useful Groovy features for Concurrency
• Related Concurrency Libraries & Tools
• GPars
• More Info
GPars - 24
©ASERT2006-2010
Concurrent Programming in Groovy
• Java concurrent programming enhancements
– Normal OO methods
– Ability to have immutable types
– Some concurrency building blocks
– Annotations with baked in goodness
• Process/Thread ease of use
– AntBuilder and GDK methods
• Closures for greater flexibility
– Enabler for concurrency
– Closure is Runnable and Callable
• Third-party libraries
– GPars, Functional Java (Actors), Multiverse, JCSP
– Cascading.groovy subproject for Hadoop clusters
– Jetlang, JPPF, GridGain, Google Collections, Gruple
– Groovy actors: http://www.groovyactors.org
GPars - 25
Thread & Process Enhancements
• DGM methods
– Thread.start{ … }
• Runtime metaprogramming
– Add custom control structures
e.g. ReentrantLock.withLock{ … }
• Process enhancements
– "ls –l".execute()
– proc1 | proc2 | proc3
– proc1.consumeProcessErrorStream()
– proc2.waitForOrKill(1000)
• AntBuilder
– ant.parallel { … }
– ant.exec() GPars - 26
Immutability options
• Built-in
• Google Collections
– Numerous improved immutable collection types
• Groovy run-time metaprogramming
• Groovy compile-time metaprogramming
– @Immutable can help us create such classes
– Also gives us @Synchronized and @Lazy
GPars - 27
import com.google.common.collect.*
List<String> animals = ImmutableList.of("cat", "dog", "horse")
animals << 'fish' // => java.lang.UnsupportedOperationException
def animals = ['cat', 'dog', 'horse'].asImmutable()
animals << 'fish' // => java.lang.UnsupportedOperationException
def animals = ['cat', 'dog', 'horse']
ArrayList.metaClass.leftShift = {
throw new UnsupportedOperationException() }
animals << 'fish' // => java.lang.UnsupportedOperationException
@Immutable...
• Java Immutable Class
– As per Joshua Bloch
Effective Java
GPars - 28
©ASERT2006-2010
public final class Punter {
private final String first;
private final String last;
public String getFirst() {
return first;
}
public String getLast() {
return last;
}
@Override
public int hashCode() {
final int prime = 31;
int result = 1;
result = prime * result + ((first == null)
? 0 : first.hashCode());
result = prime * result + ((last == null)
? 0 : last.hashCode());
return result;
}
public Punter(String first, String last) {
this.first = first;
this.last = last;
}
// ...
// ...
@Override
public boolean equals(Object obj) {
if (this == obj)
return true;
if (obj == null)
return false;
if (getClass() != obj.getClass())
return false;
Punter other = (Punter) obj;
if (first == null) {
if (other.first != null)
return false;
} else if (!first.equals(other.first))
return false;
if (last == null) {
if (other.last != null)
return false;
} else if (!last.equals(other.last))
return false;
return true;
}
@Override
public String toString() {
return "Punter(first:" + first
+ ", last:" + last + ")";
}
}
...@Immutable...
• Java Immutable Class
– As per Joshua Bloch
Effective Java
GPars - 29
©ASERT2006-2010
public final class Punter {
private final String first;
private final String last;
public String getFirst() {
return first;
}
public String getLast() {
return last;
}
@Override
public int hashCode() {
final int prime = 31;
int result = 1;
result = prime * result + ((first == null)
? 0 : first.hashCode());
result = prime * result + ((last == null)
? 0 : last.hashCode());
return result;
}
public Punter(String first, String last) {
this.first = first;
this.last = last;
}
// ...
// ...
@Override
public boolean equals(Object obj) {
if (this == obj)
return true;
if (obj == null)
return false;
if (getClass() != obj.getClass())
return false;
Punter other = (Punter) obj;
if (first == null) {
if (other.first != null)
return false;
} else if (!first.equals(other.first))
return false;
if (last == null) {
if (other.last != null)
return false;
} else if (!last.equals(other.last))
return false;
return true;
}
@Override
public String toString() {
return "Punter(first:" + first
+ ", last:" + last + ")";
}
}
boilerplate
...@Immutable
GPars - 30
©ASERT2006-2010
@Immutable class Punter {
String first, last
}
Topics
• Groovy Intro
• Useful Groovy features for Concurrency
Related Concurrency Libraries & Tools
• GPars
• More Info
GPars - 31
©ASERT2006-2010
Lightweight threads: Jetlang
• Jetlang
– A high performance threading library
– http://code.google.com/p/jetlang/
GPars - 32
import org.jetlang.fibers.ThreadFiber
import org.jetlang.core.Callback
import org.jetlang.channels.MemoryRequestChannel
import org.jetlang.channels.AsyncRequest
def req = new ThreadFiber() // or pool
def reply = new ThreadFiber()
def channel = new MemoryRequestChannel()
req.start()
reply.start()
channel.subscribe(reply, { it.reply(it.request.sum()) } as Callback)
AsyncRequest.withOneReply(req, channel, [3, 4, 5], { println it } as Callback)
sleep 1000
req.dispose()
reply.dispose()
Other High-Level Libraries: JPPF
– Open source Grid Computing platform
– http://www.jppf.org/
GPars - 33
import org.jppf.client.*
import java.util.concurrent.Callable
class Task implements Callable, Serializable {
private static final long serialVersionUID = 1162L
public Object call() {
println 'Executing Groovy'
"Hello JPPF from Groovy"
}
}
def client = new JPPFClient()
def job = new JPPFJob()
def task = new Task()
job.addTask task
def results = client.submit(job)
for (t in results) {
if (t.exception) throw t.exception
println "Result: " + t.result
}
Other High-Level Libraries: Gruple...
– http://code.google.com/p/gruple
– Simple abstraction to coordinate and synchronize
threads with ease – based on Tuplespaces
• Tuplespaces provide the illusion of a shared memory on top
of a message passing system, along with a small set of
operations to greatly simplify parallel programming
– Example Tuple:
[fname:"Vanessa", lname:"Williams", project:"Gruple"]
– Basic operations within a Tuplespace are:
• put - insert a tuple into the space
• get - read a tuple from the space (non-destructively)
• take - take a tuple from the space (a destructive read)
– Further reading: Eric Freeman, Susanne Hupfer, and
Ken Arnold. JavaSpaces Principles, Patterns, and
Practice, Addison Wesley, 1999
GPars - 34
…Other High-Level Libraries: Gruple...
GPars - 35
import org.gruple.SpaceService
def defaultSpace = SpaceService.getSpace()
defaultSpace << [fname:"Vanessa",
lname:"Williams",
project:"Gruple"]
println defaultSpace.get(fname:"Vanessa",
lname:"Williams",
project:"Gruple")
[project:Gruple, lname:Williams, fname:Vanessa]
Other High-Level Libraries: ...Gruple...
– Mandelbrot example (included in Gruple download)
GPars - 36
...
Space space = SpaceService.getSpace("mandelbrot")
Map template = createTaskTemplate()
Map task
String threadName = Thread.currentThread().name
while(true) {
ArrayList points
task = space.take(template)
println "Worker $threadName got task ${task['start']} for job ${task['jobId']
points = calculateMandelbrot(task)
Map result = createResult(task['jobId'], task['start'], points)
println "Worker $threadName writing result for task ${result['start']} for jo
space.put(result)
}
...
Other High-Level Libraries: ...Gruple
GPars - 37
Other High-Level Libraries: Cascading.groovy
– API/DSL for executing tasks on a Hadoop cluster
– http://www.cascading.org/
GPars - 38
def assembly = builder.assembly(name: "wordcount") {
eachTuple(args: ["line"], results: ["word"]) {
regexSplitGenerator(declared: ["word"], pattern: /[.,]*s+/)
}
group(["word"])
everyGroup(args: ["word"], results: ["word", "count"]) { count() }
group(["count"], reverse: true)
}
def map = builder.map() {
source(name: "wordcount") {
hfs(input) { text(["line"]) }
}
sink(name: "wordcount") {
hfs(output) { text() }
}
}
def flow = builder.flow(name: "wordcount",
map: map, assembly: assembly)
Other High-Level Libraries: GridGain…
– Simple & productive to use grid computing platform
– http://www.gridgain.com/
GPars - 39
class GridHelloWorldGroovyTask
extends GridTaskSplitAdapter<String, Integer> {
Collection split(int gridSize, Object phrase)
throws GridException {
// ...
}
Object reduce(List results) throws GridException {
// ...
}
}
import static GridFactory.*
start()
def grid = getGrid()
def future = grid.execute(GridHelloWorldGroovyTask,
"Hello World")
def phraseLen = future.get()
stop(true)
…Other High-Level Libraries: GridGain
• http://gridgain.blogspot.com/2010/10/worlds-shortest-mapreduce-
app.html
GPars - 40
words = "Counting Letters In This Phrase".split(' ')
map = new C1() { def apply(word) { word.size() } }
reduce = sumIntReducer()
println grid.forkjoin(SPREAD, yield(words, map), reduce)
// => 27
grid.forkjoin(SPREAD,yield("Counting Letters In This Phrase".split(' '),
new C1(){def apply(w){w.size()}}),sumReducer())
Multiverse STM…
– http://www.multiverse.org/
GPars - 41
import org.multiverse.api.GlobalStmInstance
import org.multiverse.api.Transaction
import org.multiverse.templates.TransactionTemplate
import org.multiverse.transactional.refs.LongRef
def from = new Account(10)
def to = new Account(10)
atomic {
from.balance -= 5
to.balance += 5
}
println "from $from.balance"
println "to $to.balance"
...
…Multiverse STM…
GPars - 42
...
void atomic(Closure block) {
atomic([:], block)
}
void atomic(Map args, Closure block) {
boolean readonly = args['readonly'] ?: false
boolean trackreads = args['trackreads'] ?: true
def txFactory = GlobalStmInstance.globalStmInstance.
transactionFactoryBuilder.
setReadonly(readonly).
setReadTrackingEnabled(trackreads).build()
new TransactionTemplate(txFactory) {
Object execute(Transaction transaction) {
block.call()
return null
}
}.execute()
}
...
…Multiverse STM
GPars - 43
class Account {
private final balance = new LongRef()
Account(long initial) {
balance.set initial
}
void setBalance(long newBalance) {
if (newBalance < 0)
throw new RuntimeException("not enough money")
balance.set newBalance
}
long getBalance() {
balance.get()
}
}
Testing multi-threaded applications: ConTest...
• Advanced Testing for Multi-Threaded Applications
– Tool for testing, debugging, and coverage-measuring
of concurrent programs (collects runtime statistics)
– Systematically and transparently (using a java agent)
schedules the execution of program threads in ways
likely to reveal race conditions, deadlocks, and other
intermittent bugs (collectively called synchronization
problems) with higher than normal frequency
– The ConTest run-time engine adds heuristically
controlled conditional instructions (adjustable by a
preferences file) that force thread switches, thus
helping to reveal concurrent bugs. You can use
existing tests and run ConTest multiple times – by
default different heuristics used each time it is run
• http://www.alphaworks.ibm.com/tech/contest
GPars - 44
...Testing multi-threaded applications: ConTest
GPars - 45
NUM = 5
count = 0
def incThread = { n -> Thread.start{
sleep n*10
//synchronized(ParalInc) {
count++
//}
} }
def threads = (1..NUM).collect(incThread)
threads.each{ it.join() }
assert count == NUM
targetClasses = ParalInc
timeoutTampering = true
noiseFrequency = 500
strength = 10000
Exception in thread "main" Assertion failed:
assert count == NUM
| | |
4 | 5
false
> groovyc ParalInc.groovy
> java -javaagent:../../Lib/ConTest.jar -cp %GROOVY_JAR%;. ParalInc
ParalInc.groovy
GPars - 46
GContracts
@Grab('org.gcontracts:gcontracts:1.0.2')
import org.gcontracts.annotations.*
@Invariant({ first != null && last != null })
class Person {
String first, last
@Requires({ delimiter in ['.', ',', ' '] })
@Ensures({ result == first + delimiter + last })
String getName(String delimiter) {
first + delimiter + last
}
}
new Person(first: 'John', last: 'Smith').getName('.')
1.8+
Testing: Spock
GPars - 47
class HelloSpock extends spock.lang.Specification {
def "length of Spock's and his friends' names"() {
expect:
name.size() == length
where:
name | length
"Spock" | 5
"Kirk" | 4
"Scotty" | 6
}
}
Topics
• Groovy Intro
• Useful Groovy features for Concurrency
• Related Concurrency Libraries & Tools
GPars
• More Info
GPars - 48
©ASERT2006-2010
Ralph Johnson: Parallel Programming Patterns…
GPars - 49
©ASERT2006-2010
http://strangeloop2010.com/talk/presentation_file/14485/Johnson-DataParallelism.pdf
…Ralph Johnson: Parallel Programming Patterns
GPars - 50
©ASERT2006-2010
http://strangeloop2010.com/talk/presentation_file/14485/Johnson-DataParallelism.pdf
GPars
• http://gpars.codehaus.org/
• Library classes and DSL sugar providing
intuitive ways for Groovy developers to
handle tasks concurrently. Logical parts:
– Actors provide a Groovy implementation of Scala-like
actors including "remote" actors on other machines
– Dataflow Concurrency supports natural shared-memory
concurrency model, using single-assignment variables
– Asynchronizer extends the Java 1.5 built-in support for
executor services to enable multi-threaded collection and
closure processing
– Parallelizer uses JSR-166y Parallel Arrays to enable
multi-threaded collection processing
– Safe a non-blocking mt-safe reference to mutable state
that is inspired by "agents" in Clojure
GPars - 51
©ASERT2006-2010
GPars: Parallel Collection Functions
GPars - 52
©ASERT2006-2010
def nums = 1..100000
def squares = nums
.collect{ it ** 2 }
.grep{ it % 7 == it % 5 }
.grep{ it % 3 == 0 }
println squares[0..3] + "..." + squares[-3..-1]
assert squares[0..3] == [36, 144, 1089, 1296]
@Grab('org.codehaus.gpars:gpars:0.10')
import static groovyx.gpars.GParsPool.withPool
def nums = 1..100000
withPool(5) {
def squares = nums.
collectParallel{ it ** 2 }.
grepParallel{ it % 7 == it % 5 }.
grepParallel{ it % 3 == 0 }
println squares[0..3] + "..." + squares[-3..-1]
assert squares[0..3] == [36, 144, 1089, 1296]
}
GPars: Transparent Parallel Collections
• Applies some Groovy metaprogramming
GPars - 53
©ASERT2006-2010
import static groovyx.gpars.GParsPool.withPool
withPool(5) {
def nums = 1..100000
nums.makeTransparent()
def squares = nums.
collect{ it ** 2 }.
grep{ it % 7 == it % 5 }.
grep{ it % 3 == 0 }
println squares[0..3] + "..." + squares[-3..-1]
assert squares[0..3] == [36, 144, 1089, 1296]
}
GPars concurrency-aware methods
Transparent Transitive? Parallel Lazy?
any { ... } anyParallel { ... } yes
collect { ... } yes collectParallel { ... }
count(filter) countParallel(filter)
each { ... } eachParallel { ... }
eachWithIndex { ... } eachWithIndexParallel { ... }
every { ... } everyParallel { ... } yes
find { ... } findParallel { ... }
findAll { ... } yes findAllParallel { ... }
findAny { ... } findAnyParallel { ... }
fold { ... } foldParallel { ... }
fold(seed) { ... } foldParallel(seed) { ... }
grep(filter) yes grepParallel(filter)
groupBy { ... } groupByParallel { ... }
max { ... } maxParallel { ... }
max() maxParallel()
min { ... } minParallel { ... }
min() minParallel()
split { ... } yes splitParallel { ... }
sum sumParallel // foldParallel +
GPars - 54Transitive means result is automatically transparent; Lazy means fails fast
GPars: Map-Reduce...
GPars - 55
©ASERT2006-2010
import static groovyx.gpars.GParsPool.withPool
withPool(5) {
def nums = 1..100000
println nums.parallel.
map{ it ** 2 }.
filter{ it % 7 == it % 5 }.
filter{ it % 3 == 0 }.
collection
}
...GPars: Map-Reduce
GPars - 56
©ASERT2006-2010
import static groovyx.gpars.GParsPool.withPool
withPool(5) {
def nums = 1..100000
println nums.parallel.
map{ it ** 2 }.
filter{ it % 7 == it % 5 }.
filter{ it % 3 == 0 }.
reduce{ a, b -> a + b }
}
GPars parallel array methods
Method Return Type
combine(initValue) { ... } Map
filter { ... } Parallel array
collection Collection
groupBy { ... } Map
map { ... } Parallel array
max() T
max { ... } T
min() T
min { ... } T
reduce { ... } T
reduce(seed) { ... } T
size() int
sort { ... } Parallel array
sum() T
parallel // on a Collection Parallel array
GPars - 57
Parallel Collections vs Map-Reduce
GPars - 58
Fork Fork
JoinJoin
Map
Map
Reduce
Map
Map
Reduce
Reduce
Map
Filter
FilterMap
GPars: Dataflows...
GPars - 59
©ASERT2006-2010
import groovyx.gpars.dataflow.DataFlows
import static groovyx.gpars.dataflow.DataFlow.task
final flow = new DataFlows()
task { flow.result = flow.x + flow.y }
task { flow.x = 10 }
task { flow.y = 5 }
assert 15 == flow.result
new DataFlows().with {
task { result = x * y }
task { x = 10 }
task { y = 5 }
assert 50 == result
}
...GPars: Dataflows...
• Evaluating:
GPars - 60
©ASERT2006-2010
import groovyx.gpars.dataflow.DataFlows
import static groovyx.gpars.dataflow.DataFlow.task
final flow = new DataFlows()
task { flow.a = 10 }
task { flow.b = 5 }
task { flow.x = flow.a - flow.b }
task { flow.y = flow.a + flow.b }
task { flow.result = flow.x * flow.y }
assert flow.result == 75
b
10 5
a
+-
*
result = (a – b) * (a + b)
x y
Question: what happens if I change the order of the task statements here?
...GPars: Dataflows...
• Naive attempt for loops
GPars - 61
©ASERT2006-2010
import groovyx.gpars.dataflow.DataFlows
import static groovyx.gpars.dataflow.DataFlow.task
final flow = new DataFlows()
[10, 20].each { thisA ->
[4, 5].each { thisB ->
task { flow.a = thisA }
task { flow.b = thisB }
task { flow.x = flow.a - flow.b }
task { flow.y = flow.a + flow.b }
task { flow.result = flow.x * flow.y }
println flow.result
}
}
// => java.lang.IllegalStateException:
A DataFlowVariable can only be assigned once.
...
task { flow.a = 10 }
...
task { flow.a = 20 }
...GPars: Dataflows...
GPars - 62
©ASERT2006-2010
import groovyx.gpars.dataflow.DataFlowStream
import static groovyx.gpars.dataflow.DataFlow.*
final streamA = new DataFlowStream()
final streamB = new DataFlowStream()
final streamX = new DataFlowStream()
final streamY = new DataFlowStream()
final results = new DataFlowStream()
operator(inputs: [streamA, streamB],
outputs: [streamX, streamY]) {
a, b -> streamX << a - b; streamY << a + b
}
operator(inputs: [streamX, streamY],
outputs: [results]) { x, y -> results << x * y }
[[10, 20], [4, 5]].combinations().each{ thisA, thisB ->
task { streamA << thisA }
task { streamB << thisB }
}
4.times { println results.val }
b
10
10
20
20
4
5
4
5
a
+-
*
84
75
384
375
...GPars: Dataflows
• Amenable to static analysis
• Race conditions avoided
• Deadlocks “typically” become repeatable
GPars - 63
©ASERT2006-2010
import groovyx.gpars.dataflow.DataFlows
import static groovyx.gpars.dataflow.DataFlow.task
final flow = new DataFlows()
task { flow.x = flow.y }
task { flow.y = flow.x }
GPars: Dataflow Sieve
GPars - 64
©ASERT2006-2010
final int requestedPrimeNumberCount = 1000
final DataFlowStream initialChannel = new DataFlowStream()
task {
(2..10000).each {
initialChannel << it
}
}
def filter(inChannel, int prime) {
def outChannel = new DataFlowStream()
operator([inputs: [inChannel], outputs: [outChannel]]) {
if (it % prime != 0) {
bindOutput it
}
}
return outChannel
}
def currentOutput = initialChannel
requestedPrimeNumberCount.times {
int prime = currentOutput.val
println "Found: $prime"
currentOutput = filter(currentOutput, prime)
}
Source: http://groovyconsole.appspot.com/script/235002
GPars: Actors...
• Actors provide explicit coordination: they
don‟t share state, instead coordinating via
asynchronous messages
– Contrasting with predefined coordination for fork/join &
map/filter/reduce & implicit coordination for dataflow
– Messages are processed one at a time normally in the
order they were sent (which is non-deterministic due to
asynchronous nature)
– Some actor systems allowing message delivery to be
prioritised; others allow for sharing some (readonly) state;
some allow remote actors for load balancing/robustness
• Not new in concept
– But has received recent publicity due to special
support in Erlang, Scala and other languages
GPars - 65
©ASERT2006-2010
…GPars: Actors...
• Class with the following lifecycle &
methods
– But also DSL sugar & enhancements
GPars - 66
©ASERT2006-2010
start()
stop()
act()
send(msg)
sendAndWait(msg)
loop { }
react { msg -> }
msg.reply(replyMsg)
receive()
join()
…GPars: Actors...
GPars - 67
©ASERT2006-2010
import static groovyx.gpars.actor.Actors.*
def decrypt = reactor { code -> code.reverse() }
def audit = reactor { println it }
def main = actor {
decrypt 'terces pot'
react { plainText ->
audit plainText
}
}
main.join()
audit.stop()
audit.join()
Source: ReGina
…GPars: Actors...
GPars - 68
©ASERT2006-2010
final class FilterActor extends DynamicDispatchActor {
private final int myPrime
private def follower
def FilterActor(final myPrime) { this.myPrime = myPrime; }
def onMessage(int value) {
if (value % myPrime != 0) {
if (follower) follower value
else {
println "Found $value"
follower = new FilterActor(value).start()
}
}
}
def onMessage(def poisson) {
if (follower) {
def sender = poisson.sender
follower.sendAndContinue(poisson, {this.stop(); sender?.send('Done
} else { //I am the last in the chain
stop()
reply 'Done'
}
}
}
Source: http://groovyconsole.appspot.com/script/242001
…GPars: Actors
GPars - 69
©ASERT2006-2010
(2..requestedPrimeNumberBoundary).each {
firstFilter it
}
firstFilter.sendAndWait 'Poisson'
Source: http://groovyconsole.appspot.com/script/242001
Dining Philosophers…
GPars - 70
Philosopher
Thinking | Eating
Philosopher
Thinking | Eating
Philosopher
Thinking | Eating
Philosopher
Thinking | Eating
Philosopher
Thinking | Eating
Fork
Available | InUse
Fork
Available | InUse
Fork
Available | InUse
Fork
Available | InUse
Fork
Available | InUse
…Dining Philosophers
GPars - 71
Philosopher
Thinking | Eating
Philosopher
Thinking | Eating
Philosopher
Thinking | Eating
Philosopher
Thinking | Eating
Philosopher
Thinking | Eating
Fork
Available | InUse
Fork
Available | InUse
Fork
Available | InUse
Fork
Available | InUse
Fork
Available | InUse
Accepted | Rejected
Take | Release Take | Release
Dining Philosophers: Actors...
GPars - 72
©ASERT2006-2010
// adapted from GPars example, repo: http://git.codehaus.org/gitweb.cgi?p=gpars.git
// file: src/test/groovy/groovyx/gpars/samples/actors/DemoDiningPhilosophers.groovy
@Grab('org.codehaus.gpars:gpars:0.10')
import groovyx.gpars.actor.*
import groovy.beans.Bindable
def names = ['socrates', 'plato', 'aristotle', 'descartes', 'nietzsche']
Actors.defaultActorPGroup.resize names.size()
class Philosopher extends AbstractPooledActor {
private random = new Random()
String name
int timesEaten = 0
def forks
@Bindable String status
void act() {
assert 2 == forks.size()
loop {
think()
forks*.send new Take()
react {a ->
react {b ->
if ([a, b].any {Rejected.isCase it}) {
[a, b].find {Accepted.isCase it}?.reply new Release()
} else {
eat()
[a, b]*.reply new Release()
}
}
}
}
}
…Dining Philosophers: Actors...
GPars - 73
©ASERT2006-2010
…
void think() {
setStatus('thinking')
sleep random.nextInt(5000)
setStatus('')
}
void eat() {
setStatus("eating ${++timesEaten}")
sleep random.nextInt(3000)
setStatus('')
}
String toString() {
switch (timesEaten) {
case 0: return "$name has starved"
case 1: return "$name has eaten once"
default: return "$name has eaten $timesEaten times"
}
}
}
final class Take {}
final class Accepted {}
final class Rejected {}
final class Release {}
…Dining Philosophers: Actors...
GPars - 74
©ASERT2006-2010
…
class Fork extends AbstractPooledActor {
String name
boolean available = true
void act() {
loop {
react {message ->
switch (message) {
case Take:
if (available) {
available = false
reply new Accepted()
}
else reply new Rejected()
break
case Release:
assert !available
available = true
break
default: throw new IllegalStateException("Cannot process the message: $message")
}
}
}
}
}
def forks = (1..names.size()).collect { new Fork(name: "Fork $it") }
def philosophers = (1..names.size()).collect {
new Philosopher(name: names[it - 1], forks: [forks[it - 1], forks[it % names.size()]])
}
…Dining Philosophers: Actors
GPars - 75
©ASERT2006-2010
…
import groovy.swing.*
import java.awt.Font
import static javax.swing.JFrame.*
def frame = new SwingBuilder().frame(title: 'Philosophers', defaultCloseOperation: EXIT_ON_CLOSE) {
vbox {
hbox {
(0..<names.size()).each { i ->
def widget = textField(id: names[i], text: names[i].center(14))
widget.font = new Font(widget.font.name, widget.font.style, 36)
philosophers[i].propertyChange = { widget.text = philosophers[i].status.center(14) }
}
}
}
}
frame.pack()
frame.visible = true
forks*.start()
sleep 1000
philosophers*.start()
sleep 10000
forks*.stop()
forks*.join()
philosophers*.stop()
philosophers*.join()
frame.dispose()
philosophers.each { println it }
socrates has eaten 3 times
plato has eaten 3 times
aristotle has eaten 6 times
descartes has eaten 2 times
nietzsche has eaten 5 times
Dining Philosophers: CSP...
GPars - 76
©ASERT2006-2010
// inspired by similar examples at the web sites below:
// http://www.cs.kent.ac.uk/projects/ofa/jcsp/
// http://www.soc.napier.ac.uk/~jmk/#_Toc271192596
@Grab('org.codehaus.gpars:gpars:0.10')
import org.jcsp.lang.*
import groovyx.gpars.csp.PAR
import groovyx.gpars.csp.ALT
import static java.lang.System.currentTimeMillis
def names = ['socrates', 'plato', 'aristotle', 'descartes', 'nietzsche']
enum ForkAction { Take, Release, Stop }
import static ForkAction.*
class Philosopher implements CSProcess {
ChannelOutput leftFork, rightFork
String name
def forks = []
private random = new Random()
private timesEaten = 0
private start = currentTimeMillis()
void run() {
while (currentTimeMillis() - start < 10000) {
think()
eat()
}
[leftFork, rightFork].each { it.write(Stop) }
println toString()
}
…
…Dining Philosophers: CSP...
GPars - 77
©ASERT2006-2010
…
void think() {
println "$name is thinking"
sleep random.nextInt(50)
}
void eat() {
[leftFork, rightFork].each { it.write(Take) }
println "$name is EATING"
timesEaten++
sleep random.nextInt(200)
[leftFork, rightFork].each { it.write(Release) }
}
String toString() {
switch (timesEaten) {
case 0: return "$name has starved"
case 1: return "$name has eaten once"
default: return "$name has eaten $timesEaten times"
}
}
}
…Dining Philosophers: CSP...
GPars - 78
©ASERT2006-2010
…
class Fork implements CSProcess {
ChannelInput left, right
private active = [0, 1] as Set
void run() {
def fromPhilosopher = [left, right]
def forkAlt = new ALT(fromPhilosopher)
while (active) {
def i = forkAlt.select()
read fromPhilosopher, i, Take
read fromPhilosopher, i, Release
}
}
void read(phil, index, expected) {
if (!active.contains(index)) return
def m = phil[index].read()
if (m == Stop) active -= index
else assert m == expected
}
}
…
…Dining Philosophers: CSP
GPars - 79
©ASERT2006-2010
…
def lefts = Channel.createOne2One(names.size())
def rights = Channel.createOne2One(names.size())
def philosophers = (0..<names.size()).collect { i ->
return new Philosopher(leftFork: lefts[i].out(),
rightFork: rights[i].out(),
name: names[i])
}
def forks = (0..<names.size()).collect { i ->
return new Fork(left: lefts[i].in(),
right: rights[(i + 1) % names.size()].in())
}
def processList = philosophers + forks
new PAR(processList).run()
Why CSP?
• Amenable to proof
and analysis
GPars - 80
Picture source: http://wotug.org/parallel/theory/formal/csp/Deadlock/
GPars: Agents...
• Agents safeguard non-thread safe objects
• Only the agent can update the underlying
object
• “Code” to update the protected object is
sent to the agent
• Can be used with other approaches
GPars - 81
©ASERT2006-2010
…GPars: Agents
GPars - 82
©ASERT2006-2010
@Grab('org.codehaus.gpars:gpars:0.10')
import groovyx.gpars.agent.Agent
def speakers = new Agent<List>(['Alex'], {it?.clone()}) // add Alex
speakers.send {updateValue it << 'Hilary'} // add Hilary
final Thread t1 = Thread.start {
speakers.send {updateValue it << 'Ken'} // add Ken
}
final Thread t2 = Thread.start {
speakers << {updateValue it << 'Guy'} // add Guy
speakers << {updateValue it << 'Ralph'} // add Ralph
}
[t1, t2]*.join()
assert new HashSet(speakers.val) ==
new HashSet(['Alex', 'Hilary', 'Ken', 'Guy', 'Ralph'])
Source: Gpars examples
GPars for testing
GPars - 83
©ASERT2006-2010
@Grab('net.sourceforge.htmlunit:htmlunit:2.6')
import com.gargoylesoftware.htmlunit.WebClient
@Grab('org.codehaus.gpars:gpars:0.10')
import static groovyx.gpars.GParsPool.*
def testCases = [
['Home', 'Bart', 'Content 1'],
['Work', 'Homer', 'Content 2'],
['Travel', 'Marge', 'Content 3'],
['Food', 'Lisa', 'Content 4']
]
withPool(3) {
testCases.eachParallel{ category, author, content ->
postAndCheck category, author, content
}
}
private postAndCheck(category, author, content) {
...
Guy Steele example in Groovy…
GPars - 84
©ASERT2006-2010
def words = { s ->
def result = []
def word = ''
s.each{ ch ->
if (ch == ' ') {
if (word) result += word
word = ''
} else word += ch
}
if (word) result += word
result
}
assert words("This is a sample") == ['This', 'is', 'a', 'sample']
assert words(" Here is another sample ") ==
['Here', 'is', 'another', 'sample']
assert words("JustOneWord") == ['JustOneWord']
assert words("Here is a sesquipedalian string of words") ==
['Here', 'is', 'a', 'sesquipedalian', 'string', 'of', 'words']
assert words(" ") == [] && words("") == []
Sequential version
Guy Steele‟s example from keynote (from slide 52 onwards for several slides):
http://strangeloop2010.com/talk/presentation_file/14299/GuySteele-parallel.pdf
…Guy Steele example in Groovy…
GPars - 85
©ASERT2006-2010
http://strangeloop2010.com/talk/presentation_file/14485/Johnson-DataParallelism.pdf
…Guy Steele example in Groovy…
GPars - 86
©ASERT2006-2010
Guy Steele‟s example from keynote (from slide 52 onwards for several slides):
http://strangeloop2010.com/talk/presentation_file/14299/GuySteele-parallel.pdf
…Guy Steele example in Groovy…
GPars - 87
©ASERT2006-2010
@Immutable class Chunk {
String s
def plus(Chunk other) { new Chunk(s + other.s) }
def plus(Segment other) {
new Segment(s + other.l, other.m, other.r)
}
}
@Immutable class Segment {
String l; List m; String r
def plus(Chunk other) { new Segment(l, m, r + other.s) }
def plus(Segment other) {
new Segment(l, m + maybeWord(r + other.l) + other.m, other.r)
}
}
class Util {
static processChar(ch) { ch == ' ' ?
new Segment('', [], '') : new Chunk(ch) }
static maybeWord(s) { s ? [s] : [] }
}
import static Util.*
...
Refactored sequential version
…Guy Steele example in Groovy…
GPars - 88
©ASERT2006-2010
def words = { s ->
def result
s.each{ ch ->
if (!result) result = processChar(ch)
else result += processChar(ch)
}
switch(result) {
case Chunk: return maybeWord(result.s)
case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) }
case null: return []
}
}
assert words("This is a sample") == ['This', 'is', 'a', 'sample']
assert words(" Here is another sample ") ==
['Here', 'is', 'another', 'sample']
assert words("JustOneWord") == ['JustOneWord']
assert words("Here is a sesquipedalian string of words") ==
['Here', 'is', 'a', 'sesquipedalian', 'string', 'of', 'words']
assert words(" ") == [] && words("") == []
Refactored sequential version
…Guy Steele example in Groovy…
GPars - 89
©ASERT2006-2010
def swords = { s ->
def result
s.each{ ch ->
if (!result) result = processChar(ch)
else result += processChar(ch)
}
result ?: new Chunk('')
}
THREADS = 4
def words = { s ->
int n = (s.size() + THREADS - 1) / THREADS
def map = new java.util.concurrent.ConcurrentHashMap()
(0..<THREADS).collect { i -> Thread.start {
def (min, max) = [[s.size(),i*n].min(), [s.size(),(i+1)*n].min()]
map[i] = swords(s[min..<max])
}}*.join()
def result = map.entrySet().sort{ it.key }.sum{ it.value }
switch(result) {
case Chunk: return maybeWord(result.s)
case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) }
}
}
Roll your own threading with ConcurrentHashMap version
…Guy Steele example in Groovy…
GPars - 90
©ASERT2006-2010
def words = { s ->
int n = (s.size() + THREADS - 1) / THREADS
def min = (0..<THREADS).collectEntries{ [it, [s.size(),it*n].min()] }
def max = (0..<THREADS).collectEntries{ [it, [s.size(),(it+1)*n].min()] }
def result = new DataFlows().with {
task { a = swords(s[min[0]..<max[0]]) }
task { b = swords(s[min[1]..<max[1]]) }
task { c = swords(s[min[2]..<max[2]]) }
task { d = swords(s[min[3]..<max[3]]) }
task { sum1 = a + b }
task { sum2 = c + d }
task { sum = sum1 + sum2 }
println 'Tasks ahoy!'
sum
}
switch(result) {
case Chunk: return maybeWord(result.s)
case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) }
}
}
DataFlow version: partially hard-coded to 4 partitions for easier reading
…Guy Steele example in Groovy…
GPars - 91
©ASERT2006-2010
GRANULARITY_THRESHHOLD = 10
THREADS = 4
println GParsPool.withPool(THREADS) {
def result = runForkJoin(0, input.size(), input){ first, last, s ->
def size = last - first
if (size <= GRANULARITY_THRESHHOLD) {
swords(s[first..<last])
} else { // divide and conquer
def mid = first + ((last - first) >> 1)
forkOffChild(first, mid, s)
forkOffChild(mid, last, s)
childrenResults.sum()
}
}
switch(result) {
case Chunk: return maybeWord(result.s)
case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) }
}
}
Fork/Join version
…Guy Steele example in Groovy…
GPars - 92
©ASERT2006-2010
THRESHHOLD = 10
def split(raw) {
raw.size() <= THRESHHOLD ? raw :
[raw[0..<THRESHHOLD]] + split(raw.substring(THRESHHOLD))
}
println GParsPool.withPool(THREADS) {
def ans = split(input).parallel.map(swords).reduce{ a,b -> a + b }
switch(ans) {
case Chunk: return maybeWord(ans.s)
case Segment: return ans.with{ maybeWord(l) + m + maybeWord(r) }
}
}
Map/Reduce version
…Guy Steele example in Groovy
GPars - 93
©ASERT2006-2010
println GParsPool.withPool(THREADS) {
def ans = input.collectParallel{ processChar(it) }.sum()
switch(ans) {
case Chunk: return maybeWord(ans.s)
case Segment: return ans.with{ maybeWord(l) + m + maybeWord(r) }
}
}
Just leveraging the algorithm‟s parallel nature
Topics
• Groovy Intro
• Useful Groovy features for Concurrency
• Related Concurrency Libraries & Tools
• GPars
More Info
GPars - 94
©ASERT2006-2010
More Information about Concurrency
• Web sites
– http://gpars.codehaus.org/
– http://g.oswego.edu/
Doug Lea's home page
– http://gee.cs.oswego.edu/dl/concurrency-interest/
– http://jcip.net/
Companion site for Java Concurrency in Practice
– http://www.eecs.usma.edu/webs/people/okasaki/pubs.html#cup98
Purely Functional Data Structures
– http://delicious.com/kragen/concurrency
Concurrency bookmark list
– http://www.gotw.ca/publications/concurrency-ddj.htm
The Free Lunch is Over, Herb Sutter
– http://manticore.cs.uchicago.edu/papers/damp07.pdf
– http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=10142
Concepts, Techniques, and Models of Computer Programming
GPars - 95
More Information about Groovy
• Web sites
– http://groovy.codehaus.org
– http://grails.codehaus.org
– http://pleac.sourceforge.net/pleac_groovy (many examples)
– http://www.asert.com.au/training/java/GV110.htm (workshop)
• Mailing list for users
– user@groovy.codehaus.org
• Information portals
– http://www.aboutgroovy.org
– http://www.groovyblogs.org
• Documentation (1000+ pages)
– Getting Started Guide, User Guide, Developer Guide, Testing
Guide, Cookbook Examples, Advanced Usage Guide
• Books
– Several to choose from ...
GPars - 96
More Information: Groovy in Action
GPars - 97
Contains a
chapter on
GPars!
Topics
• Bonus Material
– Multiverse Philosophers
– Jetlang Philosophers
– Gruple Philosophers
GPars - 98
©ASERT2006-2010
Multiverse Philosophers…
GPars - 99
//@Grab('org.multiverse:multiverse-core:0.7-SNAPSHOT')
//@Grab('org.multiverse:multiverse-alpha:0.7-SNAPSHOT')
//@Grab('org.multiverse:multiverse-groovy:0.7-SNAPSHOT')
//@GrabConfig(systemClassLoader=true, initContextClassLoader = true)
// adapted multiverse Groovy example: http://git.codehaus.org/gitweb.cgi?p=multiverse.git
// file: multiverse-groovy/src/test/groovy/org/multiverse/integration/
org/multiverse/integration/examples/DiningPhilosphersTest.groovy
import org.multiverse.transactional.refs.BooleanRef
import org.multiverse.transactional.refs.IntRef
import static MultiverseGroovyLibrary.*
def food = new IntRef(5)
def names = ['socrates', 'plato', 'aristotle', 'descartes', 'nietzsche']
def forks = (1..5).collect { new Fork(id: it, free: new BooleanRef(true)) }
def philosophers = (0..4).collect {
new Philosopher(name: names[it], food: food,
left: forks[(it + 1) % 5], right: forks[it])
}
def threads = philosophers.collect { new Thread(it) }
threads*.start()
threads*.join()
philosophers.each { println it }
class Fork {
int id
BooleanRef free
void take() { free.set(false) }
void release() { free.set(true) }
}
…Multiverse Philosophers
GPars - 100
class Philosopher implements Runnable {
String name
Fork left, right
IntRef timesEaten = new IntRef()
IntRef food
void eat() {
atomic(trackreads: true, explicitRetryAllowed: true) {
left.free.await(true)
right.free.await(true)
if (food.get() > 0) {
left.take(); right.take()
timesEaten.inc(); sleep 10; food.dec()
}
}
}
void think() {
atomic(trackreads: true, explicitRetryAllowed: true) {
left.release(); right.release()
}
sleep 10
}
void run() { 10.times { eat(); think() } }
String toString() {
switch (timesEaten) {
case 0: return "$name has starved"
case 1: return "$name has eaten once"
default: return "$name has eaten $timesEaten times"
}
}
}
Jetlang Philosophers…
GPars - 101
import org.jetlang.core.Callback
import org.jetlang.fibers.ThreadFiber
import org.jetlang.channels.*
def names = ['socrates', 'plato', 'aristotle', 'descartes', 'nietzsche']
class Philosopher implements Callback {
private random = new Random()
String name
int timesEaten = 0
String status
def forks
private channels = [new MemoryRequestChannel(), new MemoryRequestChannel()]
private req = new ThreadFiber() // or from pool
private reply = new ThreadFiber()
private responses = []
private gotFork = { it instanceof Accepted }
void start() {
assert forks.size() == 2
req.start()
reply.start()
(0..1).each{ channels[it].subscribe(reply, forks[it]) }
think()
}
String toString() {
switch (timesEaten) {
case 0: return "$name has starved"
case 1: return "$name has eaten once"
default: return "$name has eaten $timesEaten times"
}
}
…Jetlang Philosophers…
GPars - 102
…
void think() {
println(name + ' is thinking')
sleep random.nextInt(3000)
(0..1).each{ AsyncRequest.withOneReply(req, channels[it], new Take(it), this); }
}
void eat() {
timesEaten++
println toString()
sleep random.nextInt(2000)
}
void onMessage(Object message) {
responses << message
if (responses.size() == 2) {
if (responses.every(gotFork)) {
eat()
}
responses.findAll(gotFork).each {
int index = it.index
channels[index].publish(req, new Release(index), forks[index])
}
responses = []
think()
}
}
}
@Immutable class Take { int index }
@Immutable class Accepted { int index }
@Immutable class Rejected { int index }
@Immutable class Release { int index }
…
…Jetlang Philosophers
GPars - 103
…
class Fork implements Callback {
String name
def holder = []
void onMessage(message) {
def msg = message instanceof Request ? message.request : message
def index = msg.index
switch (msg) {
case Take:
if (!holder) {
holder << index
message.reply(new Accepted(index))
} else message.reply(new Rejected(index))
break
case Release:
assert holder == [index]
holder = []
break
default: throw new IllegalStateException("Cannot process the message: $message")
}
}
}
def forks = (1..names.size()).collect { new Fork(name: "Fork $it") }
def philosophers = (1..names.size()).collect {
new Philosopher(name: names[it - 1], forks: [forks[it - 1], forks[it % names.size()]])
}
philosophers*.start()
sleep 10000
philosophers.each { println it }
Gruple Philosophers…
GPars - 104
import org.gruple.SpaceService
import org.gruple.Space
class Philosopher {
private random = new Random()
String name
Space space
private timesEaten = 0
int id, num
boolean done = false
void run() {
while (true) {
think()
if (done) return
space.take(fork: id)
space.take(fork: (id + 1) % num)
eat()
space.put(fork: id)
space.put(fork: (id + 1) % num)
}
}
void think() {
println "$name is thinking"
sleep random.nextInt(500)
}
void eat() {
println "$name is EATING"
timesEaten++
sleep random.nextInt(1000)
}
…
…
socrates is thinking
nietzsche is thinking
descartes is EATING
aristotle is EATING
descartes is thinking
plato is EATING
aristotle is thinking
socrates is EATING
plato is thinking
nietzsche is EATING
socrates is thinking
nietzsche is thinking
descartes is EATING
descartes is thinking
socrates has eaten 5 times
plato has eaten 4 times
aristotle has eaten 4 times
descartes has eaten 4 times
nietzsche has eaten 5 times
…Gruple Philosophers
GPars - 105
…
String toString() {
switch (timesEaten) {
case 0: return "$name has starved"
case 1: return "$name has eaten once"
default: return "$name has eaten $timesEaten times"
}
}
}
def names = ['socrates', 'plato', 'aristotle', 'descartes', 'nietzsche']
def diningSpace = SpaceService.getSpace('Dining')
def philosophers = (0..<names.size()).collect{
new Philosopher(name: names[it], id: it, space: diningSpace, num: names.size())
}
(0..<names.size()).each{ diningSpace << [fork: it] }
sleep 500
def threads = (0..<names.size()).collect{ n -> Thread.start{ philosophers[n].run() } }
sleep 10000
philosophers*.done = true
sleep 2000
threads.join()
println()
philosophers.each{ println it }

Weitere ähnliche Inhalte

Was ist angesagt?

Programming with Python and PostgreSQL
Programming with Python and PostgreSQLProgramming with Python and PostgreSQL
Programming with Python and PostgreSQLPeter Eisentraut
 
Kotlin @ Coupang Backend 2017
Kotlin @ Coupang Backend 2017Kotlin @ Coupang Backend 2017
Kotlin @ Coupang Backend 2017Sunghyouk Bae
 
(Greach 2015) Dsl'ing your Groovy
(Greach 2015) Dsl'ing your Groovy(Greach 2015) Dsl'ing your Groovy
(Greach 2015) Dsl'ing your GroovyAlonso Torres
 
What can be done with Java, but should better be done with Erlang (@pavlobaron)
What can be done with Java, but should better be done with Erlang (@pavlobaron)What can be done with Java, but should better be done with Erlang (@pavlobaron)
What can be done with Java, but should better be done with Erlang (@pavlobaron)Pavlo Baron
 
Clojure for Java developers - Stockholm
Clojure for Java developers - StockholmClojure for Java developers - Stockholm
Clojure for Java developers - StockholmJan Kronquist
 
Clojure Intro
Clojure IntroClojure Intro
Clojure Introthnetos
 
Turtle Graphics in Groovy
Turtle Graphics in GroovyTurtle Graphics in Groovy
Turtle Graphics in GroovyJim Driscoll
 
CodeCamp Iasi 10 march 2012 - Practical Groovy
CodeCamp Iasi 10 march 2012 - Practical GroovyCodeCamp Iasi 10 march 2012 - Practical Groovy
CodeCamp Iasi 10 march 2012 - Practical GroovyCodecamp Romania
 
#살아있다 #자프링외길12년차 #코프링2개월생존기
#살아있다 #자프링외길12년차 #코프링2개월생존기#살아있다 #자프링외길12년차 #코프링2개월생존기
#살아있다 #자프링외길12년차 #코프링2개월생존기Arawn Park
 
Fun Teaching MongoDB New Tricks
Fun Teaching MongoDB New TricksFun Teaching MongoDB New Tricks
Fun Teaching MongoDB New TricksMongoDB
 
BDD - Behavior Driven Development Webapps mit Groovy Spock und Geb
BDD - Behavior Driven Development Webapps mit Groovy Spock und GebBDD - Behavior Driven Development Webapps mit Groovy Spock und Geb
BDD - Behavior Driven Development Webapps mit Groovy Spock und GebChristian Baranowski
 
python chapter 1
python chapter 1python chapter 1
python chapter 1Raghu nath
 

Was ist angesagt? (20)

Programming with Python and PostgreSQL
Programming with Python and PostgreSQLProgramming with Python and PostgreSQL
Programming with Python and PostgreSQL
 
Kotlin @ Coupang Backend 2017
Kotlin @ Coupang Backend 2017Kotlin @ Coupang Backend 2017
Kotlin @ Coupang Backend 2017
 
core.logic introduction
core.logic introductioncore.logic introduction
core.logic introduction
 
(Greach 2015) Dsl'ing your Groovy
(Greach 2015) Dsl'ing your Groovy(Greach 2015) Dsl'ing your Groovy
(Greach 2015) Dsl'ing your Groovy
 
What can be done with Java, but should better be done with Erlang (@pavlobaron)
What can be done with Java, but should better be done with Erlang (@pavlobaron)What can be done with Java, but should better be done with Erlang (@pavlobaron)
What can be done with Java, but should better be done with Erlang (@pavlobaron)
 
Poly-paradigm Java
Poly-paradigm JavaPoly-paradigm Java
Poly-paradigm Java
 
Clojure for Java developers - Stockholm
Clojure for Java developers - StockholmClojure for Java developers - Stockholm
Clojure for Java developers - Stockholm
 
Clojure Intro
Clojure IntroClojure Intro
Clojure Intro
 
Turtle Graphics in Groovy
Turtle Graphics in GroovyTurtle Graphics in Groovy
Turtle Graphics in Groovy
 
CodeCamp Iasi 10 march 2012 - Practical Groovy
CodeCamp Iasi 10 march 2012 - Practical GroovyCodeCamp Iasi 10 march 2012 - Practical Groovy
CodeCamp Iasi 10 march 2012 - Practical Groovy
 
#살아있다 #자프링외길12년차 #코프링2개월생존기
#살아있다 #자프링외길12년차 #코프링2개월생존기#살아있다 #자프링외길12년차 #코프링2개월생존기
#살아있다 #자프링외길12년차 #코프링2개월생존기
 
Presentatie - Introductie in Groovy
Presentatie - Introductie in GroovyPresentatie - Introductie in Groovy
Presentatie - Introductie in Groovy
 
Spock and Geb in Action
Spock and Geb in ActionSpock and Geb in Action
Spock and Geb in Action
 
Ruby basics
Ruby basicsRuby basics
Ruby basics
 
Fun Teaching MongoDB New Tricks
Fun Teaching MongoDB New TricksFun Teaching MongoDB New Tricks
Fun Teaching MongoDB New Tricks
 
Scala
ScalaScala
Scala
 
Groovy
GroovyGroovy
Groovy
 
BDD - Behavior Driven Development Webapps mit Groovy Spock und Geb
BDD - Behavior Driven Development Webapps mit Groovy Spock und GebBDD - Behavior Driven Development Webapps mit Groovy Spock und Geb
BDD - Behavior Driven Development Webapps mit Groovy Spock und Geb
 
Swing database(mysql)
Swing database(mysql)Swing database(mysql)
Swing database(mysql)
 
python chapter 1
python chapter 1python chapter 1
python chapter 1
 

Ähnlich wie concurrency with GPars

Groovy Introduction - JAX Germany - 2008
Groovy Introduction - JAX Germany - 2008Groovy Introduction - JAX Germany - 2008
Groovy Introduction - JAX Germany - 2008Guillaume Laforge
 
Scala - en bedre og mere effektiv Java?
Scala - en bedre og mere effektiv Java?Scala - en bedre og mere effektiv Java?
Scala - en bedre og mere effektiv Java?Jesper Kamstrup Linnet
 
The secrets of inverse brogramming
The secrets of inverse brogrammingThe secrets of inverse brogramming
The secrets of inverse brogrammingRichie Cotton
 
Golang basics for Java developers - Part 1
Golang basics for Java developers - Part 1Golang basics for Java developers - Part 1
Golang basics for Java developers - Part 1Robert Stern
 
2017 02-07 - elastic & spark. building a search geo locator
2017 02-07 - elastic & spark. building a search geo locator2017 02-07 - elastic & spark. building a search geo locator
2017 02-07 - elastic & spark. building a search geo locatorAlberto Paro
 
2017 02-07 - elastic & spark. building a search geo locator
2017 02-07 - elastic & spark. building a search geo locator2017 02-07 - elastic & spark. building a search geo locator
2017 02-07 - elastic & spark. building a search geo locatorAlberto Paro
 
java 8 Hands on Workshop
java 8 Hands on Workshopjava 8 Hands on Workshop
java 8 Hands on WorkshopJeanne Boyarsky
 
2007 09 10 Fzi Training Groovy Grails V Ws
2007 09 10 Fzi Training Groovy Grails V Ws2007 09 10 Fzi Training Groovy Grails V Ws
2007 09 10 Fzi Training Groovy Grails V Wsloffenauer
 
Introduction To Scala
Introduction To ScalaIntroduction To Scala
Introduction To ScalaPeter Maas
 
Java9 Beyond Modularity - Java 9 más allá de la modularidad
Java9 Beyond Modularity - Java 9 más allá de la modularidadJava9 Beyond Modularity - Java 9 más allá de la modularidad
Java9 Beyond Modularity - Java 9 más allá de la modularidadDavid Gómez García
 
More Stored Procedures and MUMPS for DivConq
More Stored Procedures and  MUMPS for DivConqMore Stored Procedures and  MUMPS for DivConq
More Stored Procedures and MUMPS for DivConqeTimeline, LLC
 
Scala presentation by Aleksandar Prokopec
Scala presentation by Aleksandar ProkopecScala presentation by Aleksandar Prokopec
Scala presentation by Aleksandar ProkopecLoïc Descotte
 
[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...Functional Thursday
 
awesome groovy
awesome groovyawesome groovy
awesome groovyPaul King
 

Ähnlich wie concurrency with GPars (20)

Groovy
GroovyGroovy
Groovy
 
Nantes Jug - Java 7
Nantes Jug - Java 7Nantes Jug - Java 7
Nantes Jug - Java 7
 
Groovy Introduction - JAX Germany - 2008
Groovy Introduction - JAX Germany - 2008Groovy Introduction - JAX Germany - 2008
Groovy Introduction - JAX Germany - 2008
 
Scala - en bedre og mere effektiv Java?
Scala - en bedre og mere effektiv Java?Scala - en bedre og mere effektiv Java?
Scala - en bedre og mere effektiv Java?
 
The secrets of inverse brogramming
The secrets of inverse brogrammingThe secrets of inverse brogramming
The secrets of inverse brogramming
 
Golang basics for Java developers - Part 1
Golang basics for Java developers - Part 1Golang basics for Java developers - Part 1
Golang basics for Java developers - Part 1
 
2017 02-07 - elastic & spark. building a search geo locator
2017 02-07 - elastic & spark. building a search geo locator2017 02-07 - elastic & spark. building a search geo locator
2017 02-07 - elastic & spark. building a search geo locator
 
2017 02-07 - elastic & spark. building a search geo locator
2017 02-07 - elastic & spark. building a search geo locator2017 02-07 - elastic & spark. building a search geo locator
2017 02-07 - elastic & spark. building a search geo locator
 
Summary of C++17 features
Summary of C++17 featuresSummary of C++17 features
Summary of C++17 features
 
java 8 Hands on Workshop
java 8 Hands on Workshopjava 8 Hands on Workshop
java 8 Hands on Workshop
 
2007 09 10 Fzi Training Groovy Grails V Ws
2007 09 10 Fzi Training Groovy Grails V Ws2007 09 10 Fzi Training Groovy Grails V Ws
2007 09 10 Fzi Training Groovy Grails V Ws
 
Scala - en bedre Java?
Scala - en bedre Java?Scala - en bedre Java?
Scala - en bedre Java?
 
Introduction To Scala
Introduction To ScalaIntroduction To Scala
Introduction To Scala
 
Java9 Beyond Modularity - Java 9 más allá de la modularidad
Java9 Beyond Modularity - Java 9 más allá de la modularidadJava9 Beyond Modularity - Java 9 más allá de la modularidad
Java9 Beyond Modularity - Java 9 más allá de la modularidad
 
More Stored Procedures and MUMPS for DivConq
More Stored Procedures and  MUMPS for DivConqMore Stored Procedures and  MUMPS for DivConq
More Stored Procedures and MUMPS for DivConq
 
Scala presentation by Aleksandar Prokopec
Scala presentation by Aleksandar ProkopecScala presentation by Aleksandar Prokopec
Scala presentation by Aleksandar Prokopec
 
ES6 is Nigh
ES6 is NighES6 is Nigh
ES6 is Nigh
 
[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...
 
What is new in Java 8
What is new in Java 8What is new in Java 8
What is new in Java 8
 
awesome groovy
awesome groovyawesome groovy
awesome groovy
 

Mehr von Paul King

groovy transforms
groovy transformsgroovy transforms
groovy transformsPaul King
 
functional groovy
functional groovyfunctional groovy
functional groovyPaul King
 
Make Testing Groovy
Make Testing GroovyMake Testing Groovy
Make Testing GroovyPaul King
 
Agile Testing Practices
Agile Testing PracticesAgile Testing Practices
Agile Testing PracticesPaul King
 
groovy DSLs from beginner to expert
groovy DSLs from beginner to expertgroovy DSLs from beginner to expert
groovy DSLs from beginner to expertPaul King
 
groovy and concurrency
groovy and concurrencygroovy and concurrency
groovy and concurrencyPaul King
 
Atlassian Groovy Plugins
Atlassian Groovy PluginsAtlassian Groovy Plugins
Atlassian Groovy PluginsPaul King
 
Dynamic Language Practices
Dynamic Language PracticesDynamic Language Practices
Dynamic Language PracticesPaul King
 
Make Your Builds More Groovy
Make Your Builds More GroovyMake Your Builds More Groovy
Make Your Builds More GroovyPaul King
 
Groovy Power Features
Groovy Power FeaturesGroovy Power Features
Groovy Power FeaturesPaul King
 
Make Your Testing Groovy
Make Your Testing GroovyMake Your Testing Groovy
Make Your Testing GroovyPaul King
 
Groovy Testing Sep2009
Groovy Testing Sep2009Groovy Testing Sep2009
Groovy Testing Sep2009Paul King
 
Craig Smith & Paul King Agile Tool Hacking Taking Your Agile Development ...
Craig Smith & Paul King   Agile Tool Hacking   Taking Your Agile Development ...Craig Smith & Paul King   Agile Tool Hacking   Taking Your Agile Development ...
Craig Smith & Paul King Agile Tool Hacking Taking Your Agile Development ...Paul King
 
Groovy Tutorial
Groovy TutorialGroovy Tutorial
Groovy TutorialPaul King
 
XML and Web Services with Groovy
XML and Web Services with GroovyXML and Web Services with Groovy
XML and Web Services with GroovyPaul King
 

Mehr von Paul King (16)

groovy transforms
groovy transformsgroovy transforms
groovy transforms
 
functional groovy
functional groovyfunctional groovy
functional groovy
 
Make Testing Groovy
Make Testing GroovyMake Testing Groovy
Make Testing Groovy
 
Agile Testing Practices
Agile Testing PracticesAgile Testing Practices
Agile Testing Practices
 
groovy DSLs from beginner to expert
groovy DSLs from beginner to expertgroovy DSLs from beginner to expert
groovy DSLs from beginner to expert
 
groovy and concurrency
groovy and concurrencygroovy and concurrency
groovy and concurrency
 
GroovyDSLs
GroovyDSLsGroovyDSLs
GroovyDSLs
 
Atlassian Groovy Plugins
Atlassian Groovy PluginsAtlassian Groovy Plugins
Atlassian Groovy Plugins
 
Dynamic Language Practices
Dynamic Language PracticesDynamic Language Practices
Dynamic Language Practices
 
Make Your Builds More Groovy
Make Your Builds More GroovyMake Your Builds More Groovy
Make Your Builds More Groovy
 
Groovy Power Features
Groovy Power FeaturesGroovy Power Features
Groovy Power Features
 
Make Your Testing Groovy
Make Your Testing GroovyMake Your Testing Groovy
Make Your Testing Groovy
 
Groovy Testing Sep2009
Groovy Testing Sep2009Groovy Testing Sep2009
Groovy Testing Sep2009
 
Craig Smith & Paul King Agile Tool Hacking Taking Your Agile Development ...
Craig Smith & Paul King   Agile Tool Hacking   Taking Your Agile Development ...Craig Smith & Paul King   Agile Tool Hacking   Taking Your Agile Development ...
Craig Smith & Paul King Agile Tool Hacking Taking Your Agile Development ...
 
Groovy Tutorial
Groovy TutorialGroovy Tutorial
Groovy Tutorial
 
XML and Web Services with Groovy
XML and Web Services with GroovyXML and Web Services with Groovy
XML and Web Services with Groovy
 

Kürzlich hochgeladen

AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAndrey Devyatkin
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyKhushali Kathiriya
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsNanddeep Nachan
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...apidays
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDropbox
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdflior mazor
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businesspanagenda
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century educationjfdjdjcjdnsjd
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoffsammart93
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxRustici Software
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodJuan lago vázquez
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWERMadyBayot
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobeapidays
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?Igalia
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 

Kürzlich hochgeladen (20)

AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 

concurrency with GPars

  • 1. Chicago, October 19 - 22, 2010 Dr Paul King, @paulk_asert paulk at asert.com.au Concurrency with GPars
  • 2. "Andy giveth and Bill taketh away" GPars - 2 Source:HerbSutter:http://www.gotw.ca/publications/concurrency-ddj.htm
  • 3. Why is it hard? • Many issues to deal with: – Doing things in parallel, concurrently, asynchronously • Processes, Threads, Co-routines, Events, Scheduling – Sharing/Synchronization Mechanisms • shared memory, locks, transactions, wait/notify, STM, message passing, actors, serializability, persistence, immutability – Abstractions • Shared memory on top of messaging passing • Message passing on top of shared memory • Dataflow, Selective Communication, Continuations – Data Structures and Algorithms • Queues, Heaps, Trees • Sorting, Graph Algorithms GPars - 3
  • 4. GPars - 4 Java Concurrency Features • The early years – Threads, synchronised and non-synchronised collections, synchronisation at the language level, Monitors (wait/notify), Locks, ThreadLocal, final, ... • More recent enhancements – java.util.concurrent: Executors, Thread Pools, Optimistic updates, Blocking queues, Synchronizers, Callables, Futures, Atomic operations, Deques, ... • Emerging – Fork/Join & others, Kilim, Phasers, PicoThreads ... • Leverage related APIs/technologies – Networking, real-time, GUIs, simulation, database, multimedia, operating systems, parallel processing, distribution, mobile agents, nio, ...
  • 5. Java Concurrency Best Practice? • Java Concurrency in Practice: –“If mutable threads access the same mutable state variable without appropriate synchronization, your program is broken” –“When designing thread-safe classes, good object-oriented techniques – encapsulation, immutability, and clear specification of invariants – are your best friends” GPars - 5
  • 6. Topics Groovy Intro • Useful Groovy features for Concurrency • Related Concurrency Libraries & Tools • GPars • More Info GPars - 6 ©ASERT2006-2010
  • 7. What is Groovy? GPars - 7 ©ASERT2006-2010 • “Groovy is like a super version of Java. It can leverage Java's enterprise capabilities but also has cool productivity features like closures, DSL support, builders and dynamic typing.” Groovy = Java – boiler plate code + optional dynamic typing + closures + domain specific languages + builders + metaprogramming
  • 8. Groovy Starter GPars - 8 ©ASERT2006-2010 System.out.println("Hello, World!"); // supports Java syntax println 'Hello, World!' // but can remove some syntax String name = 'Guillaume' // Explicit typing/awareness println "$name, I'll get the car." // Gstring (interpolation) def longer = """${name}, the car is in the next row.""" // multi-line, implicit type assert 0.5 == 1/2 // BigDecimal equals() assert 0.1 + 0.2 == 0.3 // and arithmetic def printSize(obj) { // implicit/duck typing print obj?.size() // safe dereferencing } def pets = ['ant', 'bee', 'cat'] // native list syntax pets.each { pet -> // closure support assert pet < 'dog' // overloading '<' on String } // or: for (pet in pets)...
  • 9. A Better Java... GPars - 9 ©ASERT2006-2010 import java.util.List; import java.util.ArrayList; class Erase { private List removeLongerThan(List strings, int length) { List result = new ArrayList(); for (int i = 0; i < strings.size(); i++) { String s = (String) strings.get(i); if (s.length() <= length) { result.add(s); } } return result; } public static void main(String[] args) { List names = new ArrayList(); names.add("Ted"); names.add("Fred"); names.add("Jed"); names.add("Ned"); System.out.println(names); Erase e = new Erase(); List shortNames = e.removeLongerThan(names, 3); System.out.println(shortNames.size()); for (int i = 0; i < shortNames.size(); i++) { String s = (String) shortNames.get(i); System.out.println(s); } } } This code is valid Java and valid Groovy Based on an example by Jim Weirich & Ted Leung
  • 10. ...A Better Java... GPars - 10 ©ASERT2006-2010 import java.util.List; import java.util.ArrayList; class Erase { private List removeLongerThan(List strings, int length) { List result = new ArrayList(); for (int i = 0; i < strings.size(); i++) { String s = (String) strings.get(i); if (s.length() <= length) { result.add(s); } } return result; } public static void main(String[] args) { List names = new ArrayList(); names.add("Ted"); names.add("Fred"); names.add("Jed"); names.add("Ned"); System.out.println(names); Erase e = new Erase(); List shortNames = e.removeLongerThan(names, 3); System.out.println(shortNames.size()); for (int i = 0; i < shortNames.size(); i++) { String s = (String) shortNames.get(i); System.out.println(s); } } } Do the semicolons add anything? And shouldn‟t we us more modern list notation? Why not import common libraries?
  • 11. ...A Better Java... GPars - 11 ©ASERT2006-2010 class Erase { private List removeLongerThan(List strings, int length) { List result = new ArrayList() for (String s in strings) { if (s.length() <= length) { result.add(s) } } return result } public static void main(String[] args) { List names = new ArrayList() names.add("Ted"); names.add("Fred") names.add("Jed"); names.add("Ned") System.out.println(names) Erase e = new Erase() List shortNames = e.removeLongerThan(names, 3) System.out.println(shortNames.size()) for (String s in shortNames) { System.out.println(s) } } }
  • 12. ...A Better Java... GPars - 12 ©ASERT2006-2010 class Erase { private List removeLongerThan(List strings, int length) { List result = new ArrayList() for (String s in strings) { if (s.length() <= length) { result.add(s) } } return result } public static void main(String[] args) { List names = new ArrayList() names.add("Ted"); names.add("Fred") names.add("Jed"); names.add("Ned") System.out.println(names) Erase e = new Erase() List shortNames = e.removeLongerThan(names, 3) System.out.println(shortNames.size()) for (String s in shortNames) { System.out.println(s) } } } Do we need the static types? Must we always have a main method and class definition? How about improved consistency?
  • 13. ...A Better Java... GPars - 13 ©ASERT2006-2010 def removeLongerThan(strings, length) { def result = new ArrayList() for (s in strings) { if (s.size() <= length) { result.add(s) } } return result } names = new ArrayList() names.add("Ted") names.add("Fred") names.add("Jed") names.add("Ned") System.out.println(names) shortNames = removeLongerThan(names, 3) System.out.println(shortNames.size()) for (s in shortNames) { System.out.println(s) }
  • 14. ...A Better Java... GPars - 14 ©ASERT2006-2010 def removeLongerThan(strings, length) { def result = new ArrayList() for (s in strings) { if (s.size() <= length) { result.add(s) } } return result } names = new ArrayList() names.add("Ted") names.add("Fred") names.add("Jed") names.add("Ned") System.out.println(names) shortNames = removeLongerThan(names, 3) System.out.println(shortNames.size()) for (s in shortNames) { System.out.println(s) } Shouldn‟t we have special notation for lists? And special facilities for list processing? Is „return‟ needed at end?
  • 15. ...A Better Java... GPars - 15 ©ASERT2006-2010 def removeLongerThan(strings, length) { strings.findAll{ it.size() <= length } } names = ["Ted", "Fred", "Jed", "Ned"] System.out.println(names) shortNames = removeLongerThan(names, 3) System.out.println(shortNames.size()) shortNames.each{ System.out.println(s) }
  • 16. ...A Better Java... GPars - 16 ©ASERT2006-2010 def removeLongerThan(strings, length) { strings.findAll{ it.size() <= length } } names = ["Ted", "Fred", "Jed", "Ned"] System.out.println(names) shortNames = removeLongerThan(names, 3) System.out.println(shortNames.size()) shortNames.each{ System.out.println(s) } Is the method now needed? Easier ways to use common methods? Are brackets required here?
  • 17. ...A Better Java GPars - 17 ©ASERT2006-2010 names = ["Ted", "Fred", "Jed", "Ned"] println names shortNames = names.findAll{ it.size() <= 3 } println shortNames.size() shortNames.each{ println it } ["Ted", "Fred", "Jed", "Ned"] 3 Ted Jed Ned Output:
  • 18. Grapes / Grab: Google collections GPars - 18 ©ASERT2006-2010 @Grab('com.google.collections:google-collections:1.0') import com.google.common.collect.HashBiMap HashBiMap fruit = [grape:'purple', lemon:'yellow', lime:'green'] assert fruit.lemon == 'yellow' assert fruit.inverse().yellow == 'lemon'
  • 19. GPars - 19 ©ASERT2006-2010 Better Design Patterns: Delegate… import java.util.Date; public class Event { private String title; private String url; private Date when; public String getUrl() { return url; } public void setUrl(String url) { this.url = url; } public String getTitle() { return title; } public void setTitle(String title) { this.title = title; } // ... public Date getWhen() { return when; } public void setWhen(Date when) { this.when = when; } public boolean before(Date other) { return when.before(other); } public void setTime(long time) { when.setTime(time); } public long getTime() { return when.getTime(); } public boolean after(Date other) { return when.after(other); } // ...
  • 20. GPars - 20 ©ASERT2006-2010 …Better Design Patterns: Delegate… import java.util.Date; public class Event { private String title; private String url; private Date when; public String getUrl() { return url; } public void setUrl(String url) { this.url = url; } public String getTitle() { return title; } public void setTitle(String title) { this.title = title; } // ... public Date getWhen() { return when; } public void setWhen(Date when) { this.when = when; } public boolean before(Date other) { return when.before(other); } public void setTime(long time) { when.setTime(time); } public long getTime() { return when.getTime(); } public boolean after(Date other) { return when.after(other); } // ... boilerplate
  • 21. GPars - 21 ©ASERT2006-2010 …Better Design Patterns: Delegate class Event { String title, url @Delegate Date when } def gr8conf = new Event(title: "GR8 Conference", url: "http://www.gr8conf.org", when: Date.parse("yyyy/MM/dd", "2009/05/18")) def javaOne = new Event(title: "JavaOne", url: "http://java.sun.com/javaone/", when: Date.parse("yyyy/MM/dd", "2009/06/02")) assert gr8conf.before(javaOne.when)
  • 22. Why Groovy? Technical Answer • Minimal learning curve • Compiles to bytecode • Java object model & integration • Annotations • "Optional" static typing • Both run-time and compile-time metaprogramming GPars - 22
  • 23. Why Groovy? Adoption Assessment • Innovators/Thought leaders – Ideas, power, flexibility, novelty, thinking community • Early adopters – Productivity benefits and collegiate community – Leverage JVM and potential for mainstream • Mainstream – Leverage existing Java skills, low learning curve – Leverage JVM and production infrastructure – Professional community – Tools, tools, tools
  • 24. Topics • Groovy Intro Useful Groovy features for Concurrency • Related Concurrency Libraries & Tools • GPars • More Info GPars - 24 ©ASERT2006-2010
  • 25. Concurrent Programming in Groovy • Java concurrent programming enhancements – Normal OO methods – Ability to have immutable types – Some concurrency building blocks – Annotations with baked in goodness • Process/Thread ease of use – AntBuilder and GDK methods • Closures for greater flexibility – Enabler for concurrency – Closure is Runnable and Callable • Third-party libraries – GPars, Functional Java (Actors), Multiverse, JCSP – Cascading.groovy subproject for Hadoop clusters – Jetlang, JPPF, GridGain, Google Collections, Gruple – Groovy actors: http://www.groovyactors.org GPars - 25
  • 26. Thread & Process Enhancements • DGM methods – Thread.start{ … } • Runtime metaprogramming – Add custom control structures e.g. ReentrantLock.withLock{ … } • Process enhancements – "ls –l".execute() – proc1 | proc2 | proc3 – proc1.consumeProcessErrorStream() – proc2.waitForOrKill(1000) • AntBuilder – ant.parallel { … } – ant.exec() GPars - 26
  • 27. Immutability options • Built-in • Google Collections – Numerous improved immutable collection types • Groovy run-time metaprogramming • Groovy compile-time metaprogramming – @Immutable can help us create such classes – Also gives us @Synchronized and @Lazy GPars - 27 import com.google.common.collect.* List<String> animals = ImmutableList.of("cat", "dog", "horse") animals << 'fish' // => java.lang.UnsupportedOperationException def animals = ['cat', 'dog', 'horse'].asImmutable() animals << 'fish' // => java.lang.UnsupportedOperationException def animals = ['cat', 'dog', 'horse'] ArrayList.metaClass.leftShift = { throw new UnsupportedOperationException() } animals << 'fish' // => java.lang.UnsupportedOperationException
  • 28. @Immutable... • Java Immutable Class – As per Joshua Bloch Effective Java GPars - 28 ©ASERT2006-2010 public final class Punter { private final String first; private final String last; public String getFirst() { return first; } public String getLast() { return last; } @Override public int hashCode() { final int prime = 31; int result = 1; result = prime * result + ((first == null) ? 0 : first.hashCode()); result = prime * result + ((last == null) ? 0 : last.hashCode()); return result; } public Punter(String first, String last) { this.first = first; this.last = last; } // ... // ... @Override public boolean equals(Object obj) { if (this == obj) return true; if (obj == null) return false; if (getClass() != obj.getClass()) return false; Punter other = (Punter) obj; if (first == null) { if (other.first != null) return false; } else if (!first.equals(other.first)) return false; if (last == null) { if (other.last != null) return false; } else if (!last.equals(other.last)) return false; return true; } @Override public String toString() { return "Punter(first:" + first + ", last:" + last + ")"; } }
  • 29. ...@Immutable... • Java Immutable Class – As per Joshua Bloch Effective Java GPars - 29 ©ASERT2006-2010 public final class Punter { private final String first; private final String last; public String getFirst() { return first; } public String getLast() { return last; } @Override public int hashCode() { final int prime = 31; int result = 1; result = prime * result + ((first == null) ? 0 : first.hashCode()); result = prime * result + ((last == null) ? 0 : last.hashCode()); return result; } public Punter(String first, String last) { this.first = first; this.last = last; } // ... // ... @Override public boolean equals(Object obj) { if (this == obj) return true; if (obj == null) return false; if (getClass() != obj.getClass()) return false; Punter other = (Punter) obj; if (first == null) { if (other.first != null) return false; } else if (!first.equals(other.first)) return false; if (last == null) { if (other.last != null) return false; } else if (!last.equals(other.last)) return false; return true; } @Override public String toString() { return "Punter(first:" + first + ", last:" + last + ")"; } } boilerplate
  • 30. ...@Immutable GPars - 30 ©ASERT2006-2010 @Immutable class Punter { String first, last }
  • 31. Topics • Groovy Intro • Useful Groovy features for Concurrency Related Concurrency Libraries & Tools • GPars • More Info GPars - 31 ©ASERT2006-2010
  • 32. Lightweight threads: Jetlang • Jetlang – A high performance threading library – http://code.google.com/p/jetlang/ GPars - 32 import org.jetlang.fibers.ThreadFiber import org.jetlang.core.Callback import org.jetlang.channels.MemoryRequestChannel import org.jetlang.channels.AsyncRequest def req = new ThreadFiber() // or pool def reply = new ThreadFiber() def channel = new MemoryRequestChannel() req.start() reply.start() channel.subscribe(reply, { it.reply(it.request.sum()) } as Callback) AsyncRequest.withOneReply(req, channel, [3, 4, 5], { println it } as Callback) sleep 1000 req.dispose() reply.dispose()
  • 33. Other High-Level Libraries: JPPF – Open source Grid Computing platform – http://www.jppf.org/ GPars - 33 import org.jppf.client.* import java.util.concurrent.Callable class Task implements Callable, Serializable { private static final long serialVersionUID = 1162L public Object call() { println 'Executing Groovy' "Hello JPPF from Groovy" } } def client = new JPPFClient() def job = new JPPFJob() def task = new Task() job.addTask task def results = client.submit(job) for (t in results) { if (t.exception) throw t.exception println "Result: " + t.result }
  • 34. Other High-Level Libraries: Gruple... – http://code.google.com/p/gruple – Simple abstraction to coordinate and synchronize threads with ease – based on Tuplespaces • Tuplespaces provide the illusion of a shared memory on top of a message passing system, along with a small set of operations to greatly simplify parallel programming – Example Tuple: [fname:"Vanessa", lname:"Williams", project:"Gruple"] – Basic operations within a Tuplespace are: • put - insert a tuple into the space • get - read a tuple from the space (non-destructively) • take - take a tuple from the space (a destructive read) – Further reading: Eric Freeman, Susanne Hupfer, and Ken Arnold. JavaSpaces Principles, Patterns, and Practice, Addison Wesley, 1999 GPars - 34
  • 35. …Other High-Level Libraries: Gruple... GPars - 35 import org.gruple.SpaceService def defaultSpace = SpaceService.getSpace() defaultSpace << [fname:"Vanessa", lname:"Williams", project:"Gruple"] println defaultSpace.get(fname:"Vanessa", lname:"Williams", project:"Gruple") [project:Gruple, lname:Williams, fname:Vanessa]
  • 36. Other High-Level Libraries: ...Gruple... – Mandelbrot example (included in Gruple download) GPars - 36 ... Space space = SpaceService.getSpace("mandelbrot") Map template = createTaskTemplate() Map task String threadName = Thread.currentThread().name while(true) { ArrayList points task = space.take(template) println "Worker $threadName got task ${task['start']} for job ${task['jobId'] points = calculateMandelbrot(task) Map result = createResult(task['jobId'], task['start'], points) println "Worker $threadName writing result for task ${result['start']} for jo space.put(result) } ...
  • 37. Other High-Level Libraries: ...Gruple GPars - 37
  • 38. Other High-Level Libraries: Cascading.groovy – API/DSL for executing tasks on a Hadoop cluster – http://www.cascading.org/ GPars - 38 def assembly = builder.assembly(name: "wordcount") { eachTuple(args: ["line"], results: ["word"]) { regexSplitGenerator(declared: ["word"], pattern: /[.,]*s+/) } group(["word"]) everyGroup(args: ["word"], results: ["word", "count"]) { count() } group(["count"], reverse: true) } def map = builder.map() { source(name: "wordcount") { hfs(input) { text(["line"]) } } sink(name: "wordcount") { hfs(output) { text() } } } def flow = builder.flow(name: "wordcount", map: map, assembly: assembly)
  • 39. Other High-Level Libraries: GridGain… – Simple & productive to use grid computing platform – http://www.gridgain.com/ GPars - 39 class GridHelloWorldGroovyTask extends GridTaskSplitAdapter<String, Integer> { Collection split(int gridSize, Object phrase) throws GridException { // ... } Object reduce(List results) throws GridException { // ... } } import static GridFactory.* start() def grid = getGrid() def future = grid.execute(GridHelloWorldGroovyTask, "Hello World") def phraseLen = future.get() stop(true)
  • 40. …Other High-Level Libraries: GridGain • http://gridgain.blogspot.com/2010/10/worlds-shortest-mapreduce- app.html GPars - 40 words = "Counting Letters In This Phrase".split(' ') map = new C1() { def apply(word) { word.size() } } reduce = sumIntReducer() println grid.forkjoin(SPREAD, yield(words, map), reduce) // => 27 grid.forkjoin(SPREAD,yield("Counting Letters In This Phrase".split(' '), new C1(){def apply(w){w.size()}}),sumReducer())
  • 41. Multiverse STM… – http://www.multiverse.org/ GPars - 41 import org.multiverse.api.GlobalStmInstance import org.multiverse.api.Transaction import org.multiverse.templates.TransactionTemplate import org.multiverse.transactional.refs.LongRef def from = new Account(10) def to = new Account(10) atomic { from.balance -= 5 to.balance += 5 } println "from $from.balance" println "to $to.balance" ...
  • 42. …Multiverse STM… GPars - 42 ... void atomic(Closure block) { atomic([:], block) } void atomic(Map args, Closure block) { boolean readonly = args['readonly'] ?: false boolean trackreads = args['trackreads'] ?: true def txFactory = GlobalStmInstance.globalStmInstance. transactionFactoryBuilder. setReadonly(readonly). setReadTrackingEnabled(trackreads).build() new TransactionTemplate(txFactory) { Object execute(Transaction transaction) { block.call() return null } }.execute() } ...
  • 43. …Multiverse STM GPars - 43 class Account { private final balance = new LongRef() Account(long initial) { balance.set initial } void setBalance(long newBalance) { if (newBalance < 0) throw new RuntimeException("not enough money") balance.set newBalance } long getBalance() { balance.get() } }
  • 44. Testing multi-threaded applications: ConTest... • Advanced Testing for Multi-Threaded Applications – Tool for testing, debugging, and coverage-measuring of concurrent programs (collects runtime statistics) – Systematically and transparently (using a java agent) schedules the execution of program threads in ways likely to reveal race conditions, deadlocks, and other intermittent bugs (collectively called synchronization problems) with higher than normal frequency – The ConTest run-time engine adds heuristically controlled conditional instructions (adjustable by a preferences file) that force thread switches, thus helping to reveal concurrent bugs. You can use existing tests and run ConTest multiple times – by default different heuristics used each time it is run • http://www.alphaworks.ibm.com/tech/contest GPars - 44
  • 45. ...Testing multi-threaded applications: ConTest GPars - 45 NUM = 5 count = 0 def incThread = { n -> Thread.start{ sleep n*10 //synchronized(ParalInc) { count++ //} } } def threads = (1..NUM).collect(incThread) threads.each{ it.join() } assert count == NUM targetClasses = ParalInc timeoutTampering = true noiseFrequency = 500 strength = 10000 Exception in thread "main" Assertion failed: assert count == NUM | | | 4 | 5 false > groovyc ParalInc.groovy > java -javaagent:../../Lib/ConTest.jar -cp %GROOVY_JAR%;. ParalInc ParalInc.groovy
  • 46. GPars - 46 GContracts @Grab('org.gcontracts:gcontracts:1.0.2') import org.gcontracts.annotations.* @Invariant({ first != null && last != null }) class Person { String first, last @Requires({ delimiter in ['.', ',', ' '] }) @Ensures({ result == first + delimiter + last }) String getName(String delimiter) { first + delimiter + last } } new Person(first: 'John', last: 'Smith').getName('.') 1.8+
  • 47. Testing: Spock GPars - 47 class HelloSpock extends spock.lang.Specification { def "length of Spock's and his friends' names"() { expect: name.size() == length where: name | length "Spock" | 5 "Kirk" | 4 "Scotty" | 6 } }
  • 48. Topics • Groovy Intro • Useful Groovy features for Concurrency • Related Concurrency Libraries & Tools GPars • More Info GPars - 48 ©ASERT2006-2010
  • 49. Ralph Johnson: Parallel Programming Patterns… GPars - 49 ©ASERT2006-2010 http://strangeloop2010.com/talk/presentation_file/14485/Johnson-DataParallelism.pdf
  • 50. …Ralph Johnson: Parallel Programming Patterns GPars - 50 ©ASERT2006-2010 http://strangeloop2010.com/talk/presentation_file/14485/Johnson-DataParallelism.pdf
  • 51. GPars • http://gpars.codehaus.org/ • Library classes and DSL sugar providing intuitive ways for Groovy developers to handle tasks concurrently. Logical parts: – Actors provide a Groovy implementation of Scala-like actors including "remote" actors on other machines – Dataflow Concurrency supports natural shared-memory concurrency model, using single-assignment variables – Asynchronizer extends the Java 1.5 built-in support for executor services to enable multi-threaded collection and closure processing – Parallelizer uses JSR-166y Parallel Arrays to enable multi-threaded collection processing – Safe a non-blocking mt-safe reference to mutable state that is inspired by "agents" in Clojure GPars - 51 ©ASERT2006-2010
  • 52. GPars: Parallel Collection Functions GPars - 52 ©ASERT2006-2010 def nums = 1..100000 def squares = nums .collect{ it ** 2 } .grep{ it % 7 == it % 5 } .grep{ it % 3 == 0 } println squares[0..3] + "..." + squares[-3..-1] assert squares[0..3] == [36, 144, 1089, 1296] @Grab('org.codehaus.gpars:gpars:0.10') import static groovyx.gpars.GParsPool.withPool def nums = 1..100000 withPool(5) { def squares = nums. collectParallel{ it ** 2 }. grepParallel{ it % 7 == it % 5 }. grepParallel{ it % 3 == 0 } println squares[0..3] + "..." + squares[-3..-1] assert squares[0..3] == [36, 144, 1089, 1296] }
  • 53. GPars: Transparent Parallel Collections • Applies some Groovy metaprogramming GPars - 53 ©ASERT2006-2010 import static groovyx.gpars.GParsPool.withPool withPool(5) { def nums = 1..100000 nums.makeTransparent() def squares = nums. collect{ it ** 2 }. grep{ it % 7 == it % 5 }. grep{ it % 3 == 0 } println squares[0..3] + "..." + squares[-3..-1] assert squares[0..3] == [36, 144, 1089, 1296] }
  • 54. GPars concurrency-aware methods Transparent Transitive? Parallel Lazy? any { ... } anyParallel { ... } yes collect { ... } yes collectParallel { ... } count(filter) countParallel(filter) each { ... } eachParallel { ... } eachWithIndex { ... } eachWithIndexParallel { ... } every { ... } everyParallel { ... } yes find { ... } findParallel { ... } findAll { ... } yes findAllParallel { ... } findAny { ... } findAnyParallel { ... } fold { ... } foldParallel { ... } fold(seed) { ... } foldParallel(seed) { ... } grep(filter) yes grepParallel(filter) groupBy { ... } groupByParallel { ... } max { ... } maxParallel { ... } max() maxParallel() min { ... } minParallel { ... } min() minParallel() split { ... } yes splitParallel { ... } sum sumParallel // foldParallel + GPars - 54Transitive means result is automatically transparent; Lazy means fails fast
  • 55. GPars: Map-Reduce... GPars - 55 ©ASERT2006-2010 import static groovyx.gpars.GParsPool.withPool withPool(5) { def nums = 1..100000 println nums.parallel. map{ it ** 2 }. filter{ it % 7 == it % 5 }. filter{ it % 3 == 0 }. collection }
  • 56. ...GPars: Map-Reduce GPars - 56 ©ASERT2006-2010 import static groovyx.gpars.GParsPool.withPool withPool(5) { def nums = 1..100000 println nums.parallel. map{ it ** 2 }. filter{ it % 7 == it % 5 }. filter{ it % 3 == 0 }. reduce{ a, b -> a + b } }
  • 57. GPars parallel array methods Method Return Type combine(initValue) { ... } Map filter { ... } Parallel array collection Collection groupBy { ... } Map map { ... } Parallel array max() T max { ... } T min() T min { ... } T reduce { ... } T reduce(seed) { ... } T size() int sort { ... } Parallel array sum() T parallel // on a Collection Parallel array GPars - 57
  • 58. Parallel Collections vs Map-Reduce GPars - 58 Fork Fork JoinJoin Map Map Reduce Map Map Reduce Reduce Map Filter FilterMap
  • 59. GPars: Dataflows... GPars - 59 ©ASERT2006-2010 import groovyx.gpars.dataflow.DataFlows import static groovyx.gpars.dataflow.DataFlow.task final flow = new DataFlows() task { flow.result = flow.x + flow.y } task { flow.x = 10 } task { flow.y = 5 } assert 15 == flow.result new DataFlows().with { task { result = x * y } task { x = 10 } task { y = 5 } assert 50 == result }
  • 60. ...GPars: Dataflows... • Evaluating: GPars - 60 ©ASERT2006-2010 import groovyx.gpars.dataflow.DataFlows import static groovyx.gpars.dataflow.DataFlow.task final flow = new DataFlows() task { flow.a = 10 } task { flow.b = 5 } task { flow.x = flow.a - flow.b } task { flow.y = flow.a + flow.b } task { flow.result = flow.x * flow.y } assert flow.result == 75 b 10 5 a +- * result = (a – b) * (a + b) x y Question: what happens if I change the order of the task statements here?
  • 61. ...GPars: Dataflows... • Naive attempt for loops GPars - 61 ©ASERT2006-2010 import groovyx.gpars.dataflow.DataFlows import static groovyx.gpars.dataflow.DataFlow.task final flow = new DataFlows() [10, 20].each { thisA -> [4, 5].each { thisB -> task { flow.a = thisA } task { flow.b = thisB } task { flow.x = flow.a - flow.b } task { flow.y = flow.a + flow.b } task { flow.result = flow.x * flow.y } println flow.result } } // => java.lang.IllegalStateException: A DataFlowVariable can only be assigned once. ... task { flow.a = 10 } ... task { flow.a = 20 }
  • 62. ...GPars: Dataflows... GPars - 62 ©ASERT2006-2010 import groovyx.gpars.dataflow.DataFlowStream import static groovyx.gpars.dataflow.DataFlow.* final streamA = new DataFlowStream() final streamB = new DataFlowStream() final streamX = new DataFlowStream() final streamY = new DataFlowStream() final results = new DataFlowStream() operator(inputs: [streamA, streamB], outputs: [streamX, streamY]) { a, b -> streamX << a - b; streamY << a + b } operator(inputs: [streamX, streamY], outputs: [results]) { x, y -> results << x * y } [[10, 20], [4, 5]].combinations().each{ thisA, thisB -> task { streamA << thisA } task { streamB << thisB } } 4.times { println results.val } b 10 10 20 20 4 5 4 5 a +- * 84 75 384 375
  • 63. ...GPars: Dataflows • Amenable to static analysis • Race conditions avoided • Deadlocks “typically” become repeatable GPars - 63 ©ASERT2006-2010 import groovyx.gpars.dataflow.DataFlows import static groovyx.gpars.dataflow.DataFlow.task final flow = new DataFlows() task { flow.x = flow.y } task { flow.y = flow.x }
  • 64. GPars: Dataflow Sieve GPars - 64 ©ASERT2006-2010 final int requestedPrimeNumberCount = 1000 final DataFlowStream initialChannel = new DataFlowStream() task { (2..10000).each { initialChannel << it } } def filter(inChannel, int prime) { def outChannel = new DataFlowStream() operator([inputs: [inChannel], outputs: [outChannel]]) { if (it % prime != 0) { bindOutput it } } return outChannel } def currentOutput = initialChannel requestedPrimeNumberCount.times { int prime = currentOutput.val println "Found: $prime" currentOutput = filter(currentOutput, prime) } Source: http://groovyconsole.appspot.com/script/235002
  • 65. GPars: Actors... • Actors provide explicit coordination: they don‟t share state, instead coordinating via asynchronous messages – Contrasting with predefined coordination for fork/join & map/filter/reduce & implicit coordination for dataflow – Messages are processed one at a time normally in the order they were sent (which is non-deterministic due to asynchronous nature) – Some actor systems allowing message delivery to be prioritised; others allow for sharing some (readonly) state; some allow remote actors for load balancing/robustness • Not new in concept – But has received recent publicity due to special support in Erlang, Scala and other languages GPars - 65 ©ASERT2006-2010
  • 66. …GPars: Actors... • Class with the following lifecycle & methods – But also DSL sugar & enhancements GPars - 66 ©ASERT2006-2010 start() stop() act() send(msg) sendAndWait(msg) loop { } react { msg -> } msg.reply(replyMsg) receive() join()
  • 67. …GPars: Actors... GPars - 67 ©ASERT2006-2010 import static groovyx.gpars.actor.Actors.* def decrypt = reactor { code -> code.reverse() } def audit = reactor { println it } def main = actor { decrypt 'terces pot' react { plainText -> audit plainText } } main.join() audit.stop() audit.join() Source: ReGina
  • 68. …GPars: Actors... GPars - 68 ©ASERT2006-2010 final class FilterActor extends DynamicDispatchActor { private final int myPrime private def follower def FilterActor(final myPrime) { this.myPrime = myPrime; } def onMessage(int value) { if (value % myPrime != 0) { if (follower) follower value else { println "Found $value" follower = new FilterActor(value).start() } } } def onMessage(def poisson) { if (follower) { def sender = poisson.sender follower.sendAndContinue(poisson, {this.stop(); sender?.send('Done } else { //I am the last in the chain stop() reply 'Done' } } } Source: http://groovyconsole.appspot.com/script/242001
  • 69. …GPars: Actors GPars - 69 ©ASERT2006-2010 (2..requestedPrimeNumberBoundary).each { firstFilter it } firstFilter.sendAndWait 'Poisson' Source: http://groovyconsole.appspot.com/script/242001
  • 70. Dining Philosophers… GPars - 70 Philosopher Thinking | Eating Philosopher Thinking | Eating Philosopher Thinking | Eating Philosopher Thinking | Eating Philosopher Thinking | Eating Fork Available | InUse Fork Available | InUse Fork Available | InUse Fork Available | InUse Fork Available | InUse
  • 71. …Dining Philosophers GPars - 71 Philosopher Thinking | Eating Philosopher Thinking | Eating Philosopher Thinking | Eating Philosopher Thinking | Eating Philosopher Thinking | Eating Fork Available | InUse Fork Available | InUse Fork Available | InUse Fork Available | InUse Fork Available | InUse Accepted | Rejected Take | Release Take | Release
  • 72. Dining Philosophers: Actors... GPars - 72 ©ASERT2006-2010 // adapted from GPars example, repo: http://git.codehaus.org/gitweb.cgi?p=gpars.git // file: src/test/groovy/groovyx/gpars/samples/actors/DemoDiningPhilosophers.groovy @Grab('org.codehaus.gpars:gpars:0.10') import groovyx.gpars.actor.* import groovy.beans.Bindable def names = ['socrates', 'plato', 'aristotle', 'descartes', 'nietzsche'] Actors.defaultActorPGroup.resize names.size() class Philosopher extends AbstractPooledActor { private random = new Random() String name int timesEaten = 0 def forks @Bindable String status void act() { assert 2 == forks.size() loop { think() forks*.send new Take() react {a -> react {b -> if ([a, b].any {Rejected.isCase it}) { [a, b].find {Accepted.isCase it}?.reply new Release() } else { eat() [a, b]*.reply new Release() } } } } }
  • 73. …Dining Philosophers: Actors... GPars - 73 ©ASERT2006-2010 … void think() { setStatus('thinking') sleep random.nextInt(5000) setStatus('') } void eat() { setStatus("eating ${++timesEaten}") sleep random.nextInt(3000) setStatus('') } String toString() { switch (timesEaten) { case 0: return "$name has starved" case 1: return "$name has eaten once" default: return "$name has eaten $timesEaten times" } } } final class Take {} final class Accepted {} final class Rejected {} final class Release {}
  • 74. …Dining Philosophers: Actors... GPars - 74 ©ASERT2006-2010 … class Fork extends AbstractPooledActor { String name boolean available = true void act() { loop { react {message -> switch (message) { case Take: if (available) { available = false reply new Accepted() } else reply new Rejected() break case Release: assert !available available = true break default: throw new IllegalStateException("Cannot process the message: $message") } } } } } def forks = (1..names.size()).collect { new Fork(name: "Fork $it") } def philosophers = (1..names.size()).collect { new Philosopher(name: names[it - 1], forks: [forks[it - 1], forks[it % names.size()]]) }
  • 75. …Dining Philosophers: Actors GPars - 75 ©ASERT2006-2010 … import groovy.swing.* import java.awt.Font import static javax.swing.JFrame.* def frame = new SwingBuilder().frame(title: 'Philosophers', defaultCloseOperation: EXIT_ON_CLOSE) { vbox { hbox { (0..<names.size()).each { i -> def widget = textField(id: names[i], text: names[i].center(14)) widget.font = new Font(widget.font.name, widget.font.style, 36) philosophers[i].propertyChange = { widget.text = philosophers[i].status.center(14) } } } } } frame.pack() frame.visible = true forks*.start() sleep 1000 philosophers*.start() sleep 10000 forks*.stop() forks*.join() philosophers*.stop() philosophers*.join() frame.dispose() philosophers.each { println it } socrates has eaten 3 times plato has eaten 3 times aristotle has eaten 6 times descartes has eaten 2 times nietzsche has eaten 5 times
  • 76. Dining Philosophers: CSP... GPars - 76 ©ASERT2006-2010 // inspired by similar examples at the web sites below: // http://www.cs.kent.ac.uk/projects/ofa/jcsp/ // http://www.soc.napier.ac.uk/~jmk/#_Toc271192596 @Grab('org.codehaus.gpars:gpars:0.10') import org.jcsp.lang.* import groovyx.gpars.csp.PAR import groovyx.gpars.csp.ALT import static java.lang.System.currentTimeMillis def names = ['socrates', 'plato', 'aristotle', 'descartes', 'nietzsche'] enum ForkAction { Take, Release, Stop } import static ForkAction.* class Philosopher implements CSProcess { ChannelOutput leftFork, rightFork String name def forks = [] private random = new Random() private timesEaten = 0 private start = currentTimeMillis() void run() { while (currentTimeMillis() - start < 10000) { think() eat() } [leftFork, rightFork].each { it.write(Stop) } println toString() } …
  • 77. …Dining Philosophers: CSP... GPars - 77 ©ASERT2006-2010 … void think() { println "$name is thinking" sleep random.nextInt(50) } void eat() { [leftFork, rightFork].each { it.write(Take) } println "$name is EATING" timesEaten++ sleep random.nextInt(200) [leftFork, rightFork].each { it.write(Release) } } String toString() { switch (timesEaten) { case 0: return "$name has starved" case 1: return "$name has eaten once" default: return "$name has eaten $timesEaten times" } } }
  • 78. …Dining Philosophers: CSP... GPars - 78 ©ASERT2006-2010 … class Fork implements CSProcess { ChannelInput left, right private active = [0, 1] as Set void run() { def fromPhilosopher = [left, right] def forkAlt = new ALT(fromPhilosopher) while (active) { def i = forkAlt.select() read fromPhilosopher, i, Take read fromPhilosopher, i, Release } } void read(phil, index, expected) { if (!active.contains(index)) return def m = phil[index].read() if (m == Stop) active -= index else assert m == expected } } …
  • 79. …Dining Philosophers: CSP GPars - 79 ©ASERT2006-2010 … def lefts = Channel.createOne2One(names.size()) def rights = Channel.createOne2One(names.size()) def philosophers = (0..<names.size()).collect { i -> return new Philosopher(leftFork: lefts[i].out(), rightFork: rights[i].out(), name: names[i]) } def forks = (0..<names.size()).collect { i -> return new Fork(left: lefts[i].in(), right: rights[(i + 1) % names.size()].in()) } def processList = philosophers + forks new PAR(processList).run()
  • 80. Why CSP? • Amenable to proof and analysis GPars - 80 Picture source: http://wotug.org/parallel/theory/formal/csp/Deadlock/
  • 81. GPars: Agents... • Agents safeguard non-thread safe objects • Only the agent can update the underlying object • “Code” to update the protected object is sent to the agent • Can be used with other approaches GPars - 81 ©ASERT2006-2010
  • 82. …GPars: Agents GPars - 82 ©ASERT2006-2010 @Grab('org.codehaus.gpars:gpars:0.10') import groovyx.gpars.agent.Agent def speakers = new Agent<List>(['Alex'], {it?.clone()}) // add Alex speakers.send {updateValue it << 'Hilary'} // add Hilary final Thread t1 = Thread.start { speakers.send {updateValue it << 'Ken'} // add Ken } final Thread t2 = Thread.start { speakers << {updateValue it << 'Guy'} // add Guy speakers << {updateValue it << 'Ralph'} // add Ralph } [t1, t2]*.join() assert new HashSet(speakers.val) == new HashSet(['Alex', 'Hilary', 'Ken', 'Guy', 'Ralph']) Source: Gpars examples
  • 83. GPars for testing GPars - 83 ©ASERT2006-2010 @Grab('net.sourceforge.htmlunit:htmlunit:2.6') import com.gargoylesoftware.htmlunit.WebClient @Grab('org.codehaus.gpars:gpars:0.10') import static groovyx.gpars.GParsPool.* def testCases = [ ['Home', 'Bart', 'Content 1'], ['Work', 'Homer', 'Content 2'], ['Travel', 'Marge', 'Content 3'], ['Food', 'Lisa', 'Content 4'] ] withPool(3) { testCases.eachParallel{ category, author, content -> postAndCheck category, author, content } } private postAndCheck(category, author, content) { ...
  • 84. Guy Steele example in Groovy… GPars - 84 ©ASERT2006-2010 def words = { s -> def result = [] def word = '' s.each{ ch -> if (ch == ' ') { if (word) result += word word = '' } else word += ch } if (word) result += word result } assert words("This is a sample") == ['This', 'is', 'a', 'sample'] assert words(" Here is another sample ") == ['Here', 'is', 'another', 'sample'] assert words("JustOneWord") == ['JustOneWord'] assert words("Here is a sesquipedalian string of words") == ['Here', 'is', 'a', 'sesquipedalian', 'string', 'of', 'words'] assert words(" ") == [] && words("") == [] Sequential version Guy Steele‟s example from keynote (from slide 52 onwards for several slides): http://strangeloop2010.com/talk/presentation_file/14299/GuySteele-parallel.pdf
  • 85. …Guy Steele example in Groovy… GPars - 85 ©ASERT2006-2010 http://strangeloop2010.com/talk/presentation_file/14485/Johnson-DataParallelism.pdf
  • 86. …Guy Steele example in Groovy… GPars - 86 ©ASERT2006-2010 Guy Steele‟s example from keynote (from slide 52 onwards for several slides): http://strangeloop2010.com/talk/presentation_file/14299/GuySteele-parallel.pdf
  • 87. …Guy Steele example in Groovy… GPars - 87 ©ASERT2006-2010 @Immutable class Chunk { String s def plus(Chunk other) { new Chunk(s + other.s) } def plus(Segment other) { new Segment(s + other.l, other.m, other.r) } } @Immutable class Segment { String l; List m; String r def plus(Chunk other) { new Segment(l, m, r + other.s) } def plus(Segment other) { new Segment(l, m + maybeWord(r + other.l) + other.m, other.r) } } class Util { static processChar(ch) { ch == ' ' ? new Segment('', [], '') : new Chunk(ch) } static maybeWord(s) { s ? [s] : [] } } import static Util.* ... Refactored sequential version
  • 88. …Guy Steele example in Groovy… GPars - 88 ©ASERT2006-2010 def words = { s -> def result s.each{ ch -> if (!result) result = processChar(ch) else result += processChar(ch) } switch(result) { case Chunk: return maybeWord(result.s) case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) } case null: return [] } } assert words("This is a sample") == ['This', 'is', 'a', 'sample'] assert words(" Here is another sample ") == ['Here', 'is', 'another', 'sample'] assert words("JustOneWord") == ['JustOneWord'] assert words("Here is a sesquipedalian string of words") == ['Here', 'is', 'a', 'sesquipedalian', 'string', 'of', 'words'] assert words(" ") == [] && words("") == [] Refactored sequential version
  • 89. …Guy Steele example in Groovy… GPars - 89 ©ASERT2006-2010 def swords = { s -> def result s.each{ ch -> if (!result) result = processChar(ch) else result += processChar(ch) } result ?: new Chunk('') } THREADS = 4 def words = { s -> int n = (s.size() + THREADS - 1) / THREADS def map = new java.util.concurrent.ConcurrentHashMap() (0..<THREADS).collect { i -> Thread.start { def (min, max) = [[s.size(),i*n].min(), [s.size(),(i+1)*n].min()] map[i] = swords(s[min..<max]) }}*.join() def result = map.entrySet().sort{ it.key }.sum{ it.value } switch(result) { case Chunk: return maybeWord(result.s) case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) } } } Roll your own threading with ConcurrentHashMap version
  • 90. …Guy Steele example in Groovy… GPars - 90 ©ASERT2006-2010 def words = { s -> int n = (s.size() + THREADS - 1) / THREADS def min = (0..<THREADS).collectEntries{ [it, [s.size(),it*n].min()] } def max = (0..<THREADS).collectEntries{ [it, [s.size(),(it+1)*n].min()] } def result = new DataFlows().with { task { a = swords(s[min[0]..<max[0]]) } task { b = swords(s[min[1]..<max[1]]) } task { c = swords(s[min[2]..<max[2]]) } task { d = swords(s[min[3]..<max[3]]) } task { sum1 = a + b } task { sum2 = c + d } task { sum = sum1 + sum2 } println 'Tasks ahoy!' sum } switch(result) { case Chunk: return maybeWord(result.s) case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) } } } DataFlow version: partially hard-coded to 4 partitions for easier reading
  • 91. …Guy Steele example in Groovy… GPars - 91 ©ASERT2006-2010 GRANULARITY_THRESHHOLD = 10 THREADS = 4 println GParsPool.withPool(THREADS) { def result = runForkJoin(0, input.size(), input){ first, last, s -> def size = last - first if (size <= GRANULARITY_THRESHHOLD) { swords(s[first..<last]) } else { // divide and conquer def mid = first + ((last - first) >> 1) forkOffChild(first, mid, s) forkOffChild(mid, last, s) childrenResults.sum() } } switch(result) { case Chunk: return maybeWord(result.s) case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) } } } Fork/Join version
  • 92. …Guy Steele example in Groovy… GPars - 92 ©ASERT2006-2010 THRESHHOLD = 10 def split(raw) { raw.size() <= THRESHHOLD ? raw : [raw[0..<THRESHHOLD]] + split(raw.substring(THRESHHOLD)) } println GParsPool.withPool(THREADS) { def ans = split(input).parallel.map(swords).reduce{ a,b -> a + b } switch(ans) { case Chunk: return maybeWord(ans.s) case Segment: return ans.with{ maybeWord(l) + m + maybeWord(r) } } } Map/Reduce version
  • 93. …Guy Steele example in Groovy GPars - 93 ©ASERT2006-2010 println GParsPool.withPool(THREADS) { def ans = input.collectParallel{ processChar(it) }.sum() switch(ans) { case Chunk: return maybeWord(ans.s) case Segment: return ans.with{ maybeWord(l) + m + maybeWord(r) } } } Just leveraging the algorithm‟s parallel nature
  • 94. Topics • Groovy Intro • Useful Groovy features for Concurrency • Related Concurrency Libraries & Tools • GPars More Info GPars - 94 ©ASERT2006-2010
  • 95. More Information about Concurrency • Web sites – http://gpars.codehaus.org/ – http://g.oswego.edu/ Doug Lea's home page – http://gee.cs.oswego.edu/dl/concurrency-interest/ – http://jcip.net/ Companion site for Java Concurrency in Practice – http://www.eecs.usma.edu/webs/people/okasaki/pubs.html#cup98 Purely Functional Data Structures – http://delicious.com/kragen/concurrency Concurrency bookmark list – http://www.gotw.ca/publications/concurrency-ddj.htm The Free Lunch is Over, Herb Sutter – http://manticore.cs.uchicago.edu/papers/damp07.pdf – http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=10142 Concepts, Techniques, and Models of Computer Programming GPars - 95
  • 96. More Information about Groovy • Web sites – http://groovy.codehaus.org – http://grails.codehaus.org – http://pleac.sourceforge.net/pleac_groovy (many examples) – http://www.asert.com.au/training/java/GV110.htm (workshop) • Mailing list for users – user@groovy.codehaus.org • Information portals – http://www.aboutgroovy.org – http://www.groovyblogs.org • Documentation (1000+ pages) – Getting Started Guide, User Guide, Developer Guide, Testing Guide, Cookbook Examples, Advanced Usage Guide • Books – Several to choose from ... GPars - 96
  • 97. More Information: Groovy in Action GPars - 97 Contains a chapter on GPars!
  • 98. Topics • Bonus Material – Multiverse Philosophers – Jetlang Philosophers – Gruple Philosophers GPars - 98 ©ASERT2006-2010
  • 99. Multiverse Philosophers… GPars - 99 //@Grab('org.multiverse:multiverse-core:0.7-SNAPSHOT') //@Grab('org.multiverse:multiverse-alpha:0.7-SNAPSHOT') //@Grab('org.multiverse:multiverse-groovy:0.7-SNAPSHOT') //@GrabConfig(systemClassLoader=true, initContextClassLoader = true) // adapted multiverse Groovy example: http://git.codehaus.org/gitweb.cgi?p=multiverse.git // file: multiverse-groovy/src/test/groovy/org/multiverse/integration/ org/multiverse/integration/examples/DiningPhilosphersTest.groovy import org.multiverse.transactional.refs.BooleanRef import org.multiverse.transactional.refs.IntRef import static MultiverseGroovyLibrary.* def food = new IntRef(5) def names = ['socrates', 'plato', 'aristotle', 'descartes', 'nietzsche'] def forks = (1..5).collect { new Fork(id: it, free: new BooleanRef(true)) } def philosophers = (0..4).collect { new Philosopher(name: names[it], food: food, left: forks[(it + 1) % 5], right: forks[it]) } def threads = philosophers.collect { new Thread(it) } threads*.start() threads*.join() philosophers.each { println it } class Fork { int id BooleanRef free void take() { free.set(false) } void release() { free.set(true) } }
  • 100. …Multiverse Philosophers GPars - 100 class Philosopher implements Runnable { String name Fork left, right IntRef timesEaten = new IntRef() IntRef food void eat() { atomic(trackreads: true, explicitRetryAllowed: true) { left.free.await(true) right.free.await(true) if (food.get() > 0) { left.take(); right.take() timesEaten.inc(); sleep 10; food.dec() } } } void think() { atomic(trackreads: true, explicitRetryAllowed: true) { left.release(); right.release() } sleep 10 } void run() { 10.times { eat(); think() } } String toString() { switch (timesEaten) { case 0: return "$name has starved" case 1: return "$name has eaten once" default: return "$name has eaten $timesEaten times" } } }
  • 101. Jetlang Philosophers… GPars - 101 import org.jetlang.core.Callback import org.jetlang.fibers.ThreadFiber import org.jetlang.channels.* def names = ['socrates', 'plato', 'aristotle', 'descartes', 'nietzsche'] class Philosopher implements Callback { private random = new Random() String name int timesEaten = 0 String status def forks private channels = [new MemoryRequestChannel(), new MemoryRequestChannel()] private req = new ThreadFiber() // or from pool private reply = new ThreadFiber() private responses = [] private gotFork = { it instanceof Accepted } void start() { assert forks.size() == 2 req.start() reply.start() (0..1).each{ channels[it].subscribe(reply, forks[it]) } think() } String toString() { switch (timesEaten) { case 0: return "$name has starved" case 1: return "$name has eaten once" default: return "$name has eaten $timesEaten times" } }
  • 102. …Jetlang Philosophers… GPars - 102 … void think() { println(name + ' is thinking') sleep random.nextInt(3000) (0..1).each{ AsyncRequest.withOneReply(req, channels[it], new Take(it), this); } } void eat() { timesEaten++ println toString() sleep random.nextInt(2000) } void onMessage(Object message) { responses << message if (responses.size() == 2) { if (responses.every(gotFork)) { eat() } responses.findAll(gotFork).each { int index = it.index channels[index].publish(req, new Release(index), forks[index]) } responses = [] think() } } } @Immutable class Take { int index } @Immutable class Accepted { int index } @Immutable class Rejected { int index } @Immutable class Release { int index } …
  • 103. …Jetlang Philosophers GPars - 103 … class Fork implements Callback { String name def holder = [] void onMessage(message) { def msg = message instanceof Request ? message.request : message def index = msg.index switch (msg) { case Take: if (!holder) { holder << index message.reply(new Accepted(index)) } else message.reply(new Rejected(index)) break case Release: assert holder == [index] holder = [] break default: throw new IllegalStateException("Cannot process the message: $message") } } } def forks = (1..names.size()).collect { new Fork(name: "Fork $it") } def philosophers = (1..names.size()).collect { new Philosopher(name: names[it - 1], forks: [forks[it - 1], forks[it % names.size()]]) } philosophers*.start() sleep 10000 philosophers.each { println it }
  • 104. Gruple Philosophers… GPars - 104 import org.gruple.SpaceService import org.gruple.Space class Philosopher { private random = new Random() String name Space space private timesEaten = 0 int id, num boolean done = false void run() { while (true) { think() if (done) return space.take(fork: id) space.take(fork: (id + 1) % num) eat() space.put(fork: id) space.put(fork: (id + 1) % num) } } void think() { println "$name is thinking" sleep random.nextInt(500) } void eat() { println "$name is EATING" timesEaten++ sleep random.nextInt(1000) } … … socrates is thinking nietzsche is thinking descartes is EATING aristotle is EATING descartes is thinking plato is EATING aristotle is thinking socrates is EATING plato is thinking nietzsche is EATING socrates is thinking nietzsche is thinking descartes is EATING descartes is thinking socrates has eaten 5 times plato has eaten 4 times aristotle has eaten 4 times descartes has eaten 4 times nietzsche has eaten 5 times
  • 105. …Gruple Philosophers GPars - 105 … String toString() { switch (timesEaten) { case 0: return "$name has starved" case 1: return "$name has eaten once" default: return "$name has eaten $timesEaten times" } } } def names = ['socrates', 'plato', 'aristotle', 'descartes', 'nietzsche'] def diningSpace = SpaceService.getSpace('Dining') def philosophers = (0..<names.size()).collect{ new Philosopher(name: names[it], id: it, space: diningSpace, num: names.size()) } (0..<names.size()).each{ diningSpace << [fork: it] } sleep 500 def threads = (0..<names.size()).collect{ n -> Thread.start{ philosophers[n].run() } } sleep 10000 philosophers*.done = true sleep 2000 threads.join() println() philosophers.each{ println it }