These are the slides of the talk we made with Stuart Marks at Devoxx Belgium 2018. This second part covers the Stream API, reduction and the Collector API.
What is the state of lambda expressions in Java 11? Lambda expressions are the major feature of Java 8, having an impact on most of the API, including the Streams and Collections API. We are now living the Java 11 days; new features have been added and new patterns have emerged. This highly technical Deep Dive session will visit all these patterns, the well-known ones and the new ones, in an interactive hybrid of lecture and laboratory. We present a technique and show how it helps solve a problem. We then present another problem, and give you some time to solve it yourself. Finally, we present a solution, and open for questions, comments, and discussion. Bring your laptop set up with JDK 11 and your favorite IDE, and be prepared to think!
5. #LambdaHOL#Devoxx
Lambda/Streams Master Class 2
Setup
Map, Filter, FlatMap
Reduction, Function combination
Collectors.toMap()
Collectors.groupingBy()
Cascading Collectors
Streaming a Map
Streaming over Indexes
6. #LambdaHOL#Devoxx
Back to the Comparator
How to deal with null names?
Comparator<Person> cmp = Comparator.comparing(Person::getLastName)
.thenComparing(Person::getFirstName)
.thenComparing(Person::getAge);
7. #LambdaHOL#Devoxx
Back to the Comparator
In fact, this comparator:
Is equivalent to this one:
Comparator.comparing(Person::getLastName)
Comparator.comparing(Person::getLastName, Comparator.naturalOrder())
8. #LambdaHOL#Devoxx
Back to the Comparator
This allows this pattern:
So, to deal with null values:
Comparator.comparing(Person::getLastName,
Comparator.nullsLast(Comparator.naturalOrder()))
Comparator.comparing(Person::getLastName,
Comparator.comparing(String::length))
9. #LambdaHOL#Devoxx
Back to the Comparator
If we need to deal with null Person objects and
null names:
Comparator.nullsLast(
Comparator.comparing(Person::getLastName,
Comparator.nullsLast(Comparator.naturalOrder())
)
)
11. #LambdaHOL#Devoxx
The LambdaHOL
You can find it here
https://github.com/stuart-marks/LambdaHOLv2
https://github.com/JosePaumard/lambda-master-class-part2
https://github.com/JosePaumard/lambda-master-class-part1
13. #LambdaHOL#Devoxx
Input Data — Sonnet
List<String> sonnet = List.of(
"From fairest creatures we desire increase,",
"That thereby beauty's rose might never die,",
"But as the riper should by time decease,",
"His tender heir might bear his memory:",
"But thou contracted to thine own bright eyes,",
"Feed'st thy light's flame with self-substantial fuel,",
"Making a famine where abundance lies,",
"Thy self thy foe, to thy sweet self too cruel:",
"Thou that art now the world's fresh ornament,",
"And only herald to the gaudy spring,",
"Within thine own bud buriest thy content,",
"And, tender churl, mak'st waste in niggarding:",
"Pity the world, or else this glutton be,",
"To eat the world's due, by the grave and thee.");
19. #LambdaHOL#Devoxx
FlatMap
• Intermediate stream operation
• Consumes one element
• May produce zero or more elements
• Compare to map: consumes one, produces
one
• How is zero-or-more represented? A stream!
• T ⇒ Stream<R>
20. #LambdaHOL#Devoxx
FlatMap
Given a list of strings...
[alfa, bravo, charlie, ...]
expand each string to a list of one-letter strings
[ [a, l, f, a], [b, r, a, v, o], [c, h, ...] ... ]
but «flatten» the nesting structure
[a, l, f, a, b, r, a, v, o, c, h, ...]
22. #LambdaHOL#Devoxx
FlatMap
[a, l, f, a, b, r, a, v, o, c, h, a, r, l, i, e, d, e, l, ...]
List<String> flatMap2() {
return alphabet.stream()
.flatMap(word -> expand(word).stream())
.collect(toList());
}
23. #LambdaHOL#Devoxx
FlatMap — Exercise
Split each line of the sonnet into words, and
then collect all the words into a single list.
To split a line into words, use
line.split(" +")
note: this returns an array, not a list or stream
24. #LambdaHOL#Devoxx
FlatMap — Solution
Split each line of the sonnet into words, and then
collect all the words into a single list.
[From, fairest, creatures, we, desire, increase, ...]
total 106 words
List<String> flatMapSolution() {
return sonnet.stream()
.flatMap(line -> Arrays.stream(line.split(" +")))
.collect(toList());
}
27. #LambdaHOL#Devoxx
Computing Factorials
Compute the factorial as a BigInteger using
streams and reduction
long number = 21;
BigInteger result = LongStream.rangeClosed(1, 21)
.mapToObj(BigInteger::valueOf)
.reduce(BigInteger.ONE, BigInteger::multiply);
// result is 51_090_942_171_709_440_000
29. #LambdaHOL#Devoxx
Function Combination
Suppose you have a shopping website where
the customer can apply a filter to limit the
products shown.
List<Product> show(Predicate<Product> predicate) {
return getAllProducts().stream()
.filter(predicate)
.collect(toList());
}
30. #LambdaHOL#Devoxx
Function Combination
Suppose you want the customer to be able to
apply two filters to the product list.
Now, how about three filters?
List<Product> show(Predicate<Product> p1, Predicate<Product> p2) {
return getAllProducts().stream()
.filter(p1.and(p2))
.collect(toList());
}
31. #LambdaHOL#Devoxx
Function Combination
Two predicates can be combined using the
Predicate.and() method.
This is all we need to write a method that
combines an arbitrary number of predicates.
Predicate<Product> p1 = ... ;
Predicate<Product> p2 = ... ;
Predicate<Product> combined = p1.and(p2);
32. #LambdaHOL#Devoxx
Function Combination
Reduction of a list over an operator applies that
operator between each element.
Predicate<Product> combine(List<Predicate<Product>> predicates) {
Predicate<Product> temp = product -> true;
for (Predicate<Product> cur : predicates) {
temp = temp.and(cur);
}
return temp;
}
33. #LambdaHOL#Devoxx
Function Combination
Reduction of a list over an operator applies that
operator between each element.
Predicate<Product> combine(List<Predicate<Product>> predicates) {
return predicates.stream()
.reduce(product -> true, Predicate::and);
}
34. #LambdaHOL#Devoxx
Function Combination
Now apply this to the original problem:
List<Product> show(List<Predicate<Product>> predicates) {
Predicate<Product> combinedPredicate = combine(predicates);
return getAllProducts().stream()
.filter(combinedPredicate)
.collect(toList());
}
35. #LambdaHOL#Devoxx
Function Combination — Exercise
An IntUnaryOperator is a functional interface that
takes an int and returns an int.
Write a method that combines an arbitrary sized
list of IntUnaryOperators into a single one.
Use streams and the IntUnaryOperator.andThen()
method.
Use your method to combine functions that add
one, multiply by two, and three.
36. #LambdaHOL#Devoxx
Function Combination — Exercise
IntUnaryOperator combine(List<IntUnaryOperator> operators) {
// TODO
}
IntUnaryOperator operator =
combine(List.of(i -> i + 1, i -> i * 2, i -> i + 3));
System.out.println(operator.applyAsInt(5));
15
37. #LambdaHOL#Devoxx
Function Combination — Solution
IntUnaryOperator combine(List<IntUnaryOperator> operators) {
return operators.stream()
.reduce(i -> i, IntUnaryOperator::andThen);
}
IntUnaryOperator operator =
combine(List.of(i -> i + 1, i -> i * 2, i -> i + 3));
System.out.println(operator.applyAsInt(5));
15
40. #LambdaHOL#Devoxx
Collectors.toMap()
Given the alphabet words, create a map whose
keys are the first letter and whose values are
the words.
Map<String, String> toMap1() {
return alphabet.stream()
.collect(toMap(word -> word.substring(0, 1),
word -> word));
}
41. #LambdaHOL#Devoxx
Collectors.toMap()
a => alfa
b => bravo
c => charlie
d => delta
e => echo
f => foxtrot
g => golf
h => hotel
i => india
j => juliet
k => kilo
l => lima
m => mike
n => november
o => oscar
42. #LambdaHOL#Devoxx
Collectors.toMap()
Now create the first-letter map using the
sonnet instead of the alphabet words.
Map<String, String> toMap1() {
return sonnet.stream()
.collect(toMap(word -> word.substring(0, 1),
word -> word));
}
43. #LambdaHOL#Devoxx
Collectors.toMap()
Exception in thread "main" java.lang.IllegalStateException: Duplicate key B (attempted merging values
But as the riper should by time decease, and But thou contracted to thine own bright eyes,)
at java.base/java.util.stream.Collectors.duplicateKeyException(Collectors.java:133)
at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:180)
at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:720)
44. #LambdaHOL#Devoxx
Collectors.toMap()
The simple (two-arg) toMap() requires that all
keys be unique. Throws exception if duplicate
keys are encountered.
To handle this, a third arg mergeFunction can be
provided. It takes the values of the duplicate
keys and returns a merged value.
45. #LambdaHOL#Devoxx
Collectors.toMap()
Use a merge function that simply returns its
first argument. “First wins.”
Map<String, String> toMap3() {
return sonnet.stream()
.collect(toMap(line -> line.substring(0, 1),
line -> line,
(line1, line2) -> line1 // merge
));
}
46. #LambdaHOL#Devoxx
Collectors.toMap()
Eight of fourteen lines remain, so some
duplicates were lost.
P => Pity the world, or else this glutton be,
A => And only herald to the gaudy spring,
B => But as the riper should by time decease,
T => That thereby beauty's rose might never die,
F => From fairest creatures we desire increase,
W => Within thine own bud buriest thy content,
H => His tender heir might bear his memory:
M => Making a famine where abundance lies,
47. #LambdaHOL#Devoxx
Collectors.toMap()
Use a “last wins” merge function.
Map<String, String> toMap4() {
return sonnet.stream()
.collect(toMap(line -> line.substring(0, 1),
line -> line,
(line1, line2) -> line2 // merge
));
}
48. #LambdaHOL#Devoxx
Collectors.toMap()
Eight entries, but some are different.
P => Pity the world, or else this glutton be,
A => And, tender churl, mak'st waste in niggarding:
B => But thou contracted to thine own bright eyes,
T => To eat the world's due, by the grave and thee.
F => Feed'st thy light's flame with self-substantial fuel,
W => Within thine own bud buriest thy content,
H => His tender heir might bear his memory:
M => Making a famine where abundance lies,
49. #LambdaHOL#Devoxx
Collectors.toMap() — Exercise
Create a map from the lines of the sonnet, with
map keys being the first letter of the line, and
values being the line. For duplicate keys,
concatenate the lines with a newline in
between.
51. #LambdaHOL#Devoxx
Collectors.toMap() — Solution
P => Pity the world, or else this glutton be,
A => And only herald to the gaudy spring,
And, tender churl, mak'st waste in niggarding:
B => But as the riper should by time decease,
But thou contracted to thine own bright eyes,
T => That thereby beauty's rose might never die,
Thy self thy foe, to thy sweet self too cruel:
Thou that art now the world's fresh ornament,
To eat the world's due, by the grave and thee.
F => From fairest creatures we desire increase,
Feed'st thy light's flame with self-substantial fuel,
W => Within thine own bud buriest thy content,
H => His tender heir might bear his memory:
M => Making a famine where abundance lies,
53. #LambdaHOL#Devoxx
Collectors.groupingBy()
The groupingBy() collector is a fancy way of
collecting a map from a stream.
In its simplest form, it takes a classifier function
to transform each stream element into a key.
Map values are a list of stream elements
classified into the same key.
Stream<T> ⇒ Map<K, List<V>>
54. #LambdaHOL#Devoxx
Collectors.groupingBy()
From the alphabet words, create a map whose
keys are the word length, and whose values are
a list of those words. First start off with toMap().
Map<Integer, List<String>> groupingBy1() {
return alphabet.stream()
.collect(
toMap(
word -> word.length(),
word -> new ArrayList<>(Arrays.asList(word)),
(list1, list2) -> { list1.addAll(list2);
return list1; }));
}
59. #LambdaHOL#Devoxx
Collectors.groupingBy() — Solution
P => [Pity the world, or else this glutton be,]
A => [And only herald to the gaudy spring,,
And, tender churl, mak'st waste in niggarding:]
B => [But as the riper should by time decease,,
But thou contracted to thine own bright eyes,]
T => [That thereby beauty's rose might never die,,
Thy self thy foe, to thy sweet self too cruel:,
Thou that art now the world's fresh ornament,,
To eat the world's due, by the grave and thee.]
F => [From fairest creatures we desire increase,,
Feed'st thy light's flame with self-substantial fuel,]
W => [Within thine own bud buriest thy content,]
H => [His tender heir might bear his memory:]
M => [Making a famine where abundance lies,]
61. #LambdaHOL#Devoxx
Cascading Collectors
The groupingBy() collector seems restrictive: it
collects stream elements into a list.
This behavior can be modified by providing a
“downstream” collector as another argument.
groupingBy(classifier, downstream)
65. #LambdaHOL#Devoxx
Collectors.mapping()
Another useful downstream collector is
mapping():
mapping(mapperFunction, downstream2)
The mapping() collector is analogous to
Stream.map(). It applies a mapper function to
an element and passes the result downstream
— to a second downstream collector.
69. #LambdaHOL#Devoxx
Cascading Collectors — Exercise
Group the lines of the sonnet by first letter, and
collect the first word of grouped lines into a set.
To extract the first word of a line, use
string.split(" +")[0]
70. #LambdaHOL#Devoxx
Cascading Collectors — Solution
Map<String, Set<Integer>> cascading3() {
return sonnet.stream()
.collect(
groupingBy(
line -> line.substring(0, 1),
mapping(line -> line.split(" +")[0], toSet())
));
} P => [Pity]
A => [And, And,]
B => [But]
T => [That, Thy, To, Thou]
F => [Feed'st, From]
W => [Within]
H => [His]
M => [Making]
71. #LambdaHOL#Devoxx
Cascading Collectors
A first set of collectors that need downstream
collectors to work:
- mapping()
- filtering()
- flatMapping()
Analogous to intermediate stream operations
72. #LambdaHOL#Devoxx
Cascading Collectors
A second set of collectors:
▪ joining()
▪ counting()
▪ groupingBy(), toMap(), toUnmodifiableMap()
▪ toList(), toSet(), toUnmodifiableList() (and set)
▪ reducing()
Analogous to terminal stream operations
73. #LambdaHOL#Devoxx
Cascading Collectors
Group lines of the sonnet by first letter, and
collect the grouped lines into a single string
separated by newlines.
Map<String, String> cascading4() {
return sonnet.stream()
.collect(groupingBy(line -> line.substring(0, 1),
joining("n")
)
);
}
74. #LambdaHOL#Devoxx
Cascading Collectors
P => Pity the world, or else this glutton be,
A => And only herald to the gaudy spring,
And, tender churl, mak'st waste in niggarding:
B => But as the riper should by time decease,
But thou contracted to thine own bright eyes,
T => That thereby beauty's rose might never die,
Thy self thy foe, to thy sweet self too cruel:
Thou that art now the world's fresh ornament,
To eat the world's due, by the grave and thee.
F => From fairest creatures we desire increase,
Feed'st thy light's flame with self-substantial fuel,
W => Within thine own bud buriest thy content,
H => His tender heir might bear his memory:
M => Making a famine where abundance lies,
75. #LambdaHOL#Devoxx
Cascading Collectors — Exercise
Generate a frequency table of letters in the
sonnet. Remember the expand() helper
method.
Hints: use flatMap(), groupingBy(), and
counting().
76. #LambdaHOL#Devoxx
Cascading Collectors — Solution
Map<String, Long> cascadingSolution2() {
return sonnet.stream()
.flatMap(line -> expand(line).stream())
.collect(groupingBy(ch -> ch, counting()));
}
A => 2
B => 2
F => 2
H => 1
M => 1
P => 1
T => 4
W => 1
=> 92
a => 28
b => 11
c => 9
d => 20
e => 68
f => 9
g => 12
' => 6
h => 33
i => 29
k => 2
l => 18
, => 15
- => 1
m => 10
. => 1
n => 29
o => 25
p => 2
r => 33
s => 30
t => 54
u => 17
v => 2
w => 11
y => 14
: => 3
78. #LambdaHOL#Devoxx
Streaming a Map
Find the most frequently occurring word from
the Sonnet
- 1st step: find one of those words
- 2nd step: find all those words in a list
79. #LambdaHOL#Devoxx
Streaming a Map
Two hints:
1) You cannot stream a map. To stream a map,
you need to get a stream of entries from its
entrySet().
Stream<Map.Entry<K, V>> stream =
map.entrySet().stream();
80. #LambdaHOL#Devoxx
Streaming a Map
Two hints:
2) There is a Stream.max() method
And Map.Entry provides comparators
stream.max(comparator)
.orElseThrow(); // max returns an Optional
stream.max(Map.Entry.comparingByValue())
.orElseThrow(); // max returns an Optional
81. #LambdaHOL#Devoxx
Inverting a Map
Suppose there are multiple maximum values
Max finds one of them
Finding all of them can be done by
converting a Map<word, count>
to a Map<count, List<word>>
83. #LambdaHOL#Devoxx
Streaming Over Indexes
Sometimes you need to process groups of
adjacent elements or a “sliding window” of
elements from a stream. The usual way of
streaming elements doesn’t handle this well.
If you have the elements in an array or random-
access list, you can work around this limitation
by streaming indexes instead of elements.
87. #LambdaHOL#Devoxx
Streaming Over Indexes — Exercise
From the alphabet list, produce a list of
overlapping sublists of length N (sliding
window)
[[alfa, bravo, charlie], [bravo, charlie, delta], [charlie, delta,
echo], [delta, echo, foxtrot], [echo, foxtrot, golf], ...
90. #LambdaHOL#Devoxx
Streaming Over Indexes — Exercise 2
Split the alphabet list into runs (sublists) of
strings of non-decreasing length, preserving
order.
That is, within each sublist, the next string
should always be the same length or longer.
[[alfa, bravo, charlie], [delta], [echo, foxtrot], [golf, ...] ...]
91. #LambdaHOL#Devoxx
Streaming Over Indexes — Solution 2
Insight: a new sublist starts when this string is
shorter than the previous string. Find the
indexes where this occurs.
[3, 4, 6, 10, 14, 15, 17, 19, 21, 23, 25]
List<Integer> breaks =
IntStream.range(1, alphabet.size())
.filter(i -> alphabet.get(i).length() <
alphabet.get(i-1).length())
.boxed()
.collect(toList());
92. #LambdaHOL#Devoxx
Streaming Over Indexes — Solution 2
We want sublists between these breaks. Run a
stream over the breaks to generate sublists.
[[delta], [echo, foxtrot], [golf, hotel, india, juliet], [kilo, lima,
mike, november], [oscar], [papa, quebec], [romeo, sierra], [tango,
uniform], [victor, whiskey], [x-ray, yankee]]
List<List<String>> sublists =
IntStream.range(0, breaks.size()-1)
.mapToObj(i -> alphabet.subList(breaks.get(i),
breaks.get(i+1)))
.collect(toList());
93. #LambdaHOL#Devoxx
Streaming Over Indexes — Solution 2
Add starting and ending indexes to break list to
pick up leading and trailing sublists.
[[alfa, bravo, charlie], [delta], [echo, foxtrot], [golf, hotel, india,
juliet], [kilo, lima, mike, november], [oscar], [papa, quebec], [romeo,
sierra], [tango, uniform], [victor, whiskey], [x-ray, yankee], [zulu]]
breaks.add(0, 0);
breaks.add(alphabet.size());
List<List<String>> sublists =
IntStream.range(0, breaks.size()-1)
.mapToObj(i -> alphabet.subList(breaks.get(i),
breaks.get(i+1)))
.collect(toList());