Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

File IO in Java 8 -- Applying the Power of Streams

12.328 Aufrufe

Veröffentlicht am

Please email hall@coreservlets.com for info on how to arrange customized courses on Java 7, Java 8, JSF 2.2, PrimeFaces, jQuery/Ajax, Android, GWT, Hadoop, and other Java EE topics onsite at YOUR location.

This section covers the use of the updated java.nio package to read and write files. In particular, it shows how the power of Java 8 Streams to simplify reading files and exploring folders. See http://www.coreservlets.com/java-8-tutorial/ for the complete tutorial, downloadable and printable PDF files, source code, exercises, and exercise solutions.

Veröffentlicht in: Software
  • Follow the link, new dating source: ❶❶❶ http://bit.ly/2u6xbL5 ❶❶❶
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • Dating for everyone is here: ❤❤❤ http://bit.ly/2u6xbL5 ❤❤❤
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier

File IO in Java 8 -- Applying the Power of Streams

  1. 1. For additional materials, please see http://www.coreservlets.com/. The Java tutorial section contains complete source code for all examples in this tutorial series, plus exercises and exercise solutions for each topic. coreservlets.com – custom onsite training Slides © 2016 Marty Hall, hall@coreservlets.com File I/O in Java 8: Applying the Power of Streams Originals of slides and source code for examples: http://courses.coreservlets.com/Course-Materials/java.html Also see Java 8 tutorial: http://www.coreservlets.com/java-8-tutorial/ and many other Java EE tutorials: http://www.coreservlets.com/ Customized Java training courses (onsite or at public venues): http://courses.coreservlets.com/java-training.html
  2. 2. For additional materials, please see http://www.coreservlets.com/. The Java tutorial section contains complete source code for all examples in this tutorial series, plus exercises and exercise solutions for each topic. coreservlets.com – custom onsite training Slides © 2016 Marty Hall, hall@coreservlets.com For customized training related to Java or JavaScript, please email hall@coreservlets.com Marty is also available for consulting and development support Taught by lead author of Core Servlets & JSP, co-author of Core JSF (4th Ed), and this tutorial. Available at public venues, or custom versions can be held on-site at your organization. • Courses developed and taught by Marty Hall – JSF 2.3, PrimeFaces, Java programming (using Java 8, for those new to Java), Java 8 (for Java 7 programmers), JavaScript, jQuery, Ext JS, Spring Framework, Spring MVC, Java EE 8 MVC, Android, GWT, custom mix of topics – Courses available in any state or country. – Maryland/DC companies can also choose afternoon/evening courses. • Courses developed and taught by coreservlets.com experts (edited by Marty) – Hadoop, Hibernate/JPA, HTML5, RESTful Web Services Contact hall@coreservlets.com for details
  3. 3. 5 Topics in This Section • More on try/catch blocks – finally, multicatch, try-with-resources • Representing file paths – Paths.get • Reading files by treating them as streams – Files.lines • Writing files – Files.write • Making file reading more flexible by using lambdas and generic types – Split into two pieces: one for Stream<String> and one for file – Combine predicates with varargs and Predicate.and • Exploring folders by using streams – Files.list, Files.walk, Files.find • File I/O in Java 7 – Files.readAllLines
  4. 4. For additional materials, please see http://www.coreservlets.com/. The Java tutorial section contains complete source code for all examples in this tutorial series, plus exercises and exercise solutions for each topic. coreservlets.com – custom onsite training Slides © 2016 Marty Hall, hall@coreservlets.com More on try/catch Blocks
  5. 5. Summary Covered earlier: basics try { statement1; statement2; ... } catch(Eclass1 var1) { ... } catch(Eclass2 var2) { ... } catch(Eclass3 var3) { ... } … New: finally try {… } catch(…) {… } finally { … } New: multicatch try {… } catch(Eclass1 | Eclass e) { … } … New: try with resources try (SomeAutoCloseable var = …) {… } catch(…) { … } …
  6. 6. 8 Finally Blocks • Idea – The finally { … } block at the end of a try/catch is called whether or not there is an exception • Motivation: resetting resources, closing sockets, other cleanup HugeDataStructure blah = ...; try { doSomethingWith(blah); ... } catch { ... } finally { blah = null; }
  7. 7. 9 Finally Blocks: Benefits • Question: difference between these two? • Answer: nested try/catch blocks – In the example on the right above, if the catch throws an exception and the entire try/catch block is inside another try/catch block, the cleanup code might not run • Same issue if there is return, break, or continue • Many developers advocate always using finally for required cleanup, even if code does not (currently) have nested exception, return statement, etc. Finally Block Code After Entire try/catch try { ... } catch(ExceptionType e) { ... } finally { doSomeCleanup(); } try { ... } catch(ExceptionType e) { ... } doSomeCleanup();
  8. 8. 10 Multicatch • Idea: can catch multiple exceptions using | – In Java 7 and later, if two different catch blocks will do the same thing, you can catch more than one in the same catch clause (but also consider catching a parent type): try { ... } catch(Eclass1 | Eclass2 e) { ... } • Example Without Multicatch With Multicatch String input = getSomeString(); int num; try { num = Integer.parseInt(input); } catch(NumberFormatException nfe) { num = someDefault; } catch(NullPointerException npe) { num = someDefault; } String input = getSomeString(); int num; try { num = Integer.parseInt(input); } catch(NumberFormatException | NullPointerException e) { num = someDefault; }
  9. 9. 11 try-with-resources: Overview • Idea – In Java 7 and later, you can declare variables that implement AutoCloseable in parens after try. • Scope of variable is scope of try/catch block • The close method of each variable is called at the end, whether or not there is an exception (i.e., as if the call to close were in a finally block) • Can declare multiple variables, separated by semicolon • Example try (BufferedReader reader = …) { readSomeDataWith(reader); ... } catch(SomeExceptionType e) { ... }
  10. 10. 12 try-with-resources: Benefits • Advantages of approach on right – Shorter and simpler – Can’t forget to call close – The reader variable is out of scope after the try/catch block finishes Without With BufferedReader reader; try { reader = ...; ... } catch(SomeExceptionType e) { ... } finally { reader.close(); } try(BufferedReader reader = ...) { ... } catch(SomeExceptionType e) { ... }
  11. 11. For additional materials, please see http://www.coreservlets.com/. The Java tutorial section contains complete source code for all examples in this tutorial series, plus exercises and exercise solutions for each topic. coreservlets.com – custom onsite training Slides © 2016 Marty Hall, hall@coreservlets.com Paths
  12. 12. 14 Idea • Path is a simpler and more flexible replacement for File class – And is main starting point for file I/O operations • Get a Path with Paths.get Path p1 = Paths.get("some-file"); Path p2 = Paths.get("/usr/local/gosling/some-file"); Path p3 = Paths.get("C:UsersGoslingDocumentssome-file"); • Notice the double backslashes above, because backslash already has meaning (escape next character) in Java strings • Paths have convenient methods – toAbsolutePath, startsWith, endsWith, getFileName, getName, getNameCount, subpath, getParent, getRoot, normalize, relativize
  13. 13. 15 Example public class PathExamples { public static void main(String[] args) { Path p1 = Paths.get("input-file.txt"); System.out.println("Simple Path"); System.out.printf("toString: %s%n%n", p1); Path p2 = p1.toAbsolutePath(); System.out.println("Absolute Path"); System.out.printf("toString: %s%n", p2); System.out.printf("getFileName: %s%n", p2.getFileName()); System.out.printf("getName(0): %s%n", p2.getName(0)); System.out.printf("getNameCount: %d%n", p2.getNameCount()); System.out.printf("subpath(0,2): %s%n", p2.subpath(0,2)); System.out.printf("getParent: %s%n", p2.getParent()); System.out.printf("getRoot: %s%n", p2.getRoot()); } }
  14. 14. 16 Example Output Simple Path toString: input-file.txt Absolute Path toString: C:eclipse-workspacejavafile-ioinput-file.txt getFileName: input-file.txt getName(0): eclipse-workspace getNameCount: 4 subpath(0,2): eclipse-workspacejava getParent: C:eclipse-workspacejavafile-io getRoot: C:
  15. 15. For additional materials, please see http://www.coreservlets.com/. The Java tutorial section contains complete source code for all examples in this tutorial series, plus exercises and exercise solutions for each topic. coreservlets.com – custom onsite training Slides © 2016 Marty Hall, hall@coreservlets.com File Reading: Overview
  16. 16. 18 Using File.lines: Idea • You can read all lines into Stream using just one method call Stream<String> lines = Files.lines(somePath); • Benefits – Can use all the cool and powerful Stream methods • map, filter, reduce, collect, etc. – Lazy evaluation • Suppose you map into uppercase, filter out the strings shorter than five characters, keep only the palindromes, then find the first. • If there is a 5-letter palindrome near the top of the file, it will never even read the rest of the file.
  17. 17. 19 Files.lines: More Details • Charset option Files.lines(path) – Uses UTF-8 Files.lines(path, someCharset) – Uses specified Charset • Throws IOException – So, you must use try/catch block or throw the exception • Stream should be closed – Most Streams do not need closing, but ones connected to I/O sources (as here) do. • Stream implements AutoCloseable – You can use try-with-resources to handle IOException and then automatically call close() at the end
  18. 18. 20 File Reading Variations • General principle – Streams help make handling large data sets more convenient and efficient – Lambdas and generic types help make code more flexible and reusable • Variation 1 – Put all code inside main; main throws Exception • Simple and easy, but not reusable • Variation 2 – Method 1 handles Stream; method 2 calls Files.lines and passes Stream to method 1 • Reusable, but each version of method 2 repeats a lot of boilerplate code • Variation 3 – Define a functional interface and a static method that can use lambdas – Method 1 handles Stream; method 2 passes filename and lambda to static method • Variation 4 – Similar to variation 3 but uses generic types so that values can be returned
  19. 19. 21 Examples: Processing Large Word List • The enable1 Scrabble™ word list – Public-domain file containing over 175,000 supposed words accepted by many US Scrabble clubs • The name comes from Enhanced North American Benchmark LExicon (ENABLE). – It is almost twice as large as the Official Scrabble Player’s Dictionary™, and contains slang, offensive words, and many obscure or questionable words – It contains no one-letter words and no super-long words, and is not endorsed in any way by Hasbro (maker of Scrabble) or Merriam Webster (publisher of The Official Scrabble Player’s Dictionary). – Details at http://www.puzzlers.org/dokuwiki/doku.php?id=solving:wordlists:about:enable_readme
  20. 20. For additional materials, please see http://www.coreservlets.com/. The Java tutorial section contains complete source code for all examples in this tutorial series, plus exercises and exercise solutions for each topic. coreservlets.com – custom onsite training Slides © 2016 Marty Hall, hall@coreservlets.com File Reading: First Variation
  21. 21. 23 Overview • Basic approach public static void main(String[] args) throws Exception { Files.lines(Paths.get("input-file")) .map(someFunction) .filter(someTest) .someOtherStreamOperation(...); } • Advantage: quick and easy – Many data analysis tasks involve one-up cases to read and analyze log files • Disadvantage: not reusable – Cannot do same tasks to Stream<String> that came from another source – Cannot test without a file – Calling main is inconvenient from other code
  22. 22. 24 Examples • Example 1: file of 4-letter words – Assume that the enable1 word list might have a few repeats, a few words in mixed case, and a few words out of alphabetical order – Produce file containing all four-letter words, in upper case, without repeats, and in alphabetical order • Example 2: all palindromes – Print out all palindromes contained in the file • Example 3: first 6-letter palindrome – Print the first 6-letter palindrome contained in the file • Example 4: q’s not followed by u’s – Count how many words have q but no qu • Example 5: x’s and y’s – Count total letters in all words that have both x and y
  23. 23. 25 Example 1: Create File of 4-Letter Words public static void main(String[] args) throws Exception { String inputFile = "enable1-word-list.txt"; String outputFile = "four-letter-words.txt"; int length = 4; List<String> words = Files.lines(Paths.get(inputFile)) .filter(word -> word.length() == length) .map(String::toUpperCase) .distinct() .sorted() .collect(Collectors.toList()); Files.write(Paths.get(outputFile), words, Charset.defaultCharset()); System.out.printf("Wrote %s words to %s.%n", words.size(), outputFile); } Resultant file AAHS AALS ABAS ABBA ABBE ABED ABET ABLE ABLY ... Files.write takes a List<String> and produces a file that contains each of the strings on a separate line. It is discussed in the next section.
  24. 24. 26 Example 2: Print All Palindromes public static void main(String[] args) throws Exception { String inputFile = "enable1-word-list.txt"; Files.lines(Paths.get(inputFile)) .filter(StringUtils::isPalindrome) .forEach(System.out::println); } Output aa aba abba aga aha ala alula ...
  25. 25. 27 Example 2: isPalindrome Helper Method public class StringUtils { public static String reverseString(String s) { return(new StringBuilder(s).reverse().toString()); } public static boolean isPalindrome(String s) { return(s.equalsIgnoreCase(reverseString(s))); } }
  26. 26. 28 Example 3: Print First 6-Letter Palindrome public static void main(String[] args) throws Exception { String inputFile = "enable1-word-list.txt"; String firstPalindrome = Files.lines(Paths.get(inputFile)) .filter(word -> word.length() == 6) .filter(StringUtils::isPalindrome) .findFirst() .orElse(null); System.out.printf("First 6-letter palindrome is %s.%n", firstPalindrome); } Output First 6-letter palindrome is denned.
  27. 27. 29 Example 4: # of Words with q not Followed by u public static void main(String[] args) throws Exception { String inputFile = "enable1-word-list.txt"; long wordCount = Files.lines(Paths.get(inputFile)) .filter(word -> word.contains("q")) .filter(word -> !word.contains("qu")) .count(); System.out.printf("%s words with q but not u.%n", wordCount); } Output 29 words with q but not u.
  28. 28. 30 Example 5: Total Letters in Words with Both x & y public static void main(String[] args) throws Exception { String inputFile = "enable1-word-list.txt"; int letterCount = Files.lines(Paths.get(inputFile)) .filter(word -> word.contains("x")) .filter(word -> word.contains("y")) .mapToInt(String::length) .sum(); System.out.printf("%,d total letters in words with " + "both x and y.%n", letterCount); } Output 8,556 total letters in words with both x and y.
  29. 29. 31 Preview of Later Variations • General principle – Streams help make handling large data sets more convenient and efficient • This was seen even in this first variation that uses Files.lines to get Stream<String> • Use of convenient Stream methods makes it relatively easy to do complex file reading tasks. Arguably as convenient as Python and Perl. • Lazy evaluation and the fact that Streams are not stored in memory all at once makes file processing efficient. – Lambdas and generic types help make code more flexible and reusable • In examples so far, code was not easily reusable • Variations 2 and especially 3 will show how lambdas can help • Variation 4 will show how generic types can help further • Followon examples will show how advanced lambda processing (combining predicates) can easily apply to file processing
  30. 30. For additional materials, please see http://www.coreservlets.com/. The Java tutorial section contains complete source code for all examples in this tutorial series, plus exercises and exercise solutions for each topic. coreservlets.com – custom onsite training Slides © 2016 Marty Hall, hall@coreservlets.com Simple File Writing
  31. 31. 33 Idea • You can write all lines in one method call List<String> lines = …; Files.write(somePath, lines, someCharset); • Recall that you can turn Stream into List with stream.collect(Collectors.toList()). • You can actually use any Iterable<String>, not just List<String>. You would think you could also use List<Object>, and the system would call toString on each Object automatically. Sadly, no. Boo. • You can write all bytes in one method call byte[] fileArray = …; Files.write(somePath, fileArray);
  32. 32. 34 The OpenOption • Both versions of Files.write optionally take an OpenOption as final argument – Files.write(somePath, lines, someCharset, someOption); – Files.write(somePath, fileArray, someOption); • Motivation – Lets you specify whether to create file if it doesn’t exist, whether to overwrite or append, and so forth – Default behavior is to create file if not there and to overwrite if it is there
  33. 33. 35 Example 1: Write Strings to File public class WriteFile1 { public static void main(String[] args) throws Exception { Charset characterSet = Charset.defaultCharset(); Path path = Paths.get("output-file-1.txt"); List<String> lines = Arrays.asList("Line One", "Line Two", "Final Line"); Files.write(path, lines, characterSet); } } • Source of output-file-1.txt after execution Line One Line Two Final Line
  34. 34. 36 Example 2: File of 4-Letter Words (Shown Earlier) public static void main(String[] args) throws Exception { String inputFile = "enable1-word-list.txt"; String outputFile = "four-letter-words.txt"; int length = 4; List<String> words = Files.lines(Paths.get(inputFile)) .filter(word -> word.length() == length) .map(String::toUpperCase) .distinct() .sorted() .collect(Collectors.toList()); Files.write(Paths.get(outputFile), words, Charset.defaultCharset()); System.out.printf("Wrote %s words to %s.%n", words.size(), outputFile); } Resultant file AAHS AALS ABAS ABBA ABBE ABED ABET ABLE ABLY ...
  35. 35. For additional materials, please see http://www.coreservlets.com/. The Java tutorial section contains complete source code for all examples in this tutorial series, plus exercises and exercise solutions for each topic. coreservlets.com – custom onsite training Slides © 2016 Marty Hall, hall@coreservlets.com Faster and More Flexible File Writing
  36. 36. 38 Overview • You often need to format Strings – Files.write does not let you format the Strings as you insert them into the file • Need higher performance for very large files – You do not want to store everything in memory as List all at once – Buffered writing writes in blocks, and is faster for very large files • Shortcut method for getting BufferedWriter Writer w = Files.newBufferedWriter(somePath, someCharset); w.write(...); • You usually wrap PrintWriter around the Writer – Writer has only simple write method, but you can do PrintWriter out = new PrintWriter(yourBufferedWriter); then use the print, println, and especially printf methods of PrintWriter out.printf(...); • printf covered in lecture on Miscellaneous Utilities
  37. 37. 39 Example 1: BufferedWriter Only public class WriteFile2 { public static void main(String[] args) throws IOException { Charset characterSet = Charset.defaultCharset(); int numLines = 10; Path path = Paths.get("output-file-2.txt"); try (BufferedWriter writer = Files.newBufferedWriter(path, characterSet)) { for(int i=0; i<numLines; i++) { writer.write("Number is " + 100 * Math.random()); writer.newLine(); } } catch (IOException ioe) { System.err.printf("IOException: %s%n", ioe); } } }
  38. 38. 40 Example Output • Source of output-file-2.txt after execution Number is 81.4612317643326 Number is 52.38736740877531 Number is 71.76545597068544 Number is 59.85194979902197 Number is 17.25041924343985 Number is 86.77057757498325 Number is 30.570152355456926 Number is 61.490142746576424 Number is 35.59135386659128 Number is 89.43130746540979
  39. 39. 41 Example 2: PrintWriter public class WriteFile3 { public static void main(String[] args) throws IOException { Charset characterSet = Charset.defaultCharset(); int numLines = 10; Path path = Paths.get("output-file-3.txt"); try (PrintWriter out = new PrintWriter(Files.newBufferedWriter(path, characterSet))) { for(int i=0; i<numLines; i++) { out.printf("Number is %5.2f%n", 100 * Math.random()); } } catch (IOException ioe) { System.err.printf("IOException: %s%n", ioe); } } }
  40. 40. 42 Example Output • Source of output-file-3.txt after execution Number is 71.95 Number is 35.75 Number is 39.52 Number is 15.04 Number is 2.50 Number is 14.58 Number is 63.06 Number is 13.77 Number is 96.51 Number is 5.27
  41. 41. For additional materials, please see http://www.coreservlets.com/. The Java tutorial section contains complete source code for all examples in this tutorial series, plus exercises and exercise solutions for each topic. coreservlets.com – custom onsite training Slides © 2016 Marty Hall, hall@coreservlets.com File Reading: Second Variation
  42. 42. 44 Simple Script vs. Reusable Method • For simple script, do everything in main public static void main(String[] args) throws Exception { Files.lines(Paths.get("input-file")) .map(someFunction) .filter(someTest) .someOtherStreamOperation(...); } • For reusable methods, break processing into two pieces – First method takes a Stream<String> • This can be tested and reused independently of the file – Second method calls Files.lines and passes the result to the first method • But also has try/catch block to handle problems and to automatically close the file stream when done
  43. 43. 45 Why Split the Processing? • Why use two methods? – One that processes a Stream – One that uses Files.lines to build a Stream<String>, and passes it to first method • Benefits to splitting – Simpler testing. You can test first method with simple Stream created with Stream.of or someList.stream(). – More reusable. The first method can be used for Streams created from other sources. – More flexible. The first method can take a Stream<T>, where T is a generic type, and thus can be used for a variety of purposes, not just String processing. – Better error handling. Uses try/catch blocks instead of main throwing Exception. – Better memory usage. Stream is closed when done.
  44. 44. 46 Variation 2: General Approach public static void useStream(Stream<String> lines, ...) { lines.filter(...).map(...)...; } public static void useFile(String filename, ...) { try(Stream<String> lines = Files.lines(Paths.get(filename))) { SomeClass.useStream(lines, ...); } catch(IOException ioe) { System.err.println("Error reading file: " + ioe); } }
  45. 45. 47 Example: Printing All Palindromes public class FileUtils { public static void printAllPalindromes(Stream<String> words) { words.filter(StringUtils::isPalindrome) .forEach(System.out::println); } public static void printAllPalindromes(String filename) { try(Stream<String> words = Files.lines(Paths.get(filename))) { printAllPalindromes(words); } catch(IOException ioe) { System.err.println("Error reading file: " + ioe); } } }
  46. 46. 48 Example: Printing All Palindromes public static void main(String[] args) { String filename = "enable1-word-list.txt"; if (args.length > 0) { filename = args[0]; } testAllPalindromes(filename); } public static void testAllPalindromes(String filename) { List<String> testWords = Arrays.asList("bog", "bob", "dam", "dad"); System.out.printf("All palindromes in list %s:%n", testWords); FileUtils.printAllPalindromes(testWords.stream()); System.out.printf("All palindromes in file %s:%n", filename); FileUtils.printAllPalindromes(filename); } Output All palindromes in list [bog, bob, dam, dad]: bob dad All palindromes in file enable1-word-list.txt: aa aba ...
  47. 47. 49 Pros/Cons of First Attempt • First method: good – Can be tested with any Stream<String>, not only with file – Depending on operations used, could be rewritten to take a Stream<T> • Second method: good – Filename passed in, not hardcoded – Errors handled explicitly – Stream closed automatically • Second method: bad – Contains lots of tedious boilerplate code that must be repeated for each application • Hint for next variation: we had same problem when using Arrays.sort
  48. 48. For additional materials, please see http://www.coreservlets.com/. The Java tutorial section contains complete source code for all examples in this tutorial series, plus exercises and exercise solutions for each topic. coreservlets.com – custom onsite training Slides © 2016 Marty Hall, hall@coreservlets.com File Reading: Third Variation
  49. 49. 51 Third Variation: Overview • Previous variation – Method that takes Stream<String> or Stream<T> and performs general Stream ops • Good in every way – Method that takes filename, makes Stream<String>, passes it to first method • Good in some ways: the filename is passed in, errors are handled explicitly, and the Stream is always closed • But bad in some ways: repeats the Path creation, Stream creation, and error handling each time • New variation – Method that takes Stream<String> or Stream<T> and performs general Stream ops • Exactly the same as above – Method that takes filename, then calls static method with that filename and a lambda or method reference designating the above method • Requires us to create a new functional interface with abstract method that takes a Stream<String> and static method that does the boilerplate code
  50. 50. 52 Use Lambdas to Reuse Repeated Code • New interface: StreamProcessor – Abstract method takes a Stream<String> – Static method takes filename and instance of the interface (usually as a lambda), calls Files.lines, and passes result to the abstract method. Uses try/catch block and try-with-resources. • Stream-processing method – Same as before: processes Stream<String> • File-processing method – Calls static method with two arguments: • Filename • Lambda designating the method that should get the Stream<String> that will come from the file
  51. 51. 53 Variation 3: General Approach public static void useStream(Stream<String> lines) { lines.filter(...).map(...)...; } public static void useFile(String filename) { StreamProcessor.processFile(filename, SomeClass::useStream); }
  52. 52. 54 Variation 3: StreamProcessor Interface @FunctionalInterface public interface StreamProcessor { void processStream(Stream<String> strings); public static void processFile(String filename, StreamProcessor processor) { try(Stream<String> lines = Files.lines(Paths.get(filename))) { processor.processStream(lines); } catch(IOException ioe) { System.err.println("Error reading file: " + ioe); } } }
  53. 53. 55 Variation 1: Printing All Palindromes public static void main(String[] args) throws Exception { String inputFile = "enable1-word-list.txt"; Files.lines(Paths.get(inputFile)) .filter(StringUtils::isPalindrome) .forEach(System.out::println); }
  54. 54. 56 Variation 2: Printing All Palindromes public class FileUtils { public static void printAllPalindromes(Stream<String> words) { words.filter(StringUtils::isPalindrome) .forEach(System.out::println); } public static void printAllPalindromes(String filename) { try(Stream<String> words = Files.lines(Paths.get(filename))) { printAllPalindromes(words); } catch(IOException ioe) { System.err.println("Error reading file: " + ioe); } } }
  55. 55. 57 Variation 3: Printing All Palindromes public static void printAllPalindromes(Stream<String> words) { words.filter(StringUtils::isPalindrome) .forEach(System.out::println); } public static void printAllPalindromes(String filename) { StreamProcessor.processFile(filename, FileUtils::printAllPalindromes); }
  56. 56. 58 Printing All Palindromes (Test Code) public static void testAllPalindromes(String filename) { List<String> testWords = Arrays.asList("bog", "bob", "dam", "dad"); System.out.printf("All palindromes in list %s:%n", testWords); FileUtils.printAllPalindromes(testWords.stream()); System.out.printf("All palindromes in file %s:%n", filename); FileUtils.printAllPalindromes(filename); } Output All palindromes in list [bog, bob, dam, dad]: bob dad All palindromes in file enable1-word-list.txt: aa aba ...
  57. 57. 59 Printing N-Length Palindromes public static void printPalindromes(Stream<String> words, int length) { words.filter(word -> word.length() == length) .filter(StringUtils::isPalindrome) .forEach(System.out::println); } public static void printPalindromes(String filename, int length) { StreamProcessor.processFile(filename, lines -> printPalindromes(lines, length)); }
  58. 58. 60 Printing N-Length Palindromes (Test Code) public static void testPalindromes(String filename, int... lengths) { List<String> testWords = Arrays.asList("rob", "bob", "reed", "deed"); for(int length: lengths) { System.out.printf("%s-letter palindromes in list %s:%n", length, testWords); FileUtils.printPalindromes(testWords.stream(), length); System.out.printf("%s-letter palindromes in file %s:%n", length, filename); FileUtils.printPalindromes(filename, length); } } Output 3-letter palindromes in list [rob, bob, reed, deed]: bob 3-letter palindromes in file enable1-word-list.txt: aba ... 4-letter palindromes in list [rob, bob, reed, deed]: deed 4-letter palindromes in file enable1-word-list.txt: abba ...
  59. 59. For additional materials, please see http://www.coreservlets.com/. The Java tutorial section contains complete source code for all examples in this tutorial series, plus exercises and exercise solutions for each topic. coreservlets.com – custom onsite training Slides © 2016 Marty Hall, hall@coreservlets.com File Reading: Fourth Variation
  60. 60. 62 Overview • Previous variation – Method that takes Stream<String> or Stream<T> and performs general Stream ops • No value returned – Method that takes filename, then calls static method with that filename and a lambda or method reference designating the above method • This assumes that the above method has void return type • New variation – Method that takes Stream<String> or Stream<T> , performs general Stream ops, and returns a value – Method that takes filename, then returns a value that is the result of calling a static method with that filename and a lambda or method reference designating the above method • This method now returns whatever the above method would return
  61. 61. 63 Variation 4: General Approach public static SomeType getValueFromStream(Stream<String> lines) { return(lines.filter(...).map(...)...); } public static SomeType getValueFromStream (String filename) { return(StreamAnalyzer.analyzeFile(filename, SomeClass::getValueFromStream); }
  62. 62. 64 Variation 4: StreamAnalyzer Interface @FunctionalInterface public interface StreamAnalyzer<T> { T analyzeStream(Stream<String> strings); public static <T> T analyzeFile(String filename, StreamAnalyzer<T> analyzer) { try(Stream<String> lines = Files.lines(Paths.get(filename))) { return(analyzer.analyzeStream(lines)); } catch(IOException ioe) { System.err.println("Error reading file: " + ioe); return(null); } } }
  63. 63. 65 Example: First Palindrome public static String firstPalindrome(Stream<String> words) { return(words.filter(StringUtils::isPalindrome) .findFirst() .orElse(null)); } public static String firstPalindrome(String filename) { return(StreamAnalyzer.analyzeFile(filename, FileUtils::firstPalindrome)); }
  64. 64. 66 First Palindrome (Test Code) public static void testFirstPalindrome(String filename) { List<String> testWords = Arrays.asList("bog", "bob", "dam", "dad"); String firstPalindrome = FileUtils.firstPalindrome(testWords.stream()); System.out.printf("First palindrome in list %s is %s.%n", testWords, firstPalindrome); firstPalindrome = FileUtils.firstPalindrome(filename); System.out.printf("First palindrome in file %s is %s.%n", filename, firstPalindrome); } Output First palindrome in list [bog, bob, dam, dad] is bob. First palindrome in file enable1-word-list.txt is aa.
  65. 65. For additional materials, please see http://www.coreservlets.com/. The Java tutorial section contains complete source code for all examples in this tutorial series, plus exercises and exercise solutions for each topic. coreservlets.com – custom onsite training Slides © 2016 Marty Hall, hall@coreservlets.com Combining Predicates
  66. 66. 68 Overview: Common Pattern • Finding one value stream.filter(test1).filter(test2).filter(test3)... .findFirst().orElse(null); • Finding a List of values stream.filter(test1).filter(test2).filter(test3)... .collect(Collectors.toList()); • Problem in both cases – You do not know how many filter operations will be done • Solution – Use varargs • Allow any number of Predicates • Combine them into a single Predicate that can be used in a single call to filter • No need to limit this part to Stream<String>
  67. 67. 69 Predicate Combiner public class FileUtils { @SafeVarargs public static <T> Predicate<T> combinedPredicate (Predicate<T>... tests) { Predicate<T> result = e -> true; for(Predicate<T> test: tests) { result = result.and(test); } return(result); } ... } @SafeVarargs is difficult to understand. The issue is that it is not always safe to use varargs for generic types: the resultant array can have runtime type problems if you modify entries in it. But, if you only read the values and never modify them, varargs is perfectly safe. @SafeVarargs says “I am not doing anything dangerous, please suppress the compiler warnings”. For details, see http://docs.oracle.com/javase/8/docs/technotes/guides/language/non-reifiable-varargs.html
  68. 68. 70 Directly Applying Predicate Combiner (Main Methods) @SafeVarargs public static int letterCount(Stream<String> words, Predicate<String>... tests) { Predicate<String> combinedTest = FileUtils.combinedPredicate(tests); return(words.filter(combinedTest) .mapToInt(String::length) .sum()); } @SafeVarargs public static Integer letterCount(String filename, Predicate<String>... tests) { return(StreamAnalyzer.analyzeFile(filename, stream -> letterCount(stream, tests))); }
  69. 69. 71 Directly Applying Combiner (Test Code) public static void testLetterCount(String filename) { List<String> testWords = Arrays.asList("hi", "hello", "hola"); System.out.printf("In list %s:%n", testWords); int sum1 = FileUtils.letterCount(testWords.stream(), word -> word.contains("h"), word -> !word.contains("i")); printLetterCountResult(sum1, "contain h but not i"); System.out.printf("In file %s:%n", filename); int sum2 = FileUtils.letterCount(filename, StringUtils::isPalindrome); printLetterCountResult(sum2, "are palindromes"); int sum3 = FileUtils.letterCount(filename, word -> word.contains("q"), word -> !word.contains("qu")); printLetterCountResult(sum3, "contain q but not qu"); int sum4 = FileUtils.letterCount(filename, word -> true); printLetterCountResult(sum4, "are in English language"); } private static void printLetterCountResult(int sum, String message) { System.out.printf(" %,d total letters in words that %s.%n", sum, message);
  70. 70. 72 Directly Applying Combiner (Results) In list [hi, hello, hola]: 9 total letters in words that contain h but not i. In file enable1-word-list.txt: 417 total letters in words that are palindromes. 163 total letters in words that contain q but not qu. 1,570,550 total letters in words that are in English language.
  71. 71. 73 Indirectly Applying Combiner: firstMatch @SafeVarargs public static <T> T firstMatch(Stream<T> elements, Predicate<T>... tests) { Predicate<T> combinedTest = FileUtils.combinedPredicate(tests); return(elements.filter(combinedTest) .findFirst() .orElse(null)); } @SafeVarargs public static String firstMatch(String filename, Predicate<String>... tests) { return(StreamAnalyzer.analyzeFile(filename, stream -> firstMatch(stream, tests))); }
  72. 72. 74 Applying firstMatch public static void testFirstMatch(String filename) { List<Integer> testNums = Arrays.asList(1, 10, 2, 20, 3, 30); Integer match1 = FileUtils.firstMatch(testNums.stream(), n -> n > 2, n -> n < 10, n -> n % 2 == 1); System.out.printf("First word in list %s that is greater " + "than 2, less than 10, and odd is %s.%n", testNums, match1); String match2 = FileUtils.firstMatch(filename, word -> word.contains("q"), word -> !word.contains("qu")); System.out.printf("First word in file %s with q but " + "not u is %s.%n", filename, match2); } Output First word in list [1, 10, 2, 20, 3, 30] that is greater than 2, less than 10, and odd is 3. First word in file enable1-word-list.txt with q but not u is buqsha.
  73. 73. 75 Indirectly Applying Combiner: allMatches @SafeVarargs public static <T> List<T> allMatches(Stream<T> elements, Predicate<T>... tests) { Predicate<T> combinedTest = FileUtils.combinedPredicate(tests); return(elements.filter(combinedTest) .collect(Collectors.toList())); } @SafeVarargs public static List<String> allMatches(String filename, Predicate<String>... tests) { return(StreamAnalyzer.analyzeFile(filename, stream -> allMatches(stream, tests))); }
  74. 74. 76 Applying allMatches public static void testAllMatches(String filename) { List<Integer> testNums = Arrays.asList(2, 4, 6, 8, 10, 12); List<Integer> matches1 = FileUtils.allMatches(testNums.stream(), n -> n > 5, n -> n < 10); System.out.printf("All numbers in list %s that are " + "greater than 5 and less than 10: %s.%n", testNums, matches1); List<String> matches2 = FileUtils.allMatches(filename, word -> word.contains("q"), word -> !word.contains("qu")); System.out.printf("All words in file %s with q " + "but not u: %s.%n", filename, matches2); } Output All numbers in list [2, 4, 6, 8, 10, 12] that are greater than 5 and less than 10: [6, 8]. All words in file enable1-word-list.txt with q but not u: [buqsha, buqshas, faqir, ...].
  75. 75. For additional materials, please see http://www.coreservlets.com/. The Java tutorial section contains complete source code for all examples in this tutorial series, plus exercises and exercise solutions for each topic. coreservlets.com – custom onsite training Slides © 2016 Marty Hall, hall@coreservlets.com Exploring Folders
  76. 76. 78 Idea • Get all files in a folder – Files.list • Get all files in and below a folder – Files.walk • Get matching files in and below a folder – Files.find • With Files.walk above, you usually manually apply a Predicate by using filter, and thus only process certain files. • Files.find simplifies that: you also pass in a BiPredicate that takes a Path and a BasicFileAttributes object, and Files.find returns only the Paths that pass the test.
  77. 77. 79 Example 1: Printing Files in Folder public class FolderUtils public static void printAllPaths(Stream<Path> paths) { paths.forEach(System.out::println); } public static void printAllPathsInFolder(String folder) { try(Stream<Path> paths = Files.list(Paths.get(folder))) { printAllPaths(paths); } catch(IOException ioe) { System.err.println("IOException: " + ioe); } }
  78. 78. 80 Example 1: Printing Files in Folder (Continued) public static void printPaths(Stream<Path> paths, Predicate<Path> test) { paths.filter(test) .forEach(System.out::println); } public static void printPathsInFolder(String folder, Predicate<Path> test) { try(Stream<Path> paths = Files.list(Paths.get(folder))) { printPaths(paths, test); } catch(IOException ioe) { System.err.println("IOException: " + ioe); } }
  79. 79. 81 Printing Files in Folder: Test Code public static void listExamples() { System.out.println("All files in project root"); FolderUtils.printAllPathsInFolder("."); System.out.println("Text files in project root"); FolderUtils.printPathsInFolder(".", p -> p.toString().endsWith(".txt")); } All files in project root ..classpath ..project ..settings .coreservlets .dzone-programming- language-list.txt .enable1-word-list.txt .four-letter-words.txt .input-file.txt .output-file-1.txt .output-file-2.txt .output-file-3.txt .unixdict.txt Text files in project root .dzone-programming- language-list.txt .enable1-word-list.txt .four-letter-words.txt .input-file.txt .output-file-1.txt .output-file-2.txt .output-file-3.txt .unixdict.txt
  80. 80. 82 Example 2: Printing Files in Tree public static void printAllPathsInTree(String rootFolder) { try(Stream<Path> paths = Files.walk(Paths.get(rootFolder))) { printAllPaths(paths); } catch(IOException ioe) { System.err.println("IOException: " + ioe); } } public static void printPathsInTree(String rootFolder, Predicate<Path> test) { try(Stream<Path> paths = Files.walk(Paths.get(rootFolder))) { printPaths(paths, test); } catch(IOException ioe) { System.err.println("IOException: " + ioe); } } Files.walk also has options where you can limit the depth of the tree searched and where you can specify FileVisitOptions.
  81. 81. 83 Printing Files in Tree: Test Code public static void walkExamples() { System.out.println("All files under project root"); FolderUtils.printAllPathsInTree("."); System.out.println("Java files under project root"); FolderUtils.printPathsInTree(".", p -> p.toString().endsWith(".java")); } All files under project root . ..classpath ..project ..settings ..settingsorg.eclipse.jdt.core.prefs .coreservlets .coreservletsfolders .coreservletsfoldersFolderExamples.class .coreservletsfoldersFolderExamples.java .coreservletsfoldersFolderUtils.class .coreservletsfoldersFolderUtils.java .coreservletsjava7 .coreservletsjava7FileUtils.class .coreservletsjava7FileUtils.java ...
  82. 82. 84 Example 3: Printing Matching Files in Tree public static void findPathsInTree(String rootFolder, BiPredicate<Path,BasicFileAttributes> test) { try(Stream<Path> paths = Files.find(Paths.get(rootFolder), 10, test)) { printAllPaths(paths); } catch(IOException ioe) { System.err.println("IOException: " + ioe); } } In call above to Files.find, 10 is the maximum depth searched.
  83. 83. 85 Printing Matching Files in Tree: Test Code public static void findExamples() { System.out.println("Java files under project root"); FolderUtils.findPathsInTree(".", (path,attrs) -> path.toString().endsWith(".java")); System.out.println("Folders under project root"); FolderUtils.findPathsInTree(".", (path,attrs) -> attrs.isDirectory()); System.out.println("Large files under project root"); FolderUtils.findPathsInTree(".", (path,attrs) -> attrs.size() > 10000); } Java files under project root ... .coreservletsfoldersFolderExamples.java .coreservletsfoldersFolderUtils.java ... Folders under project root . ..settings .coreservlets ... Large files under project root .enable1-word-list.txt .four-letter-words.txt .unixdict.txt
  84. 84. For additional materials, please see http://www.coreservlets.com/. The Java tutorial section contains complete source code for all examples in this tutorial series, plus exercises and exercise solutions for each topic. coreservlets.com – custom onsite training Slides © 2016 Marty Hall, hall@coreservlets.com File I/O in Java 7
  85. 85. 87 Differences from Java 8 • Read lines into List instead of Stream List<String> lines = Files.readAllLines(somePath, someCharset); • Streams were not yet available in Java 7. Although returning a List is much less convenient than the current approach of returning a Stream, the Java 7 way was still substantially better than the Java 6 way. • Need way of reading large files – Files.newBufferedReader • Since Lists do not support lazy evaluation like Streams do, reading large files with Files.readAllLines is inefficient because entire file contents is in memory all at once, and because you read entire file even if the data you need is near the top.
  86. 86. 88 Simple File Reading in Java 7 • You can read all lines into List in one method call List<String> lines = Files.readAllLines(somePath, someCharset); • You can read all bytes into array in one method call byte[] fileArray = Files.readAllBytes(file); • Strings can easily be made from byte arrays: String fileData = new String(Files.readAllBytes(file)); • Minor caveats – You have to explicitly specify a Charset, even if you will use the default for the JDK • Charset cset1 = Charset.defaultCharset(); • Charset cset2 = Charset.forName("UTF-8"); – You still have to catch IOException
  87. 87. 89 Advantages of Java 8 Approach • Java 8 Stream<String> lines = Files.lines(path); • Java 7 List<String> lines = Files.readAllLines(path, charset); • Stream version far better – Massive memory savings • Does not store entire file contents in one huge list, but processes each line as you go along – Potentially much faster • You can stop partway through, and rest of file is never processed (due to lazy evaluation of Streams) – Many convenient filtering and transformation methods • You can chain these method calls together
  88. 88. 90 File Reading: Example public class ReadFile1 { public static void main(String[] args) throws Exception { String file = "input-file.txt"; Charset characterSet = Charset.defaultCharset(); Path path = Paths.get(file); List<String> lines = Files.readAllLines(path, characterSet); System.out.printf("Lines from %s: %s%n", file, lines); } }
  89. 89. 91 File Reading: Example Output • Source of input-file.txt First line Second line Third line Last line • Output of example code from previous slide Lines from input-file.txt: [First line, Second line, Third line, Last line]
  90. 90. 92 Some Simple Java 7 Utilities • FileUtils.getLines("filename") – Reading file into a List<String> • FileUtils.writeLines("filename", list) – Writing file from a List<String> public class FileUtils { public static List<String> getLines(String file) throws IOException { Path path = Paths.get(file); return(Files.readAllLines(path, Charset.defaultCharset())); } public static Path writeLines(String file, List<String> lines) throws IOException { Path path = Paths.get(file); return(Files.write(path, lines, Charset.defaultCharset())); } }
  91. 91. 93 Minor Variation of ReadFile1 (Using Utility Method) public class ReadFile1A { public static void main(String[] args) throws Exception { String file = "input-file.txt"; List<String> lines = FileUtils.getLines(file); System.out.printf("Lines from %s: %s%n", file, lines); } } • Output – Same as ReadFile1. E.g.: Lines from input-file.txt: [First line, Second line, Third line, Last line]
  92. 92. 94 Minor Variation of WriteFile1 (WriteFile1 Shown in File Writing Section) public class WriteFile1A { public static void main(String[] args) throws IOException { List<String> lines = Arrays.asList("Line One", "Line Two", "Final Line"); FileUtils.writeLines(“output-file-1.txt", lines); } } • Source of output-file-1.txt after execution Line One Line Two Final Line
  93. 93. 95 Lower-Level but More Flexible File Reading • You sometimes need only part of the file – Files.readAllLines reads everything, which is wasteful • Stores entire file in memory • No way to stop if you find the info you need early in file • Need higher performance for very large files – Buffered reading reads in blocks; faster for very large files • Shortcut method for getting BufferedReader – Files.newBufferedReader(somePath, someCharset) • BufferedReader has readLine method – Returns a String • Can chop the String into pieces using StringTokenizer (weak but simple) or String.split (much more powerful but requires knowledge of regular expressions) • Details on parsing in lectures on network programming
  94. 94. 96 Files.newBufferedReader Rarely Needed in Java 8 • When Files.newBufferedReader beneficial – When you don’t want to read full line as String • E.g., if reading single characters or arrays of characters • Files.lines preferable most other times – Files.lines returns a Stream, and Streams use lazy evaluation • Instead of entire file going into memory in advance as with Files.readAllLines, each line is processed as you go along • You can quit partway through, so if you find the data you need early on, you never have to read the rest of the file – As shown earlier, Streams have many convenient filtering and transformation methods, whereas using BufferedReader requires lower-level and less convenient techniques
  95. 95. 97 Example public class ReadFile2 { public static void main(String[] args) throws Exception { String file = "input-file.txt"; Charset characterSet = Charset.defaultCharset(); Path path = Paths.get(file); try(BufferedReader reader = Files.newBufferedReader(path, characterSet)) { System.out.printf("Lines from %s:%n", file); String line; while ((line = reader.readLine()) != null) { System.out.println(line); } } catch (IOException ioe) { System.err.printf("IOException: %s%n", ioe); } } } This approach using Files.newBufferedReader would be useful in Java 7 if the input file was large, so that all the lines are not stored in memory at once. In Java 8, you would just do Files.lines(path).forEach(System.out::println).
  96. 96. 98 Example Output • Source of input-file.txt First line Second line Third line Last line • Output of example code from previous slide Lines from input-file.txt: First line Second line Third line Last line
  97. 97. For additional materials, please see http://www.coreservlets.com/. The Java tutorial section contains complete source code for all examples in this tutorial series, plus exercises and exercise solutions for each topic. coreservlets.com – custom onsite training Slides © 2016 Marty Hall, hall@coreservlets.com Wrap-Up
  98. 98. 10 Summary: try/catch Blocks • finally blocks try { ... } catch(SomeExceptionType e) { ... } finally { ... } • multicatch try {… } catch(ExceptionType1 | ExceptionType2 e) { … } • try with resources try (SomeAutoCloseable var = ...) { ... } catch(SomeExceptionType e) { ... }
  99. 99. 10 Summary: File I/O in Java 8 • Use Path to refer to file location Path somePath = Paths.get("/path/to/file.txt"); • Read all lines into a Stream Stream<String> lines = Files.lines(somePath); – Can now use filter, map, distinct, sorted, findFirst, collect, etc. – You get benefits of lazy evaluation – Put Stream processing in separate method, consider making reusable utility • Write List or other Iterable into a file Files.write(somePath, someList, someCharset); • Get Writer for more flexible output Files.newBufferedWriter(somePath, someCharset) – Use write method, or, more often, wrap in PrintWriter and use printf • Explore and search folders and subfolders – Files.list, Files.walk, Files.find
  100. 100. 10 Summary: File I/O in Java 7 • Use Path to refer to file location Path somePath = Paths.get("/path/to/file.txt"); • Read all file lines into a List List<String> lines = Files.readAllLines(somePath, someCharset); • Write List or other Iterable into a file Files.write(somePath, someList, someCharset); • Minor utilities shown (not builtin) – List<String> lines = FileUtils.getLines("filename"); – FileUtils.writeLines("filename", someList); • Lower-level but more flexible readers & writers Files.newBufferedReader(somePath, someCharset) – To read, use readLine method Files.newBufferedWriter(somePath, someCharset) – To write, use write method or wrap in PrintWriter and use printf
  101. 101. For additional materials, please see http://www.coreservlets.com/. The Java tutorial section contains complete source code for all examples in this tutorial series, plus exercises and exercise solutions for each topic. coreservlets.com – custom onsite training Slides © 2016 Marty Hall, hall@coreservlets.com Questions? More info: http://courses.coreservlets.com/Course-Materials/java.html – General Java programming tutorial http://www.coreservlets.com/java-8-tutorial/ – Java 8 tutorial http://courses.coreservlets.com/java-training.html – Customized Java training courses, at public venues or onsite at your organization http://coreservlets.com/ – JSF 2, PrimeFaces, Java 7 or 8, Ajax, jQuery, Hadoop, RESTful Web Services, Android, HTML5, Spring, Hibernate, Servlets, JSP, GWT, and other Java EE training Many additional free tutorials at coreservlets.com (JSF, Android, Ajax, Hadoop, and lots more)

×