Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Iterators Must Go

                             Andrei Alexandrescu




                                                  ...
This Talk

           The STL
         •
         • Iterators
         • Range-based design
         • Conclusions




   ...
What is the STL?




                                                3 / 52
c 2009 Andrei Alexandrescu
Yeah, what is the STL?

         • A good library of algorithms and data structures.




                                 ...
Yeah, what is the STL?

         • A ( good|bad) library of algorithms and data structures.




                          ...
Yeah, what is the STL?

         • A ( good|bad|ugly) library of algorithms and data
           structures.




          ...
Yeah, what is the STL?

         • A ( good|bad|ugly) library of algorithms and data
           structures.

         • it...
Yeah, what is the STL?

         • A ( good|bad|ugly) library of algorithms and data
           structures.

         • it...
What the STL is

         • More than the answer, the question is important in the
           STL

         • “What would ...
STL is nonintuitive




                             6 / 52
c 2009 Andrei Alexandrescu
STL is nonintuitive

         • The same way the theory of relativity is nonintuitive

         • The same way complex num...
Nonintuitive

         • “I want to design the most general algorithms.”

         • “Sure. What you obviously need is som...
Nonintuitive

       • “I want to design the most general algorithms.”

       • “Sure. What you obviously need is somethi...
Fundamental vs. Incidental in STL

         •     Algorithms defined for the narrowest interface
             possible

   ...
Fundamental vs. Incidental in STL

         • F: Algorithms defined for the narrowest interface
           possible

      ...
Fundamental vs. Incidental in STL

         • F: Algorithms defined for the narrowest interface
           possible

      ...
Fundamental vs. Incidental in STL

         • F: Algorithms defined for the narrowest interface
           possible

      ...
Fundamental vs. Incidental in STL

         • F: Algorithms defined for the narrowest interface
           possible

      ...
STL: The Good

         • Asked the right question

         • General

         • Efficient

         • Reasonably extensi...
STL: The Bad

         • Poor lambda functions support
                     Not an STL problem
                     High o...
STL: The Ugly

         • Attempts at for_each et al. didn’t help

         • Integration with streams is tenuous

       ...
STL: The Ugly

         • Attempts at for_each et al. didn’t help

         • Integration with streams is tenuous

       ...
What’s the Deal with Iterators?




                                                     12 / 52
c 2009 Andrei Alexandrescu
Iterators Rock

         • They broker interaction between containers and
           algorithms

                         ...
Warning Sign #1

         • C++ Users Journal around 2001 ran an ad campaign
                     “Submit an article to CU...
Warning Sign #1

         • C++ Users Journal around 2001 ran an ad campaign
                     “Submit an article to CU...
Warning Sign #2


   File copy circa 1975:


       #include <stdio.h>
       int main() {
          int c;
          whil...
Warning Sign #2
   Fast forward 20 years, and. . .
       #include <iostream>
       #include <algorithm>
       #include ...
(forgot the try/catch around main)




                                                                  17 / 52
c 2009 An...
Something, somewhere, went terribly wrong.




                                                                 18 / 52
c ...
Warning Sign #3

         • Iterators are brutally hard to define

         • Bulky implementations and many gotchas

     ...
Warning Sign #4

         • Iterators use pointer syntax & semantics

         • Integration with pointers for the win/los...
Final nail in the coffin

         • All iterator primitives are fundamentally unsafe

         • For most iterator types, ...
Ranges




                                      22 / 52
c 2009 Andrei Alexandrescu
Enter Ranges

         • To partly avoid these inconveniences, ranges have
           been defined

         • A range is a...
They made an interesting step in a good direction.




                                                                   ...
Things must be taken much further.




                                                                  25 / 52
c 2009 An...
Look, Ma, no iterators!

         • How about defining ranges instead of iterators as the
           primitive structure fo...
Defining Ranges

         • All of <algorithm> should be implementable with
           ranges, and other algorithms as well...
Input/Forward Ranges




       template<class T> struct InputRange {
          bool empty() const;
          void popFron...
Verifiable?

       template<class T> struct ContigRange {
          bool empty() const { return b >= e; }
          void p...
Find


       // Original version per STL
       template<class It, class T>
       It find(It b, It e, T value) {
       ...
Design Question

         • What should find with ranges look like?

                1. Return a range of one element (if ...
Design Question

         • What should find with ranges look like?

                1. Return a range of one element (if ...
Design Question

         • What should find with ranges look like?

                1. Return a range of one element (if ...
Find


       // Using ranges
       template<class R, class T>
       R find(R r, T value) {
          for (; !r.empty();...
Elegant Specification




       template<class R, class T>
       R find(R r, T value);



   “Reduces the range r from le...
Bidirectional Ranges




       template<class T> struct BidirRange {
          bool empty() const;
          void popFron...
Reverse Iteration

       template<class R> struct Retro {
          bool empty() const { return r.empty(); }
          vo...
How about find_end?


       template<class R, class T>
       R find_end(R r, T value) {
          return retro(find(retr...
find_end with iterators sucks

       // find_end using reverse_iterator
       template<class It, class T>
       It find...
More composition opportunities

             Chain: chain several ranges together
         •
                     Elements...
How about three-iterators functions?


       template<class It1, It2>
       void copy(It1 begin, It1 end, It2 to);
     ...
“Where there’s hardship, there’s opportunity.”




                                               – I. Meade Etop




    ...
3-legged algos ⇒ mixed-range algos



       template<class R1, class R2>
       R2 copy(R1 r1, R2 r2);

         • Spec: ...
3-legged algos ⇒ mixed-range algos

       template<class R1, class R2>
       void partial_sort(R1 r1, R2 r2);

         ...
But wait, there’s more



       vector<double> v1, v2;
       deque<double> d;
       partial_sort(v1, chain(v2, d));
   ...
But wait, there’s even more



       vector<double> vd;
       vector<string> vs;
       // sort the two in lockstep
    ...
Output ranges


         • Freedom from pointer syntax allows supporting different
           types


       struct OutRan...
Back to copying stdin to stdout


       #include <...>
       int main() {
          copy(istream_range<string>(cin),
   ...
Infinite ranges

         • Notion of infinity becomes interesting with ranges

         • Generators, random numbers, serie...
has_size

         • Whether a range has an efficiently computed size is
           another independent trait

            ...
A Twist

         • Can <algorithm> be redone with ranges?

         • D’s stdlib offers a superset of <algorithm> in modu...
Conclusion

         • Ranges are a superior abstraction

         • Better checking abilities (not perfect still)

      ...
Announcement




           Please note: “The D Programming
          Language” soon to appear on Safari’s
               ...
Andrei Alexandrescu, Ph %@! D




     Please note: Andrei will soon be offering
     training and consulting services. Co...
Nächste SlideShare
Wird geladen in …5
×

iterators-must-go

iterators-must-go

  1. 1. Iterators Must Go Andrei Alexandrescu 1 / 52 c 2009 Andrei Alexandrescu
  2. 2. This Talk The STL • • Iterators • Range-based design • Conclusions 2 / 52 c 2009 Andrei Alexandrescu
  3. 3. What is the STL? 3 / 52 c 2009 Andrei Alexandrescu
  4. 4. Yeah, what is the STL? • A good library of algorithms and data structures. 4 / 52 c 2009 Andrei Alexandrescu
  5. 5. Yeah, what is the STL? • A ( good|bad) library of algorithms and data structures. 4 / 52 c 2009 Andrei Alexandrescu
  6. 6. Yeah, what is the STL? • A ( good|bad|ugly) library of algorithms and data structures. 4 / 52 c 2009 Andrei Alexandrescu
  7. 7. Yeah, what is the STL? • A ( good|bad|ugly) library of algorithms and data structures. • iterators = gcd(containers, algorithms); 4 / 52 c 2009 Andrei Alexandrescu
  8. 8. Yeah, what is the STL? • A ( good|bad|ugly) library of algorithms and data structures. • iterators = gcd(containers, algorithms); • Scrumptious Template Lore • Snout to Tail Length 4 / 52 c 2009 Andrei Alexandrescu
  9. 9. What the STL is • More than the answer, the question is important in the STL • “What would the most general implementations of fundamental containers and algorithms look like?” • Everything else is aftermath • Most importantly: STL is one answer, not the answer 5 / 52 c 2009 Andrei Alexandrescu
  10. 10. STL is nonintuitive 6 / 52 c 2009 Andrei Alexandrescu
  11. 11. STL is nonintuitive • The same way the theory of relativity is nonintuitive • The same way complex numbers are nonintuitive 6 / 52 c 2009 Andrei Alexandrescu
  12. 12. Nonintuitive • “I want to design the most general algorithms.” • “Sure. What you obviously need is something called iterators. Five of ’em, to be precise.” 7 / 52 c 2009 Andrei Alexandrescu
  13. 13. Nonintuitive • “I want to design the most general algorithms.” • “Sure. What you obviously need is something called iterators. Five of ’em, to be precise.” • Evidence: No language has supported the STL “by chance.” ◦ In spite of relentless “feature wars” ◦ C++, D the only ones ◦ Both were actively designed to support the STL • Consequence: STL very hard to understand from outside C++ or D 7 / 52 c 2009 Andrei Alexandrescu
  14. 14. Fundamental vs. Incidental in STL • Algorithms defined for the narrowest interface possible • Broad iterator categories as required by algorithms • Choice of iterator primitives • Syntax of iterator primitives 8 / 52 c 2009 Andrei Alexandrescu
  15. 15. Fundamental vs. Incidental in STL • F: Algorithms defined for the narrowest interface possible • Broad iterator categories as required by algorithms • Choice of iterator primitives • Syntax of iterator primitives 8 / 52 c 2009 Andrei Alexandrescu
  16. 16. Fundamental vs. Incidental in STL • F: Algorithms defined for the narrowest interface possible • F: Broad iterator categories as required by algorithms • Choice of iterator primitives • Syntax of iterator primitives 8 / 52 c 2009 Andrei Alexandrescu
  17. 17. Fundamental vs. Incidental in STL • F: Algorithms defined for the narrowest interface possible • F: Broad iterator categories as required by algorithms • I: Choice of iterator primitives • Syntax of iterator primitives 8 / 52 c 2009 Andrei Alexandrescu
  18. 18. Fundamental vs. Incidental in STL • F: Algorithms defined for the narrowest interface possible • F: Broad iterator categories as required by algorithms • I: Choice of iterator primitives • I: Syntax of iterator primitives 8 / 52 c 2009 Andrei Alexandrescu
  19. 19. STL: The Good • Asked the right question • General • Efficient • Reasonably extensible • Integrated with built-in types 9 / 52 c 2009 Andrei Alexandrescu
  20. 20. STL: The Bad • Poor lambda functions support Not an STL problem High opportunity cost • Some containers cannot be supported E.g. sentinel-terminated containers E.g. containers with distributed storage • Some iteration methods cannot be supported 10 / 52 c 2009 Andrei Alexandrescu
  21. 21. STL: The Ugly • Attempts at for_each et al. didn’t help • Integration with streams is tenuous allocator • One word: 11 / 52 c 2009 Andrei Alexandrescu
  22. 22. STL: The Ugly • Attempts at for_each et al. didn’t help • Integration with streams is tenuous allocator • One word: • Iterators suck ◦ Verbose ◦ Unsafe ◦ Poor Interface 11 / 52 c 2009 Andrei Alexandrescu
  23. 23. What’s the Deal with Iterators? 12 / 52 c 2009 Andrei Alexandrescu
  24. 24. Iterators Rock • They broker interaction between containers and algorithms m + n implementations instead of • “Strength reduction:” m·n • Extensible: there’s been a flurry of iterators ever since STL saw the light of day 13 / 52 c 2009 Andrei Alexandrescu
  25. 25. Warning Sign #1 • C++ Users Journal around 2001 ran an ad campaign “Submit an article to CUJ!” “No need to be an English major! Just start your editor!” “We’re interested in security, networking, C++ techniques, and more!” 14 / 52 c 2009 Andrei Alexandrescu
  26. 26. Warning Sign #1 • C++ Users Journal around 2001 ran an ad campaign “Submit an article to CUJ!” “No need to be an English major! Just start your editor!” “We’re interested in security, networking, C++ techniques, and more!” “Please note: Not interested in yet another iterator” • How many of those published iterators survived? 14 / 52 c 2009 Andrei Alexandrescu
  27. 27. Warning Sign #2 File copy circa 1975: #include <stdio.h> int main() { int c; while ((c = getchar()) != EOF) putchar(c); return errno != 0; } 15 / 52 c 2009 Andrei Alexandrescu
  28. 28. Warning Sign #2 Fast forward 20 years, and. . . #include <iostream> #include <algorithm> #include <iterator> #include <string> using namespace std; int main() { copy(istream_iterator<string>(cin), istream_iterator<string>(), ostream_iterator<string>(cout,quot;nquot;)); } 16 / 52 c 2009 Andrei Alexandrescu
  29. 29. (forgot the try/catch around main) 17 / 52 c 2009 Andrei Alexandrescu
  30. 30. Something, somewhere, went terribly wrong. 18 / 52 c 2009 Andrei Alexandrescu
  31. 31. Warning Sign #3 • Iterators are brutally hard to define • Bulky implementations and many gotchas • Boost includes an entire library that helps defining iterators • The essential primitives are like. . . three? ◦ At end ◦ Access ◦ Bump 19 / 52 c 2009 Andrei Alexandrescu
  32. 32. Warning Sign #4 • Iterators use pointer syntax & semantics • Integration with pointers for the win/loss • However, this limits methods of iteration ◦ Can’t walk a tree in depth, need ++ with a parameter ◦ Output iterators can only accept one type: ostream_iterator must be parameterized with each specific type to output, although they all go to the same place 20 / 52 c 2009 Andrei Alexandrescu
  33. 33. Final nail in the coffin • All iterator primitives are fundamentally unsafe • For most iterator types, given an iterator ◦ Can’t say whether it can be compared ◦ Can’t say whether it can be incremented ◦ Can’t say whether it can be dereferenced • Safe iterators can and have been written ◦ At a high size+speed cost ◦ Mostly a good argument that the design hasn’t been cut quite right 21 / 52 c 2009 Andrei Alexandrescu
  34. 34. Ranges 22 / 52 c 2009 Andrei Alexandrescu
  35. 35. Enter Ranges • To partly avoid these inconveniences, ranges have been defined • A range is a pair of begin/end iterators packed together • As such, a range has higher-level checkable invariants • Boost and Adobe libraries defined ranges 23 / 52 c 2009 Andrei Alexandrescu
  36. 36. They made an interesting step in a good direction. 24 / 52 c 2009 Andrei Alexandrescu
  37. 37. Things must be taken much further. 25 / 52 c 2009 Andrei Alexandrescu
  38. 38. Look, Ma, no iterators! • How about defining ranges instead of iterators as the primitive structure for iteration? • Ranges should define primitive operations that do not rely on iteration • There would be no more iterators, only ranges • What primitives should ranges support? Remember, begin/end are not an option If people squirrel away individual iterators, we’re back to square one 26 / 52 c 2009 Andrei Alexandrescu
  39. 39. Defining Ranges • All of <algorithm> should be implementable with ranges, and other algorithms as well • Range primitives should be checkable at low cost • Yet, ranges should not be less efficient than iterators 27 / 52 c 2009 Andrei Alexandrescu
  40. 40. Input/Forward Ranges template<class T> struct InputRange { bool empty() const; void popFront(); T& front() const; }; 28 / 52 c 2009 Andrei Alexandrescu
  41. 41. Verifiable? template<class T> struct ContigRange { bool empty() const { return b >= e; } void popFront() { assert(!empty()); ++b; } T& front() const { assert(!empty()); return *b; } private: T *b, *e; }; 29 / 52 c 2009 Andrei Alexandrescu
  42. 42. Find // Original version per STL template<class It, class T> It find(It b, It e, T value) { for (; b != e; ++b) if (value == *b) break; return b; } ... auto i = find(v.begin(), v.end(), value); if (i != v.end()) ... 30 / 52 c 2009 Andrei Alexandrescu
  43. 43. Design Question • What should find with ranges look like? 1. Return a range of one element (if found) or zero elements (if not)? 2. Return the range before the found element? 3. Return the range after the found element? 31 / 52 c 2009 Andrei Alexandrescu
  44. 44. Design Question • What should find with ranges look like? 1. Return a range of one element (if found) or zero elements (if not)? 2. Return the range before the found element? 3. Return the range after the found element? • Correct answer: return the range starting with the found element (if any), empty if not found 31 / 52 c 2009 Andrei Alexandrescu
  45. 45. Design Question • What should find with ranges look like? 1. Return a range of one element (if found) or zero elements (if not)? 2. Return the range before the found element? 3. Return the range after the found element? • Correct answer: return the range starting with the found element (if any), empty if not found Why? 31 / 52 c 2009 Andrei Alexandrescu
  46. 46. Find // Using ranges template<class R, class T> R find(R r, T value) { for (; !r.empty(); r.popFront()) if (value == r.front()) break; return r; } ... auto r = find(v.all(), value); if (!r.empty()) ... 32 / 52 c 2009 Andrei Alexandrescu
  47. 47. Elegant Specification template<class R, class T> R find(R r, T value); “Reduces the range r from left until its front is equal with value or r is exhausted.” 33 / 52 c 2009 Andrei Alexandrescu
  48. 48. Bidirectional Ranges template<class T> struct BidirRange { bool empty() const; void popFront(); void popBack(); T& front() const; T& back() const; }; 34 / 52 c 2009 Andrei Alexandrescu
  49. 49. Reverse Iteration template<class R> struct Retro { bool empty() const { return r.empty(); } void popFront() { return r.popBack(); } void popBack() { return r.popFront(); } E<R>::Type& front() const { return r.back(); } E<R>::Type& back() const { return r.front(); } R r; }; template<class R> Retro<R> retro(R r) { return Retro<R>(r); } template<class R> R retro(Retro<R> r) { return r.r; // klever } 35 / 52 c 2009 Andrei Alexandrescu
  50. 50. How about find_end? template<class R, class T> R find_end(R r, T value) { return retro(find(retro(r)); } • No more need for rbegin, rend • Containers define all which returns a range retro(cont.all()) • To iterate backwards: 36 / 52 c 2009 Andrei Alexandrescu
  51. 51. find_end with iterators sucks // find_end using reverse_iterator template<class It, class T> It find_end(It b, It e, T value) { It r = find(reverse_iterator<It>(e), reverse_iterator<It>(b), value).base(); return r == b ? e : --r; } • Crushing advantage of ranges: much terser code • Easy composition because only one object needs to be composed, not two in sync 37 / 52 c 2009 Andrei Alexandrescu
  52. 52. More composition opportunities Chain: chain several ranges together • Elements are not copied! Category of range is the weakest of all ranges Zip: span several ranges in lockstep • Needs Tuple Stride: span a range several steps at once • Iterators can’t implement it! Radial: span a range in increasing distance from its • middle (or any other point) 38 / 52 c 2009 Andrei Alexandrescu
  53. 53. How about three-iterators functions? template<class It1, It2> void copy(It1 begin, It1 end, It2 to); template<class It> void partial_sort(It begin, It mid, It end); template<class It> void rotate(It begin, It mid, It end); template<class It, class Pr> It partition(It begin, It end, Pr pred); template<class It, class Pr> It inplace_merge(It begin, It mid, It end); 39 / 52 c 2009 Andrei Alexandrescu
  54. 54. “Where there’s hardship, there’s opportunity.” – I. Meade Etop 40 / 52 c 2009 Andrei Alexandrescu
  55. 55. 3-legged algos ⇒ mixed-range algos template<class R1, class R2> R2 copy(R1 r1, R2 r2); • Spec: Copy r1 to r2, returns untouched portion of r2 vector<float> v; list<int> s; deque<double> d; copy(chain(v, s), d); 41 / 52 c 2009 Andrei Alexandrescu
  56. 56. 3-legged algos ⇒ mixed-range algos template<class R1, class R2> void partial_sort(R1 r1, R2 r2); • Spec: Partially sort the concatenation of r1 and r2 such that the smallest elements end up in r1 • You can take a vector and a deque and put the smallest elements of both in the array! vector<double> v; deque<double> d; partial_sort(v, d); 42 / 52 c 2009 Andrei Alexandrescu
  57. 57. But wait, there’s more vector<double> v1, v2; deque<double> d; partial_sort(v1, chain(v2, d)); sort(chain(v1, v2, d)); • Algorithms can now operate on any mixture of ranges seamlessly without any extra effort • Try that with iterators! 43 / 52 c 2009 Andrei Alexandrescu
  58. 58. But wait, there’s even more vector<double> vd; vector<string> vs; // sort the two in lockstep sort(zip(vs, vd)); • Range combinators allow myriads of new uses • Possible in theory with iterators, but the syntax would explode (again) 44 / 52 c 2009 Andrei Alexandrescu
  59. 59. Output ranges • Freedom from pointer syntax allows supporting different types struct OutRange { typedef Typelist<int, double, string> Types; void put(int); void put(double); void put(string); } 45 / 52 c 2009 Andrei Alexandrescu
  60. 60. Back to copying stdin to stdout #include <...> int main() { copy(istream_range<string>(cin), ostream_range(cout, quot;nquot;)); } • Finally, a step forward: a one-liner that fits on one line1 ostream_range does not need to specify string • 1 slide limitations notwithstanding 46 / 52 c 2009 Andrei Alexandrescu
  61. 61. Infinite ranges • Notion of infinity becomes interesting with ranges • Generators, random numbers, series, . . . are infinite ranges • Infinity is a trait distinct from the five classic categories; any kind of range may be infinite • Even a random-access range may be infinite! • Statically knowing about infinity helps algorithms 47 / 52 c 2009 Andrei Alexandrescu
  62. 62. has_size • Whether a range has an efficiently computed size is another independent trait list.size, endless debate on) • (Index entry: • Even an input range can have a known size, e.g. take(100, rndgen) which takes 100 random numbers ◦ take(100, r) has length 100 if r is infinite ◦ length min(100, r.size()) if r has known length ◦ unknown length if r is finite with unknown length 48 / 52 c 2009 Andrei Alexandrescu
  63. 63. A Twist • Can <algorithm> be redone with ranges? • D’s stdlib offers a superset of <algorithm> in modules std.algorithm and std.range (google for them) • Ranges pervade D: algorithms, lazy evaluation, random numbers, higher-order functions, foreach statement. . . • Some opportunities not yet tapped—e.g. filters (input/output ranges) • Check “The Case for D” in Doctor Dobb’s Journal, coming soon 49 / 52 c 2009 Andrei Alexandrescu
  64. 64. Conclusion • Ranges are a superior abstraction • Better checking abilities (not perfect still) • Easy composition • Range-based design offers much more than a port of iterator-based functions to ranges • Exciting development taking STL one step further 50 / 52 c 2009 Andrei Alexandrescu
  65. 65. Announcement Please note: “The D Programming Language” soon to appear on Safari’s Rough Cuts. 51 / 52 c 2009 Andrei Alexandrescu
  66. 66. Andrei Alexandrescu, Ph %@! D Please note: Andrei will soon be offering training and consulting services. Contact him for details. 52 / 52 c 2009 Andrei Alexandrescu

×