Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Type Profiler: Ambitious Type Inference for Ruby 3

2.792 Aufrufe

Veröffentlicht am

@ RubyKaigi Takeout 2020

Veröffentlicht in: Technologie
  • Loggen Sie sich ein, um Kommentare anzuzeigen.

Type Profiler: Ambitious Type Inference for Ruby 3

  1. 1. Type Profiler: Ambitious Type Inference for Ruby 3 Yusuke Endoh (@mametter) RubyKaigi Takeout 2020 1
  2. 2. Goals of Ruby 3’s Static Analysis •Make Ruby programming easier • Bug detection before execution • Completion and document in IDE … with no type annotation in code! (Ruby 3 Type Challenge) 2 42 + "str" Is this a bug? 42.ti| Do you mean: 42.times {|i| | }
  3. 3. Ruby 3 will provide three “type” items 3 # app.rbs def inc: (Integer) -> Integer ① Type Signature Format (RBS) # app.rb def inc(n) n+1 end Ruby code ② Type Inference (ruby-type-profiler) ③ Type Check (Steep, Sorbet, RDL, …)
  4. 4. Ruby 3 Development Experience 4 gem lib.rb lib.rbs app app.rb app.rbs 42.ti| 42 + "str" Is this a bug? Do you mean: 42.times {|i| | } ② Type Inference • Generate an RBS prototype • Simply check the code ③ Type Checker • Verify .rb and .rbs • Serve as LSP server You can also write RBS manually if you want Type-guided linter Dynamic type checker More dedicated dev. environments Monitor and Harness Ruby code in run-time Code formatter by leveraging types Rubymine, Tabnine, and other development tools may use type informationhttps://github.com/pocke/rubocop-typedhttps://github.com/ruby/rbs/blob/master/ lib/rbs/test/type_check.rb RBS may inspire other dreams Today we talk about ①RBS
  5. 5. Agenda ➔Type Profiler: Type Inference for Ruby 3 •Demo •How to use Type Profiler •Future plan 5
  6. 6. What is Type Profiler Abstract ("type-level") interpreter of Ruby Generates an RBS prototype by gathering what types a method accepts and returns 6 def foo(n) n.to_s end foo(42) Integer (not 42) String (not "42") def foo: (Integer) -> String
  7. 7. Why is the name "Type Profiler"? •Just for a historical reason • The initial version was runtime analysis (profiling) • Now it is a bit confusing with a normal profiler •Invite suggestions for the name • Should it start with "S"? (Steep, Sorbet, …) 7
  8. 8. Difference from traditional type system •Traditional type systems use intra-procedural (per-method) analysis • Can't handle unannotated method parameter well • … especially when there are many classes that respond to foo 8 def f(x) x.foo end What type is "x"? class Foo def foo; ...; end end class Bar def foo; ...; end end class Baz def foo; ...; end end ...
  9. 9. Difference from traditional type system •Solutions 1. Write type annotation → Avoid this 2. Infer type based on its usage → Too strict or too conservative • foo must be unique • or, structural type inference? 3. Use "inter-procedural" analysis • Pros: More powerful analysis • Cons: Slow and hard to control (Challenging) 9 def f(x: Foo) x.foo end def f(x) x.foo end f(Foo.new) "x" is a Foo! def f(x) x.foo end "x" is an object that responds to foo that accepts no argument and returns the same type of a return value of this method
  10. 10. There are many, many topics (but omit) Theoretical issues • Recursion and closures[RubyKaigi 2019] • Type-changing variable assignment • Container types and destructive operations[Osaka Ruby Kaigi 2019] • Flow-sensitive analysis[EuRuKo 2019] • Context-insensitive analysis[PPL 2019] • Aid of escape analysis • Cumbersome "untyped" type • Meta-programming features • etc, etc. Practical issues • Trade-off between precision and performance [Nagoya Ruby Kaigi 2019] • Tuple-like and sequential array • Method-local container type [Osaka Ruby Kaigi 2019] • Diagnosis features[Ruby 3 Summit] • Unreachable method analysis • Limitation of byte code • Super-rich Ruby features • Too complex Ruby features • etc, etc. 10
  11. 11. Agenda •Type Profiler: Type Inference for Ruby 3 ➔Demo • Simple case • Real-world program case • Library case •How to use •Future Plan 11
  12. 12. Demo 1: ao.rb •Simple case • A 3D renderer (~300 LoC) • Written by Hideki Miura • Original version was written by Syoyo Fujita https://code.google.com/archive/p/aobench/ •Analysis time < 1 sec. 12
  13. 13. Demo 1: ao.rb 13 class Vec attr_accessor x : Float attr_accessor y : Float attr_accessor z : Float def initialize : (Float, Float, Float) -> Float def vadd : (Vec) -> Vec def vsub : (Vec) -> Vec def vcross : (Vec) -> Vec def vdot : (Vec) -> Float def vlength : -> Float def vnormalize : -> Vec end NEW! attr_accessor Formerly, "def x=" and "def x"
  14. 14. Demo 1: ao.rb 14 class Sphere attr_reader center : Vec attr_reader radius : Float def initialize : (Vec, Float) -> Float def intersect : (Ray, Isect) -> Vec? end class Isect attr_accessor t : Float attr_accessor hit : bool attr_accessor pl : Vec attr_accessor n : Vec def initialize : -> Vec end NEW! optional type Formerly, "Vec | NilClass" NEW! bool type Formerly, "TrueClass | FalseClass"
  15. 15. Demo 2: Goodcheck • Real-world program case • A customizable linter for Ruby (~2000 LoC) • It has "hand-written" RBS • Analysis time < 30 sec. • Note: It requires many libraries which have no RBS • activesupport, concurrent-ruby, cgi, optparse, etc. • Type Profiler analyzed not only Goodcheck but also them • In future, we expect they have own RBS • Type Profiler can use RBS instead of the code itself 15
  16. 16. Manually reformatted to make comparison easy Demo 2: Goodcheck 16 class Goodcheck::Trigger attr_reader patterns : Array[(Goodcheck::Pattern::Literal | Goodcheck::Pattern::Regexp | Goodcheck::Pattern::Token)?] attr_reader globs : Array[Goodcheck::Glob?] attr_reader passes : Array[Array[untyped]] attr_reader fails : Array[Array[untyped]] attr_reader negated : bool ... end class Goodcheck::Trigger attr_reader patterns : Array[pattern] attr_reader globs: Array[Glob] attr_reader passes: Array[String] attr_reader fails: Array[String] attr_reader negated: bool ... end type Goodcheck::pattern = Pattern::Literal|Pattern::Regexp|Pattern::Token RBS inferred by Type Profiler Hand-written by Soutaro Wrong guess Type alias Not so bad? Extra optionalRedundant namescope
  17. 17. Demo 2: Goodcheck 17 class Goodcheck::Pattern::Token attr_reader source : untyped attr_reader case_sensitive : true attr_reader variables : {} def initialize : (source: untyped, variables: {}, case_sensitive: true) -> true @regexp : Regexp def regexp : -> Regexp def test_variables : (untyped) -> bool def self.expand : (untyped, untyped, ?depth: Integer) -> Array[Regexp] def self.regexp_for_type : (name: untyped, type: :__, scanner: untyped) -> Regexp? def self.compile_tokens : (untyped, {}, case_sensitive: true) -> Regexp @@TYPES : {} end class Goodcheck::Pattern::Token attr_reader source: String attr_reader case_sensitive: bool attr_reader variables: Hash[Symbol, VarPattern] def initialize: (source: String, variables: Hash[Symbol, VarPattern], case_sensitive: bool) -> void def regexp: -> ::Regexp def self.expand: (String, String, ?depth: Integer) -> Array[::Regexp] def self.regexp_for_type: (name: Symbol, type: Symbol, scanner: StringScanner) -> ::Regexp def self.compile_tokens: (String source, Hash[Symbol, VarPattern] variables, case_sensitive: bool) -> void @@TYPES: Hash[Symbol, ^(String) -> ::Regexp] end RBS inferred by Type Profiler Hand-written by Soutaro Too specfic Failed to track the elements C lib constant (Lack of RBS) void is intended I think not so bad
  18. 18. Demo 3: diff-lcs •Real-world library case Famous algorithmic library •Hand-written entry point •Analysis time < 1 sec. 18 https://bestgems.org/ require_relative "lib/diff/lcs" class T; end Diff::LCS.diff([T.new]+[T.new], [T.new]+[T.new]) {}
  19. 19. Demo: diff-lcs 19 class Diff::LCS::Change include Comparable attr_reader action : String attr_reader position : Integer attr_reader element : (Array[T] | T)? def self.valid_action? : (String) -> untyped def initialize : (String, Integer, (Array[T] | T)?) -> nil def inspect : -> String ... def == : (untyped) -> bool def <=> : (untyped) -> Integer? def adding? : -> bool def deleting? : -> bool def unchanged? : -> bool def changed? : -> bool def finished_a? : -> bool def finished_b? : -> bool end predicate methods
  20. 20. Agenda •Type Profiler: Type Inference for Ruby 3 •Demo ➔ How to use Type Profiler • Planned experience • Specific usage •Future plan 20
  21. 21. Typical usage and experience (Plan) 1. Write an entry point program (if needed) 2. Apply Type Profiler 3. Partially write RBS for wrong-guessed methods 4. Re-apply Type Profiler 21 lib.rb app.rb Type Profiler lib.rbs (may include wrong guesses) partial RBS for difficult methods lib.rbs (final) ① ② ③ ④ "Partial RBS specification" has been implemented
  22. 22. How to use TP specifically Will be written until the RubyKaigi Takeout! (hopefully) https://github.com/mame/ruby-type-profiler 22 I have written the document
  23. 23. Agenda •Type Profiler: Type Inference for Ruby 3 •Demo •How to use Type Profiler ➔Future plan • Recent updates • Future plan • Conclusion 23
  24. 24. Recent updates • Improve cosmetics (attr_*, optional, bool, …) • Import Array and Hash methods from RBS • Type variable • Support Enumerable, Enumerator, and Struct • Support global variables • Improve flow-sensitive analysis • Improve analysis performance • Fix many, many bugs 24
  25. 25. Future plan • Until Ruby 3: Make it possible for plain Ruby code • Support partial RBS specification • Write document and release a gem • Continue to experiment, improve, etc, etc. • After the release: Support Sinatra app, Rails app… • Maybe need dedicated hard-coding for the frameworks • Concern is almost a language extension • ActiveRecord is super-meta feature • Please go easy on 🙏 25
  26. 26. Acknowledgment •Hideki Miura •Matz, Akr, Ko1 •Soutaro Matsumoto •Katsuhiro Ueno •Eijiro Sumii •Sorbet developers (Stripe, Shopify, etc.) •Jeff Foster 26
  27. 27. Conclusion •Type Profiler is crawling to you • Inviting suggestions for the tool name! • Ask me anything: @mametter (Twitter) 27

×