Diese Präsentation wurde erfolgreich gemeldet.
Die SlideShare-Präsentation wird heruntergeladen. ×

Prairie DevCon 2015 - Crafting Evolvable API Responses

Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Wird geladen in …3
×

Hier ansehen

1 von 40 Anzeige

Prairie DevCon 2015 - Crafting Evolvable API Responses

Herunterladen, um offline zu lesen

Web frameworks help you build an API quickly but most have little support for dealing with an API that needs to evolve, forcing you to prematurely version your API. But many industry professionals are telling us not to version. How can we avoid it? Take back control of the content you send over the wire. API responses are the "user interface" of your API and should be crafted with same attention to detail that cause designers to fret over color choices, shadows and highlights. In this talk I’ll show techniques that can be used to build responses that are easier to evolve and highlight the types of practices that encourage breaking changes and force you to version your API.

Web frameworks help you build an API quickly but most have little support for dealing with an API that needs to evolve, forcing you to prematurely version your API. But many industry professionals are telling us not to version. How can we avoid it? Take back control of the content you send over the wire. API responses are the "user interface" of your API and should be crafted with same attention to detail that cause designers to fret over color choices, shadows and highlights. In this talk I’ll show techniques that can be used to build responses that are easier to evolve and highlight the types of practices that encourage breaking changes and force you to version your API.

Anzeige
Anzeige

Weitere Verwandte Inhalte

Diashows für Sie (20)

Anzeige

Ähnlich wie Prairie DevCon 2015 - Crafting Evolvable API Responses (20)

Aktuellste (20)

Anzeige

Prairie DevCon 2015 - Crafting Evolvable API Responses

  1. 1. Crafting Evolvable API Responses
  2. 2. Who am I? • Twitter: @darrel_miller • http://www.bizcoder.com/ Solve API Problems Fast
  3. 3. Our Journey Today • Focus on API responses • Versioning is painful • Why we think we need it? • How can we avoid it? • What if you can’t?
  4. 4. Objects over the wire
  5. 5. We have been here before • CORBA, DCOM • SOAP, WSDL • DTOs • JSON
  6. 6. The ASP.NET Web API Project Template public class ValuesController : ApiController { // GET api/values public IEnumerable<string> Get() { return new string[] { "value1", "value2" };} // GET api/values/5 public string Get(int id) { return "value"; } // POST api/values public void Post([FromBody]string value) { } // PUT api/values/5 public void Put(int id, [FromBody]string value) { } // DELETE api/values/5 public void Delete(int id) { } }
  7. 7. The ASP.NET Web API Starter Tutorial public class ProductsController : ApiController { //… public IEnumerable<Product> GetAllProducts() { return products; } public IHttpActionResult GetProduct(int id) { var product = products.FirstOrDefault((p) => p.Id == id); if (product == null) { return NotFound(); } return Ok(product); } } http://www.asp.net/web-api/overview/getting-started-with-aspnet-web-api/tutorial-your-first-web-api
  8. 8. ServiceStack public class ReqstarsService : Service { public List<Reqstar> Get(AllReqstars request) { return Db.Select<Reqstar>(); } }
  9. 9. NancyFX public class SampleModule : Nancy.NancyModule { public SampleModule() { Get["/"] = _ => "Hello World!"; } }
  10. 10. Python Flask @app.route('/todo/api/v1.0/tasks', methods=['GET']) def get_tasks(): return jsonify({'tasks': tasks})
  11. 11. Rails # Returns the resource from the created instance variable # @return [Object] def get_resource instance_variable_get("@#{resource_name}") end
  12. 12. But, what’s the problem?
  13. 13. Which objects to map? • Domain objects • How do we hide content we don’t want to expose? • How do we create different views of data? • Changes to domain model cause changes to API • DTOs • whole lot of busy work • Only indirect control over serialization process
  14. 14. Automatic Serialization • All the properties • Missing values, unrecognized properties • Non standard data types: datetime, timespan • Does null mean unspecified, or explicitly null? • Empty collection or no collection • Capitalization • Links • Cycles • Changes to behavior in the frameworks • Security Risks
  15. 15. The whole app is useless because the API added a new preference
  16. 16. APIs Break because Serialization Libraries Change/Fix things
  17. 17. Take back control
  18. 18. Use a DOM to build your document dynamic jspeaker = new JObject(); jspeaker.name = speakerInfo.Name; jspeaker.bio = speakerInfo.Bio; dynamic links = new JObject(); dynamic iconLink = new JObject(); iconLink.href = speakerInfo.ImageUrl; links.icon = iconLink; dynamic sessionsLink = new JObject(); sessionsLink.href = SessionsLinkHelper.CreateLink(request, speakerInfo).Target; links[LinkHelper.GetLinkRelationTypeName<SessionsLink>()] = sessionsLink; jspeaker["_links"] = links; return new DynamicHalContent(jspeaker);
  19. 19. What to do with your new found freedom?
  20. 20. Resources and Representations /api/order/34 /api/order/35 /api/order/35/invoice.html application/json application/ld+json /api/order/35/invoice.pdf application/json application/ld+json application/pdf text/html /api/PrintQueue /api/CurrentWeather /api/LeaderBoard
  21. 21. Anatomy of an HTTP representation 200 OK HTTP/1.1 Server: Microsoft-HTTPAPI/2.0 Content-Length: 44 Content-Type: text/plain Acme-perf-database-cost: 120ms The quick brown fox jumped over the lazy dog
  22. 22. Let’s build some payloads
  23. 23. { "description" :"There is a hole in my bucket" } The smallest thing that is actionable
  24. 24. { "description" :"There is a hole in my bucket", "steps_to_reproduce" : "Pour water in bucket. Lift bucket off ground. Look for water dripping", "date_reported": "2012-04-21T18:25:43-05:00" } Just enough data to solve the problem
  25. 25. { "description" :"The font is too big", "application_name" : "Wordament", "application_version" : "1.2.0.2392", "environment_osversion" : "NT4.0", "environment_memory_free" : "100 MB", "environment_diskspace_free" : "1.5 GB", "reported_by_user" : “Bob Bing” } Why do they need that data?
  26. 26. { "description" :"The font is too big", "application_name" : "Wordament", "application_version" : "1.2.0.2392", "environment" : { "osversion" : "NT4.0", "memory_free" : "100 MB", "diskspace_free" : "1.5 GB" } } Attribute Groups
  27. 27. { "description" :"The font is too big", "history" : [ {"status" : "reported", "date" :"2014-02-01"}, {"status" : "triaged", "date" :"2014-02-04"}, {"status" : "assigned", "date" :"2014-02-12"}, {"status" : "resolved", "date" :"2014-02-19"}, ] } Attribute Groups for multiple instances
  28. 28. { "description" :"The font is too big", "reported_by_user" : { "name" : "Bob Bing", "email" : "bob@acme.com", "twitter" : "@bobbing", "date_hired" : "2001-01-21" } } Attribute Groups for related data
  29. 29. { "description" :"The font is too big", "reported_by_user_url" : "http://api.acme.com/users/75" } Linking related data
  30. 30. { "description" :"The font is too big", “links" : [ { "href" :"http://api.acme.com/users/75", "rel": "reportedbyuser" }, { "href" :"http://api.acme.com/users/24", "rel": "assignedtouser" } ] } Multiple Links
  31. 31. { "issues" : [ { "description" :"There is a hole in my bucket" } ] } Beware of structural changes { "issue" : { "description" :"There is a hole in my bucket" } } Item in a Array Item as a Property
  32. 32. Naming Your Conventions • Empty arrays • Nulls • Single item as array or object • Camel case, snake case • Vocabulary • Protocols
  33. 33. Media Type Scope application/json application/vnd.acme.issue+json application/issue+json Too broad Too narrow Ideal application/hal+json Partial application/vnd.acmeapi+json Compromise Vocabularies : Profiles, Namespaces, Schemas, Ontologies
  34. 34. Learn from others application/hal+json application/ld+json application/vnd.mason+json application/vnd.siren+json application/vnd.amundsen-uber+json application/vnd.github.v3+json application/vnd.heroku+json application/vnd.api+json application/atom+xml application/voicexml+xml application/vnd.collection+json application/activity+json application/http-problem application/json-home Generic Hypermedia API Specific Task Specific
  35. 35. { "description" :"The font is too big", "_embedded" : { "reportedByUser" : { "name" : "Bob Bing", "email" : "bob@acme.com", "_links" : { "self" : {"href" :"http://api.acme.com/users/75"}} } } Meet application/hal+json
  36. 36. { "collection": { "links": [], "items": [ { "data": [ { "name": "Title", "value": "rntttLearning from Noda Time: a case study in API design and open source (good, bad and ugly)rntt“ }, { "name": "Timeslot", "value": "04 December 2013 16:20 - 17:20“ }, { "name": "Speaker", "value": "Jon Skeet“ } ], "links": [ { "href": "http://conference.hypermediaapi.com/speaker/6", "rel": "http://tavis.net/rels/speaker" }, { "href": "http://conference.hypermediaapi.com/session/133/topics", "rel": "http://tavis.net/rels/topics" } ], "href": "http://conference.hypermediaapi.com/session/133" } ], "query": [], "template": { "data": [] }, "version": "1.0" } } Meet application/vnd.collection+json
  37. 37. Version as a Last Resort • Version the payload • HTML DOCTYPE, Collection+Json • Version the resource • /api/documents/invoice.v2/758 • Version the media type • application/vnd.github.v3+json • Version the API • /api/v2/documents/invoice/758
  38. 38. Wrap up • Understand the limitations of “objects over the wire” • Consider taking back control of your representations • Think in terms of messages, instead of objects • Build software that is designed to survive change • Believe that versioning is an admission of failure
  39. 39. Image Credits • Package - https://flic.kr/p/3mrNyn • Freedom - https://flic.kr/p/4vwRDw • Treasure map - https://flic.kr/p/7jDJwi • Handshake - https://flic.kr/p/nbAu8Y • Telephone - https://flic.kr/p/7Q8bMd • Blindfolded Typing - https://flic.kr/p/53Q3JE • Magic Trick - https://flic.kr/p/7T8zk5 • Donut machine - https://flic.kr/p/anncxf • GT-R Steering Wheel - https://flic.kr/p/fDUSDk • Anatomy - https://flic.kr/p/6bfUZn • Shapes - https://flic.kr/p/3aKUAq • Payloaders - https://flic.kr/p/dTU9sN • Birds on a Wire - https://flic.kr/p/4YdfK

Hinweis der Redaktion

  • - Developer advocate for Runscope.
    - Cloud based solutions for API performance monitoring
    Microsoft MVP
    Book
  • Two parts to API design, resource identification and representation design
    designing representations aka HTTP response.
    - Versioning sucks
    So many ways to do it
    Starting to hear the message, don’t version
    Roy says middle finger
    Layer7 say don’t.
    The problem is we all make mistakes.
    - Why do we think we need it
  • One reason we think we need versioning is because we do
    Objects over the wire
    Pass object graph
    Infrastructure converts it
    Hope the client understands it
    Pre-arranged agreement between client/server
    The details are key to avoiding versioning.
    But Web APIs are not the same as local APIs

    Objects over the wire. The idea is that you construct some kind of object graph in your favourite language in your web application, that contains the data that you want to deliver to the clients. And then you pass that object graph to some piece of infrastructure code that converts it into a wire format that hopefully will be consumable by the client. The challenge is, for this to work, there has to be some kind of pre-arranged agreement between sender and receiver. The details of that agreement are the key to avoiding versioning.

  • - Trying to do this since the mid 1990’s CORBA, DCOM
    - When the web won, SOAP was invented
    simplified to DTOs
    - REST was rediscovered and redefined to use JSON to send objects over the wire

    JSON has helped. It makes the easy stuff really easy. But the more work we do in JSON the more we are starting to see the re-invention of the complexity. JSON-Schema, JSON Pointer, JSON Path, patch formats, namespacing.

    Fallacy of REST has no contracts. We are just pretending they are there, but they are implicit and unwritten.

    So what does “objects over the wire” look like in code, in the current incarnation.


    We have been trying to do this since the mid 1990’s. CORBA, DCOM were attempts to allow access to remote objects.
    When it was recognized that the web has won, SOAP was invented to try and do Objects over the wire on HTTP
    Some smart people realized that “objects over the wire” was never really going to succeed so they simplified to DTOs over the wire.
    Types over the wire, but sharing types creates coupling. SOAP used a similar concept but called them data contracts
    Only works if you have tight control over both sides of the wire.
  • - The default behavior is to return CLR types
    let the framework decide how to convert it to representation on the wire
    What does Ienumerable<string> look like on the wire? Does every platform do it the same way?
    What does string look like on the wire? JSON spec has changed recently for simple values.
    POSTing string. So many StackOverflow questions on posting string of XML that doesn’t work.
    The worlds simplest API and multiple interoperability issues.
  • In this example we move from returning a native CLR data type to a custom object. Product
    Note IHTTPActionResult
    Starts to break down when the information we are trying to send is not in the payload
    Returning a object limits how we can craft the response.
    But this is not just Microsoft doing silly things
  • What are the odds that List<Reqstars> creates the same wire representation as List<Reqstars> in Web API
    Returning objects is hiding a critical information needed to ensure interoperability
  • Even the really cool frameworks.
    I wonder what this will look like on the wire.
    It is a very common pattern.
    Every framework provider will tell you, “oh but you can customize the response if you want”
    …but that’s not where they lead you.
  • It is also not a problem that is limited to the .net space.
    This python example is slightly better, because at least we know it is going to be sent as a JSON representation
    and the mapping from objects to JSON is fairly straight forward
  • And just as one last example, rails does the same thing.
  • So what, you say… most of those are minor implementation details.
    What’s the fundamental problem with objects over the wire?
  • Even with DTOs, you only have indirect control over the serialization process
  • If you address supports suite no you are going to get it whether there is a value or not.
    Sending data, what does it expect? Mandrill
    How do you format dates? iJson was a new standard developed to help interoperability
    In a geneology app, if Date of death is null, are we missing that data or is he just not dead yet?
    How you handle links in your server implementation is your business, how you format them on the wire is everyone’s business
    .Net objects don’t follow capitalization rules of JSON. What about word separation?
    Cycles can cause some serializers to die randomly.
    The great YAML fiasco

    When dealing with Automatic Serialization there are so many unknowns
    Being aware of what are the rules of your representations is absolutely critical.
    When you know it is a rule, you start to think about the long term effects and the cross platform concerns.
    When you are “just serializing your objects as JSON” you get a false sense of security.

  • This is what happens when you are not clear about your contract.
  • The poor guys writing the serializers. They break people when they fix things.
  • What’s a DOM….
    - Here is just one example of using the JSON.NET Jobject and dynamic.
    - There are many ways to do this.
    - helper methods allow you to establish conventions in your representation formats.
    - Instead of domain objects to DTOs,
    - Do domain objects/linq projections to DOM.
    - You define your own conventions instead of a serializer doing it for you.
    And maybe you don’t even have to define your own conventions, Maybe the conventions of an existing media type meet your needs
  • The problem with not using serializers, now you have the problem of having think about how to structure your responses.
    So, let’s start with some fundamentals.
  • Clear up confusion around terms.
    Mapping Resources to entities can be limiting to your design.
    A different URI should almost always be a different resource.
    Ideally if you want two URLs for the same resource then one should redirect to the other.
    Prevents cache pollution
    A Resource can have multiple representations, or just a single one.
    There is nothing wrong with creating distinct resources for available formats
    Resources can be completely dynamic concepts

  • You can change that reason phrase…
    Headers are metadata about the response
    Performance – Headers can be processed and interpreted without having to interpret the entire request body. Cross cutting concerns
    Timing – Headers arrive first and can actually be used to abort the reception of the body, or prepare for the body, whilst downloading bytes
    Naming – don’t use x-, do use company- for custom headers
    Content - Links, json, Unicode?
    In the future http/2 will support header compression
  • Often at the root of an API. Another example : Atom service document
  • Building an Issue Tracking System

    Really simple minimal documents make it really easy to get started
  • - Optional attributes can be added to the root object
    - Consider the use-case, not the server object model.
    - What would a serializer do?
    - Objects include everything. Guide client devs down the right path
    - You can always add, it’s much hard to take away.
    - There are reasons to group attributes, until you have a reason, probably not worth it. Structural changes are tricky.
    - Clients shouldn’t depend on order
    - Name it based on most natural conventions for the format
  • How much semantics does the client really need? Is it going to do calcs on the memory free?
    Consider cultural issues, like formatting, language.
    Should a date be pretty formatted and localized?
    Should an address be concatenated or separate fields
    Should a name be two fields, or formatted firstname + lastname or last name comma first name
  • Grouping together can have a variety of advantages
    saved a few bytes
    potentially more human readable
    allows definition of mandatory fields within a group, without the group itself being mandatory
    - group becomes a candidate for linking instead of embedding.
    It might map to an object, it might not.
  • Groups are useful for supporting multiple instances of a set of data.
  • Embed related data to reduce round trips
    When updating an issue, we would not update the contents of the reportedByUser.
    - Related information can also be included by providing a link.
    - Consider volatility and reusability of data.
    - beware of embedding short lived data inside long lived data.
  • // Links can be very handy for pointing at information that changes rarely and can easily be cached locally.
    // There are many ways of representing links. Sometimes a link is presented as an object.
  • Multiple objects can be displayed like this…
  • Making a your concept a property allows for a issue.v2,
    …or allows for defining a media type that can hold many different concepts
    Sometimes you might want to make something a collection, even though today there is only one.
    Don’t make the root of your JSON object an array (must Ignore policy)
  • “Media Type” is the name you give to your conventions
    Content-Type is the header to identify conventions to the client
    There is a lot of nitty gritty details to think about when defining a media type from scratch
    Better to layer new types on old ones.
    Profile is an attempt to separate the efforts.
  • Distinction between what is here and what is elsewhere
  • A list of things

×