5 Heresies for a Better World: some playful challenges to everyone's assumptions about building for the modern web.
(From some time back in 2008, so some of the references have been forgotten by now. The points about having to think, and not just following the crowd without thinking, and cats being evil and about to make us all obsolete slaves, are still pretty relevant thought.)
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Web heresies
1. 5 Heresies
James Aylett
Kittens with Jetpacks
Thursday, 27 October 2011
2. 5 Heresies
For A Better World
James Aylett
Kittens with Jetpacks
Thursday, 27 October 2011
3. 1: Don’t use a web
framework
Thursday, 27 October 2011
Web frameworks make it easier to do the repetitive tasks involved in creating a web app
that weren’t terribly difficult in the first place. Which saves time, but has some problems.
So what actually happens when you start using the latest, sexiest framework?
4. Thursday, 27 October 2011
Whee! This Nightshade framework is so cool. I wrote my web app in 10 minutes. Give me
lots of money!
5. Thursday, 27 October 2011
Then you do the usual things with a new site - you squash bugs...
7. Thursday, 27 October 2011
And next you start getting some real users (ie not your mum), and things get a bit
slower, and your investors come calling because they can’t get on your site to look at the
attractive coeds. So you do the sensible thing and add some more machines in, praying
to god that you understood the idea of shared-nothing and that your database isn’t too
shabby.
9. Thursday, 27 October 2011
Then you get more bugs, and more users, and more bugs - and eventually you’re not
fixing your own bugs, you’re diving into the framework and fixing theirs.
11. Thursday, 27 October 2011
And, if you’re really lucky, you get enough users that you end up with a room full of
servers, and learn about all the nasty details of how mysql prevents deadlocking, or how
your storage server distributed file blocks across spindles, or how web browsers fuck up
TCP congestion control algorithms. And of course you start running into those weird rare
bits of the framework that don’t actually scale for you.
13. Thursday, 27 October 2011
And this goes on until your database freezes solid, your site goes down for days on end,
and your users remember they didn’t actually need it in the first place. Your investors
start screaming...
14. Thursday, 27 October 2011
... and if you’re really famous, so does Michael Arrington.
15. Thursday, 27 October 2011
In the best case scenario, one of your investors was Y Combinator, and you rewrite the
entire thing in arc.
18. Thursday, 27 October 2011
Frameworks give you a number of really import things: a templating system which isn’t
the one you want, a URI dispatcher, which doesn’t match your URI scheme, an ORM
that’s less powerful than the one you would have used otherwise (and that’s certainly
less powerful than just using your database directly), and finally...
19. Thursday, 27 October 2011
... an octopus higher up your call stack. The problem is that while frameworks save you a
tonne of time, YOU DON’T UNDERSTAND YOUR FRAMEWORK, unless you wrote it. At
some point it will stop being a help and start being a hindrance, because their problem
space is not yours.
20. Thursday, 27 October 2011
Besides, it’s lazy. Standing on the shoulders of giants is pretty awesome, but it’s really a
waste of time unless you’re looking for something high up.
21. Thursday, 27 October 2011
Most websites aren’t up there, and while you’re looking around, your competitors are
eating you. Or the shrubs, depending on how good they are.
22. Thursday, 27 October 2011
So let’s go back to our new framework Nightshade. It’s got all the standard stuff...
25. Thursday, 27 October 2011
But many are increasing complexity of the framework without actually giving YOU any
advantages.
26. Thursday, 27 October 2011
Features become optional, which generally either increases the length of the callstack
before it hits your code or at best makes it harder to just pick up an instance of the
framework and figure out what’s going on, because some core section has been replaced
somewhere in a config file.
28. Thursday, 27 October 2011
Until we pass the Zawinski threshold, and it starts reading your email.
29. Thursday, 27 October 2011
Of course, that’s pretty helpful in a web app, so people don’t really notice.
30. Thursday, 27 October 2011
And features keep on being added until someone has a REALLY smart idea of adding
31. Thursday, 27 October 2011
a user scripting system. At the most extreme, this is itself a framework that allows you
to use a DSL rather than writing code in your base language. Typically this starts in the
templating layer, but if it’s designed well, hell you might as well start using it for
triggers in your models, or for flexible configuration, or to run deployment scripts.
32. Thursday, 27 October 2011
And then someone has the REALLY smart idea of just reimplementing the entire
framework in terms of the scripting system...
33. Thursday, 27 October 2011
Which of course makes perfect sense. Except that then you’ve just obeyed Greenspan’s
10th Law, which really just means ...
34. Thursday, 27 October 2011
you’ve built emacs. And we’ve probably got enough of them already.
35. Thursday, 27 October 2011
However NONE of this actually helps you once you start to hit serious scale. For that
you’re going to need architecture.
36. Thursday, 27 October 2011
And I’m afraid for that you’re going to have to think.
37. 2: Don’t build an API
Thursday, 27 October 2011
Look, you could have built an API. But you chose to build a website. If you have decent
URI design, and use microformats, and you were only thinking of building a read-only
API anyway, then just don’t bother. Which is probably just as well, because you’d only
get it wrong.
39. Thursday, 27 October 2011
In fact, APIs are notoriously difficult to get right. Here’s are just a few ways you can get
them wrong.
40. Thursday, 27 October 2011
Firstly, you could use XML. It’s not that XML’s a bad language for moving data around;
it’s really not. But there are lots of exciting ways of getting it wrong. Actually, it’s
difficult to see why. This guy’s Tim Bray, one of the guys who invented XML. He said:
41. “Anyone who can’t make
[...] well-formed XML is
an incompetent fool”
- Tim Bray
Thursday, 27 October 2011
Which is true. Unfortunately we’re all incompetent fools. I am, you are, and lots of
important people reckon XML’s too hard to generate. They include James Clark, and this
guy:
42. Thursday, 27 October 2011
Henri Sivonen, who went to the lengths of coming up with a list of things to do and not
to to make sure you get XML right. Turns out there are 19 points! So maybe trying to get
all them right isn’t such a great idea.
43. Thursday, 27 October 2011
Another popular choice for moving data around these days is JSON. JSON’s pretty simple,
although there are a couple of simple mistakes you can make. But there are good
libraries for it (although to be fair, if everyone used good XML libraries, that wouldn’t be
a problem either). However there’s an interesting wrinkle that comes out of using JSON.
44.
Thursday, 27 October 2011
JSON isn’t HTML. It’s NOTHING TO DO WITH HTML. But these little HTML named entity
escapes have a tendency to creep into JSON outputs of APIs, often because the APIs were
originally intended for internal use in loading little chunks of data to quickly spit out into
innerHTML. This doesn’t help very much when people start using it for other things.
45. Thursday, 27 October 2011
And then there’s REST. No one understands REST. Okay, Roy T. Fielding probably
understands it, but given that he makes it looks like industrial plumbing, we’re probably
not going to get it right. The trouble is, you really don’t want to use SOAP (or you’ll
make Mark Baker cry), so you’re going to have to try. If you get it wrong...
46. Thursday, 27 October 2011
... and REST purists will burn you in effigy. And you’re limiting the ability of larger sites
to use you without killing your servers.
47. Thursday, 27 October 2011
But there is a chance that some big site will come along, have a look at your lovely API,
and
48. Thursday, 27 October 2011
Suck all of your data out. But there aren’t many that will do that, except in Russia and
China, so you can probably ignore that. Of course, lots of other startups probably want
your data, but they’re going to be subtle about it - they’ll buy a couple of hackers, and a
botnet. In Russia or China. So you could just block those countries. And if you do that, of
course you don’t really need to bother internationalising and localising your API. Or web
site.
49. Thursday, 27 October 2011
Oh - except that Richard Ishida will eat your children. Here he is, demonstrating how to
direct traffic in both France and Japan at the same time.
50. Thursday, 27 October 2011
So APIs require you to think. This may become a theme. Of course, you don’t have to
think.
51. Thursday, 27 October 2011
You could not build an API, as I suggested. Or I suppose you could hire me.
53. Thursday, 27 October 2011
Lots of people can do this. Maybe not that many of them work for MSN. But more
important than making things pretty is making them useful - ignore the flashy bits for a
moment and concentrate on the CONTENT. The DATA.
57. Thursday, 27 October 2011
Of course, if you take this perfectly-designed site, suited to its purpose, there is a risk
that by making it pretty THEN
60. 4: Don’t build websites
Thursday, 27 October 2011
You know, it’s just a lot easier not to bother building websites. I’m not even talking
about using Flash, or Silverlight, or JavaFX. They still live within the browser. There’s
some problems serving stuff over HTTP in the first place, and certainly one of these
problems is the browsers. Or should I say the user agents.
61. Thursday, 27 October 2011
You’ve all heard of these user agents. Hopefully you make sure your websites work with
them. But you don’t have to worry too much about them implementing HTTP differently.
62. Thursday, 27 October 2011
This is still true as we think about some more unusual web browsers.
63. Thursday, 27 October 2011
Although they may surprise you with their support, or lack of, for CSS, Javascript, bits of
HTML, SSL support.
64. Thursday, 27 October 2011
Then we have automated processes, one of which is probably the most important user
agent your site will encounter.
65. Thursday, 27 October 2011
This is the point where you really want to be speaking HTTP correctly yourself, because
Google will reward you. You’ll get cookies or something.
66. Thursday, 27 October 2011
There are a whole load of other things that talk to your website. Here’s a couple of
interesting ones.
67. Thursday, 27 October 2011
Package managers for your operating system, be it Windows, Mac OS, Linux, whatever.
apt is interesting because it deliberately has an unusual interpretation of HTTP which bit
me recently when I tried to reorganise a URI tree. It still seems to obey the rules though.
68. Thursday, 27 October 2011
These guys might not. If they’re not raping your data, they might be trying to take your
site down if you’re successful enough. It does happen. Buffer overruns, web process
exploits - and that’s just at the HTTP layer. You’ll also get lower-level attacks at the TCP
and IP layers. If you think you haven’t got competitors who’ll stoop to this, you still have
to cope with random hackers.
69. Thursday, 27 October 2011
We’re starting to get interesting things on the internet as well. Clocks, refrigerators,
robots...
70. Thursday, 27 October 2011
... rabbits. At least some of these can’t speak HTTP properly. Some of them can’t speak
TCP properly, because they have abbreviated network stacks because they only have 1K
of memory. Some of them are rabbits.
71. Thursday, 27 October 2011
Oh, and there are these crazy people who’ll just telnet straight to your web server or
load balancer and start typing away.
73. Thursday, 27 October 2011
And we can make mistakes; instead of typing GET /girlfriend I could make a typo and
instead try to PET your girlfriend. Although that might not be a mistake. The correct
response in this case is 405. Or 301 if you’ve sent her to a convent.
74. Thursday, 27 October 2011
Then there’s stuff we haven’t even dreamed of. Or we’ve dreamed it, but we don’t
believe it yet. And this is beginning to feel a bit like a medieval map of places we don’t
know very well, which is a fairly accurate description of all the things that can hammer
on your web server.
76. Thursday, 27 October 2011
This guy, if I remember correctly, is the roman God Janus. He’s a gatekeeper. His head
points in both directions, and he enjoys close harmony singing. He’s a pretty good
analogy for the routers and gateways that make up the internet. I had this whole way of
explaining packet routing using a banana, but it kept on getting more complex, and in
the end it was going to take half an hour all by itself and at the end we’d have had
something closer to a pie than a banana, so you’re going to have to take my word for it
that Janus is sometimes going to chew up your packets, drop them on the floor, or
drown out all the local Januses with his off-key singing.
77. Thursday, 27 October 2011
Then there’s the topology of the network between you and the people trying to use your
website.
78. Thursday, 27 October 2011
And those users might have their computers set up weirdly. Most of these things will
bite you even if you don’t build websites, if you just build desktop software that
functions over the network. Life would be easier if you just built little desktop
applications that ignore the network. Or you could just play guitar in a band, because
you only really need three chords.
79. Thursday, 27 October 2011
Yet again, you’re going to have to think. Unless you get really good at guitar.
80. 5: Be evil; get a cat
Thursday, 27 October 2011
Firstly, because being evil is fun.
81. Thursday, 27 October 2011
And cats are evil. This one’s about to steal your watch.
83. Thursday, 27 October 2011
The good news is that cats are smarter than you, so actually you don’t have to think
after all. Unfortunately they are also more evil, so when they rev the social structure of
the universe...