SlideShare verwendet Cookies, um die Funktionalität und Leistungsfähigkeit der Webseite zu verbessern und Ihnen relevante Werbung bereitzustellen. Wenn Sie diese Webseite weiter besuchen, erklären Sie sich mit der Verwendung von Cookies auf dieser Seite einverstanden. Lesen Sie bitte unsere Nutzervereinbarung und die Datenschutzrichtlinie.
SlideShare verwendet Cookies, um die Funktionalität und Leistungsfähigkeit der Webseite zu verbessern und Ihnen relevante Werbung bereitzustellen. Wenn Sie diese Webseite weiter besuchen, erklären Sie sich mit der Verwendung von Cookies auf dieser Seite einverstanden. Lesen Sie bitte unsere unsere Datenschutzrichtlinie und die Nutzervereinbarung.
A crash course in the history, theory, and implementation of the internet of today. Learn about TCP/IP, headers, packets, HTTP both 1 and 2, and all the glorious technology that makes web developers tick. Finally take a dive into sockets and peek a bit into hardware, and understand why no matter how fat your vm is why you still run out of socket connections. All the things DARPA folks wish you knew before you wrote that first webapp.
We're going to start in a way that might feel a bit backwards. I'm going to start by talking about the implementation of the internet
How do you get those cat gifs on your system?
I often laugh quite a bit when I talk about the internet, because it often feels as though the entire thing is held together with duct tape and string. And yet the internet is surprisingly robust for what it does.
the global Internet consists of tens of thousands of interconnected networks run by service providers, individual companies, universities, governments, and others. Open standards enable this network of networks to communicate.
Uniform Resource Locator
Uniform Resource Locators were defined in RFC 1738 in 1994 by Sir Tim Berners-Lee,
The format combines the pre-existing system of domain names (created in 1985) with file path syntax,
An IP (Internet Protocol) address is a numerical label assigned to a device that uses the IP protocol to communicate. An IP address is used for host or network interface identification and location addressing Every computer connected to a network has one, you have one, I have one, we all have an IP!
So we have a starting address – our computer's ip, and the address we want to get to. So we'll chop up our request into things called "packets" and send them on our network but wait – we have this weird url thing – there's no "ip address" there. how do we know where to go?
so we take that domain name part – the rest is for the request itself – and we'll say "I know this name, what is my address? Where do I go get this"
This is like looking things up in a telephone book – only none of you really do that
TCIP is the name for the protocol we use on the internet – we talked about the IP part – that's your internet protocol part, your address
TCP is something else – it actually takes this data you're going to send _ cause that's what you're doing, sending some text that is a request to another server – and chops it up and adds all this magical information
The NIC may use one or more of the following techniques to indicate the availability of packets to transfer: Polling is where the CPU examines the status of the peripheral under program control. Interrupt-driven I/O is where the peripheral alerts the CPU that it is ready to transfer data. NICs may use one or more of the following techniques to transfer packet data: Programmed input/output is where the CPU moves the data to or from the NIC to memory. Direct memory access (DMA) is where some other device other than the CPU assumes control of the system bus to move data to or from the NIC to memory. This removes load from the CPU but requires more logic on the card. In addition, a packet buffer on the NIC may not be required and latency can be reduced.
This nic does the hardware heavy lifting of sending
Ethernet network controllers typically support 10 Mbit/s Ethernet, 100 Mbit/s Ethernet, and 1000 Mbit/s Ethernet
then there's also fiber optic cables
SO you need to make data flow between networks – using our glorious router
The network can balance the load across various pieces of equipment on a millisecond-by-millisecond basis. If there is a problem with one piece of equipment in the network while a message is being transferred, packets can be routed around the problem, ensuring the delivery of the entire message.
so those packets come from your computer, and flow to our router
the router is going to look at the ip and make some decisions based on that ip
these routers are using the subnet to inspect the packet to see if it's staying local or going out, then if it's going out it uses some thing called a routing table to decide where to send your stuff, picks a router, and sends it off
You connect on a local network through a modem – the type of modem is different (cable modem, dial up modem…wait does that still happen? Digitial subscriber) That hooks into your ISP who you pay to connect up (internet service provider)
and there are multiple routers at each level of this!
SO all these major fiber optic cables meet up and voila, we have a backbone
hese data routes are hosted by commercial, government, academic and other high-capacity network centers, the Internet exchange points and network access points, that exchange Internet traffic between the countries, continents and across the oceans. Internet service providers, often Tier 1 networks, participate in Internet backbone traffic by privately negotiated interconnection agreements, primarily governed by the principle of settlement-free peering.
Backbone providers of roughly equivalent market share regularly create agreements called peering agreements, which allow the use of another's network to hand off traffic where it is ultimately delivered. Usually they do not charge each other for this, as the companies get revenue from their customers regardless. Transit agreements Backbone providers of unequal market share usually create agreements called transit agreements, and usually contain some type of monetary agreement. Regulation Antitrust authorities have acted to ensure that no provider grows large enough to dominate the backbone market. In the United States, the Federal Communications Commission has decided not to monitor the competitive aspects of the Internet backbone interconnection relationships as long as the market continues to function well.
so the providers of this – that keep the huge fiber optic cables in shape and provide huge router farms – they are getting money from this
you pay the isp who pays the backbone provider to get your data out – these are normally big telecoms
there are three tiers of backbone providers – tier 1 are the big boys with transatlantic cables tier 2 are the smaller but still extensive tier 3 are generally much small – the last mile provider
level 3 was one of the largest in Europe and the us that was just bought by centurylink
How this all fits together
this is the part you get involved with
dns tells you what ip to look up, ip tells you where to go, tcp is the transmission of the data, http does your request and response, and html marks up what you see your browser
there, the internet
via the magic of html, our browser shows us hilarious cat pictures!! OMG SO CUTE
"A series of tubes" is a phrase coined originally as an analogy by then-United States Senator Ted Stevens (R-Alaska) to describe the Internet in the context of opposing network neutrality. On June 28, 2006, he used this metaphor to criticize a proposed amendment to a committee bill.
We can't start talking about the internet without talking about all the precursors that made it possible Without all these things coming together in the 19th century we just wouldn't be where we are today.
why am I bothering with this "precursors" thing? Because the internet is actually formed from 3 inventions that stacked on top of each other to create what we know as the "internet" today
And what is a network? an interconnected group of computers
the "world wide web" www we know and love is result of 3 fantastic things that happened and stacked together
Yes yes Al, they all took you out of context
But dude, hire a better speech writer
So there were many many interesting papers and writings happening in the late 1950s
the cold war was in full swing and everyone was worried about the bomb
and a research project agency – that still exists! And has done other cool stuff like GPS! came into existence
here's a little piece of history – sputnik was responsible for arpa being founded so you could say that the internet is all the Russian's fault
Paul Baran: The one hurdle packet switching faced was AT&T. They fought it tooth and nail at the beginning. They tried all sorts of things to stop it. They pretty much had a monopoly in all communications. And somebody from outside saying that there’s a better way to do it of course doesn’t make sense. They automatically assumed that we didn’t know what we were doing. I asked them if they wanted to be early members so they could learn technology as we went along. They said no. I said, Well, why not? And they said, Because packet switching won’t work. They were adamant. As a result, AT&T missed out on the whole early networking experience.
This was a time which was pretty much ARPA-derived, in the sense that the money for computers and for networking computers was coming from the government, and from pretty enlightened leadership there. The idea of Arpanet was that it was going to basically join up computational resources. It was not set up primarily to do e-mail—but the computational-resource connection turned out to be not so important, and the e-mail turned out to be the killer app. These were people who were just trying those two experiments, one to try to make the computational resources blend, and the other to stay in touch with each other conveniently. You were inventing in all directions, with no particular certainty what was going to play out.
In 1969, ARPA gave the job of building “interface message processors” (I.M.P.’s), otherwise known as “nodes” or “packet switches”—the crucial hardware for sending and receiving bursts of data—to Bolt, Beranek & Newman.
More than a year and a half later there were really no fully operational sites. And the reason was that, in order to get on, you had to implement interfaces, you had to build protocols, you had to connect it to your operating systems, you had to connect it to your applications. It was a job for wizards. My conclusion was that we needed to do something to stimulate people. So I talked to ARPA about doing a demonstration, and they made arrangements with the organizers of the very first International Conference on Computer Communication.
Bob Metcalfe: Imagine a bearded grad student being handed a dozen AT&T executives, all in pin-striped suits and quite a bit older and cooler. And I’m giving them a tour. And when I say a tour, they’re standing behind me while I’m typing on one of these terminals. I’m traveling around the Arpanet showing them: Ooh, look. You can do this. And I’m in U.C.L.A. in Los Angeles now. And now I’m in San Francisco. And now I’m in Chicago. And now I’m in Cambridge, Massachusetts—isn’t this cool? And as I’m giving my demo, the damned thing crashed. And I turned around to look at these 10, 12 AT&T suits, and they were all laughing. And it was in that moment that AT&T became my bête noire, because I realized in that moment that these sons of bitches were rooting against me. To this day, I still cringe at the mention of AT&T. That’s why my cell phone is a T-Mobile. The rest of my family uses AT&T, but I refuse.
by the way – the "routers" they use – the IMPS from BNN cost $82,200 – half a million dollars in todays money
Abhay Bhushan and published as RFC 114 on 16 April 1971. – ftp – in 1980 it moved to tcp/ip
On January 1, 1983, known as flag day, NCP was officially rendered obsolete when the ARPANET changed its core networking protocols from NCP to the more flexible and powerful TCP/IP protocol suite, marking the start of the modern Internet
NCP preceded the Transmission Control Protocol (TCP) as a transport layer protocol used during the early ARPANET. NCP was a simplex protocol that utilized two port addresses, establishing two connections, for two-way communications. An odd and an even port were reserved for each application layer application or protocol. The standardization of TCP and UDP reduced the need for the use of two simplex ports for each application down to one duplex port
Telnet was an ad hoc protocol with no official definition until March 5, 1973, the name actually referred to Teletype Over Network Protocol
no one should really use this – it's unencrypted and… well – bad things can happen
Vint Cerf: One of the features of this Arpanet is that the machines that were connected to it were time-shared. The idea of leaving files for each other was pretty common in the time-sharing world. A guy named Ray Tomlinson, at Bolt, Beranek & Newman, figured out a way to cause a file to be transferred from one machine through the Net to another machine and left in a particular location for someone to pick up. He said, I need some symbol that separates the name of the recipient from the machine that the guy’s files are on. And so he looked around for what symbols on the keyboard were not already in use, and found the “@” sign. It was a tremendous invention.
Experimental email transfers between separate computer systems began shortly after the creation of the ARPANET in 1969. Ray Tomlinson is generally credited as having sent the first email across a network, initiating the use of the "@" sign to separate the names of the user and the user's machine in 1971, when he sent a message from one Digital Equipment Corporation DEC-10 computer to another DEC-10. The two machines were placed next to each other. Tomlinson's work was quickly adopted across the ARPANET, which significantly increased the popularity of email. Tomlinson is internationally known as the inventor of modern email. Initially addresses were of the form, username@hostname but were extended to "firstname.lastname@example.org" with the development of the Domain Name System (DNS).
Originally, the entire ARPANET was managed by the military. But network operators realized that a centralized network would eventually become unmanageable if it continued to grow. They decided that the network should be reorganized as a decentralized "network of networks."
Under this scheme, different networks would be controlled by different organizations, but all the networks able to communicate using shared standards, forming a shared "internet." The military asked the computer scientists Robert Kahn and Vint Cerf to develop new networking standards to make this possible.
The result was a set of standards known as TCP/IP. These standards specified the basic format of data packets transmitted across the internet. On January 1, 1983, the ARPANET switched to using TCP/IP, marking the birth of the modern internet. The switch to TCP/IP didn't make much difference from a user perspective — applications like email and Telnet worked about the same as they had before. But the new standard paved the way for much faster network growth by lowering the barrier to entry for new networks.
By the time the ARPANET was decommissioned in 1990, it was just one of many networks that comprised the internet. Today, the internet is made up of more than 40,000 different networks. These networks still communicate with each other using the TCP/IP standards Cerf and Kahn developed in the 1970s.
Apple II IBM Personal Computer IBM Clones . 0 n 1982 "The Computer" was named Machine of the Year by Time magazine
suddenly we had a computer people's homes, in their schools – not just mainframes that academics telnetted into!
The original HTTP proposal by Tim Berners-Lee was designed with simplicity in mind as to help with the adoption of his other nascent idea: the World Wide Web. The strategy appears to have worked: aspiring protocol designers, take note.
file transfer functionality, ability to request an index search of a hypertext archive, format negotiation, and an ability to refer the client to another server.
From this period of rapid experimentation, a set of best practices and common patterns began to emerge, and in May 1996 the HTTP Working Group (HTTP-WG) published RFC 1945, which documented the "common usage" of the many HTTP/1.0 implementations found in the wild. Note that this was only an informational RFC: HTTP/1.0 as we know it is not a formal specification or an Internet standard!
The work on turning HTTP into an official IETF Internet standard proceeded in parallel with the documentation effort around HTTP/1.0 and happened over a period of roughly four years: between 1995 and 1999. In fact, the first official HTTP/1.1 standard is defined in RFC 2068, which was officially released in January 1997, roughly six months after the publication of HTTP/1.0. Then, two and a half years later, in June of 1999, a number of improvements and updates were incorporated into the standard and were released as RFC 2616. The HTTP/1.1 standard resolved a lot of the protocol ambiguities found in earlier versions and introduced a number of critical performance optimizations: keepalive connections, chunked encoding transfers, byte-range requests, additional caching mechanisms, transfer encodings, and request pipelining.
The primary focus of HTTP/2 is on improving transport performance and enabling both lower latency and higher throughput. The major version increment sounds like a big step, which it is and will be as far as performance is concerned, but it is important to note that none of the high-level protocol semantics are affected: all HTTP headers, values, and use cases are the same. Any existing website or application can and will be delivered over HTTP/2 without modification: you do not need to modify your application markup to take advantage of HTTP/2. The HTTP servers will have to speak HTTP/2, but that should be a transparent upgrade for the majority of users. The only difference if the working group meets its goal, should be that our applications are delivered with lower latency and better utilization of the network link!
In 1980, physicist Tim Berners-Lee, a contractor at CERN, proposed and prototyped ENQUIRE, a system for CERN researchers to use and share documents. In 1989, Berners-Lee wrote a memo proposing an Internet-based hypertext system. Berners-Lee specified HTML and wrote the browser and server software in late 1990. That year, Berners-Lee and CERN data systems engineer Robert Cailliau collaborated on a joint request for funding, but the project was not formally adopted by CERN. In his personal notes from 1990 he listed "some of the many areas in which hypertext is used" and put an encyclopedia first
Mosaic was built at the University of Illinois. I was an undergrad student, but I was also a staff member at the National Center for Supercomputing Applications, which is basically a federally funded research institute. When Al Gore says that he created the Internet, he means that he funded these four national supercomputing centers. Federal funding was critical. I tease my libertarian friends—they all think the Internet is the greatest thing. And I’m like, Yeah, thanks to government funding.
images – became netscape which became mozilla which became firefox
Building on Tim Berner-Lee’s initial browser prototype, a team at the National Center of Supercomputing Applications (NCSA) decided to implement their own version. With that, the first popular browser was born: NCSA Mosaic. One of the programmers on the NCSA team, Marc Andreessen, partnered with Jim Clark to found Mosaic Communications in October 1994. The company was later renamed Netscape, and it shipped Netscape Navigator 1.0 in December 1994. By this point, it was already clear that the World Wide Web was bound to be much more than just an academic curiosity.
Lynx is a customizable text-based web browser for use on cursor-addressable character cell terminals. As of May 2017, it is the oldest web browser still in general use and active development, having started in 1992.
Lynx was a product of the Distributed Computing Group within Academic Computing Services of the University of Kansas
The Internet Engineering Task Force (IETF) develops and promotes voluntary Internet standards, in particular the standards that comprise the Internet protocol suite (TCP/IP). It is an open standards organization, with no formal membership or membership requirements. All participants and managers are volunteers, though their work is usually funded by their employers or sponsors. The IETF started out as an activity supported by the U.S. federal government, but since 1993 it has operated as a standards development function under the auspices of the Internet Society, an international membership-based non-profit organization.
The first IETF meeting was attended by 21 U.S.-government-funded researchers on 16 January 1986. It was a continuation of the work of the earlier GADS Task Force. Representatives from non-governmental entities were invited to attend starting with the fourth IETF meeting in October 1986. Since that time all IETF meetings have been open to the public.
I wasn't born until 1979 but I lived this my father was a bit of a geek – self taught programmer
we had an apple 2e an amiga/commodore
then a mac emulator a macintosh performa and in college EVERYONE HAD FREE DIAL UP and pine!
and my first windows pc
It's hard to image but I lived in a time where I was the only child in my class with a computer at home. And all I did on it was play games
The internet of today is built on a lot of theory and a lot of straws and duct tape.
But really the internet is built out of turtles
all the way down
The internet is actually a STACK of technologies, just as it relied on a huge stack of technologies to be implemented.
This is why there isn't one "inventor" of the internet Because the internet is the network that connects The computers And use hypertext and email to communicate
BGP to tell the router what routes to use
routers in general
bgp is telling you what other routers you can talk to ip is telling you your final destination – but also your subnet so we know if we need other routers
The routing and transferring of data by means of addressed packets so that a channel is occupied during the transmission of the packet only, and upon completion of the transmission the channel is made available for the transfer of other traffic
As they traverse network nodes, such as switches and routers, packets are received, buffered, queued, and transmitted (stored and forwarded), resulting in variable latency and throughput depending on the link capacity and the traffic load on the network.
also check your cables and your nic physical hardware is always your limitation!!!
As with all other communications protocol, TCP/IP is composed of layers: IP - is responsible for moving packet of data from node to node. IP forwards each packet based on a four byte destination address (the IP number). The Internet authorities assign ranges of numbers to different organizations. The organizations assign groups of their numbers to departments. IP operates on gateway machines that move data from department to organization to region and then around the world. TCP - is responsible for verifying the correct delivery of data from client to server. Data can be lost in the intermediate network. TCP adds support to detect errors or lost data and to trigger retransmission until the data is correctly and completely received. Sockets - is a name given to the package of subroutines that provide access to TCP/IP on most systems.
On the battlefield a communications network will sustain damage, so the DOD designed TCP/IP to be robust and automatically recover from any node or phone line failure. This design allows the construction of very large networks with less central management. However, because of the automatic recovery, network problems can go undiagnosed and uncorrected for long periods of time.
maintained by the Internet Engineering Task Force (IETF)
So your application level is the basic data you want to send in most http applications this is your http page INLUDING the headers section
the transport is how you’re sending it – UDP and TCP are the most popular
the Internet layer is the “IP” layer – with the header telling the system what address (ip) to send the data to and what port to take to then you get a frame header and footer on the actual piece of data the packet being sent
If you ever see tcp jokes, this is why! tcp is VERY chatty! this also shows what socket calls are underneath – now these are the C versions, but they map to PHP sockets as well
The protocol was designed by David P. Reed in 1980 user datagram protocol
than just tcp and IP – ther’esa lot more to it than that – when we work with sockets we work at the transport and internet layers – making connections, using supported protocols, and using transports to talk to other layers The tools that most programmers use for sockets are cross OS and you all have a bunch of old guys in beards to thank for writing it ;)
The Domain Name System (DNS) is a hierarchical decentralized naming system for computers, services, or other resources connected to the Internet or a private network. It associates various information with domain names assigned to each of the participating entities. Most prominently, it translates more readily memorized domain names to the numerical IP addresses needed for locating and identifying computer services and devices with the underlying network protocols. By providing a worldwide, distributed directory service, the Domain Name System is an essential component of the functionality on the Internet, that has been in use since 1985.
The Stanford Research Institute (now SRI International) maintained a text file named HOSTS.TXT that mapped host names to the numerical addresses of computers on the ARPANET.
In 1984, four UC Berkeley students, Douglas Terry, Mark Painter, David Riggle, and Songnian Zhou, wrote the first Unix name server implementation for the Berkeley Internet Name Domain, commonly referred to as BIND.
The Hypertext Transfer Protocol (HTTP) is an application protocol for distributed, collaborative, and hypermedia information systems. HTTP is the foundation of data communication for the World Wide Web.
Hypertext is structured text that uses logical links (hyperlinks) between nodes containing text. HTTP is the protocol to exchange or transfer hypertext.
Development of HTTP was initiated by Tim Berners-Lee at CERN in 1989. Standards development of HTTP was coordinated by the Internet Engineering Task Force (IETF) and the World Wide Web Consortium (W3C), culminating in the publication of a series of Requests for Comments (RFCs). The first definition of HTTP/1.1, the version of HTTP in common use, occurred in RFC 2068 in 1997, although this was made obsolete by RFC 2616 in 1999 and then again by the RFC 7230 family of RFCs in 2014.
A later version, the successor HTTP/2, was standardized in 2015, and is now supported by major web servers and browsers over TLS using ALPN extension where TLS 1.2 or newer is required.
get head post put delete connect options trace patch
HTML is a markup language that web browsers use to interpret and compose text, images, and other material into visual or audible web pages. Default characteristics for every item of HTML markup are defined in the browser, and these characteristics can be altered or enhanced by the web page designer's additional use of CSS. Many of the text elements are found in the 1988 ISO technical report TR 9537 Techniques for using SGML, which in turn covers the features of early text formatting languages such as that used by the RUNOFF command developed in the early 1960s for the CTSS (Compatible Time-Sharing System) operating system: these formatting commands were derived from the commands used by typesetters to manually format documents. However, the SGML concept of generalized markup is based on elements (nested annotated ranges with attributes) rather than merely print effects, with also the separation of structure and markup; HTML has been progressively moved in this direction with CSS.
Berners-Lee considered HTML to be an application of SGML. It was formally defined as such by the Internet Engineering Task Force (IETF) with the mid-1993 publication of the first proposal for an HTML specification
HTML5 in the Web Hypertext Application Technology Working Group (WHATWG), which became a joint deliverable with the W3C in 2008, and completed and standardized on 28 October 2014
MPTCP (Multipath TCP), which aims to develop a protocol that will allow the simultaneous use of multiple paths to send data, and TCPINC (TCP increased security) which is working on developing an extension to provide unauthenticated encryption and integrity protection of TCP streams.
develp a UDP-based, stream-multiplexing, encrypted transport protocol. The QUIC working group works closely with the HTTPbis working group, especially on the QUIC mapping for HTTP/2.
The Internet Engineering Task Force (IETF) is the premier Internet standards body, developing open standards through open processes. The IETF is a large open international community of network designers, operators, vendors, and researchers concerned with the evolution of the Internet architecture and the smooth operation of the Internet. The technical work of the IETF is done in Working Groups, which are organized by topic into several Areas. Much of the work is handled via mailing lists. The IETF holds meetings three times per year.
I’m a freelance developer – doing primarily C and C++ dev but available for other stuff Also do a lot of open source
Aurora Eos Rose is the handle I’ve had forever – greek and roman goddesses of the dawn and aurora rose from sleeping beauty
Welcome to the internet
WELCOME TO THE INTERNET
We have packets!
Duck tape and string and https://www.internetsociety.org/
It's the beginning of the ColdWar
Advanced Research Projects Agency (ARPA) is founded
It's renamed to DARPA in 1972 to confuse us all
The US wanted a system that would withstand first attack and still run
Telephones wouldn't work, they were too centralized
A decentralized system would be able to keep running
ARPA was trying to get computers to work with, remember this is the age of
Paul Baran comes up with the idea of Packet Switching in 1960, Davies does it in
BobTaylor wanted to work on the computers at MIT and UC Berkley and Rand and
SDC – using one terminal – in 1966
ARPANET IS BORN
Packet Switches was contracted out
BNN delivered a prototype in 8 months
It took a month to get switches hooked up at both Stanford and UCLA
October 29, 1969, at 10:30 - “Talked to SRI host to host.”
no one really notices
THINGS ARE BORN FROM ARPANET
TCP/IP (1974 – 1983 flag day)
World Wide Web Project (1989)
THEORY OF THE INTERNET
What are all the pieces?
WHAT ISTHE INTERNET?
Everybody now has a computer in their pocket
Packet switching breaks data into chunks, or “packets,” and lets each one take its
own path to a destination, where they are re-assembled (rather than sending
everything along the same path, as a traditional telephone circuit does).
Proposed independently by Paul Baran in 1960 and Donald Davies in 1965
LAYERS OF FUN
IP – forwards packets based on a destination address
TCP – verifies the correct delivery of data from client to server with error and lost
Network Sockets – subroutines that provideTCP/IP (and UDP and some other
support) on most systems
The DOD section DARPA built ARPANET which ran onTCP/IP and the protocols
are maintained by IETF
TCP -Transmission Control Protocol
IP – Internet Protocol
UDP - User Datagram Protocol