Networking and resource loading
How HTTP requests work
Learn what happens when you type a URL and press enter, from the DNS lookup and TCP handshake to sending the HTTP request and parsing the response HTML.
This lesson describes how browsers perform page requests using the HTTP/1.1 protocol.
“What happens when you type a URL and press enter?” is a classic interview question. Let’s answer it properly.
I assume you type an actual URL, not a search term. This lesson follows macOS and Linux. Windows does some things slightly differently.
Building the URL
If you type just a domain, like flaviocopes.com, the browser adds the protocol for you. It used to default to http://. Modern browsers try https:// first.
DNS lookup
The browser needs the server IP address, a set of numbers like 104.26.10.78 (IPv4). It starts with a DNS lookup.
First it checks the local DNS cache. Chrome has a DNS page at chrome://net-internals/#dns where you can clear it. If nothing is there, the browser asks the system resolver through the gethostbyname POSIX system call.
gethostbyname
gethostbyname first looks in /etc/hosts. If the domain is listed there, the lookup ends.
Otherwise the system sends a UDP request to the DNS server set in the system preferences. Two popular public ones are 8.8.8.8 (Google) and 1.1.1.1 (Cloudflare). Most people use the one their provider gives them.
If that server doesn’t have the answer cached, it asks the root DNS server, a system of 13 named servers replicated across the planet. The root doesn’t know every domain. It knows where the top-level domain (TLD) servers are: .com, .it, .pizza. For flaviocopes.com it returns the address of the .com server, and our resolver caches it.
The TLD server knows the authoritative name servers for the domain. Your registrar sent them there when you bought the domain. There are usually a few, for redundancy:
ns1.dreamhost.comns2.dreamhost.comns3.dreamhost.com
The resolver asks the first one for the IP of the domain. That’s the source of truth. Now we have the IP address.
TCP handshake
With the IP address, the browser opens a TCP connection to the server. TCP needs a short handshake before data can flow. Then we can send the request.
Sending the request
The request is a plain text document with 3 parts: the request line, the headers, and the body.
The request line holds the HTTP method, the resource location, and the protocol version:
GET / HTTP/1.1
The headers are field: value pairs. Host is the only mandatory one in HTTP/1.1, and Connection is almost always there with it:
Host: flaviocopes.com
Connection: close
Host says which domain we want. Connection is close unless we want to keep the connection open for more requests. Other common headers are Origin, Accept, Accept-Encoding, Cookie, Cache-Control and Dnt. A blank line ends the headers.
The body is optional. A GET request has none. POST requests use it a lot, often with JSON data.
The response
The server sends back a response. It starts with a status code and message:
200 OK
Other common ones:
404 Not Found
403 Forbidden
301 Moved Permanently
500 Internal Server Error
304 Not Modified
401 Unauthorized
Then come the response headers and the body. Since we asked for a page, the body is HTML.
Parsing the HTML
The browser parses the HTML and repeats this exact process for every resource the page needs: CSS files, images, the favicon, JavaScript files, and so on.
Rendering is the topic of the next module. What matters now is that this process is not just for HTML pages. It’s for anything served over HTTP.
Lesson completed