How HTTP requests work
What happens when you type an URL in the browser, from start to finish
- The HTTP protocol
- I analyze URL requests only
- Things relate to macOS / Linux
- DNS Lookup phase
- TCP request handshaking
- Sending the request
- The response
- Parse the HTML
This article describes how browsers perform page requests using the HTTP/1.1 protocol
If you ever did an interview, you might have been asked: “what happens when you type something into the Google search box and press enter”.
It’s one of the most popular questions you get asked. People just want to see if you can explain some rather basic concepts and if you have any clue how the internet actually works.
In this post, I’ll analyze what happens when you type an URL in the address bar of your browser and press enter.
It’s a very interesting topic to dissect in a blog post, as it touches many technologies I can dive into in separate posts.
This is tech that is very rarely changed, and powers one the most complex and wide ecosystems ever built by humans.
The HTTP protocol
First, I mention HTTPS in particular because things are different from an HTTPS connection.
I analyze URL requests only
Modern browsers have the capability of knowing if the thing you wrote in the address bar is an actual URL or a search term, and they will use the default search engine if it’s not a valid URL.
I assume you type an actual URL.
When you enter the URL and press enter, the browser first builds the full URL.
If you just entered a domain, like flaviocopes.com
, the browser by default will prepend HTTP://
to it, defaulting to the HTTP protocol.
Things relate to macOS / Linux
Just FYI. Windows might do some things slightly differently.
DNS Lookup phase
The browser starts the DNS lookup to get the server IP address.
The domain name is a handy shortcut for us humans, but the internet is organized in such a way that computers can look up the exact location of a server through its IP address, which is a set of numbers like 222.324.3.1
(IPv4).
First, it checks the DNS local cache, to see if the domain has already been resolved recently.
Chrome has a handy DNS cache visualizer you can see at chrome://net-internals/#dns
If nothing is found there, the browser uses the DNS resolver, using the gethostbyname
POSIX system call to retrieve the host information.
gethostbyname
gethostbyname
first looks in the local hosts file, which on macOS or Linux is located in /etc/hosts
, to see if the system provides the information locally.
If this does not give any information about the domain, the system makes a request to the DNS server.
The address of the DNS server is stored in the system preferences.
Those are 2 popular DNS servers:
8.8.8.8
: the Google public DNS server1.1.1.1
: the CloudFlare DNS server
Most people use the DNS server provided by their internet provider.
The browser performs the DNS request using the UDP protocol.
TCP and UDP are two of the foundational protocols of computer networking. They sit at the same conceptual level, but TCP is connection-oriented, while UDP is a connectionless protocol, more lightweight, used to send messages with little overhead.
How the UDP request is performed is not in the scope of this tutorial
The DNS server might have the domain IP in the cache. If not, it will ask the root DNS server. That’s a system (composed of 13 actual servers, distributed across the planet) that drives the entire internet.
The DNS server does not know the address of each and every domain name on the planet.
What it knows is where the top-level DNS resolvers are.
A top-level domain is the domain extension: .com
, .it
, .pizza
and so on.
Once the root DNS server receives the request, it forwards the request to that top-level domain (TLD) DNS server.
Say you are looking for flaviocopes.com
. The root domain DNS server returns the IP of the .com TLD server.
Now our DNS resolver will cache the IP of that TLD server, so it does not have to ask the root DNS server again for it.
The TLD DNS server will have the IP addresses of the authoritative Name Servers for the domain we are looking for.
How? When you buy a domain, the domain registrar sends the appropriate TDL the name servers. When you update the name servers (for example, when you change the hosting provider), this information will be automatically updated by your domain registrar.
Those are the DNS servers of the hosting provider. They are usually more than 1, to serve as backup.
For example:
ns1.dreamhost.com
ns2.dreamhost.com
ns3.dreamhost.com
The DNS resolver starts with the first, and tries to ask the IP of the domain (with the subdomain, too) you are looking for.
That is the ultimate source of truth for the IP address.
Now that we have the IP address, we can go on in our journey.
TCP request handshaking
With the server IP address available, now the browser can initiate a TCP connection to that.
A TCP connection requires a bit of handshaking before it can be fully initialized and you can start sending data.
Once the connection is established, we can send the request
Sending the request
The request is a plain text document structured in a precise way determined by the communication protocol.
It’s composed of 3 parts:
- the request line
- the request header
- the request body
The request line
The request line sets, on a single line:
- the HTTP method
- the resource location
- the protocol version
Example:
GET / HTTP/1.1
The request header
The request header is a set of field: value
pairs that set certain values.
There are 2 mandatory fields, one of which is Host
, and the other is Connection
, while all the other fields are optional:
Host: flaviocopes.com
Connection: close
Host
indicates the domain name which we want to target, while Connection
is always set to close
unless the connection must be kept open.
Some of the most used header fields are:
Origin
Accept
Accept-Encoding
Cookie
Cache-Control
Dnt
but many more exist.
The header part is terminated by a blank line.
The request body
The request body is optional, not used in GET requests but very much used in POST requests and sometimes in other verbs too, and it can contain data in JSON format.
Since we’re now analyzing a GET request, the body is blank and we’ll not look more into it.
The response
Once the request is sent, the server processes it and sends back a response.
The response starts with the status code and the status message. If the request is successful and returns a 200, it will start with:
200 OK
The request might return a different status code and message, like one of these:
404 Not Found
403 Forbidden
301 Moved Permanently
500 Internal Server Error
304 Not Modified
401 Unauthorized
The response then contains a list of HTTP headers and the response body (which, since we’re making the request in the browser, is going to be HTML)
Parse the HTML
The browser now has received the HTML and starts to parse it, and will repeat the exact same process we did for all the resources required by the page:
- CSS files
- images
- the favicon
- JavaScript files
- …
How browsers render the page then is out of the scope, but it’s important to understand that the process I described is not just for the HTML pages, but for any item that’s served over HTTP.
→ I wrote 17 books to help you become a better developer:
- C Handbook
- Command Line Handbook
- CSS Handbook
- Express Handbook
- Git Cheat Sheet
- Go Handbook
- HTML Handbook
- JS Handbook
- Laravel Handbook
- Next.js Handbook
- Node.js Handbook
- PHP Handbook
- Python Handbook
- React Handbook
- SQL Handbook
- Svelte Handbook
- Swift Handbook
Also, JOIN MY CODING BOOTCAMP, an amazing cohort course that will be a huge step up in your coding career - covering React, Next.js - next edition February 2025