Skip to content

Python, create a network request

New Course Coming Soon:

Get Really Good at Git

Python offers us the urllib standard library package to create network requests.

Create a request using:

from urllib import request
url = 'https://dog.ceo/api/breeds/list/all'

response = request.urlopen(url)
content = response.read()

print(content)

You can also use the with statement to simplify

from urllib import request
url = 'https://dog.ceo/api/breeds/list/all'

with request.urlopen(url) as response:
   content = response.read()

print(content)

The response is a sequence of bytes, as you will notice because the response is wrapped in a b'' string:

b'{"message":{"affenpinscher":[],"african":[],"airedale":[],"akita":[],"appenzeller":[],"australian":["shepherd"],"basenji":[],"beagle":[],"bluetick":[],"borzoi":[],"bouvier":[],"boxer":[],"brabancon":[],"briard":[],"buhund":["norwegian"],"bulldog":["boston","english","french"]},"status":"success"}'

Decode it to a UTF-8 encoded string using content.decode('utf-8')

This gets the HTML content from my website flavicopes.com:

from urllib import request

url = 'https://flaviocopes.com'

with request.urlopen(url) as response:
   content = response.read().decode('utf-8')

print(content)

You can parse the response as JSON, using the json standard library module:

from urllib import request
import json

url = 'https://dog.ceo/api/breeds/list/all'

with request.urlopen(url) as response:
   content = response.read()

data = json.loads(content)
print(data['status'])

If you need to specify query parameters, use the urllib.parse() method to build the query string:

from urllib import request, parse
url = 'https://api.thecatapi.com/v1/images/search'

parms = {
    'limit' : 5,
    'page' : 1,
    'order' : 'Desc'
}

querystring = parse.urlencode(parms)

with request.urlopen(url + '?' + querystring) as response:
   content = response.read().decode('utf-8')

print(content)

This is the built-in urllib package.

For convenience purposes, you might want to use the requests package, not part of the Python standard library but quite popular.

Are you intimidated by Git? Can’t figure out merge vs rebase? Are you afraid of screwing up something any time you have to do something in Git? Do you rely on ChatGPT or random people’s answer on StackOverflow to fix your problems? Your coworkers are tired of explaining Git to you all the time? Git is something we all need to use, but few of us really master it. I created this course to improve your Git (and GitHub) knowledge at a radical level. A course that helps you feel less frustrated with Git. Launching May 21, 2024. Join the waiting list!
→ Get my Python Handbook
→ Get my Python Handbook

Here is how can I help you: