HTTP requests with Python (the basics)

4 min readMay 16, 2021

What is HTTP (in a nutshell)

In a nutshell, HTTP is just a way for computers (i.e. your computer and a server) to talk to each other. You can think of it as the ‘WhatsApp’ of computers.

In nerd terms, HTTP stands for Hyper Text Transfer Protocol. HTTP follows what we call as the client-server model, where a client (for example, your browser) opens a connection between your computer and a server to make a request, then waits until it receives a response.

Most of us aren’t aware of this as it’s abstracted away from us by our browsers so that we can just click and drag anywhere on a website for a smooth User Experience.

Maybe it’s no use for us Average Joes on what the heck is HTTP but not for people who code, especially those who make apps that is connected to the internet. Knowing HTTP is part of a developer toolbox, though you don’t really need to always think about it.

CRUD operations

Some smart people aren’t really good at naming things, eh? But anyway, CRUD stands for: Create, Read, Update, Delete.

So what’s the connection?

CRUD actually is used by the client (i.e. your browser) to tell the server what is it’s intention. It’s like “Hey server, I need this to happen, can you do it for me?”, then it waits for the response of the server.

For example, you entered YouTube.com to your browser. Your browser then sends a GET request to a YouTube’s server to retrieve some file. Basically, it’s asking for files like html, css, js, image, video, etc. so that it can render the YouTube website to your screen. If the YouTube server is fine with this request, it will accept the request, and send to your browser the requested files.

So that’s the R or Read part in CRUD, “gimme something!”

But what if you want to comment on a video?

This is when your browser sends a POST (or sometimes PUT) request to YouTube’s server. It’s like saying “Hey YouTube, I want this comment for this video to be saved to your database, can you save it for me?”. And again, your browser waits for the response, and if YouTube is fine with this request, it will accept the comment, save it to it’s database, and responds to the browser confirming the request, “all done!”

So this is the C or Create part of CRUD, “save this!”

And for UPDATE and DELETE requests, you know how it goes.

Requests library

OK, enough for the definitions, let’s get straight into it.

In Python, instead of letting the browser do the requests for us, we can directly communicate to the server ourselves via the Requests library.

“Requests: HTTP for humans”, this is the tagline of the Requests library and it clearly speaks what it does.

To start working with it, let’s do a simple pip install.

$ pip3 install requests

Most Mac/Linux folks pip installs this way:

C:\Users> pip install requests

Making Requests

Let’s get through each of the CRUD operations with requests.

For these examples, we will be using ‘JSON Placeholder’ as a mock API. Here’s the base URL:

https://jsonplaceholder.typicode.com

OK, so first, let’s import requests and start getting some fake posts data:

import requestsURL = 'https://jsonplaceholder.typicode.com'
res = requests.get(URL + '/posts')
print(res.json())

So what’s going on here?

First, we saved our base URL on a variable so that we don’t have to copy-paste it repeatedly, then we sent a simple get request to the JsonPlaceholder server in just a single line. The /posts is called an End point which basically tells the server where to get the data from it’s resources.

Then finally, we print the JSON data.

If you’re unsure what’s JSON, you can check out Mozilla’s guide, but basically, it looks like Python dictionaries and it is what’s being exchanged between clients and servers. In a nutshell, this is the ‘message’ computers send to each other.

By running the above code, we will get something like this:

[
  {
    "userId": 1,
    "id": 1,
    "title": "sunt aut facere repellat provident occaecati excepturi optio reprehenderit",
    "body": "quia et suscipit\nsuscipit recusandae consequuntur expedita et cum\nreprehenderit molestiae ut ut quas totam\nnostrum rerum est autem sunt rem eveniet architecto"
  },
  {
    "userId": 1,
    "id": 2,
    "title": "qui est esse",
    "body": "est rerum tempore vitae\nsequi sint nihil reprehenderit dolor beatae ea dolores neque\nfugiat blanditiis voluptate porro vel nihil molestiae ut reiciendis\nqui aperiam non debitis possimus qui neque nisi nulla"
  },
  {
    "userId": 1,
    "id": 3,
    "title": "ea molestias quasi exercitationem repellat qui ipsa sit aut",
    "body": "et iusto sed quo iure\nvoluptatem occaecati omnis eligendi aut ad\nvoluptatem doloribus vel accusantium quis pariatur\nmolestiae porro eius odio et labore et velit aut"
  },...

This is what we call ‘JSON Data’.

And that’s basically the data that represents the posts on a website. When your writing programs let’s say for front-end, you’ll expect to get these data from the server and you build the look and feel of the app from these data.

Here’s how you do POST, PUT, and DELETE with requests:

r = requests.post(URL + '/posts',
     data={
         'title': 'How HTTP work?',
         'body': 'Lorem ipsum...',
         'userId': '1'
     })
r = requests.put(URL + '/posts/1',
     data={
         'id': 1,
         'title': 'How HTTP work?',
         'body': 'HTTP is used by...'
     })
r = requests.delete(URL + '/posts/1')

Downloading files with Python

So what are some other things we can use requests for?

A LOT!

For everything that relates to the internet, requests can be useful in one way or the other.

For example, we can download files like this one, which is a fun little comics about Python:

import requests# fetch the file
URL = 'https://imgs.xkcd.com/comics/python.png'
res = requests.get(URL)# save the file
with open('antigravity.png', 'wb') as file:
    file.write(res.content)
    print('Downloaded successfully!')

Make sure to check out the comics 😝

Other common uses of Requests:

Web Scraping
API integration

Me on social media (follow me!):