⚡Python Requests Library & FastAPI

Python Requests Library

How to Use Python Requests with REST APIs

First, you’ll need to have the necessary software; make sure you have Python and pip installed on your machine. Then, head over to the command line and install the python requests module with pip:

pip install requests

Now you’re ready to start using Python Requests to interact with a REST API, make sure you import the Requests library into any scripts you want to use it in:

import requests

How Request Data With GET

The GET method is used to access data for a specific resource from a REST API; Python Requests includes a function to do exactly this.

import requests
response = requests.get("http://api.open-notify.org/astros.json")
print(response)
>>>> Response<200>

The response object contains all the data sent from the server in response to your GET request, including headers and the data payload. When this code example prints the response object to the console it simply returns the name of the object’s class and the status code the request returned (more on status codes later).

While this information might be useful, you’re most likely interested in the content of the request itself, which can be accessed in a few ways:

response.content() # Return the raw bytes of the data payload
response.text() # Return a string representation of the data payload
response.json() # This method is convenient when the API returns JSON

How to Use Query Parameters

Queries can be used to filter the data that an API returns, and these are added as query parameters that are appended to the endpoint URL. With Python Requests, this is handled via the params argument, which accepts a dictionary object; let’s see what that looks like when we use the Open Notify API to GET an estimate for when the ISS will fly over a specified point:

query = {'lat':'45', 'lon':'180'}
response = requests.get('http://api.open-notify.org/iss-pass.json', params=query)
print(response.json())

The print command would return something that looks like this:

{
'message': 'success',
'request': {
'altitude': 100,
'datetime': 1590607799,
'latitude': 45.0,
'longitude': 180.0,
'passes': 5
},
'response': [
{'duration': 307, 'risetime': 1590632341},
{'duration': 627, 'risetime': 1590637934},
{'duration': 649, 'risetime': 1590643725},
{'duration': 624, 'risetime': 1590649575},
{'duration': 643, 'risetime': 1590655408}
]
}

How to Create and Modify Data With POST and PUT

In a similar manner as the query parameters, you can use the data argument to add the associated data for PUT and POST method requests.

# Create a new resource
response = requests.post('https://httpbin.org/post', data = {'key':'value'})
# Update an existing resource
requests.put('https://httpbin.org/put', data = {'key':'value'})

How to Access REST Headers

You can also retrieve metadata from the response via headers. For example, to view the date of the response, just specify that with the `headers` property:

print(response.headers["date"]) 
'Wed, 11 June 2020 19:32:24 GMT'

For open APIs that covers the basics. However, many APIs can’t be used by just anyone. For those, let’s go over how to authenticate to REST APIs.

How to Authenticate to a REST API

So far you’ve seen how to interact with open REST APIs that don’t require any authorization. However, many REST APIs require you to authenticate to them before you can access specific endpoints, particularly if they deal with sensitive data.

There are a few common authentication methods for REST APIs that can be handled with Python Requests. The simplest way is to pass your username and password to the appropriate endpoint as HTTP Basic Auth; this is equivalent to typing your username and password into a website.

requests.get(
'https://api.github.com/user', 
auth=HTTPBasicAuth('username', 'password')
)

A more secure method is to get an access token that acts as an equivalent to a username/password combination; the method to get an access token varies widely from API to API, but the most common framework for API authentication is OAuth. Here at Nylas, we use three-legged OAuth to grant an access token for user accounts that is restricted to scopes that define the specific data and functionality that can be accessed. This process is demonstrated in the Nylas

Once you have an access token, you can provide it as a bearer token in the request header: this is the most secure way to authenticate to a REST API with an access token:

my_headers = {'Authorization' : 'Bearer {access_token}'}
response = requests.get('http://httpbin.org/headers', headers=my_headers)

There are quite a few other methods to authenticate to a REST API, including digest, Kerberos, NTLM, and AuthBase. The use of these depends on the architecture decisions of the REST API producer.

Use Sessions to Manage Access Tokens

Session objects come in handy when working with Python Requests as a tool to persist parameters that are needed for making multiple requests within a single session, like access tokens. Also, managing session cookies can provide a nice performance increase because you don’t need to open a new connection for every request.

session = requests.Session()
session.headers.update({'Authorization': 'Bearer {access_token}'})
response = session.get('https://httpbin.org/headers')

How to Handle HTTP Errors With Python Requests

API calls don’t always go as planned, and there’s a multitude of reasons why API requests might fail that could be the fault of either the server or the client. If you’re going to use a REST API, you need to understand how to handle the errors they output when things go wrong to make your code more robust. This section covers everything you need to know about handling HTTP errors with Python Requests.

The Basics of HTTP Status Codes

Before we dive into the specifics of Python Requests, we first need to take a step back and understand what HTTP status codes are and how they relate to errors you might encounter.

All status codes fall into one of five categories.

1xx Informational – Indicates that a request has been received and that the client should continue to make the requests for the data payload. You likely won’t need to worry about these status codes while working with Python Requests.
2xx Successful – Indicates that a requested action has been received, understood, and accepted. You can use these codes to verify the existence of data before attempting to act on it.
3xx Redirection – Indicates that the client must make an additional action to complete the request like accessing the resource via a proxy or a different endpoint. You may need to make additional requests or modify your requests to deal with these codes.
4xx Client Error – Indicates problems with the client, such as a lack of authorization, forbidden access, disallowed methods, or attempts to access nonexistent resources. This usually indicates configuration errors on the client application.
5xx Server Error – Indicates problems with the server that provides the API. There are a large variety of server errors and they often require the API provider to resolve.

How to Check for HTTP Errors With Python Requests

The response objects has a status_code attribute that can be used to check for any errors the API might have reported. The next example shows how to use this attribute to check for successful and 404 not found HTTP status codes, and you can use this same format for all HTTP status codes.

response = requests.get("http://api.open-notify.org/astros.json")
if response.status_code == 200:
    print("The request was a success!")
    # Code here will only run if the request is successful
elif response.status_code == 404:
    print("Result not found!")
    # Code here will react to failed requests

To see this in action, try removing the last letter from the URL endpoint, the API should return a 404 status code.

If you want requests to raise an exception for all error codes (4xx and 5xx), you can use the raise_for_status() function and catch specific errors using Requests built-in exceptions. This next example accomplishes the same thing as the previous code example.

try:
    response = requests.get('http://api.open-notify.org/astros.json')
    response.raise_for_status()
    # Additional code will only run if the request is successful
except requests.exceptions.HTTPError as error:
    print(error)
    # This code will run if there is a 404 error.

TooManyRedirects

Something that is often indicated by 3xx HTTP status codes is the requirement to redirect to a different location for the resource you’re requesting. This can sometimes result in a situation where you end up with an infinite redirect loop. The Python Requests module has the TooManyRedirects error that you can use to handle this problem. To resolve this problem, it’s likely the URL you’re using to access the resource is wrong and needs to be changed.

try:
    response = requests.get('http://api.open-notify.org/astros.json')
    response.raise_for_status()
    # Code here will only run if the request is successful
except requests.exceptions.TooManyRedirects as error:
    print(error)

You can optionally use the request options to set the maximum number of redirects:

response = requests.get('http://api.open-notify.org/astros.json', max_redirects=2)

Or disable redirecting completely within your request options:

response = requests.get('http://api.open-notify.org/astros.json', allow_redirects=False)

ConnectionError

So far, we’ve only looked at errors that come from an active server. What happens if you don’t receive a response from the server at all? Connection errors can occur for many different reasons, including a DNS failure, refused connection, internet connectivity issues or latency somewhere in the network. Python Requests offers the ConnectionError exception that indicates when your client is unable to connect to the server.

try:
    response = requests.get('http://api.open-notify.org/astros.json') 
    # Code here will only run if the request is successful
except requests.ConnectionError as error:
    print(error)

This type of error might be temporary, or permanent. In the former scenario, you should retry the request again to see if there is a different result. In the latter scenario, you should make sure you’re able to deal with a prolonged inability to access data from the API, and it might require you to investigate your own connectivity issues.

Timeout

Timeout errors occur when you’re able to connect to the API server, but it doesn’t complete the request within the allotted amount of time. Similar to the other errors we’ve looked at, Python Requests can handle this error with a Timeout exception:

try:
    response = requests.get('http://api.open-notify.org/astros.json', timeout=0.00001)
    # Code here will only run if the request is successful
except requests.Timeout as error:
    print(error)

In this example, the timeout was set as a fraction of a second via the request options. Most APIs are unable to respond this quickly, so the code will produce a timeout exception. You can avoid this error by setting longer timeouts for your script, optimizing your requests to be smaller, or setting up a retry loop for the request. This can also sometimes indicate a problem with the API provider. One final solution is to incorporate asynchronous API calls to prevent your code from stopping while it waits for larger responses.

How to Make Robust API Requests

As we’ve seen, the Requests module elegantly handles common API request errors by utilizing exception handling in Python. If we put all of the errors we’ve talked about together, we have a rather seamless way to handle any HTTP request error that comes our way:

try:
    response = requests.get('http://api.open-notify.org/astros.json', timeout=5)
    response.raise_for_status()
    # Code here will only run if the request is successful
except requests.exceptions.HTTPError as errh:
    print(errh)
except requests.exceptions.ConnectionError as errc:
    print(errc)
except requests.exceptions.Timeout as errt:
    print(errt)
except requests.exceptions.RequestException as err:
    print(err)

⚡️ FastAPI

So far, we have learned how to use APIs to get data from external sources. So, we have been on the client side asking for data. We were not particularly interested in how our requests were being processed. We were just interested in the response Json data we would be receiving. But from now on, we will switch roles and be on the server side. We will process the incoming API requests on our back-end code and send back a response object to the client side. We will not be consuming APIs, but we will be creating them. For that, we will be using a Python library called FastAPI.

FastAPI is a modern, fast (high-performance), web framework for building APIs with Python 3.7+ based on standard Python type hints. It is built on top of Starlette and pydantic, and is one of the fastest Python web frameworks available.

Here are some key features of FastAPI:

Fast: Very high performance, on par with NodeJS and Go (thanks to Starlette and Pydantic). One of the fastest Python frameworks available.
Fast to code: Increase the speed to develop features significantly.
Fewer bugs: Reduce about 40% of human (developer) induced errors.
Fewer lines of code: Achieve more with fewer lines of code.
Fewer files: A single file for a small project.
Auto-validation of request parameters using Python 3.6+ type hints.
Automatic generation of OpenAPI documentation (including example requests and responses) using pydantic models and Python 3.6+ type hints.
Automatic generation of Swagger UI from the OpenAPI documentation.
Async support with async/await syntax.
WebSockets support.
GraphQL support with automatic schema generation.
Dependency injection and easy integration with other libraries.

Installing FastAPI

To use FastAPI, you will need to install it first. You can do this using pip, the Python package manager:

pip install fastapi
pip install "uvicorn[standard]"

Example FastAPI Project

Once you have FastAPI installed, you can start using it to build your API. Here are the steps you can follow to start a new FastAPI project:

Create a new directory for your project and navigate to it.
Create a new virtual environment for your project. This is optional, but recommended, as it will allow you to isolate the packages you install for this project from other projects on your system. You can do this using python -m venv env, where env is the name of your virtual environment.
Activate the virtual environment by running source env/bin/activate on Linux or macOS, or env\Scripts\activate.bat on Windows.
Install FastAPI and any other dependencies your project needs using pip. For example, to install FastAPI, you can run pip install fastapi.
Create a new file for your API, such as main.py.
Import FastAPI and create a new instance of the FastAPI class.
Define your API endpoints using FastAPI's routing decorators (e.g. @app.get(), @app.post(), etc.).
Run the API using uvicorn.run(). For example: uvicorn.run(app, host="0.0.0.0", port=8000). This will start the API on port 8000 of your local machine.

Here's an example of what your main file might look like:

from fastapi import FastAPI
from typing import Union
import uvicorn

app = FastAPI()

@app.get("/")
def read_root():
    return {"Hello": "World"}

@app.get("/items/{item_id}")
def read_item(item_id: int, q: Union[str, None] = None):
    return {"item_id": item_id, "q": q}
    
if __name__ == "__main__":
    uvicorn.run(app, host="0.0.0.0", port=8000)

Run it

To start the API, simply run the main file using Python: python main.py. This will start the API and make it available at http://localhost:8000.

Alternatively, you can run the server from your terminal with the command:

uvicorn main:app --reload

INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [28720]
INFO:     Started server process [28722]
INFO:     Waiting for application startup.
INFO:     Application startup complete

The command uvicorn main:app --reload refers to:

main: the file main.py (the Python "module").
app: the object created inside of main.py with the line app = FastAPI().
--reload: make the server restart after code changes. Only do this for development.

Check it

Open your browser at http://127.0.0.1:8000/items/5?q=somequery.

You will see the JSON response as:

{"item_id": 5, "q": "somequery"}

You already created an API that:

Receives HTTP requests in the paths / and /items/{item_id}.
Both paths take GET operations (also known as HTTP methods).
The path /items/{item_id} has a path parameter item_id that should be an int.
The path /items/{item_id} has an optional str query parameter q.

Interactive API docs

Now go to http://127.0.0.1:8000/docs.

You will see the automatic interactive API documentation (provided by Swagger UI):

Example upgrade

Now modify the file main.py to receive a body from a PUT request.

Declare the body using standard Python types, thanks to Pydantic.

from typing import Union

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()


class Item(BaseModel):
    name: str
    price: float
    is_offer: Union[bool, None] = None


@app.get("/")
def read_root():
    return {"Hello": "World"}


@app.get("/items/{item_id}")
def read_item(item_id: int, q: Union[str, None] = None):
    return {"item_id": item_id, "q": q}


@app.put("/items/{item_id}")
def update_item(item_id: int, item: Item):
    return {"item_name": item.name, "item_id": item_id}

The server should reload automatically (because you added --reload to the uvicorn command above).

Interactive API docs upgrade

Now go to http://127.0.0.1:8000/docs.

The interactive API documentation will be automatically updated, including the new body:

Click on the button "Try it out", it allows you to fill the parameters and directly interact with the API:

Then click on the "Execute" button, the user interface will communicate with your API, send the parameters, get the results and show them on the screen:

Recap

In summary, you declare once the types of parameters, body, etc. as function parameters.

You do that with standard modern Python types.

You don't have to learn a new syntax, the methods or classes of a specific library, etc.

Just standard Python 3.7+.

For example, for an int:

item_id: int

or for a more complex Item model:

item: Item

...and with that single declaration you get:

Editor support, including:
- Completion.
- Type checks.
Validation of data:
- Automatic and clear errors when the data is invalid.
- Validation even for deeply nested JSON objects.

For a more complete example including more features, see the Tutorial - User Guide.

Fast API Tutorial Video #1

Here is an hour-long video which covers all the basics of FastAPI library, make sure to carefully watch and code-along:

FastAPI Tutorial Video #2

Once you finish watching/coding-along the video above, check out this video as well. This one will help you consolidate the things you have learned in the first video and will teach you some more advanced concepts (as always, try to code along the video):

PreviousWeek 14 NextIntroduction to Statistics

Last updated 1 year ago