March 28, 2017
HomeArticles and TutorialsPython REST API example

Python REST API example

By  Articles and Tutorials, Code Snippets, Computer Networks, Computer Security  0 Comments

Introduction

In order for two networked computers to exchange data, a protocol has to be used. A protocol is an agreed method to identify computers (ex. PC, iPhone), applications (ex. browser, web server) and resources (ex. webpage, image, database table). A protocol also facilitates secure (ex. encryption) and reliable (ex. data loss protection) communication. The world wide web (i.e www) heavily relies on HTTP (i.e. hyper text transfer protocol) protocol. For example, a browser uses HTTP to request a webpage hosted on a web server. The web server returns the requested page back for display. The browser uniquely identifies a webpage resource using a URL (i.e. uniform resource locator). This sounds familiar when we type the address of a website in the browser address bar. The URL fully identifies a resource using a hostname (or IP address), port number and path to the resource. The host name identifies the destination computer. The port number identifies the service (ex. web, ftp) followed by the path to the resource. To demonstrate that, take a look at the following example

Python
1
http://www.8bitavenue.com:80/2016/11/python-mutex-example/

  • Protocol: http
  • Host name: www.8bitavenue.com
  • Port number: 80 but since 80 is the default port number for HTTP we can safely remove :80 from the URL
  • Path to resource: /2016/11/python-mutex-example

Note that the path to the resource should not necessarily match the corresponding local file system path on the server. The translation from URL path to system path is taken care of by the web server application. An example showing this translation is provided later in the implementation section.

HTTP commands

We learned that a browser communicates with a web server using HTTP protocol by adopting client server model (as opposed to a peer to peer model). Let us now briefly describe some of the methods that the HTTP protocol provides to facilitate communication.

  • GET method is used to request a resource. It only retrieves data, for example, a browser uses it to download then display a webpage.
  • POST method is used to send data to the destination computer. For example, a browser uses it to submit a web form.
  • PUT method creates a new resource if it does not exist, otherwise it updates that resource.
  • DELETE method is used to delete a resource.

Please note that as we browse the web, GET and POST commands are used the most.

HTTP request vs response

Let us be more specific and put things into perspective. A web browser retrieving a webpage, that is an application level communication. On the HTTP protocol level, a browser sends an HTTP request, the web server returns an HTTP response. With that said, pay attention to the following points:

  • An HTTP method (ex. POST, GET) is part of an HTTP request
  • A status code is part of an HTTP response to indicate what happened with the request
  • Query parameters are part of an HTTP request. They are used as input and can be sent either in request body or added to the URL
  • A request or response contains a headers section to include extra information (meta data). For example, the content type is one of the most important header information when dealing with HTTP response because it specifies the data format being returned
  • Both request and response have a body section containing the data (payload) being sent or received in a given format (JSON in our case)

Python web service

Now that we have a basic idea how the web works, The next step is to utilize HTTP in order to implement data exchange between computers. In other words, we are going to implement a simple Python web service and develop a client to consume it. Instead of retrieving data in HTML format as in the browser case, we are going to send and receive data in JSON format.That is a naive way to think of a web service. A web service basically exposes an API through endpoints.

API vs end point

Lets us quickly clarify what API and end point are. These terms are frequently mentioned when discussing web services. API (application programming interface) is a general term, it can refer to web services, libraries, methods and functions. Without being fancy in the definition, It refers to specifying service name, input parameters, their types and any returned results. APIs allow us glue components together and interconnect systems. End point is one end of the communication channel. It is just another way to refer to the URL of the web service. It is also important to note that web service API typically have more than one end point. Each end point serves a specific purpose. All web service end points share a common base URL. Oh by the way, did we just say JSON? What on eath is JSON format?

JSON Format

Some times, when terms are repeated over and over, we often tend to lose track. For that reason, let us clarify few basic points:

  • Text vs binary: JSON data is typically saved to disk in text format. A binary file is an array of bytes representing some sort of custom data (ex. image pixels). On the other hand, bytes in a text file represent characters in a certain encoding.
  • ASCII vs UTF8: Characters in a JSON text file can be encoded in ASCII (plain text) or UTF8 (if you want to use international character sets). Note that ASCII and UTF8 are just example encoding schemes. You can use whatever encoding suitable for your application.
  • JSON vs XML: that is how we structure the content of the file. In this post, we are going to use JSON format. This is not the place to debate which is better JSON or XML, however JSON seems to win the competition on the web due to simplicity, lightweight and adoption.

JSON (JavaScript Object Notation) is a subset of the JavaScript language syntax where:

  • Data is represented in name value pairs separated by commas
  • Objects are held in curly braces
  • Arrays are held in square brackets

JSON Example

Here is an example

Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
{
   "cars": [
      {
         "id":"01",
         "model": "Honda",
         "color": "White"
      },
      {
         "id":"02",
         "model": "Ford",
         "color": "Black"
      }
   ]
}

The JSON file above contains information about two cars. Each car object has three fields. It resembles a database table. Car objects are the rows and fields are the columns. Try to copy the JSON text above and paste it to this online JSON viewer to visualize how the file is structured. We agreed that HTTP is going to be used to send and receive data in JSON format so we are ready now to talk about REST.

What is REST?

REST (REpresentational State Transfer) is a set of conventions to structure a web service. A web service that conforms to these conventions is called a RESTful web service. Fine details of REST constraints (i.e. client server, stateless, cacheable, layered, uniform interface, on demand) are not covered in this article. For more information, please see the references at the end of this article. So what is covered here then? I am going to cover basic design tips and implement a simple RESTful web service along with the corresponding client in Python.

Nouns verbs and representations

When designing RESTful API, we should pay attention to nouns, verbs and representations. Nouns refer to resources (ex. database table) exposed through the API. Verbs are HTTP methods (ex. POST, GET) applied to resources. Representation refers to data format (ex. json, xml, html). In our case, we are going to apply the following design tips.

  • Take out verbs out of web service URL and dedicate two base URLs for each resource
  • Use HTTP commands to apply verbs to nouns in order to reduce number of base URLs
  • Use plural nouns as opposed to singular for better intuition and clarity

Taking the same cars example mentioned earlier.

  • GET on /cars retrieves all cars
  • GET on /cars/id retrieves a specific car with id
  • POST on /cars add a new car
  • PUT on /cars/id updates a specific car with id
  • DELETE on /cars/id deletes a specific car with id

As you can see, only two base URLs are used (/cars and /cars/id). HTTP methods are used to reduce the number of base URLs. For example, no need for base URLs like /getCars and addCar. Also, we used plural noun /cars not /car.

Base URL

In base URL, we may include service name and version number. Service name provides a name space in case of hosting more than one service on the server. Version number is handy if we want to update the web service without breaking legacy functionality. Here is an example:

Python
1
http://hostname.com/dealership/cars/v1.0

In our implementation, we are not going to use a service name nor version. We are going to keep things simple so that the demonstration is less distracting.

Python REST API server

We are going to use web.py web server Python module. If you do not have it installed, you can do that using the command:

Python
1
sudo easy_install web.py or sudo pip install web.py

If you do not have pip or easy_install then you need to install it first.

For the sake of clarity, It is not our attention to have a fancy implementation. Typically, a web service is implemented using a production quality python rest api framework such as Django or Flask. Also, a database is used to back the web service for persisting data. In our case, the focus is on exposing an API through a web service and consuming that service. For that reason, web.py is more than enough to do the job. For persistence, we are going to use in memory python list. We are not going to use any database nor save to disk. If you turn off the server then you will loose data but who cares ? it is a demo. Here is our implementation commented well so there is no need for extra explanation.

Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
# Import web.py web server module
import web
 
# Makes dealing with json format easy
import json
 
# GET    on /cars lists all cars
# GET    on /cars/10 retrieves car with id = 10
# POST   on /cars creates a car
# PUT    on /cars/10 updates car with id = 10
# DELETE on /cars/10 deletes car with id = 10
 
# Regular expressions are used to extract car id
# Here is an explanation of that:
# /cars followed by optional / followed by a capture group
# The syntax for the capture group is (?P...)
# Inside the capture group we have <car_id>[0-9]+
# meaning the ID is a number one or more
# The capture group can be 0 or more because it is followed by ?
 
# urls is used by the web.py module
# For each end point there is a method that handles
# it in the HandleRequest class
urls = (r'/cars/?(?P<car_id>[0-9]+)?', 'HandleRequest')
 
# We are going to use a dictionary for demonstration only.
# Think of it as an in memory database
cars = []
 
    #--------------------------------------------
 
# This class is used to handle requests
class HandleRequest():
    
    # Initialization code
    def __init__(self):
     # Set response content type (json) and
     # encoding (utf8) for all requests
        web.header('Content-Type', 'text/json; charset=utf-8')
      
    #--------------------------------------------
                              
# Handle HTTP GET
    def GET(self, car_id=None):
     # If no id is provided list all cars
        if car_id is None:
print "---- Listing all cars"
# json.dumps converts the array of dictionaries cars
# into a json text format
return json.dumps(cars)
        # Otherwise send back the requested car object
        else:
         # Search for the requested car
print "---- Retrieving car with id = ", car_id
for car in cars:
if int(car['id']) == int(car_id):
return json.dumps(car)
# return empty json
return "{}"
 
    #--------------------------------------------
# Handle HTTP POST
    def POST(self, car_id=None):
     # Append the car object sent in the POST request
     # to the cars array. The sent data is saved in
     # web.data(), json.loads converts the json text
     # data into a dictionary. The cars array is an
     # array of dictionaries, each dictionary stores
     # information about one car.
        if car_id is None:
# Make sure we do not add car if id exists
found = False
mycar = json.loads(web.data())
for car in cars:
if int(car['id']) == int(mycar['id']):
found = True
if found == False:
cars.append(mycar)
print "---- Adding car : ", web.data()
print "---- Cars added : ", cars
return web.data()
        else:
         # When posting, no id should be provided
            raise web.badrequest()
    #--------------------------------------------
 
# Handle HTTP PUT
    def PUT(self, car_id=None):
        if car_id is None:
         # You can not update a car if the id does not exit
            raise web.badrequest()
        else:
         print "---- Updating car with id = ", car_id
         # Get car from request
         mycar = json.loads(web.data())
         for car in cars:
if int(car['id']) == int(car_id):
car['color'] = mycar['color']
car['model'] = mycar['model']
print "Avaiable cars : ", cars
return web.data()
    #--------------------------------------------
 
# Handle HTTP DELETE
    def DELETE(self, car_id=None):
        # You can not delete a car if id does not exist
        if car_id is None:
            raise web.badrequest()
        else:
         print "---- Deleting car with id = ", car_id
         # Get car from request
         for car in cars:
if int(car['id']) == int(car_id):
cars.remove(car)
print "Avaiable cars : ", cars
return json.dumps(cars)
 
    #--------------------------------------------
 
# Start web server
app = web.application(urls, globals())
 
# Start script
if __name__ == "__main__":
    app.run()

Python REST API client

Consuming a web service is not hard, we just need to send the appropriare HTTP requests. We can easily do that using a browser or the curl command. In python, we can use built in libraries to invoke HTTP, however there is an easy way. We are going to use (Kenneth Reitz) request library which is a wrapper for Python built in HTTP support.

Request library

The requests library by Kenneth Reitz allows us to invoke HTTP commands like never before. Here is a quick tutorial.

    • If you do not have it installed, you can do so by issuing the command

Python
1
sudo easy_install requests

    • Import the library

Python
1
import requests

    • To do a GET

Python
1
r = requests.get('http://hostname.com/cars')

    • To do a POST

Python
1
r = requests.post('http://hostname.com/cars', data = json)

    • Other commands

Python
1
2
3
4
r = requests.put('hostname.com/cars/1', data = {'key':'value'})
r = requests.delete('hostname.com/cars/2')
r = requests.head('http://hostname.com/cars')
r = requests.options('http://hostname.com/cars')

    • You can pass parameters in the URL

Python
1
r = requests.get('http://hostname.com/cars', params={'key1': 'value1', 'key2': 'value2'})

    • The response object encapsulate a lot of useful information

Python
1
print r.text, r.encoding, r.headers, r.json, r.status_code

That is more than enough to implement our own client. The code snippet below consumes the web service that we implemented earlier:

Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
# Import the requests library
import requests
 
# Prints few response parameters
def printResponse():
print "Status code  : ", r.status_code
print "Encoding     : ", r.encoding
print "JSON         : ", r.json()
print "Content type : ", r.headers["Content-Type"]
print ""
 
 
# Define 2 car objects in json format
car1 = {"id": 1, "model": "Honda", "color": "White" }
car2 = {"id": 2, "model": "Ford", "color": "Black" }
 
# HTTP POST: Add first car
r = requests.post('http://localhost:8080/cars', json=car1)
print "--- Adding car 1\n"
printResponse()
 
# Add second car
r = requests.post('http://localhost:8080/cars', json=car2)
print "--- Adding car 2\n"
printResponse()
 
 
# HTTP GET: get all cars
r = requests.get("http://localhost:8080/cars/");
print "--- Retrieving all cars\n"
printResponse()
 
# Print retrieved cars
for car in r.json():
    print('{} {} {}\n'.format(car['id'], car['model'], car['color']))
    
# GET the second car
r = requests.get("http://localhost:8080/cars/2");
print "--- Retrieving car 2\n"
printResponse()
 
# Print retrieved car
print('\n{} {} {}\n'.format(r.json()['id'], r.json()['model'], r.json()['color']))
 
 
# PUT to update first car
car3 = {"id": 1, "model": "New Honda", "color": "New Black" }
r = requests.put("http://localhost:8080/cars/1", json=car3);
print "--- Updating car 1\n"
printResponse()
 
# GET the first car to see if it was updated
r = requests.get("http://localhost:8080/cars/1");
print "--- Retrieving car 1\n"
printResponse()
 
# Print retrieved car
print('{} {} {}\n'.format(r.json()['id'], r.json()['model'], r.json()['color']))
 
# Delete first car
r = requests.delete("http://localhost:8080/cars/1");
print "--- Deleting car 1\n"
printResponse()
 
# HTTP GET: get all cars
r = requests.get("http://localhost:8080/cars/");
print "--- Retrieving all cars\n"
printResponse()

Testing a web service

To debug a web service, a web browser may be used but it is not easy to generate all HTTP methods. There are dedicated tools for working with web services that one can try (ex. postman). A simpler and flexible solution is to use the curl command. Here is a list of curl commands to test the web service implemented earlier.

Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# Add car 1
curl -X POST  -H "Content-Type: application/json" -d '{"id": 1, "model": "Honda", "color": "White" }' -i http://localhost:8080/cars/
 
# Add car 2
curl -X POST  -H "Content-Type: application/json" -d '{"id": 2, "model": "Ford", "color": "Black" }' -i http://localhost:8080/cars/
 
# Retrieve all cars
curl -X GET  -H "Content-Type: application/json" -i http://localhost:8080/cars/
 
# Get car 2
curl -X GET  -H "Content-Type: application/json" -i http://localhost:8080/cars/2
 
# Update car 2
curl -X PUT  -H "Content-Type: application/json" -d '{"id": 2, "model": "New Ford", "color": "New Black" }' -i http://localhost:8080/cars/2
 
# Delete car 2
curl -X DELETE http://localhost:8080/cars/2
 
# Retrieve all cars
curl -X GET  -H "Content-Type: application/json" -i http://localhost:8080/cars/

Securing a web service

This is a complex topic and very important, however it is beyond the scope of this article. Users connecting to a private web service must be authenticated before permission is granted. If you want to extend the code in this post to support authentication, you may check the following:

  1. Secure the web service. Check out this article
  2. Use authentication in the requests library. Check out this article

Summary

In this Python REST API tutorial, we discussed how the HTTP protocol is used to exchange data between computers. We also described the format of an HTTP URL. Under the hood, HTTP uses requests to send data and responses to retrieve data. Web services can utilize HTTP by exposing resources via API end points. The main theme of this article was REST, a popular web service design style. We also implemented an example RESTful Python web service and client. We ended the article by suggesting how to test and secure a web service.

References

Check the following articles for more information about REST API design and implementation in Python.

That is all for today. Thanks for visiting. Please leave your comments below.

More from my site

Tags:, ,

About Author

Mohammed Abualrob

Software Engineer @ Cisco

Add a Comment

Your email address will not be published. Required fields are marked *