REST API Best practices

Understanding best practices for designing RESTful API’s

The concept of REST is to separate the API structure into logical resources. These resources are manipulated using HTTP requests where the method (GET, POST, PUT, PATCH, DELETE) has specific meaning.

Why should one conform to REST standards?

First of all, REST is an architectural style, and one of its principles is to leverage on the standardized behavior of the protocol underlying it, so if you want to implement a RESTful API over HTTP, you have to follow HTTP strictly for it to be RESTful. You’re free to not do so if you think it’s not adequate for your needs, nobody will curse you for that, but then you’re not doing REST. You’ll have to document where and how you deviate from the standard, creating a strong coupling between client and server implementations, and the whole point of using REST is precisely to avoid that and focus on your media types. By conforming to the REST standard, everyone using your API who knows how the different HTTP methods work and you don’t have to document or explain what the methods should do for a given resource.

Keep in mind that REST is an architectural style focused on long term evolution of your API. To do it right will add more work now, but will make changes easier and less traumatic later. That doesn’t mean REST is adequate for everything and everyone. If your focus is the ease of implementation and short term usage, just use the methods as you want. You can do everything through POST if you don’t want to bother about clients choosing the right methods.

HTTP Method Properties

Safe Methods
Request methods are considered “safe” if their defined semantics are essentially read-only; i.e., the client does not request, and does not expect, any state change on the origin server as a result of applying a safe method to a target resource.

This definition of safe methods does not prevent an implementation from including behavior that is potentially harmful, that is not entirely read-only, or that causes side effects while invoking a safe method. For example, a safe request initiated by selecting an advertisement on the Web will often have the side effect of charging an advertising account.

Idempotent Methods
A request method is considered “idempotent” if the intended effect on the server of multiple identical requests with that method is the same as the effect for a single such request.  Of the request methods defined by this specification, PUT, DELETE, and safe request methods are idempotent.Like the definition of safe, the idempotent property only applies to what has been requested by the user; a server is free to log each request separately, retain a revision control history, or implement other non-idempotent side effects for each idempotent request.Idempotent methods are distinguished because the request can be repeated automatically if a communication failure occurs before the client is able to read the server’s response.  For example, if a client sends a PUT request and the underlying connection is closed before any response is received, then the client can establish a new connection and retry the idempotent request.  It knows that repeating the request will have the same intended effect, even if the original request succeeded, though the response might differ.

Cacheable Methods
Request methods can be defined as “cacheable” to indicate that responses to them are allowed to be stored for future reuse;  In general, safe methods that do not depend on a current or authoritative response are defined as cacheable.

HTTP METHODS

Method Description Safe Idempotent Cacheable
GET Transfer a current representation of the target resource Yes Yes Yes
HEAD Same as GET, but only transfer the status line and header section Yes Yes Yes
POST Perform resource-specific processing on the request payload No No No (Can use cache-control headers)
PUT Replace all current representations of the target resource with the request payload No Yes No (Can use cache-control headers)
DELETE Delete all current representations of the target resource No Yes No
OPTIONS Describe the communication options for the target resource Yes Yes No
PATCH Apply a set of changes described in the    request payload to the resource identified by the Request-URI No No No (Can use cache-control headers)

1. Use nouns and not verbs for resources

What can I make a resource? Resources should be nouns (not verbs) that make sense from the perspective of the API consumer. Although your internal models may map neatly to resources, it isn’t necessarily a one-to-one mapping. The key here is to not leak irrelevant implementation details out to your API consumer.

Once you have your resources defined, you need to identify what actions apply to them and how those would map to your API. RESTful principles provide strategies to handle CRUD actions using HTTP methods mapped as follows:

Resource POST GET PUT DELETE
/groups Create a new group Returns list of groups Bulk update of groups if allowed. (405 Method not allowed otherwise) Delete all groups if allowed. (405 Method not allowed otherwise)
/groups/{group id} Method not allowed (405) Returns the group with the given group id Updates the group with {group id} Deletes the group with the given group id

2. GET and Query Parameters should not alter state

Use PUT, POST and DELETE methods  instead of the GET method to alter the state.
Do not use GET for state changes:

GET /users/711?activate or
GET /users/711/activate

3. Use PLURAL nouns

Do not mix up singular and plural nouns. Keep it simple and use only plural nouns for all resources.

/users instead of /user
/groups instead of /group

4. Use sub resources for relations

If a relation can only exist within another resource, use sub resources to represent the relation

GET /groups/{group id}/members Returns a list of members in group with given id
GET /groups/{group id}/members/{member id} Returns user with given user id for given group id

5. Handling actions that dont fall under CRUD category

  • Use PATCH to apply the set of changes described in the patch document (request payload). For example an activate/deactivate group action could be handled via a PATCH to the resource as follows
    PATCH  /groups/{group id}
    {
    “action”:”activate|deactivate”
    }
  • Treat it like a sub-resource with RESTful principles. For example, the activate/deactivate group action can be handled via a PUT to the sub resource as follows
    PUT  /groups/{group id}/status
    {
    “status”:”active”|”inactive”
    }
  • Sometimes you really have no way to map the action to a sensible RESTful structure. For example, a multi-resource search doesn’t really make sense to be applied to a specific resource’s endpoint. In this case, /search would make the most sense even though it isn’t a resource. This is OK – just do what’s right from the perspective of the API consumer and make sure it’s documented clearly to avoid confusion.

6.  Use HTTP headers for specifying serialization formats

Both, client and server, need to know which format is used for the communication. The format has to be specified in the HTTP-Header.

Content-Type defines the request format.
Accept defines a list of acceptable response formats.

7.  Result filtering, sorting and searching

Complex result filters, sorting requirements and advanced searching (when restricted to a single type of resource) can all be easily implemented as query parameters on top of the base URL.

Filtering
Use a unique query parameter for each field that implements filtering.
GET /groups?status=activeReturns a list of active groups

Sorting:
Similar to filtering, a generic parameter sort can be used to describe sorting rules. Accommodate complex sorting requirements by letting the sort parameter take in list of comma separated fields, each with a possible unary negative to imply descending sort order. For example,
GET /groups?sort=status,-nameReturns list of groups in ascending order of status; Within the same status, groups returned will be sorted by name in descending order

Searching:
Sometimes basic filters aren’t enough and you need the power of full text search. When full text search is used as a mechanism of retrieving resource instances for a specific type of resource, it can be exposed on the API as a query parameter on the resource’s endpoint. Let’s say q. Search queries should be passed straight to the search engine (elastic search) and API output should be in the same format as a normal list result.
GET /groups/?status=active&sort=-name&q=test –  Return list of all active groups sorted by name (desc) which contain ‘test’ in their names

8.  Aliases for common queries

To make the API experience more pleasant for the average consumer, consider packaging up sets of conditions into easily accessible RESTful paths. For example, when  querying for mostactive,recommended groups etc, we can have endpoints like

GET /groups/mostactive         –    Returns list of mostactive groups
Default values can be used for the sort parameters.

  • Field selection

The API consumer doesn’t always need the full representation of a resource (mobile clients for example). The ability select and chose returned fields goes a long way in letting the API consumer minimize network traffic and speed up their own usage of the API. Use a fields query parameter that takes a comma separated list of fields to include.
GET /groups/?fields=id,name,owner,status&status=active&sort=-name

9.  Pagination

The right way to include pagination details today is using the Link header introduced by RFC 5988. An API that uses the Link header can return a set of ready-made links so the API consumer doesn’t have to construct links themselves.

GET /groups?offset=20&limit=20

The response should include pagination information through links as below

{
“start”: 1,
“count”: 20,
“totalCount”: 100,
“totalPages”: 5,
“links”: [
{
“href”: “https: //<url>/offset=40&limit=20”,
“rel”: “next”
},
{
“href”: “https: //<url>/offset=0&limit=20”,
“rel”: “previous”
}
]
}

10.  Authentication

Using HTTP basic authentication
The most simple way to deal with authentication is to use HTTP basic authentication. We use a special HTTP header where we add ‘username:password’ encoded in base64. It is very easy to retrieve the username and password from a basic authentication. This authentication scheme should not be used on plain HTTP, but only through SSL/TLS.

Using HMAC
One of the downsides of basic authentication is that we need to send over the password on every request. Also, it does not safeguard against tampering of headers or body by an eavesdropper. Here, we just concatenate the HTTP verb and the actual URL. We could add other information as well, like the current timestamp, a random number, or the md5 of the message body in order to prevent tampering of the body, or prevent replay attacks. Next, we generate a hmac and pass it in the request header. The server than validates the digest from the request, headers and payload using the client secret to ensure that the client making the request is genuine

OAuth
One of the downsides of hmac is that there are no more passwords, but just plain secrets. When a user’s secret gets out, everyone could use that secret to access the account of that user. A common way to prevent this is to use temporary tokens. Client “asks” the server for a “token” and “secret”, and with these token and secret, it is allowed to access protected resources of a user.

OAuth is a mechanism that allows you to create temporary tokens. It is a common used scheme for authentication and authorization.

11.  Caching

HTTP provides a built-in caching framework! All you have to do is include some additional outbound response headers and do a little validation when you receive some inbound request headers. However, its important to note that the caching behavior across browsers may vary when using the cache control headers. There are two ways of handling caching using conditional GET requests

Time-based Approach (Last-Modified, If-UnModified-Since)
A time-based conditional request ensures that only if the requested resource has changed since the browser’s copy was cached will the contents be transferred. If the cached copy is the most up-to-date then the server returns the 304 response code.

Content-based Approach (ETag, If-Match)
A content based conditional request is similar to a time based request except that the value is a digest of the resources contents (for instance, an MD5 hash). This allows the server to identify if the cached contents of the resource are different to the most recent version.

12.  Response Handling (Success and errors)

Successful Response
The response codes returned for API’s should be standard http codes with additional information in the response headers or payload wherever necessary.

Error Response
An API should provide a useful error message in a known consumable format in case of errors in API’s. The representation of an error should be no different than the representation of any resource, just with its own set of fields. A JSON error body should provide a few things for the developer – a useful error message, a unique error code (that can be looked up for more details in the docs) and possibly a detailed description. JSON output representation for something like this would look like:

{
“code” : 1234,
“message” : “Something bad happened :(“,
“description” : “More details about the error here”
}

Both successful and error response can be handled using the HTTP Status codes.

Here’s a list of Common Http Status Codes

How to use HTTP status codes for successful/error response in APIs?

Let us consider a resource ‘group’ which has properties id, name, description and status; time keeping fields like create_time and update_time are also available. A group is created in active state by default. The name and description cannot be empty, have a limit on the number of characters and also have a restriction on allowed characters.

1. Creating a new group
POST /api/v1/groups
{
“name”:”Test Group”,
“description”:”Test Description”
}
Response Code

Http Status Code Error Code Description
201 Successfully created group. Include a Location header in response to locate newly created resource
Location:/api/v1/groups/1
422 ID-GP-001 The name is empty.
422 ID-GP-002 The description is empty.
422 ID-GP-003 Invalid characters in name.
422 ID-GP-004 Invalid characters in description.
422 ID-GP-005 Name too long.
422 ID-GP-006 Description too long.
415 ID-GE-001 Unsupported Media Type.
500 ID-GE-002 Application encountered unexpected error

Successful Response Payload

{
“id”: “1”,
“name”: “Test Group”,
“description”: “Test Description”,
“status”: “active”,
“created_time”: “2014-04-18 14:12:47”,
“updated_time”: “2014-04-30 14:12:47”
}

Note: The response of a POST request is not cached by default. Though it is possible to control the cache behavior with cache control response headers, the behavior of various browsers to these headers in case of POST, PUT methods is different; so its a better practice to let the client cache the resource with a GET call using the URI in the Location header.

2. Updating an existing group
PUT /api/v1/groups/1
{
“name”:”Test Updated Group”,
“description”:”Test Updated Description”
}

Response Code

Http Status Code Error Code Description
200 or 204 Successfully updated group. 200 or 204 are valid status codes. Ensure the status codes are used uniformly across API’s
422 ID-GP-001 The name is empty.
422 ID-GP-002 The description is empty.
422 ID-GP-003 Invalid characters in name.
422 ID-GP-004 Invalid characters in description.
422 ID-GP-005 Name too long.
422 ID-GP-006 Description too long.
404 ID-GP-007 Group with specified id not found in system.
415 ID-GE-001 Unsupported Media Type.
500 ID-GE-002 Application encountered unexpected error

Note: To get the updated resource, do a GET on the URI

3. Deleting an existing group
DELETE /api/v1/groups/1

Response Code

Http Status Code Error Code Description
200 or 204 Successfully updated group. 200 or 204 are valid status codes. Ensure the status codes are used uniformly across API’s
404 ID-GP-007 Group with specified id not found in system.
415 ID-GE-001 Unsupported Media Type.
500 ID-GE-002 Application encountered unexpected error

13.  Pretty print response(dont minify) and support compression

An API that provides white-space compressed output isn’t very fun to look at from a browser. Although some sort of query parameter (like ?pretty=true) could be provided to enable pretty printing, an API that pretty prints by default is much more approachable. The cost of the extra data transfer is negligible, especially when you compare to the cost of not implementing gzip.

Consider some use cases: What if an API consumer is debugging and has their code print out data it received from the API – It will be readable by default. Or if the consumer grabbed the URL their code was generating and hit it directly from the browser – it will be readable by default. These are small things. Small things that make an API pleasant to use!

14.  Auto loading related resource representations

There are many cases where an API consumer needs to load data related to (or referenced) from the resource being requested. Rather than requiring the consumer to hit the API repeatedly for this information, there would be a significant efficiency gain from allowing related data to be returned and loaded alongside the original resource on demand. We can use an embed (or expand) parameter to include the related resource details in the response

GET /groups/{group id}/?embed=members

Why do we need ’embed’ when we have ‘fields’ query parameter?

Let’s say we are fetching all the groups in the system with an option to fetch members selectively (we need only a few selected fields in groups and members).
On the API front, we need to fetch all the groups first using a query(1) and for each group we need to fire another query to fetch all the member data(N) resulting in a total of N+1 queries. To optimize this we need to use ‘Get and Stitch’ approach where we fire one query to get all the group information and another query to get all the member information for all groups fetched using IN clause. Once the results are fetched, we try to associate the members to the group in memory. In order to support this optimization, we would need to differentiate between fetching properties of a resource and fetching related resources of a resource; thats where embed comes in

Response
{
“name”: “Test Group”,
“owner”: “201206”,
“members”: [
{
“memberid”: “2017709”,
“displayName”: “Roger”,
“firstName”: “Roger”,
“lastName”: “Federer”,
“links”: {
“link”: [
{
“rel”: “self”,
“href”: “https://<url>/profile?accountId=2017709&#8221;,
“title”: “Profile details”
},
{
“rel”: “permalink”,
“href”: “https://<url>/profile/2017709&#8221;,
“title”: “Profile page”
}
]
}
}
]
}

15.  Rate limiting

To prevent abuse, it is standard practice to add some sort of rate limiting to an API. RFC 6585 introduced a HTTP status code 429 Too Many Requests to accommodate this.

However, it can be very useful to notify the consumer of their limits before they actually hit it. This is an area that currently lacks standards but has a number of popular conventions using HTTP response headers.

At a minimum, include the following headers

  • X-Rate-Limit-Limit- The number of allowed requests in the current period
  • X-Rate-Limit-Remaining – The number of remaining requests in the current period
  • X-Rate-Limit-Reset – The number of seconds left in the current period

16.  API Versioning

The most common ways of versioning API’s are

1. Having the version information in the URL
This is very easy to implement. When an API is changed, the URL to access the API should include the new version.
When implementing versioning using URL, we need to ensure all HATEOAS links return the right version information

2. Using custom headers
We can use the same URL as before but add a header such as “api-version: 2”
The HTTP spec gives us a means of requesting the resource we’d like by way of the accept header

3. Versioning media types using accept header
Modify the accept header to specify the version, for example “Accept: application/vnd.groups.v2+json”
Clients accessing the API have to carefully construct the request and configure the accept header appropriately

URL Versioning is used by a lot of popular API’s like twitter, Github etc though it is not without its issues

17.  API Documentation

When exposing API’s to consumers it is very important to document all API’s. Documentation should at a minimum include a description of the API, response and error codes, sample success response and error response. There are a lot of open source tools like Swagger, Enunciate,miredot etc available which enable client and documentation systems to update at the same pace as the server. The documentation of methods, parameters and models are tightly integrated into the server code, allowing APIs to always stay in sync making it easier to deploy and manage APIs

Advertisements
This entry was posted in Uncategorized. Bookmark the permalink.

8 Responses to REST API Best practices

  1. Pingback: How to: Understanding REST: Verbs, error codes, and authentication | SevenNet

  2. Pingback: How to: RESTful URL design for search | SevenNet

  3. Pingback: Solution: Understanding REST: Verbs, error codes, and authentication #dev #it #computers | Technical information for you

  4. Pingback: Fixed RESTful URL design for search #dev #it #asnwer | Good Answer

  5. Pingback: Resolved: RESTful URL design for search #fix #development #it | IT Info

  6. Adam says:

    Thanks for this post, it was incredibly useful. Really thorough and well written. I have a question about a system that would have multiple shared collections, like a blog. If I had a collection ‘posts’ for blog posts, and each member had a collection ‘comments’, I could GET all comments for a given post using a uri such as ‘posts/123/comments’. But what if I also had a collection ‘users’ where each member could have comments on individual posts or even create entire posts themselves? Would it be a good design to have a uri like ‘/users/1/posts’ to get all of the users posts? And ‘/users/1/comments/’ to access all of that users comments? Presumably this could lead to URIs like ‘/users/1/posts/123/comments’?

    Like

    • From what you have described, each user can have several blog posts and each post will have comments from different users. So, there is a relationship between user and post and post and comments. You can design your endpoints like
      /users/1/posts – To get all the posts of a user
      /users/1/posts/1/comments – To get all the comments for a post
      If you want to get the comments made by a particular user, you can use a query param
      /users/1/posts/1/comments?userId=[user id] – To get all the comments written by a particular user on a post.

      Now, if you have a requirement where you want to get all the comments made by a particular user, you might have to support another endpoint
      /comments?userId=[user id] which will go through all the comments of all posts and filter out the posts made by the user.

      Please let me know if you need further clarification.

      Like

  7. Really nice to see how to do more than simple crud with rest. Thanks. I’m still wondering how to deal with creating a resource representing a user inside another resource. Lets say I’m making a sport league API with teams and members. A member is the representation of a user in a team where there can be information related to this user inside this team like shirt number or stats.

    When a user joins a team, a new member should be created, so that’s a new resource, but it’s also modifying the team by adding a member. The user would normally just click on join without providing more info. The other consideration is that you should not create a member for someone else. So a post on “/teams/teamID/members” would be weird. So perhaps a patch on the team with a join action would be fine, but it mainly creates a new resource.

    How do you deal with that situation in a RESTful API ?

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s