API Design & Testing State of the Art

This post was originally published on the Nordic APIs blog as “API Design & Testing State of the Art”. Ensuring proper APIs functionality raises all the following questions:

• How do you decide which endpoints you’re going to expose with your API?
• How do you create all the documentation that developers will use to interact with your API?
• What is the best way to verify that your API is working as specified by the documentation, and in keeping with the developer’s expectations?
• Who should be involved in modelling APIs inside your organization?

By the time you finish reading this article you should be able to answer all these questions. API design and testing are very hot spaces right now. This is because they are the primary means of guaranteeing that APIs not only reflect business processes, but are being used in a way that will generate value to both consumers and providers.

We’ll first cover the API design space, presenting the tools and standards that will help you offer the best possible developer experience. Then we’ll dive into the API testing space, helping you gain complete understanding of what you can test, and how to ensure that your API will keep working repeatedly, and according to expectations. Finally, various industry experts will share their opinions on what the future holds.

API Design #

API design is the act of modeling an API taking into account three very different factors:

• Usability: how the API will be used, by whom, and what specific use cases need to be addressed.
• Security: what information is handled by the API; how it should be stored and transported; who can access what data — and when.
• Performance: how many consumers the API will have, how long can they wait for a response, and what internal systems affect the API speed.

Designing an API can start in many ways, from brainstorming to diving directly into the code. However, it should always produce great documentation that can be used by developers, and, if possible, programmatically by consumer apps. It’s considered a good practice to start an API by documenting it and then using the documentation to generate the code that will implement it.

This article describes three tools that let an API designer start by describing how an API works, then how to feed tools that will generate the code that makes it work.

Swagger #

Swagger is the work of Reverb Technologies, a company created in 2008 that builds language-related products. One of those products, Wordnik, was the main reason they created Swagger. Wordnik API support requests were increasing and they couldn’t find an easy way to document the API and support the SDK at the same time.

developer.wordnik.com was the very first Swagger-powered interactive documentation website. It helped Reverb define what Swagger came to be, and iterate on its original implementation until open-sourcing it in 2011. There are now over 40 server-side integrations with Swagger and an estimated 10,000 APIs are using it to power their documentation.

[Swagger] is successful, in my opinion, largely because it is language agnostic, vendor neutral, and simple. It doesn’t try to solve all use cases, and is opinionated to a degree. This helps keep the specification lean and very effective. — Tony Tam, Reverb CEO

Swagger is an interesting approach to API design because it allows developers to write the API definition only once, and it automatically generates server code, client code, and the final documentation. Swagger-generated documentation is interactive. This means that developers can immediately try to call API endpoints without having to write any code.

There are different third-party tools following the Swagger specification to help developers better implement their APIs. One of these tools, Apigee-127, lets you model your API using the Swagger editor and generate the code that actually implements the API methods described in the documentation. This is an interesting approach because it lets you easily iterate on your API design until you’re satisfied with the result.

RAML #

RAML was initiated in 2013 by MuleSoft when they became “frustrated with the lack of a clean, clear way to describe practically-RESTful APIs such that you could build great experiences around that description”, says Uri Sarid, MuleSoft CTO. RAML was created with the goal of assisting the API Design process with a special focus on letting you reuse the documentation.

RAML’s main goals are to provide a specification that allows you to create documentation that is humanly readable, simple to understand, and can be broken down by patterns. By meeting these goals, RAML offers a unique combination of features that allows developers to easily extend it by building tools.

RAML isn’t a product […], it’s really about an API developer wanting the ability to “whiteboard” an API, build that API, then manage and provision it. — Sumit Sharma, MuleSoft Director of API Solutions

RAML development is used by a broad group of companies including MuleSoft, Intuit, Box, PayPal and Cisco. This has contributed to the creation of a consensual specification that is now in use by numerous large-size companies. According to SOA Software, RAML is among a short list of API specification formats gaining adoption in the enterprise space.

There are now hundreds of API portals using RAML and a combination of compatible tools. One of these tools, the API Designer lets you easily edit RAML using your browser. This tool, combined with a code generation utility like the JAX-RS Codegen, will help you quickly iterate on your API design.

API Blueprint #

Apiary, an API design company, created the API Blueprint at the turn of 2012/2013 because they found that there wasn’t a common and simple way to describe APIs. API Blueprint is unique because it uses the popular Markdown format that most developers love.

Our goal […] was to create a fundamentally very human-readable/writable format that was easy to use, but also powerful in what could be generated out of it. — Jakub Nesetril, Apiary CEO

The API Blueprint format is available to use and modify under the MIT license which lets any individual or company build on top of it, even commercially. Among these are SmartBear, a company that builds API-related tools. SmartBear offers an API Blueprint for their popular SoapUI Pro tool which lets enterprises easily test their APIs.

There are now more than 100,000 API Blueprint-powered documents in the wild, making it probably the most used API documentation standard. The number of available tools also makes its adoption very easy and straightforward, letting you render documentation into a number of formats, generate code, test API implementations, and even generate API Blueprint documents from existing code.

API Testing #

API testing is often associated with API design because it’s a way of closing the iteration loop. After your API is modelled, you generate the server code and then run exhaustive tests to understand how it will perform in the real world.

You should focus on three factors when testing your API:

• Usability: how easily can your API be used by developers. This kind of testing is often performed by multiple developers with different backgrounds.
• Performance: how fast your API can respond to requests. This test can be performed by an automated testing tool like the ones described in this article.
• Compliance: how compliant your API is to the documentation. (Can developers easily follow the documentation and make requests to your API?)

Remember that testing is an ongoing process that you should run repeatedly in an automatic fashion, if possible. If you don’t proactively test your API someone else will — and, if they find any issues, it can create a negative marketing impact on your organization.

Let’s look at two tools that help you go from API design to API testing in almost no time. We describe POSTMAN, a browser-based application, and Runscope, which offers cloud-based testing.

POSTMAN #

POSTMAN was created by Abhinav Asthana in 2012 as a strategy for working more easily with APIs. Abhinav was working at Yahoo! at the time, and couldn’t find any tool that would let him easily test API calls and share the results with other developers.

At that time I tried out tools like Curl and I wasn’t very happy with them. […] What I saw as a basic problem was the inability to communicate to other developers how a particular API worked. — Abhinav Asthana, POSTMAN Founder

POSTMAN is available as an installable Chrome app, which is now being used by more than 600,000 developers. POSTMAN also offers some interesting tools such as Newman which lets you run POSTMAN test collections from the command line and the Interceptor, which makes it easy to intercept and inspect any HTTP call made with Chrome.

Runscope #

Runscope was launched in 2013 as a way to easily inspect API calls traffic. John Sheehan joined forces with Frank Stratton to solve common problems they were finding while working on their previous jobs at IFTTT and Twilio.

Runscope was born out of the frustration I was experiencing working with so many different APIs […] Every day a new one would break, have performance issues, or be impossible to integrate with. — John Sheehan, Runscope CEO

Runscope is now built and maintained by a team of 13 and they’re growing rapidly as they see increasing demand. John Sheehan believes that APIs are “the one technology that is common to every language and framework and every part of the stack from infrastructure to end user”, especially on enterprises.

The Future #

There are still many challenges to be solved in the API design and Testing space. This requires that tools evolve to bring more value to the table by allowing people with different skills to collaborate in the definition and verification of APIs. Kin Lane, who’s been working in the technology space for more than 20 years, and is best known as the API Evangelist, believes that “[…] we are going to need a robust stack that helps everyone participate in the API design process, including non-developer business owners”. Abhinav Asthana, from POSTMAN, also agrees that “if all stakeholders who are involved in the development and testing of APIs can collaborate faster, APIs are going to be more robust”.

Another big challenge is how developers should cope with the growing number of APIs that organizations are releasing. With so many releases there will be new “API products [and] ensuring that they can create synergy with the existing APIs can be extremely difficult”, says Jason Harmon, Head of API Design at PayPal. API design applications should provide features that let people define business logic without having to worry how different endpoints talk to each other.

We should all be working hard on business logic and let tooling take care of the plumbing. — Tony Tam, Reverb CEO

APIs should also be treated like any other product where you are “[…] using metrics & feedback to understand customer behavior, and use it to make product decisions about how to evolve your API”, says Jakub Nesetril from Apiary. By following a metric-centered approach you can better align your API with the needs of developers and, in the end, improve the way your customers interact with your app.

A good question when speaking of metrics and customer behavior is what is considered success. In the product development space success is often associated with more actions being performed, or with less time being spent to perform them. While this is still a blurred vision in the API space, John Sheehan from Runscope affirms that “[…] it’s going to be possible to test and monitor an API in the future without having to define up front what success looks like”.

As you can see, the future looks very bright and we’ll almost surely be looking at better tools in the next few years. API Design and Testing is playing a bigger role in businesses and is expanding out of the technical realm.

Conclusion #

API design is not a job exclusively for technical people. Because APIs can affect how businesses operate, different people inside organizations should be able to contribute to the process of modelling APIs. You should now have a good idea about which tools you can use to design your API and, at the same time, produce ready to use documentation.

We’ve covered three ways to design APIs that range from a more technical approach to using a document format that any person can read and understand. Swagger is interesting because it lets you easily generate server and consumer code while not being difficult to understand. RAML is more positioned at the enterprise level, and offers a very comprehensive standard that lets you specify almost all aspects related with an API. The API Blueprint standard is probably the easiest, as you can use Markdown to write your API specification which is, at the same time, its documentation.

On the API testing space we covered POSTMAN, a browser based testing tool that also lets you share API calls with your co-workers, making it easy to collaborate on API consumers. We also covered Runscope, a powerful cloud-based tool that also lets you automate API testing and alerts you when something is not working as expected.