background image background image background image

API Quality is more than meets the eye

Benny Boye Johansen

Head of OpenAPI, Saxo Bank

As Saxo is working to create the vision for our next version of OpenAPI, we are looking for many sources of inspiration as to what needs to get improved. One initiative is a collaboration with the Department of Digitalization at Copenhagen Business School with the purpose to assess API quality and how different ways of working, different design methodologies, and different forms of governance may affect delivery speed and API quality.While the project is just getting started, it has already provided me with a first "aha" moment, or insight as you may say.

One of the first activities was to gauge the current quality of our OpenAPI. Instead of just relying on our own gut feeling about what may constitute a “good” API, the CBS team wanted to derive a more objective measure. Being quite thorough and academic, they scoured a long list of 59 resources, including developer blogs, academic articles, and vendor material to identify attributes which were mentioned as important to API quality. They subsequently grouped these into categories, themes, and indicators, and presented these to a number of OpenAPI front-runner companies, such as Saxo for feedback and refinement.

Their findings will later be published and detailed in a CBS working paper, but I have been allowed to provide a small overview/summary.

OpenAPI Quality Indicators

The table below summarises CBS’ early findings. All stated measures are divided into two broad categories (Product attributes and Process Attributes). Each category comprises several themes (Documentation, Security etc.), each of which then includes a number of quality indicators, the number of which are included is indicated in brackets.

 Product AttributesProcess Attributes 
 Documentation (3):

Is the documentation correct, comprehensive, etc.?

API Inventory (3):
Are there processes and tools in place to support proper inventorying of the API landscape?

Security (6):

Are security requirements satisfied?

Guides and Policies (4):
Are there design guides and testing guides and are they followed?

Design Conformity (14):

Does the API conform to internal standards, and “best practices”?

Internal Tooling (4):
Is there tooling to aid design, ensure conformity and adherence?

REST API (3):

Does the API conform to common REST design patterns?

Testing (13):
Is adequate testing done at relevant stages in the development process?

Scalable (9):

Is the API scalable and is performance monitored?

API Governance (7):
How is the API development governed and aligned with company strategy?

Metrics (15):

Are relevant technical and commercial metrics readily available?

Approach (8):
API First, Code-First and integration into the general s/w development lifecycle. Does the chosen approach match with expected outcome?

Standard software components (4):

Does the offering include a developer portal, an API Gateway etc.?

Development Process (9):
Are common best practices for s/w development followed?

Industry Standards Adherence (5):
If relevant, does the API conform to established industry standards?
 

So how good is your API really?

The interesting part – for me at least – was that on a day-to-day basis my team of OpenAPI Platform managers primarily worry about a small subset of the comprehensive list, and mostly about the attributes on the left in the table.

We constantly ask ourselves:

  • Does our API follow the agreed design principles, is it “fit for purpose”, is there consistency within and across domains?
  • Is the API stable, performant etc?
  • Do we have sufficient documentation, sample code, and support in our demo environments?

But what about all the other stuff? Why are we not thinking about that, and why would we not have come up with these measures, if you had asked any of us how we would assess API quality?

Perhaps it has to do with the "Four Levels of Competence" model, first developed by Noel Buch in the 1970s, which states that in almost all learning we go through four levels, as shown in the table below:

 Competency levelsWhat it means 
Unconscious incompetenceWe are really poor at something, and we don’t even know it, so we don’t do anything to address it. 
Conscious incompetenceWe are poor at something, and we know that we have to improve. 
Conscious competenceWe understand how to do something (well), but it still requires effort. 
Unconscious competenceWe are so competent in this area, that it is second nature and can be performed easily. 
Source: https://en.wikipedia.org/wiki/Four_stages_of_competence


Note that according to this, we will have a blind spot to what we are really poor at (unconscious incompetence) but also what we are really good at (unconscious competence).

This reminds me of a job interview I had a long time ago. The interview went quite well until we got to the stereotypical question of “so what are you good at”? I remember giving a passionate description of a problem I had recently solved (to my own great satisfaction), and I was pretty happy with that answer until a colleague praised me for something, which I actually thought was super easy and did not think much of. I then realised that my answer at the job interview probably was not my real strength – it was just what I was conscious about at the time, and in this case consciously competent at.

Time for an API 360?

The comprehensive and somewhat academic and theoretical work eventually published by CBS is an important reminder for both developers and consumers of OpenAPIs.

It is very natural for a team like mine to sometimes have a too narrow focus. It is easy for any engineering team to be too much “inside out” rather than “outside in” and forget what matters most for our clients. As discussed above, we are by default also limited to focus on what we are conscious about.

For us, a checklist like the above (even in its skeleton form) is a good reminder to consider all of the indicators of good API design and good API development practices.

The same goes for potential OpenAPI consumers, such as our partners. It is quite tempting to only assess the product you can see directly on our OpenAPI developer portal. “Do they have an API for this, does it appear to be well documented, do they have available samples etc?” But they are probably well advised to also consider some of the more invisible qualities: “Have they got adequate security measures in place”, “Do they have proper software development and testing practices”, “Is the underlying system scalable” etc.

In summary, developing and managing a comprehensive OpenAPI supporting hundreds of partners and millions of trades from over 180 countries is a multifaceted challenge. To be successful, both as providers and consumers  one must consider many factors and remember that there is more than meets the eye when it comes to a true assessment of OpenAPI quality.

Disclaimer

The Saxo Bank Group entities each provide execution-only service and access to Analysis permitting a person to view and/or use content available on or via the website. This content is not intended to and does not change or expand on the execution-only service. Such access and use are at all times subject to (i) The Terms of Use; (ii) Full Disclaimer; (iii) The Risk Warning; (iv) the Rules of Engagement and (v) Notices applying to Saxo News & Research and/or its content in addition (where relevant) to the terms governing the use of hyperlinks on the website of a member of the Saxo Bank Group by which access to Saxo News & Research is gained. Such content is therefore provided as no more than information. In particular no advice is intended to be provided or to be relied on as provided nor endorsed by any Saxo Bank Group entity; nor is it to be construed as solicitation or an incentive provided to subscribe for or sell or purchase any financial instrument. All trading or investments you make must be pursuant to your own unprompted and informed self-directed decision. As such no Saxo Bank Group entity will have or be liable for any losses that you may sustain as a result of any investment decision made in reliance on information which is available on Saxo News & Research or as a result of the use of the Saxo News & Research. Orders given and trades effected are deemed intended to be given or effected for the account of the customer with the Saxo Bank Group entity operating in the jurisdiction in which the customer resides and/or with whom the customer opened and maintains his/her trading account. Saxo News & Research does not contain (and should not be construed as containing) financial, investment, tax or trading advice or advice of any sort offered, recommended or endorsed by Saxo Bank Group and should not be construed as a record of our trading prices, or as an offer, incentive or solicitation for the subscription, sale or purchase in any financial instrument. To the extent that any content is construed as investment research, you must note and accept that the content was not intended to and has not been prepared in accordance with legal requirements designed to promote the independence of investment research and as such, would be considered as a marketing communication under relevant laws.

Please read our disclaimers:
Notification on Non-Independent Investment Research (https://www.home.saxo/legal/niird/notification)
Full disclaimer (https://www.home.saxo/legal/disclaimer/saxo-disclaimer)
Full disclaimer (https://www.home.saxo/legal/saxoselect-disclaimer/disclaimer)

Saxo Bank A/S (Headquarters)
Philip Heymans Alle 15
2900
Hellerup
Denmark

Contact Saxo

Select region

International
International

Trade responsibly
All trading carries risk. Read more. To help you understand the risks involved we have put together a series of Key Information Documents (KIDs) highlighting the risks and rewards related to each product. Read more

This website can be accessed worldwide however the information on the website is related to Saxo Bank A/S and is not specific to any entity of Saxo Bank Group. All clients will directly engage with Saxo Bank A/S and all client agreements will be entered into with Saxo Bank A/S and thus governed by Danish Law.

Apple and the Apple logo are trademarks of Apple Inc, registered in the US and other countries and regions. App Store is a service mark of Apple Inc. Google Play and the Google Play logo are trademarks of Google LLC.