Web and cloud services allow third-party access by exposing application programming interfaces, but many developers and customers do not adequately secure the keys to the cloud and their data, experts say
Dark Reading
Attackers over the past three years have begun to actively target the digital keys used to secure the Internet infrastructure. Stuxnet’s creators stole code-signing keys and then used them to allow the malware to more easily evade host-based security. An alleged Iranian hacker broke into a partner of registry Comodo and bought Secure Sockets Layer (SSL) keys for major domains to eavesdrop on activists. And unknown attackers stole important information on RSA’s SecureID token, a device that generates one-time keys to strengthen online security.
The unique codes that applications in the cloud use to identify one another could be next, security experts say.
So-called API keys are used by Web and cloud services to identify third-party applications using the services. If service providers are not careful, an attacker with access to the key can cause a denial-of-service or rack up fees on behalf of the victim.
“It was created as a fairly nonauthoritative identifier — it was only there to identify applications or the application’s use of an API,” says K. Scott Morrison, chief technology officer of Layer7 Technologies, a provider of Web security and governance products. “The problem is that developers have started using API keys for stuff that matters.”
The problem is not any inherent weakness in the keys, but that developers use them for security when they ought not, he says. In many implementations, the keys are used to identify users, even though the technology was not meant as a way to authorize access to data. And after expanding the power of the keys, developers do not treat them as critical assets. Instead, companies fail to keep track of the keys, e-mailing them around and storing them on desktop hard drives.
“They shouldn’t be used for anything that matters, but people do. And when they do, they don’t take it as far as they need to,” Morrison says. “It’s kind of the worst of both worlds.”
During a presentation at the RSA Security Conference earlier this year, Morrison stressed the danger in the misuse and mishandling of API keys. The warning was repeated at the recent SOURCE Boston conference by application gateway maker Vordel. An improper implementation that allows simple access to an API via use of a secret key can allow attackers to have unmitigated access if the key can be sniffed out or stolen from an authorized user’s computer, said Jeremy Westerman, Vordel’s director of product management, at the conference.
“There is a need to protect these cloud API keys,” Westerman said. “There is a lot of awareness in the industry about protecting, say, SSL keys … Unfortunately, protecting API keys has not reached that level of awareness.”
Cloud and Web service developers must first follow best practices in opening up their APIs to third parties. In return, third-party developers need to handle the keys in a secure manner and not, for example, encode a nonobfuscated key into an application.
[Microsoft Research report shows how risky single sign-on can be without solid integration and better support from Web service providers like Google and Facebook. See Web Services Single Sign-On Contain Big Flaws.]
Communicating best practices can go a long way to fixing the issues, says Mark O’Neill, Vordel’s chief technology officer.
“The SaaS [software-as-a-service] providers expect you to protect these keys, but they don’t tell you how to protect the keys,” O’Neill says.
Companies that have API keys should treat them as valued assets, he says. The keys should be handled in much the same way as code-signing keys and other encryption material.
API keys were first used by Google, Yahoo, and other early pioneers of Web services. However, as the model moved from standalone sites to Web 2.0 mashups and the companies exposed their services for use by other websites, the weaknesses of API keys quickly became evident. Companies began to implement different schemes for application and user authentication, including OAuth, the Security Assertion Markup Language (SAML), and hashed-based authentication codes (HMACs).
The stronger authentication methods should be used for securing sensitive data, and each token should have a reasonable expiration time. In addition, because secret keys are occasionally exchanged, communications should always be over SSL, says Gregory Brail, vice president of technology for Web technology and services firm Apigee.
“The developer needs to understand the limitations and understand the best practices around implementing API keys,” he says.
Developers should still use API keys, Brail says. They should just use them for their proper function and use other tools as the situation demands.
“I’m not saying that there is nothing that can go wrong here; I’m saying that this is not a reason to throw away your API keys,” Brail says. “They are an important part of your whole security system.”
A popular usage of the Vordel Gateway is to take a legacy SOAP service, and deploy it as a REST API for consumption by mobile apps. Mobile apps can consume REST APIs, but you are unlikely to call a SOAP service from a mobile app. REST-to-SOAP conversion is very simple to achieve using a Gateway, because the Gateway can expose REST APIs which map to SOAP services, dynamically creating the SOAP request based on the REST API call.
In the video below, you can see an Android app running in the Android emulator, which is calling a REST API on the Vordel Gateway. At the Gateway, you can see the REST API request is being dynamically converted to SOAP. All without writing any code
When you zoom in, you can see the steps which convert the REST API to the legacy SOAP service call:
Enabling an API Portal with Vordel. In this video we see how the Vordel API Gateway provides the building blocks which customers use to create an API Portal.
I often think it’s ironic that while the mission of REST is to simplify Web development, REST itself is beset with seemingly complex jargon and architecture patterns. I say “seemingly complex” because, once you look into REST architecture in depth, it actually is simple. In some ways, it’s almost too simple. It’s easy to rack your brains about some REST pattern, but then realize: It’s just how the Web works. I’m reminded of the line from Moliere about the bourgeois gentleman who spends years trying to understand how he could speak in “prose”, then he exclaims “Good heavens! For more than forty years I have been speaking prose without knowing it”. So it is with REST. HTTP verbs, URLs, and resources are just what we’ve been doing for years.
There are various levels of REST though, most famously categorized in the Richardson Maturity Model. At the top of this maturity model is HATEOAS. Actually I think “maturity model” is a bit of a misleading name, because ideally you don’t want to start with a very brittle URL and then “mature” it to HATEOAS, since that puts undue requirements onto your client applications. It is better to start with HATEOAS principles from the start.
But what is HATEOAS (and how do you pronounce it?). Well, it stands for Hypertext As The Engine Of Application State. As with everything REST, the concepts come from Roy Fielding’s paper. The core idea is that the HTTP server serves out links which guide the client as to what they can do with a particular resource. So, for example, if I do a GET on a product listed in a catalog, I receive back a series of links which are the actions I can perform on it. I may be able to get the price of the product via one of the links, order it via another link, get back a description of it via another link, or get back its weight via another link. My application may also may be provided an “up” link back to the main list of products. The powerful concept is that the links are all the actions my app can take on the resource. So, if I am not allowed to order the product, then I will not be given the “order” link. The client app can then take this into account (“I wasn’t served up the “order” link for this particular product, therefore I will not provide the user with the ability to order it”). If you can order it, and do order it, then you may not be served up the “order” link for that product anymore, but (because you’ve ordered it) you may be served a link to view your order. This is the stateful aspect (it “knows” you’ve placed the order, and in fact that information is conveyed by the hypertext itself).
Another example is iteration through a set (for example, a list of products). As I am returned back each batch of results (e.g. the first 50, the next 50, etc), I get back a link to the previous batch of results (unless it’s the first batch, then there is no “previous” link provided) and to the next batch (unless it’s the last batch, then there is no “next” link). In this case also, all possible links are provided back to me. So it is also stateful (when I am on the second batch of results, it “knows” to give me link back to the first batch).
Providing all possible links, right inside the hypertext response, is a powerful concept. It’s very different from the SOAP/WSDL world, where you must look at the WSDL in order to find out what actions (operations) are available to you. With HATEOAS, there is no “WSDL”, instead the possible actions are in the response. The SOAP analogy would be where each SOAP response contained a WSDL that listed all of the subsequent operations the client could invoke. Another way HATEOAS is different is that, in the SOAP/WSDL world, if the WSDL changes, then SOAP/WSDL client apps must often be rebuilt. With HATEOAS, the service provider can add another capability, and it comes back as a new link in the hypertext responses, but this does not “break” anything. Similarly, removing a capability translates to removing a link, which should already be handled gracefully by the application.
With HATEOAS, the (hypertext) responses guide the client through their interaction with the resource, by providing links to the actions they can perform, so therefore the hypertext pages themselves become the engine of state. Hence the term, Hypertext As The Engine Of Application State.
[ Regarding pronunciation of HATEOAS, I've heard it pronounced like a breakfast cereal or chip snack ("hate -o's") and also I've heard the edgier "hate yo' ass". ]
So, how does an API Gateway cater for HATEOAS? A key requirement is not to break it. As in medicine, “first do no harm”. Consider what an API Gateway does: It provides external-facing APIs to the public, then translates them to back-end (usually on-premise) API calls.
Take for example the following JSON returned by a server to a client. The client has done a GET to get product information. The response includes a link to order the product, and it also an “up” link to go back to a higher-level view.
We see in the response above that the address of the back-end API server (myBackEndAPI:8000) is inside the JSON. When an API Gateway is used, clients go through the API Gateway, and should not go to the back-end API server directly [in fact, usually the API Gateway is cloud-based, and the API implementation is on-premises not directly accessible except through the API Gateway]. Therefore, the API Gateway must selectively replace the address of the back-end API server with the public-facing API server address, and it must do so throughout the responses, whether they are JSON, XML, or another format.
When requests come back from the client, it must also selectively change this address information, not only in the request URL but also in the payloads being POSTed or PUT to the REST API.
So how is this done? In the case of Vordel’s API Gateway, such content substitution is provided as standard, and is implemented in a high-performance manner so that the URL replacement required to support HATEOAS will not impact on the user experience for your users. Really, what is involved here is analogous to how an application gateway always operates, with the extra stipulation that it must convert information within messages also. But, substituting information within messages is simple with Vordel’s API Gateway, and already widely used such as in simple search/replace scenarios. All of this enables Vordel to support HATEOAS for our API Gateway customers.
Enabling OAuth To A Google JSON API Using The Vordel API Gateway
OAuth is a very popular way to manage access to APIs. Amongst other things, it enables users to log into an application (which could be an application on a phone or tablet) and then that application calls lightweight APIs on their behalf. From a user’s point of view, they accessed the application, and it all “just worked”. They didn’t have to enter any more passwords. From an implementation perspective, it’s more complex of course. But an API Gateway makes it simple to call lightweight APIs using OAuth.
When I say “Lightweight APIs” in the paragraph above, I mean REST Web Services which generally use JSON. And what is JSON? JSON is a lightweight alternative to XML, which is increasingly used to bypass heavyweight XML parsing (and, in some cases, to bypass the security controls of the XMLHttpRequest object often used in AJAX).
JSON and REST can be seen as more efficient optimizations of what went before. OAuth is far from being “just another authentication mechanism”. To understand OAuth involves understanding how it is used. For APIs, it generally involves an application (e.g. an Android app) accessing the API on the user’s behalf. Phil Hunt has an excellent explanation of this aspect of OAuth here:
OAuth2 supports a range of new use case scenarios. Many do not directly involve a user or a browser, but rather define a client application acting on behalf of a resource owner’s (e.g. the user) behalf using only HTTP to access REST based services in a lightweight fashion. From an authorization perspective, OAuth2 use cases introduce the new capability that client applications, each with their own identity, act on behalf of a users that own resources and can perform service calls with a specified “scope”.
[ It's also worth following the link above to Phil Hunt's piece if you're ever wondered why the HTTP header "Authorization" is not called "Authentication"]
So, all in all, there are a lot of new technologies at play here: OAuth, JSON, APIs, and even REST. And there is also a new concept, the API Gateway. An API Gateway applies the security mechanisms such as OAuth, which is required to authenticate to an API. But, beyond this, it also provides:
Monitoring of the API usage
Alerting if the API is not responding correctly
An audit trail of API access
Validation of API content
Redaction and tokenization of content being sent in API calls
Mapping to identity stores
Putting it all together, here is a demo of how a user can authorize the Vordel API Gateway to access a JSON API with OAuth on their behalf.
In the video below, I’m using an Android tablet to authorize the Vordel API Gateway to access an API on my behalf. In the Android environment, I am already logged in with Google, and the API Gateway leverages this to ask me do I want to access the API using my Google logon. In this case the API is a Google URL-shortening API. When I authorize it, then from that point on the Vordel API Gateway is enabled to access this API on my behalf. At the API Gateway level, you can see in the video that the API usage and access is managed. The API Gateway administrator can add controls such as JSON validation, detection of private data leakage, and tokenization (replacement of private data with an opaque token). At no point does the user have to worry about creating OAuth headers – this is all taken care of at the API Gateway.
This is the first in a series of videos on the API Gateway topic – watch this space for more!
An often-overlooked aspect of security is the Audit Trail. In the case of a REST API, we want to know not only that a REST API call was blocked, but why it was blocked.
Let’s take a look at the Real-Time Monitoring from the Vordel Gateway, deployed to manage a REST API. We see the orange spike indicating that an API call was blocked:
The key to looking up the Audit Trail is the message ID. Vordel users will be familiar with this ID as the ${id} attribute which is automatically created for each message in the Gateway. In this case, I highlight the message ID for the offending message and copy it:
Then I tab over to the Audit Trail and paste the Message ID into the search form:
When I press the Search button, I can see the message content, including the SQL Injection attempt which I have circled. The Vordel Gateway detected and blocked this attack against the REST API.
So, it’s important to know not only that a REST API call was blocked, but why it was blocked and what the REST API call actually was. This is the value of an audit trail. In addition, the Audit Trail logs may be signed, and the key used to sign them may be stored on a HSM (Hardware Security Module). All of the screenshots were taken from the Vordel evaluation image, which you can request more from info@vordel.com
A very common question asked by Vordel Gateway users is “How do I call out to an API or Web Service from within the Gateway, then use the response in a request to another destination?”. I notice that people often gravitate to the “Web Service Filter” or “Call Internal Service” filter for this, but that’s not actually how it’s done.
The way to do this is to use the “Connect to URL” filter. In the example below, I am calling out to a REST API from within a circuit on the Vordel Gateway. You can see that because the initial request to the Gateway is a POST (it’s a SOAP message coming in) and the call out to the REST API is a GET, I’m using a “Set HTTP Verb” to set the verb to “GET” before making the request to the REST API (a REST STS in this case, to request a SAML Assertion). I then am taking the response from that service, and using it in the construction of a request to the destination Web Service, which is an SP (Service Provider) that expects a SAML Assertion to be sent to it.
It’s quite easy, in this way, to string together many calls to different APIs or Web Services.
Note that if you want to store the initial request, make the callout “off to the side” to another service, then restore the initial request (i.e. not overwrite it with the response from the service you called “off to the side”) then that’s where the “Store” and “Restore” filters are used.
It’s natural to concentrate on the New, New Thing, but in the case of authentication web APIs, based on HTTP, there are options. There is certainly HMAC authentication for APIs, as shown in this Vordel Gateway case study. But remember that HTTP authentication still exists, and can be used for authentication to an API. If you want to test this, you can pick up the free Vordel SOAPbox tool which, although it has “SOAP” in its name, can be used to test REST APIs also. Here is how you do this:
Firstly click up on the title bar for your API call:
You’ll now see the “Request Settings” dialog. Notice that it’s a GET request (not a POST in the case of SOAP). I then choose the lower “Security” tab and the upper “HTTP Authentication” tab, and configure my parameters there. Notice also that mutual SSL and Kerberos are options too.
As enterprise architects meet the growing tide of API demand, where to now for the SOA infrastructures they have invested in over the past 5 years?
In this webinar, Vordel and our guest speaker, Randy Heffner, Vice President and Principal Analyst with Forrester Research, Inc. examine the evolution from SOA to API and outline how enterprise architects can leverage the investments they have made in SOA Application Gateways as part of a SOA infrastructure to ensure their organization efficiently manages their APIs and delivers on the overall API Strategy.
What you will learn:
Best practice reference architecture for API Delivery
How to leverage existing SOA infrastructure to manage API delivery
Identify challenges for securing and managing API delivery
