Web services are not new, but they are evolving into something that is increasingly important for software development companies to deliver. While you can enable your applications to access services provided by other developers, the real key to long term success is to create your own web service APIs to allow other developers to access your systems.
Why? Because doing so puts your software and data at the centre of an ecosystem of interlinked systems instead of living on the periphery, consuming services provided by others. That makes your system the controlling hub which others connect to; makes your data and business logic the vital resource that other come to rely on.
The changing landscape
Mechanisms for transferring data between systems have been around almost as long as computers: techniques such as file transfer (CSV, fixed-length text, etc.) long predate the Internet or even LANs, although with the arrival of those, network exchange of such things gradually supplanted sneakernet mediated transfers on magnetic tape or floppy disk.
That said, the uses to which these interfaces are being put is broadening dramatically.
The adoption of SOAP web services, starting almost 20 years ago, resulted not only in rapid growth of those kind of machine-to-machine interactions. It also produced an entirely new use of the technology, allowing web page content to be dynamically updated with changing data from servers – Ajax – leading to the richness of modern single-page web applications, which compare well in terms of usability with traditional desktop GUI applications – pushing many of those into the dreaded “legacy” category – while returning to a centralised control of IT resources that makes the management of complex systems tractable in a way that they had not been since the proliferation of PCs sent mainframes the way of the dinosaurs.
In a similar way, the emergence of RESTful web services, alongside the accumulation of vast amounts of data in the systems behind those new, but increasingly pervasive, web applications, has resulted in an explosion of new ways the technology is being employed.
Instead of web services being used purely in a system-to-system context, allowing one corporate application to access the monthly sales transactions, or customer addresses, or stock levels in another, they are now facilitating such things as insurance quotation aggregation, or airline flight comparison services and similar.
These however are not the main game-changer.
What has changed?
What has changed is the way users of systems are being provided with new ways of accessing their data – and the accompanying logic – within those systems.
Don’t like LinkedIn’s interface? If enough users agree with you, that turns into a business opportunity for a third-party software developer to build a different interface which uses LinkedIn’s RESTful API to deliver access via a different one.
An example might be a CRM application where the sales people can see the LinkedIn profiles and posts of customers as they access their records within the CRM system, possibly while talking to them on the phone.
Do LinkedIn object? Certainly not! What that third party (or internal corporate) developer is doing is enhancing LinkedIn’s product with an alternative interface (and quite possibly driving the sales people to form LinkedIn connections with their customers). LinkedIn’s value proposition is not based on their web interface or their mobile app, but on the professional connections that users form between one another and the posts they make to it. Allowing users alternative ways to access that enhances and compliments LinkedIn’s offering, detracting nothing from it.
The same is true of the wide range of service providers offering the same kind of thing: from Google’s vast range of APIs to Microsoft’s interface to Office 365, almost every major player employs them: Amazon, Facebook, LinkedIn, Salesforce, Twitter, Wikipedia, PayPal and many, many more. These companies have recognised that opening up access to their systems leverages their value and adds to the options available to their users.
There is, of course, one critical issue. Allowing users to sign into these services through third-party applications is all very well, but we don’t want users sharing their credentials with every app on the market!
The solution to that issue is OAuth.
OAuth (currently OAuth 2.0, which is simpler to implement that the original OAuth 1.0, although some providers like Twitter still use a version of that: OAuth 1.0a) is a mechanism for allowing a third-party application to act on a user’s behalf without having to know their credentials.
Without going into the technicalities in depth, essentially what happens is that a third-party application requiring access to a user’s data in a service will first direct the user to a login page for that service. The user can then login to the service and specify what scope of access they wish to grant the third-party application to their data.
The important point here is that this login-and-grant process occurs entirely outside of the third-party application: it is strictly between the user and the service which holds their data. In the case of web applications (for which the OAuth protocol was originally designed) this is generally achieved by opening a new browser window on the user’s device.
If the user grants some level of access, the service then returns an authorization code to the third-party application.
The third-party application then makes a request of the service for an access token, supplying with it not only that authorization code, but also its own client secret, known only to it and the service, which allows it to identify itself to the service. (Actually there are a number of other aspects to such identification: the client id and authorized redirect URLs, but we are keeping things simple here.)
If all is hunky-dory, the service will issue an access token, which will identify the extent and duration of the allowed access, and which the third-party will send along with each and every request it makes to the service on the user’s behalf.
Let’s look at a slightly different example which most of us will have encountered at some point or another.
When a user visits a store-front website and makes a purchase there, they may well be wary – and rightly so! – of giving their credit or debit card details to the unknown and untrusted web application involved in order to make the payment. However on many such websites they will have the option to use “PayPal Checkout – the safer, easier way to pay“.
The buyer can use PayPal to arrange payment for their purchase, additionally benefitting from PayPal’s “Buyer Protection” to ensure that if they don’t get what they thought they were buying, they will be able get a refund.
The vending web application will pop-up a new browser window pointing at PayPal’s login page (if the user is already logged into PayPal, PayPal’s application – not the vendor’s – may skip this step: that “Stay logged in for faster checkout” checkbox at work) and after logging in will be asked to authorize the transaction. If they do so, the sale can proceed.
This may not seem as though they are “delegating access” to the vending application to act on their behalf, but that is exactly what happens, using the OAuth 2.0 protocol. In this case the vendor first “Creates a payment” with PayPal, giving details of the purchase using an HTTP call to a RESTful web service (PayPal’s Payments API) from which it receives back a transaction ID, then pops-up a new browser window pointing to a PayPal sign-in page for the user to log into their PayPal account, passing it that transaction ID so that PayPal’s system can associate the authorization request with the previously notified transaction.
There the user can log in and choose from whatever payment methods they have registered with PayPal (or if they do not already have a PayPal account they can enter their payment method details directly).
Importantly this interaction is strictly between the user and PayPal – the vendor is not involved other than in initiating it. Once the user clicks the “Pay Now” button, PayPal redirects that browser window to the vendor’s “Redirect URL” informing them that payment has been authorized, so they can pop-up a “Payment Complete!” message and process the order.
PayPal collects the payment through whatever payment method the user has chosen and passes that onto the vendor, taking a small fee for the service.
All of this is done through PayPal’s REST API using OAuth 2.0 authentication and authorization. The vendor’s web application never sees the user’s credentials or just what the payment method was. The user only has to trust PayPal, not some web application they are encountering for the first time.
In this case the access is limited to effecting payment of a single transaction, but the underlying mechanism is the one discussed above (although without an access token ever needing to be issued, since the scope is so limited).
On the basis of that trust, PayPal’s relatively simple service (although to be fair, there is a bit more to them than just that) has built it into a $83 billion company in less than 20 years since its founding in 1998 (source: The Wall Street Journal).
So how can you do that?
If your company has one or more systems holding valuable user data you should be building secure RESTful APIs for them now.
It may seem like a daunting task, but the way to eat a whole elephant is one bite at a time – baby steps will get you started and lead to a powerful position of market advantage.
A step at a time
First create a RESTful API for some – possibly trivial – aspect of your system. It could be something such as a lookup list on payment methods or states or delivery methods and offer only read-only access. To look at one way of of doing this see our article: Creating RESTful JSON web services in DataFlex.
Once you have that working, you will need to secure access to it before exposing anything potentially sensitive, so a server certificate will be required to enable HTTPS. The cost of these ranges from very cheap (or even free) to several hundred dollars a year, so be sure to shop around for the deal which best suits your requirement.
Once TLS/HTTPS is set up you can add a simple passcode (an “API Key”) to allow access to the API. This might simply be be passed in a dedicated HTTP header field – say “APIKey” – at this stage.
Checking that the correct key has been passed with each call to the API should be isolated in its own section of code (probably a single function) which will be passed whatever credentials are to be checked (in this initial case, simply the API key) which will return a true or false result, as you will most probably need to make that checking more sophisticated as things progress.
Now you have an up-and-running secure RESTful web API – time to start adding functionality. What functionality that is will depend on the nature of the system involved. You should employ a little imagination at this stage, asking yourself what kind of information within your systems an external system, or remote user, might be interested in having access to.
The beauty of such an API is that once you have one in place, it is simple to extend, adding more functionality as required. You can think of it as a thin layer sitting on top of your system providing access to the underlying functionality – making calls to it as required. That may involve reusing code which is currently embedded in your existing system and at this point you should therefore break that out into libraries of some sort which can then be shared by the API and the existing system, if it is not already. Any differences in how that is to be served up by the API can either be built into the API directly, or into an intermediate library.
Having reached this point, whenever you find yourself in the situation of having to interface with a third party system of whatever kind you can simply say: We have a RESTful API you can connect to… they don’t have to know that your API doesn’t have that functionality built into it yet – adding it for any reasonable requirement will generally be quick and easy.
At this point you can start to add more sophisticated authentication, assigning each application requiring access to your system its own unique “client id” as well as a “client secret”, eventually reaching the stage when you can implement full OAuth authentication which will allow third-party developers to offer alternative ways for existing users to access and use their data held in your systems.
The Commercial Advantage
Having an API for third party developers to connect to immediately establishes your system’s place in a wider world. It becomes more than simply a stand-alone system, but potentially the core of multiple systems and applications based around its data and business logic.
The value of the data its users hold in it is enhanced by having the potential for being employed in new and different ways, based on the trust that they have in your original system.
You get to control what data is exposed and how – if at all – it can be modified by different external applications.
As well as increasing its value to users, it will also help ensure its longevity. If your system – through its RESTful API – becomes the centre of an ecosystem of interconnected applications, it becomes harder to replace, helping to ensure that it will not suddenly be swept aside by some management “new broom” with ideas about replacing “legacy” with shiny new toys.
In this context, the term “legacy” itself stops being a dirty word. Instead it comes to mean “tried and trusted” – something that has been around for a long time because it is good at doing its job and is something which can evolve and grow to meet changing conditions and challenges. Creating such a RESTful API is not only a part of that evolution, but it positively facilitates further such evolution. It allows that reliable business functionality which your users have come to depend on to move forward in ways that you probably cannot even foresee today.
Perhaps most importantly, if you do not provide such a capability, eventually a competitor will. You need to be creating this kind interface today, even if you do not see any pressing requirement for it yet: by the time such a requirement becomes obvious, it may well be too late to start playing catch-up.