Our VDF Ajax JSON interface
(If you want to skip all the waffle fascinating technical explanations and go directly to the sample, it is here.)
Motivation for Using JSON as the Communication Protocol
The motivation for the VDF Ajax JSON Interface example and this accompanying article (or set of articles really) stems from a presentation that I (Mike) gave at the Data Access Synergy 2011 conference in Orlando, Florida. The subject of that presentation was JSON, but with 40 minutes available, and JSON being such an elegantly simple format, I felt that there was scope for demonstrating it in action, rather than just explaining it. To show that JSON was capable of being used to represent even very complex data I picked on one of the most complex web service interchanges I had close to hand: the Request method of the VDF Ajax library.
It wasn’t entirely easy to get working, but I managed to get it all together in time for my presentation and everything worked on the day – which I am sure anybody who has delivered technical presentations will know is not always the case. However when I looked out at the audience at the end of the lecture, I was met with a hall-full of uncomprehending, glazed-eyed faces. Oh dear! Mike’s gone over the top again!
So… this is an attempt to rectify that … excess?… by providing an explanation that people can take at their own pace, rather than having it packed breathlessly into the final 15 minutes of a conference presentation.
Our Approach to Using JSON as the Communication Protocol
VDF has been capable of returning JSON data from its web services (whether using the Ajax library or not) since the release of Visual DataFlex 2009 (VDF 15.0), however those web services can normally only (as at time of writing and VDF 16.1 at least) be invoked using XML/SOAP, or, if they take only primitive data types (i.e. not struct or array types) as input parameters, by using “name=value” pairs with either the HTTP GET (with the parameters in the query string of the URL) or HTTP POST (with the parameters in the HTTP BODY) requests.
To use JSON data as input to, as well as output from, VDF web services, we had to use a number of techniques which are interesting (if in some cases arguably inadvisable – see Sture’s health warning) in their own right. For that reason we have broken the various explanations down into a number of pages dealing with different aspects of the sample:
- The JSON protocol
- The VDF Ajax web service
- The JSON Parser
- Some degree of introspection (modifying Visual DataFlex)
- The JSON web service
We first had to find a way, within a VDF web service, of being able to take JSON data and make sense of it – to parse it in effect. In this regard we were fortunate in that Data Access’s John Tuohy had – partially in response to some of us complaining about not being able to use JSON data in VDF – written a JSON parser, which, with a little modification, filled that requirement.
Next we needed to find a way of automatically placing that data into one or more variables of the (complex) data types that the VDF Ajax web service took as its inputs. John’s parser places its decoded JSON data in a very generic structure known as a “ValueTree”, which is a recursive “struct” made up of a string “value” and an array of child ValueTrees. It turns out that there already existed a VDF command, developed for getting the data returned by web service client calls into VDF data structures, called “ValueTreeDeserializeParameter“. This we co-opted, renaming it “MoveValueTreeToStruct” to make it more understandable in this context.
The key to making these pieces work together was being able to tell the JSON parser about the structures it was to place its JSON data into. That could not be determined from the JSON data alone, because the order of the properties within a JSON object is not guaranteed, while the “MoveValueTreeToStruct” command requires them to be in exactly the order they appear in the target struct.
This is where Some Degree of Introspection comes in. Sture Andersen worked out a way of determining (recursively) the elements of a VDF struct data type and making that available to the program at run time, allowing the parser to put the decoded JSON data in the right places in the ValueTree to satisfy the MoveDataToStruct command.
Then we built a second web service, to be placed in the same web application as the VDF Ajax web service, but for each of the Ajax operations requiring complex input, it would instead accept a string (containing the JSON data) which it could then translate into VDF struct variables before calling its counterpart in the Ajax web service with them.
The outcome of all that can be seen as our VDFAjaxJsonInterface.
Access to the Code
We have not made the entire sample available for download (although that may change if there is much demand), but if you would like a copy please feel free to get in touch, either by phone (the number at the bottom of the page) or by e-mail.
We must point out that the need for all of this is only temporary, since at some point in the not too distant future Data Access will almost certainly build “JSON-in” capability directly into the product, making this complicated series of work-arounds unnecessary.
Finally, we would like to thank a number of people whose various contributions – begged, borrowed, stolen (or, in the last resort, bought and paid for) – made this sample possible.
Obviously the team at Data Access Worldwide for the Visual DataFlex product itself, but especially John Tuohy for the original JSON parser. The VDF Ajax team at Data Access Europe for the excellent library that we are so ruthlessly abusing here – most notably Eddy Kleinjan and Harm Wibier. Lastly Sture Andersen of Sture ApS for his creative and insightful solution to the problem of how to get Visual DataFlex to understand its own data structures.