RSS

Category Archives: contract-orientation

Judging XINS: The Good, the Bad and the Ugly

Looking Back

Back in 2002 or 2003, while working at an internet service provider (Wanadoo Netherlands, now called Online), we were constantly integrating frontend applications created by one team with backend applications created by another. To improve integration, our development team created a technology that would evolve into the contract-first development tool now known as XINS. It worked like a charm for the people involved (architects, developers, IT opeations, management and testers), getting nothing but praise.

But that is a while ago, and the landscape has changed; so how does XINS fare after about 8 years? Looking back, what is arguably good, bad and ugly about it?

The Good

  • The contract first approach really pays off in terms of quality and productivity. Once the contract is agreed upon by the involved developers and/or development teams, there’s suddenly a lot less room for discussion. And it’s fail fast; the contract is validated at both sides (client and server). This approach makes XINS unique and a real productivity monster. Developers are able to focus on the actual implementation code and hardly have to deal with after-the-implementation-discussions on integration issues;
  • Strong operational excellence (like failover, load balancing, error handling, logging, etc.) increases productivity and improves overall quality of these aspects. Part of the functionality is built into the framework, the rest is generated from the specifications.
  • From specifications, test forms can be generated. This is not only great for developers testing their own stuff, but also for other development teams, for testers and operations. It’s easy to peek at XINS-based applications.
  • Performance has always been very good, with automated performance tests in place to detect regressions. The overhead of the XINS runtime is typically less than a millisecond.
  • As long as it goes over HTTP, XINS supports various transport protocols, like JSON, XML-RPC, SOAP and the default: a simple browser-friendly protocol (HTTP GET/POST in, XML out).

The Bad

  • XINS still uses DTDs for validation of the specification files, instead of using the much more powerful XML Schema standard (or even RelaxNG).
  • The XML-based specification files use custom file name extensions, like .fnc, for function definitions, .typ for types, etc. This typically confuses editors and disables syntax highlighting.
  • Unit testing of function implementation classes is hard, they they require generated code (their superclasses). Nowadays, such classes would normally be POJOs.
  • The default protocol has some quirks: e.g. it requires XML support in the browser (which Safari does not offer), it is impossible to send empty strings (because these are interpreted as nulls) and errors are returned using the 200 OK status, which is not in accordance with the HTML spec, etc.
  • XINS 2.3 still uses various non-standard data type classes; this will be fixed in XINS 3.0. Examples of such data types include the PropertyReader (will become Map<String,String> in XINS 3) and the Element class (XINS 3 will use the W3C DOM API instead).

The Ugly

  • XINS tries to be too much at once; it not only generates stuff (code, documentation and test forms), but it also compiles compiles Java code, generates Javadoc, produces a WAR file, enforces an approach to application configurations, etc.
  • As a tool, XINS does not fit in well in the current ecosystem of (Java) build tools and IDEs. It has its own source directory structure, it has its own build system, it does not integrate directly with Maven, etc.
  • All transport protocols are supported, as long as it is HTTP. Local calls are supported, but then everything needs to be wrapped in HttpServletRequest and HttpServletResponse objects. Especially for binary data, this introduces a lot of overhead: binary data is converted to an inefficient format (Base64 or hex), transported in the inefficient format (overhead!) and then converted back to the binary format. Note that this involves storing the data in memory (more than) twice on the server side!
  • XINS hardly evolves: the community is minimal and development is slow.

Crowning the Successor

Overall, XINS is a productivity monster that addresses a topic no other tool appears to cover as good. But it has its share of issues; and integration with other processes and tools makes it less developer-friendly.

There is definitely room for an improved contract-first technology, especially if it would offer the following properties and features:

  1. Strong generation of code, documentation and browser-accessible test forms, similar to XINS.
  2. A runtime environment (client- and server-side) that provides strong operational excellence, similar to XINS.
  3. No attempt to be a one-size-fits-all; instead, make it easy to integrate this technology in existing contexts (tools, processes and technologies).
  4. Easy to pick up for current Java developers; this requires easy integration with modern build tools and IDEs.
  5. Transport-independence, supporting local calls, HTTP (including browser compatibility and efficient binary data handling), as well as other transports, with solid data type conversion.
  6. Strong integration with an existing component technology, such as Jigsaw or OSGi.
  7. Reusing an existing specification technology, such as WebIDL.

Conclusion

There is plenty of room for improving XINS, especially when it comes to adaptation to the current Java ecosystem. Still, in its current form, XINS is a mature technology that really shines when it comes to productivity and operational excellence.

Advertisements
 

The Importance of Contract-First Development

In enterprise IT projects, software services are typically consumed by other teams or even organisations than those who programmed those services. For example, a Billing system can be instructed by a User Profile system to allocate billing details for a new user profile. Or a Web Frontend system can ask a Session Management system for the data pertaining to a certain session ID.

This is typical service-orientation.

Expectation management
A challenge in such a service-oriented approach is making sure that all involved parties – and that can be or become a lot more than just 2 – have the same expectations from the service. In my experience this is the number one source of frustrations and issues.

Before a service is built or extended, the parties involved at that point in time typically reach agreement what the new functionality should do and how it should be accessed. It is then quite common to have those teams work in parallel, because time-to-market is key.

But even if the teams do not work in parallel, when the service provider delivers functionality for testing and integration, expectations from the other parties can be very hard to meet. For example, service consumers may have expected different formats or may have expected necessary transformations would be done by the service provider, while the service provider is pushing the requirement to the callers.

When this happens, project deadlines and stress levels are at stake.

And the more consumers and the more time it takes for the service provider to deliver a first version, the higher the risk.

Working with contracts
The common solution to this is to work with a contract that defines the agreements between the provider(s) and the consumer(s). Such contracts vary from informal (e.g. Word-documents lacking strict interface definitions) to formal (WSDL with very strict constraints).

Such contracts can be written at the start or they can be generated from existing code.

Generating a contract from existing code is generally a bad idea. The contract is then focused on the implementation, which is subject to change. Then if the implementation changes, what happens to the interface? If it should also change, this violates the common principle of Separation of Concerns. And it is clearly the wrong way around, the contract is the agreement, not the implementation chosen by the provider.

Another issue with contract generation is that the contract can be very much focused on the implementation, including specific data types tailored towards an implementation within a certain platform or programming language. This has an impact on the callers and, in the long term, also on the consumer, since the latter may be forced to stick with legacy implementation choices.

An implementation-independent contract
A first step towards a complete solution to this challenge is to author a contract that is implementation-independent. Such a contract can be agreed upon from the start by all involved parties. Real separation of concerns: the contract is the common agreement, while providers and consumers are free to make their own choices within the bounds of the contract.

This may appear a complete solution.

However, in practice, contracts change, possibly even during the initial implementation project. So what should be done in such a case? Is the contract still leading or will the contract by bypassed by reality?

Based on my experience in professional environments I dare say that if the contract can be violated, it will be. With impact on stability/reliability and on stress levels. In the battle between contract-orientation and project deadlines, the former is easily sacrificed.

Contract-first development
The only way to really guarantee that you can keep the promise of an independent contract is to integrate your contract-orientation in your development process, making the implementation(s) dependant on the contract, preferrably on both sides of the contract (providers as well as consumers).

One way to achiebe this is to make your code link to the contract somehow, for example by having a URL to your WSDL files in all implementations, making sure that failures to adhere to the contract result in errors.

Validation at runtime
To avoid the contract being bypassed, implementations should check all service invocations to make sure the contract is respected.

This can be achieved by generating server- and client-side code from the contract. Having access to the contract (in some form) the implementation then checks all aspects of the invocation. Not only the  preconditions (consumer requests), but also postconditions (provider responses). If either a consumer or a provider violates to the contract, that must be treated as a fatal violation. Only then your contract will actually be leading.

Validation on the server versus on the client
So if you want to validate your contract, where should you do it: on the client (consumer) or on the server (provider)?

The best approach to this is to do it in both locations, for good reasons:

  1. Separation of concerns: both ends should assume their responsibility, which is to adhere to the contract and consider violations fatal. Neither end can be sure the other end will validate the contract, so they should not assume that.
  2. The provider and the consumer may have access to different versions of the contract. Validation on both ends will likely catch any incompatible differences in the contract versions.

Your contract is your treasure!
Now when you have accomplished a true contract-oriented development process, you will find the contract is your gold mine. The contract is typically orders of magnitude simpler than implementations, making it easy to read and change. But since it describes an extremely important integration point so well (concise) and so correctly, it is easy to reuse it for all sorts of of purposes. For example:

  • generation of documentation, e.g. Word-documents or web pages, etc.
  • generation of forms or programs for manual use (including human testing)
  • generation of unit-, performance- and load-tests
  • generation of IDE projects, such as Visual Studio, Eclipse, NetBeans, XCode, etc.

To enable certain possibilities, the formal part of the contract needs an additional informal part. The informal part can include examples and human-readable descriptions. They can also help humans understand the contract and its purpose faster and more easily.

Examples of contract-orientation
While WSDL is an example of a contract definition language for web services, it does not provides a full contract-orientation process, requiring the contract to be held.

The only fully contract-oriented projects I am aware of are those I initiated: XINS and Logdoc. The first (XINS) is an open-source web services framework (that generates server- and client-side Java code), while the latter (Logdoc) implements contracts for logging, again including a Java-implementation.

Do you have experience with contract-orientation? Do you know of other projects that put contract-orientation in the heart of their approach? Please let me know in the comments!

 
3 Comments

Posted by on 17 April 2010 in contract first, contract-orientation