Minimal IT - Minimal integration 12: integration summarised

20 December 2005

Minimal integration 12: integration summarised

Effective systems integration requires a clear idea of what you are trying to achieve and an awareness of the common problems to avoid.

Over the years I have seen, and caused, many problems with integration. Most of these problems come from a handful of common mistakes. We make integration hard by using the wrong technologies, or by burdening ourselves with grandiose strategies. We cause dependency problems when we do no not separate systems well, or when we mix integration code with business code. We delude ourselves by believing in integration tools too much, or by pressuring package vendors to provide integration solutions that they can not then deliver.

We can make integration simple and effective if we are clear about what we are trying to achieve, and avoid these common problems. Here's a summary of the key points I have covered over the past few weeks.

Consider integration as not merely connecting systems, but connecting them in such a way that they can be changed independently. It should be possible to change the technology, timing, internal processing and internal data of any system without impacting another.
Define clear boundaries between systems. Base these on differences of management responsibility and technical differences. It is more important that boundaries are clearly defined than that they reflect a rigorous logical model of functionality.
Define interfaces to move whole meaningful business transactions between systems. Have a clear definition of the logical unit of integration (LUI) in any interface.
Use XML to encode data as it passes from one system to another. XML copes with the technical differences between platforms, and the complicated structures required to fully encode an LUI.
Use only a simple subset of XML's features.
Understand that XML can be processed in different ways, either as an easy-to-use object, or as a fast stream of data.
Standardise on a small number of appropriate data transfer technologies, such as messaging middleware, file transfer, and web services. Do not use technologies that force a high degree of dependence between systems, such as direct database access or calls to remote objects.
Consider the overall integration architecture as a bus, in which standards allow any system to connect to any other system. Allow for hubs which centralise some integration activity. Recognise that some aspects of integration, such as business meaning, are effectively point-to-point.
Base your integration strategy on standards, not tools. Be very wary of the claims of integration tool vendors. A simple approach and hand coding are often more effective.
Write simple informal documentation. Rely on it during development, and keep it up to date. Do not be diverted into grandiose documentation schemes.
Keep integration code separate from business code. Structure the code so that it is easy to change the data transfer methods.
Apply these rules to integrating packaged software. Expect to build an additional interface to bridge between your standards and whatever the vendor supports best. Do not abdicate the integration to the vendor.

I believe that this approach to integration is as simple as it can be, and as complicated as it needs to be, for effective systems integration.