SOA and System Integration

There is nothing mystical about service-oriented architecture

Integration is essentially application development, and it is guided by the same principles as all software development projects. Successful system integration requires a well-structured organization and simple technologies.

SOA, or service-oriented architecture, refers to architecture in which existing systems are integrated using public service interfaces. A sound integration solution is based on open and proven technologies, a distributed implementation model and a clear interface design.

Good system integration requires clear responsibilities

The first thing to do when building a distributed integration model should be to clarify the responsibilities of system owners within the organization. Aside from network infrastructure, the gray zone of integration between the systems should be kept to a minimum. Each system owner should be responsible for the interfaces of their systems and their interoperability with the interfaces of other systems.

The task of integration support is assigned to an organization that ensures that agreed practices are followed and offers technical support. A dedicated integration organization may not be necessary if system interfaces are clear enough.

The opposite of a distributed integration model is a large, centralized, enterprise-wide integration solution. In a solution like this, the responsibilities for the business logic of systems and the technical aspects of integrations can become blurred. This means that a distinct integration system emerges alongside actual business systems, requiring its own maintenance organization. A support organization like this does not generate any added value for business, but it does increase costs and adds unwanted complexity.

System integration is application development

The products on offer for integration purposes often claim to be a cure-all solution. Yet products such as ESBs are ultimately just application development and runtime environments used to implement the actual integration solutions. Implementation here means programming, which is why ESB products should be compared to other programming environments. Based on my experience with ESB products, they do not stand comparison with other application development and runtime environments and the expectations of the gains to be had are often unrealistic.

When comparing solutions, the key points to consider are the characteristics of the programming language used, the testability of the solutions to be developed, and how well the environment supports good programming practices. It matters greatly for the cost and quality of the final solution whether an open or closed, or simple or complex technology is chosen. You do not want to tie your integration model to particular product choices before establishing what is needed.

An ill-advised, product-driven solution model adds unnecessary complexity. This will show on the technological balance sheet as costly maintenance, poor quality, slow progress and being tied to products and solutions that are expensive to get rid of. By contrast, the simplification and continuous development of solutions are investments that will start paying for themselves immediately and enable the continuous rationalization of operations.

The chosen software development environment has a bearing on development costs. The characteristics of programming languages are important, as are the license and support fees for the development and runtime environments and the hardware they require. One of the key selection criteria is the ecosystem that emerges around technologies, involving open-source libraries created by the user community, support networks and, and the actual user base. Closed solutions typically have considerably smaller user communities than open and free development environments. Furthermore, finding expertise and standard components is difficult in closed environments.

The Internet is built on open and proven technologies

The simplest, and thus also the easiest, cheapest and least error-prone technologies can be found in the solutions on which the Internet is based. No corporation can boast internal solutions that are as tried and optimized as the Internet technologies, which have to withstand a constant flood of hundreds of millions of users, evolve continuously and be secure to use.

The ones to use for system integration are simple Internet technologies such as the light REST/JSON interfaces and simple message queue systems without ESB features. The provisioning, securing and routing of services can be handled using standard technologies such as simple HTTP servers and DNS name servers instead of the closed ESB and directory services. There are a number of viable open-source platforms that have millions of users and the credibility to last well into the future. They do not live and die by the individual product choices of corporations.

Integration is primarily an administrative challenge

The technical challenges of integrations are far smaller than the challenges related to their administration and coordination. Setting up distinct development and support models and integration server applications just for integration purposes adds overhead and complexity and slows down development.

Integration is application development. As in application development in general, your best bet is to use good practices and tools, simple technologies and architectures, and competent development teams.

Jari Mäkelä, Chief Architect

Jari Mäkelä has over 15 years of experience on software development and challenging system integrations in different environments.