On Integration of Complex Systems

· Thoughts and opinions are my own, albeit cybernetically enhanced.

Integration is the process of putting together parts of a system into varying levels of assembly, observing the assembled object for behavior and performance, and feeding back those observations to engineering for architectural refinement.

If you build anything, you are doing integration. Integration isn’t a process that merely adds value. You cannot build a product without integration, whether it is consciously part of the engineering process or not.

“We don’t need to do systems engineering.” You are already doing systems engineering. You see a need and say “we need a thing that solves this problem” (user story, requirement); you design the thing (architecture), implement it (implementation), then run it either once on your machine or directly in production (integration) before getting feedback from the operator of your solution (acceptance testing).

The question of integration is not “will there be integration?” but “how intentionally will I perform integration?” For solo developers and smaller organizations in their initial stages, a “big bang” approach is sometimes entirely reasonable. The project is small enough and the required velocity high enough that throwing everything together at the highest level to check its emergent behavior presents an acceptable level of risk. Acceptable risk, which tends to vary with complexity, is a good signal for deciding on a reasonable integration approach for a project.

Risk is not simply a linear “time vs money” decision. It involves understanding the general processes behind integration approaches, how those processes can be interpreted in terms of time and money costs, how these costs evolve over time, and making a decision for your particular organization and product based on uncertain information. Big-bang integration can provide extremely fast feedback on whether the system is working but can be expensive in the long run because it becomes harder to perform root-cause analyses and because the actual integration and testing work takes a long time as tests expand in scope. Bottom-up approaches can take a long time to get running across an organization due to the many possible layers of integration that can occur within complex systems, but in the long run, they can lead to faster testing cycles and lower overall costs for the organization. Overall, there is no single rule by which big-bang integration is faster and costs more money while bottom-up integration is slower and costs less money. The approach needs to be justifiable for the particular organization and product.

There are hybrid approaches that can work as an organization grows and its product matures. Consider an integration process that starts with a big-bang approach to provide fast iterative cycles while the product is relatively immature and less complex. Feedback is fast; root-cause analyses are still relatively fast because of the lower complexity; and for the purposes of research and development or proof of concept, this is a reasonable approach. As the organization matures, the requirements and architecture will increase in scope, and this is where starting to build out a layered integration approach can be useful. Organizations can keep full-system testing while building out lower levels of integration testing that can provide the necessary feedback for engineering and release decisions.

Picking which layers of integration to begin intentionally testing is generally not straightforward and probably shouldn’t be. It actually isn’t logical to always begin at the very bottom and work monotonically to the top. Each integration layer that gets tested should add value to the engineering process in some way. That can mean serving as a test that protects against known regressions in subassemblies. It can mean checking that binary execution works for release blobs that get built on particular architectures. It can even mean testing implicit contracts between software components. The point isn’t to be exhaustive—if there’s anything to learn in testing, it’s that exhaustive testing is a losing engineering strategy. The point is to make sure you can justify why you did the engineering work that you did.

Integration doesn’t just span software and systems. It can also span domains such as hardware and electrical engineering. If it’s a part of the system, it is a part of integration. In general, the person responsible for integration needs to have the domain expertise to understand how to integrate those parts, be able to quickly learn or find the necessary documentation to come up to speed, or have access to subject matter experts.

As a quick summary, here’s why being intentional about integration can help with engineering: