This article is outlining the cell-based architecture, which was published as an open specification on GitHub summer-2018. Our approach creates a pragmatic and technology-neutral reference architecture that addresses the requirement for agility. It can be instantiated to create an effective and agile method for digital enterprises, deployed in private, public, or hybrid cloud environments.

When I present the new architecture at technology events, one common question is the reason we are defining a new reference architecture in an already crowded market.  This article lists the motivating factors.  

As a technology provider, My company has been involved in the integration space for more than a decade, helping over 500 customers implement various digital-driven solutions. Paul (the co-author of Cell-based Architecture) and I have been deeply involved in most of the implementations.  Paul and I also have more than 20 years of experience in the distributed computing space. The above factors led us to discover the mismatch between existing architecture patterns and what enterprises truly require. 

Layered and Centralized Architecture

Over the past two decades, we have used layered and centralized architecture patterns, which have resulted in creating many centers of excellence (CoE) groups and technology stacks. These layers have created gates, which, in turn, have slowed the rapid movement and flexibility that agile teams require for iterative development and continuous delivery. Owing to this, even organizations that practice agile methodologies are only partly agile (agile or fast waterfall). Our approach aims to minimize centralized dependencies and move integration into the agile development pipeline, making organizations more responsive to business needs. Based on the results from the most recent State of Agile Report, only four percent of companies benefiting from the actual result of agility actually spend time and money on practicing agile methodologies. 

Brownfield and Greenfield

With the introduction of microservices architectures (MSA) and cloud-native infrastructures, architects started designing architectures entirely as greenfield (I would call these PowerPoint architectures). However, the reality is that many enterprises need brownfield systems and data, which are required to run the business. As existing reference architectures address either greenfield or brownfield, we realized that enterprises require a hybrid architecture approach that can reuse existing systems and data as well as build new systems in greenfield. 

Reference Implementations

We conducted precise research before defining the new architecture and, during that exercise, we identified that most existing reference architectures were reference implementations. The rationale behind claiming those as reference implementations were owing to defining implementations based on a vendor-specific technology. We wanted to make the new reference architecture ultimately based on technology and be vendor-neutral. That way, an architect could refer to the defined reference architecture and build solutions based on the desired technologies that already exist in the enterprise or plan to introduce them in the future. 

Architecture and Technology Agility

Existing reference architectures are also limited in architectural agility in that they block adopting new or comparable technologies for projects. As the CoEs have standardized technologies across the organization, each project team has to stick to those specific technologies even if they are not the ideal fit for the project. This underutilization of technology is a side effect of the centralized layered architecture and it led us to think about the need for a decentralized and modular architecture. 

The Gap Between Architecture, Implementation, and Deployment

To a certain extent, agile methodologies helped to narrow the gap between architecture, development, and deployment. However,  there is a disconnect between those three steps in the application lifecycle; this results in broken systems for the business. Establishing seamless connectivity in each lifecycle stage with a common architecture construct is crucial to improve agility (by having an iterative architecture and continuous delivery). Our approach combines the application and system architecture using the same architecture construct and this naturally connects the architect, developer, and DevOps engineer. 

Governance and Dependency Management

While microservices and containers increased the flexibility of development and deployment for applications, both patterns increased complexity by introducing many moving parts. With this in mind, we wanted to implement a light governance framework with observability, which would enable dependency management, impact analysis, and a monitoring mold into the architecture.    

We started the project by conducting thorough research on the market. Scholarly papers published by various research institutes, books, and online content from other vendors were some of the sources for the content. This research-validated that our motivation factors were real. 

We narrowed down our research into four areas — quantum computing, Kubernetes, microbiology, and systems biology — to find an entry point for the project. Quantum computing and Kubernetes led us to think more about decentralized architectures and the market’s movement towards the same. Microbiology and systems biology gave us insight into how independent and self-operated units (cells) connect and build numerous extensive systems. Also, there were other exciting characteristics of biological cells mapped to the architecture construct we were looking at, such as how the membrane of a cell provides a clear cell boundary and protects the cell’s internals, as well as creating a single communication channel. 

After the preliminary research work, we looked at the reality for the enterprise: mainly the disconnectivity of business expectations and the technical definitions. We concluded that regardless of the service development style (SOA or MSA), organizations seek solutions for business problems from services, which essentially require connecting multiple technical services and exposing a business savvy API. As a result, (microservices are required to connect using a composite service or an API gateway. 

Conclusion

In summary, our motivation factors behind introducing this architecture pattern were based on teething issues faced by architects while moving to the microservices and cloud-native approach, which requires connecting with a large number of monolithic applications that already reside in the enterprise.  

In the next article of this series, I will give an overview of cell-based architecture.