How to Transition Incrementally to Microservice Architecture

Revenue cycle management (RCM) is a complicated process that involves several steps and considerable data flow. A software development organization (SDO) that was building an RCM application quickly realized the complexity of this task. Specifically, the cyclomatic complexity of the application was in the thousands. The SDO could not easily scale the application or add features without having an impact on the entire code base.

Back to Top

In the early 2000s, no clear means was available to overcome this challenge until, per the suggestion of a consultant, the SDO began to discuss how to simplify the system by isolating the RCM steps into smaller, independent services. Later, this idea became known as microservice architecture (MSA), which has recently been touted as a promising software architecture alternative. Generally, an architecture style denotes a plausible and reusable pattern of solutions, backed by experience, to a set of known programming problems.¹⁰ The architecture conveys a highly abstracted conceptual model of structure and behavior of the software, given its design goals and constraints. The choice of the architecture has an impact on the ways the software will be implemented and how its development can be organized. A judiciously chosen architecture style helps reduce technical debt and enhances software efficiency and quality.⁹

Generally, a MSA solution is founded on the idea of “orchestrating” the software, comprising loosely coupled and independent “services.”^13,18,22 In an MSA solution, each service is responsible for a dedicated, well-defined, and scalable function, which may and often is expected to serve, at the same time, other applications. Credit card processing, product rating, and checkout business functions are examples of microservices in e-business. Each service is developed, tested, and deployed independently without making strict assumptions as to how the deployment of the service will affect its use as part of other applications. Under MSA, some services are developed internally, while others are developed by third parties and linked to the final application through APIs through service orchestration. From a historical perspective, MSAs are a testament to the software community’s aspiration to develop and manage highly modular, well-organized software assets.³

In the past, software has been built mainly around a tightly coupled code-base called a monolith architecture. This style mainly organizes software as a modular single tier or, later, as a horizontally isolated n-tier architecture (Internet stack). Software built using a monolith architecture faces significant challenges to its feature growth, scalability, or performance when the software continues to evolve, often under tight time pressure.² As new requirements emerge or innovative technologies are adopted, the “legacy” code often fails to support a rapid implementation of such changes.

Recent technological advances have created versatile software ecosystems to develop and deploy microservices. For example, Docker, a container platform, provide a means to operate system-level virtualization to package software in lightweight “containers” orchestrated by Kubernetes.⁵ A growing number of software service startups now offer support services for container deployment, management, and security. This also allows small SDOs to increasingly deploy software assets previously available for large SDOs, such as Amazon.

Moving to MSA, however, is neither easy nor risk free. It calls for a strategic, disciplined approach that avoids the disruption of current operations and user experience with software. Many SDOs remain uncertain of the benefits of the transition and remain on the sidelines.¹⁵ The published success stories of transition offer often unfounded and far too positive claims and give the appearance that a decoupling will automatically lead to scalability, flexibility, and decreased time to market. The most publicly known successful examples come from large and born-digital operations, such as Amazon, whereas most SDOs that consider the transition lack the resources, skills, and project management capabilities of such software giants. Therefore, companies with legacy systems need to carefully consider alternatives to accomplish the transition.

Per our field study, most SDOs with significant software assets select an incremental strategy to decrease transition risks and ensure a smooth transition. Under the incremental strategy, the monolith legacy software and the new, modular MSA-based software will be simultaneously present for a significant period. This calls for a deeper appreciation of their joint impact on software assets, people, the organization, and managerial practices.⁸ SDO management must prepare for a wide array of changes at multiple levels of the organization while the transition unfolds and to assess the impact of the benefits and inevitable risks. We next identify some of those changes and related challenges identified in a field study that focused on leading industry practices during a successful MSA transition.

Back to Top

Study Design

The field study data was collected between February 2018 and June 2019. The study relied on semi-structured interviews that focused on changes, challenges, and opportunities during the architectural transition and the impacts of the transition on SDO’s management of software assets and their software process and organization. The data was collected from nine SDOs that had experienced an incremental transition in different forms and stages. The SDOs were of varying sizes and operated in six industries. The study included 23 interviews with 31 software experts, whose titles included CTO, Global Director, Senior/Principal Architects, Application Architects, and Lead Developers. The data collection progressed through three iterative rounds. In the first round, we sought to understand the impact of emerging technologies on software development. During this round, informants unanimously identified MSA as a critical architectural trend. In the second round, we sought to identify the motivations and decision logics that guided the selection of architectural styles and transitions. During the third round, we sought to identify and understand pivotal challenges in operating co-existing monoliths and MSA architectures and solutions. We also validated our study findings and conclusions among a subset of informants. The transition strategies and challenges discussed here were inductively derived from the transcripts. We coded data simultaneously with the data collection to ensure higher validity and reliability of emerging themes, guide follow-up interviews, and identify saturation in data and code. Further details of data collection and analysis as well as examples of coding trees are reported in Appendix A available online at https://dl.acm.org/doi/10.1145/3378064.

Back to Top

Incremental Transition

During incremental architectural transition, microservices are introduced in a piecemeal fashion while software assets are iteratively and successively re-architected. Table 1 provides a summary of the benefits, risks, and organizational impacts of the incremental transition to MSA, as derived from our field study. The benefits and drawbacks of the two most common alternative strategies—big bang and zero alternative, no transition—are reported in Appendix B available online at https://dl.acm.org/doi/10.1145/3378064. In the incremental strategy, the choice of applications to be re-architected and the way microservices are spun out from monolith code base are guided primarily by the determination of whether the re-architecting provides recognizable value to the business and immediate benefits to the application users. Business needs drive which services will be isolated and developed independently. Under the incremental strategy, the final goal is not necessarily to decouple the system fully unless it comes with clearly defined and measurable advantages. Overall, the goal is to continually balance transition risks²³ with expected benefits.^1,7,12

Table 1. Incremental transition to MSA.

The four phases of incremental transition. Typically, an incremental transition process unfolds through four phases: identify goals for transition; identify the scope and level of architectural changes; prepare for resource readiness; and change critical development practices. The phases are distinguished by:

• Different decision makers who initiate and are responsible for the types of changes related to phase;
• Different mechanisms that allow decision makers to carry out the transition in that phase; and
• Different outcomes for each phase.

The mechanisms are defined organizational capabilities that are used to make related decisions or to implement these decisions. SDOs will deploy such mechanisms when making and executing decisions about the transition directions while developing their software assets. The accompanying figure provides a summary of the principles that separate the phases and the logical dependencies between them. The top-down arrows in the figure illustrate the increasingly granular focus of the incremental transition as it moves across the phases. Initially, the leadership must put mechanisms in place that help to identify the gaps between an SDO’s strategic goals and the extent to which the current software development practices align with them. These mechanisms help to identify the limitations that must be overcome and the opportunities that need to be capitalized on during the MSA transition. The strategies in an incremental transition that help to achieve goals the most often involve scalable applications, enhanced application flexibility, and velocity-improved time to market. Based on identified gaps, the SDO leadership needs to establish specific strategic goals for incremental transition.

Figure. Incremental transition process phases.

During the next phase, mechanisms need to be established to identify a select set of applications and related services as targets for decoupling. This determines the scope of the architectural choice guidelines and, potentially, the impact of the proposed service splits. Such impacts can be evaluated only by knowledgeable software architects. In the next phase, project managers need to reorganize software teams to execute the splits. This calls for establishing new roles and responsibilities to properly develop, deploy, and maintain microservices. The mechanisms here relate to the knowledge and skills to reorganize teams and execute related organizational change. In the final phase, software developers must learn and make the transition to new development practices, and software managers will acquire new responsibilities to manage service vendor relationships. The mechanism here aims to change local software development principles and provides the means to change project management practices and related development guidelines, which then result in new software development practices, such as DevOps.^3,9

The bottom-up arrows depict the expected impact of each phase, that is, learning-based feedback loops. For example, a change in development practices is likely to have an impact on the teams and their skills and behaviors. A proper team structure will drive a feasible separation of services, while each decoupling step needs to be checked against the established goals to ensure the split will bring business value.

Understanding the role and impact of all four phases in regard to transition outcomes is a vital precursor to a successful MSA transition. Phasing activities and related role changes help SDOs to gradually prepare for and manage the incremental transition effectively. In particular, if each phase is properly planned and managed, and related tasks are addressed prior to moving to the next phase, the transition is likely to be more successful in balancing the risks and benefits. Each phase involves multiple critical activities. Table 2 provides a summary of the key activities and facets as well as their impacts on the organization.

Table 2. Incremental transition phases, activities, key facets, and organizational impact.

1. Identify strategic goals. The SDO must identify salient strategic goals that are currently not being properly met due to faults in software assets and related organizational capabilities. Understanding the readiness for and urgency of the MSA transition is vital for a successful transition, or “burning platform.” SDO managers should lead the transition by asking for valid business reasons to change the architecture of software assets. Typically, these goals relate to software velocity, scalability or flexibility.¹⁷ Only such well-defined strategic goals warrant the effort to engage in the transition and calls for the SDO to carry out the following activities:

• Recognize strategic goals for the transition. Successful SDOs typically set up well-defined and measurable goals for MSA transition. These goals guide decision making and implementation of the organizational and technological changes to achieve them.^11,16 One SDO, for example, established a strategic goal of providing innovative solutions to their customers by responding to new demands in a more timely manner. This goal became the key driver for the MSA transition, which involved decoupling highly embedded and dependent services from the code base for better responsiveness and flexibility. To accomplish this, the SDO prioritized the development of modular microservices to support key business functions, enabling experimental development and innovation.
• Identify pain points in SDO operations and assets. The legacy system’s features and related practices that obstruct the realization of strategic goals need to be identified. As one executive stated, “I always ask why they want to use microservices. What problem do they plan to fix with moving to microservices architecture?” For example, if the strategic goal is to increase innovativeness of solutions and improve the customer experience, bottlenecks that impede the achievement of this goal must be identified. Tightly coupled services are time consuming to change due to the dependencies and require complex and slow testing arrangements. Decoupling such services allows simpler testing, faster code change, and easier monitoring of performance and user behavior.
• Map microservice(s) to pain points. A set of services must be identified as a cause of bottlenecks. By implementing a microservice in lieu of a monolith augmentation, the new service should help to achieve the identified goals. Consider a ride-hailing service similar to Uber, whereby a service is provided by using GPS coordinates for pick up and drop off, while the prices for the service are determined by dynamic supply and demand information. The data needed to run such service will have high volume, as the application must be able to compare historic data of supply and demand quickly and combine it with applicable customer characteristics to determine the price on the spot. The processing of such high volume data, while using a monolith, could easily put significant constraints on which types of pricing outcomes can be provided in real time while offering such a service. Therefore, using Apache’s Kafka Stream application as a microservice to handle the data would emerge as an appealing alternative microservice solution that would better meet system requirements.

1. Identify architectural change. The second phase helps the technical leadership to engage in critical technical decisions that relate to the scope of the architectural change needed. This change occurs by establishing a clear understanding of the technical scope and risks of the transition. Services to be isolated need to be identified and their dependencies accounted for. This helps an SDO to conduct a proper initial scoping of the transition. In this phase, it is important not only to assess the complexity that new microservices add to the software management but also to identify those parts of the monolith that should not be decoupled, as the benefits would be marginal or the risks would outweigh them. The following activities are conducted in the architectural change phase:

• Assess the complexity of microservice(s). Technical leadership needs to identify and agree on services to be split and prioritize the decoupling schedule. This requires a detailed assessment of the complexity that each split introduces to the application. Understanding and mitigating possible risks, organizational tolerance for such risks, and expected benefits are important elements to consider. For example, messaging between microservices will quickly grow in complexity and become error prone. If a code freeze is necessary until the services are developed and integrated, customers need to be notified whether this is expected and whether the reported enhancements will be delayed.
• Service split rules/decisions. The order of services to be decoupled should be dictated by service characteristics and related business value. In customer-facing services, if the service is deployed independently, rapid development and deployment can soon realize business value. The services to be split should be large enough and further divided as the need arises to avoid unnecessary complexity. Leadership must understand the value of proper scoping and avoid the creation of “minimonoliths,” which often result from poorly defined cross-boundary services. A common approach is to split two to three services in a single development effort and to leave everything else in the monolith’s core. The splitting rules are not, however, always driven by business logic. They also are affected by inherent constraints of database design and data dependencies. As one interviewee stated, “The code is actually not that difficult to pull apart; it is almost always [that] the data is [sic] so much harder to pull apart because you have to know which service owns which pieces of the data.”

When a microservice has been identified for separation, either building it internally in an isolated fashion or searching for a third-party service provider is a viable option. For example, if a product rating service is added to a commercial sales site, it can be developed in-house. Alternatively, a third-party may offer a fully tested, feature-enhanced product rating service ready to be deployed. In such a situation, it is important to understand and anticipate the future maintenance, customization options, and consequences for data ownership before committing to any final decision.

• Contain the monolith core. The main business logic of the application is not likely to change dramatically over time. Therefore, it is often inefficient and unnecessary to break it fully into services, especially if no scale-related bottlenecks are present. All identified services need to be interfaced with the remaining core and data shared accordingly. One microservice developer described the significance of keeping the core as follows: “You’ll always have to deal with monoliths and part of the legacy system but you are not adding onto monolith.”

1. Advance resource readiness. The identified architectural scope will drive the extent to which this phase will permeate the entire organization. If the implementation team is inexperienced with MSA transition, the order of service separation should start with a less mission-critical one. For example, a product-rating service in an e-commerce application would be a reasonable choice in contrast to the checkout or payment function. The first service isolation needs to be treated as a showcase project to demonstrate the business benefits and to obtain the organization’s buy-in. Therefore, a proper assembly and management of teams will be crucial for the long-term success of the transition. The architectural change phase helps to identify the needs for proper composition of the teams responsible for developing these first services. The phase consists of activities focused on building mechanisms that hone individuals’ skills and enables successful team formation, as described here:

• Re-organize teams. A software development team’s organization typically mirrors the modular software organization expressed in Conway’s Law:⁶ An organization that designs software will produce a software organization that structurally represents the organization’s current division of labor. A shift to MSA seeks to break the present software organization, and, thus, it is necessary to divide the software teams differently to produce a new structure in the organization. Developers need to be reorganized into smaller, independent, service-focused units with less dependencies and narrower communication structures.¹⁴ The new organization needs to isolate critical independent, multi-application services within their autonomous development units. The new team structure puts pressure on maintaining the legacy system and how its current social organization operates when parts of software functionality are split into autonomous services with new responsible DevOps teams. If the organization does not have significant experience with carrying out such a radical re-organization, it is advisable to start with just one service team responsible for a non-critical service to get a sense how manage and organize such teams.
• Identify and train key developers. Another challenge is to identify team members who will truly advocate for the idea of MSA and are willing to undergo training and learning. Developers must learn new skills and need to remain dedicated to carrying out the change while untried technologies are integrated into a technology portfolio. Those with good business domain knowledge (especially of the services to be decoupled) and a natural drive to learn technical skills are the best candidates. The selected candidates should be fully removed from their monolith-related responsibilities to create a fully capable team that can effectively decouple, develop, deploy, and maintain the service as an independent unit. Because both architecture styles will co-exist, it is important to understand that all remaining developers will not be interested in learning new skills. These developers should be allocated to maintain the legacy code. In some situations, if all developers are interested in working on microservices, choosing to rotate between the developer roles is advisable to prevent issues between the team members.

Acquiring critical MSA design and implementation skills, such as continuous delivery, represents a radical departure from the past. Training for the DevOps skills and associated code management must be conducted. Outside consultants are commonly invited initially to guide developers to execute the first split smoothly and to manage the code change. The training should cover the skills to build relationships with external service providers that engage in the transition effort. Developers should attend professional conferences to build up developers’ critical knowledge base and create learning networks.

• Shift roles and responsibilities. Large SDOs commonly use established consulting firms for internal talent development. Smaller SDOs encourage self-based and paced learning but hire external consultants to lead the initial process of decoupling code and assimilating the chosen technologies. Some infrastructural and testing support roles in the new environment will be less desired, as development and operations are now largely run on and managed in the cloud. The transition team needs to involve members from business operations so that the business side understands why these changes take place. This is also necessary to create successful DevOps practices³ that place autonomous development teams at the forefront of not only developing services but also of testing and deploying them with higher velocity.²³
• Obtain buy-in. The success of the initial service split will create momentum for the architectural transition and help to obtain a buy-in from developers and business units. The MSA transition generally induces a change within the SDO to support the creation of modular and independent operational units responsible for specific microservices. Because MSA assumes that there are more software components to develop, test, deploy, and manage, DevOps practices need to be introduced prior to forming teams. This grants time to erect the proper infrastructure and document and develop proper metrics to measure MSA transition outcomes.¹⁰

1. Change software development practices. This phase focuses on changing key development mechanisms and technologies with the goal of finding appropriate technology to implement selected microservices. This phase also establishes standards for quality assurance and prepares the organization for more complex service vendor management. Key activities include:

• Support hybrid architectures. Although the teams that support MSA solutions and the monolith core will be separated by relatively limited and well-defined interfaces, they need to understand the impact of their work on one another. Well-defined service responsibilities will guarantee more rapid fault recognition and resolution. With the isolated services, the need for agility will change significantly. Management needs to introduce associated changes in project management methods to properly entrain sets of activities related to the core and microservice development. They must run agile DevOps teams to become more reactive in microservice development. The monolith application benefits from stability and requires more predictive long-term support and related practices.
• Establish due process to find the right technology. Developing microservices grants teams higher flexibility to choose “fit” technologies for services for which they are responsible. The chosen technologies, however, need to be formally evaluated at the higher level to ensure that their use aligns with the holistic needs of the organization. In this way, SDOs can better control diffusion and deployment of varied technologies and mitigate the risks of growing heterogeneity, poor choices, and cost escalation. If the selected technology proves to be a poor fit for the service, it can be relatively easily replaced if proper microservice specifications have been used. In this regard, one microservice developer stated, “We build this process in such a way that we make it okay to drop a new technology if we see that it’s not working out.” Most SDOs encourage experimenting, and failing early is accepted as part of the process, which ultimately promotes innovative service development.
• Manage vendors. The cloud platforms necessary for the development and deployment of microservices introduce myriad potential new vendors within the MSA ecosystem. Coordinating service deployment, managing contracts and compliance, and understanding and mitigating associated risks become significant new challenges. SDO management needs to prepare for and consider vendor issues prior to starting the transition.
• Establish minimal software quality. The proliferation of diverse technologies comes with advantages, as it helps to search for the best solutions, given particular implementation challenges. This diversity, however, needs to be matched with the call for quality and the need to minimize errors. Establishing quality standards and delivering consistently high quality across deployed technologies in this new environment remain significant challenges. One developer observed: “We have all these different microservices written in different languages, giving us different challenges during deployment … some will be more buggy than others when the teams deploy them.”
• Monitoring user behavior. The real-time understanding of user interactions with the orchestrated application calls for constant instrumentation, monitoring, and analysis of large volumes of user stream data. In MSA environments, data scientists often become as important as business analysts for understanding customer needs and identifying service enhancements.
• Establish communication structure. Monoliths, due to a large number of unaccounted dependencies, often call for communications among all involved developers. Microservices should cut down communication overhead, as they rely primarily on a narrow band of asynchronous communications established through interface specifications. Yet, SDO management needs to allow “enough” interactions among developers across teams to avoid building additional dependencies manifested in “mini-monoliths.” Co-located teams identified this as a huge challenge, and an analyst stated, “Keeping the walls up around each microservice, I think is certainly more challenging when you have like a co-located focused product team versus a distributed team. I think the distributed team lends itself, kind of forces you to better keep those services decoupled.”

Back to Top

Conclusion

MSA is not a silver bullet. It will not solve all persistent problems of managing software assets. For example, decoupling monoliths and isolating critical microservices will not fix the problems that arise from designing a flawed system or those emerging from writing poor-quality code. An incremental transition, however, can realize better business value derived from software assets if guided by the realistic and measurable goals and the SDO is aware of the cascading impact of the transitioning to the entire organization.

The SDO must have a solid reason for the MSA transition. If a realistic strategic goal can be established for the MSA transition, then the SDO needs to prepare and build the mechanisms that it can, make competently necessary decisions, and implement them properly. The transition strategy should be incremental unless the organization has significant resources and deep software experience, or the full transition will be necessary due to drastic operational failures or other business reasons.

The SDO needs to state clear, measurable benefits for service isolation that outweigh related risks and higher organizational complexity. The SDO should not decouple service without understanding its benefits for more flexible development, scalable applications, or enhanced time to market. The SDO must also establish stringent technological discipline with well-founded, standardized splitting rules. It should execute the service splitting iteratively, following established priorities, as determined by business needs.

The key to success in an MSA transition is to understand and deal with the softer social underbelly of the organization. MSA is fundamentally about a deep change in the SDO’s structure, minds, and hearts, triggered by a new technological opportunity. It shapes the organization in a holistic and punctuated manner and results, over time, in a deep transformation of the organization’s status quo, whereby its structures, roles, responsibilities, skills, incentives, and routines all are affected. SDO managers should not expect these incumbent and deeply entrenched structures to adapt to MSA automatically or by fiat. The SDO needs to be prepared for continued change and to learn to adjust the organization in a disciplined manner to the new architectural regime. Poor execution will create confused and disheartened employees, unfit applications, and, ultimately, the loss of competitiveness.