Microservices and Digital Transformation – a Microservices Devil's Advocate

Five Common Misconceptions About Microservices

Misconception #1: Microservices Architecture is Simple

Truth: The primary goal of a microservice is to implement the functionality of a single, large monolithic application as a set of distributed components that may or may not be located on the same machine (physical or virtual). This allows each microservice to be developed, tested, deployed, scaled, and updated individually without needing to impact the entire system. However, properly orchestrating all these components requires a well-planned and more complex architectural design.

An Example of Microservices Architecture

Source

Recommendations: The complexity of microservices сan be countered with proper planning and architecture to enable scaling and backend data persistence. As a result, a lot of work is involved in designing a system meeting all these requirements. Yet, when done correctly, each microservice will have clean, functional boundaries, networking, and security.

Misconception #2: Microservices Are Cost-Effective

Truth: There is typically a higher cost associated with implementing a microservices architecture than a monolithic one because the microservices architecture adds more moving parts to the system. For instance, a small team of developers is not enough to keep all the services up and running around the clock — maintaining, deploying, and monitoring a few dozen services is challenging. As a result, you might need a larger and more diverse pool of technical specialists, including DevOps, to join the team.

Source

Recommendations: Although monolithic solutions are generally easier to manage and more cost-efficient than microservices, they fall short on crucial objectives like modularity, availability, and scalability. Even though replacing a monolith system is expensive, complex, and might include some hidden costs, many companies report a significant advantage by getting increased flexibility with an easier path to adding new features, simpler management, and simplified scaling as the organization grows.

Benefits Organizations Experienced after Moving to Microservices

Source

Misconception #3: Microservices Are Always “Small”

Truth: Although a microservice is smaller in code size than its single monolithic counterpart, the size of a microservice depends on its boundary context, which can still be hundreds (if not thousands) of lines of code. The size and complexity of an application always remain the same, no matter how it is approached.

Recommendations: Do not fall for the misconception that microservices should be as tiny as possible when implementing the microservices architecture. Of course, microservices are smaller than monolithic since they break a complex solution into separate modules, but they shouldn’t be too small either. Such a way of thinking can lead to multiple issues when creating the architecture and introduce artificial limitations for what can otherwise be the approach your project needs.

Misconception #4: Microservices Scale Easily and Automatically

Truth: Breaking a monolithic application into components allows scaling the individual parts based on workload or application. However, this process requires additional skill sets and new systems in place.

Recommendations: For scaling up or down, you can use an orchestrator such as Kubernetes to manage multiple services simultaneously and enrich your resource pool with new skills. Microservices shift complexity to operations where automation can occur and is critical.

Misconception #5: All Apps Should be Converted to Microservices

Truth: Even though microservices solve many of today's problems, rewriting your monolithic application can be complicated. It requires meticulous planning and dissecting of the existing application into smaller, autonomous services that can operate independently while maintaining overall system functionality, which is all highly demanding.

Recommendations: To accomplish this task, you have to set a realistic budget and timeline, acquire the proper tools for automation, have a robust system for monitoring and detecting microservice issues, and ensure that you have enough experts with the right technical skillset to handle all the tasks associated with microservices.

All in all, to reach this objective, you can always keep the following questions in mind:

• Have I set a realistic budget (both in time and money) to support this effort?
• Will simply adding additional nodes to my existing cluster solve any performance problems I am currently experiencing?
• Do I have the proper tools currently in place (i.e., an automated Continuous Integration and Continuous Deployment pipeline) to make automation a reality?
• Do I have a robust enough system in place for monitoring and detecting issues with my microservices?
• Does my team currently have the right technical skill set to move to microservices and do we have sufficient staff to manage all these microservices, their intercommunication, lifecycle, and deployment?

The Challenges of Adopting Microservices

All transformations need time, money, and commitment to be successful, and migration to microservices is no exception. The benefits of microservices might be easier to realize with a solid understanding of the challenges. However, since adopting new technologies like microservices boost innovation and modern virtual architectures, why are so many organizations still not embracing microservices as the standard architecture? Here are the most widespread reasons.

Complexity of Multiple Technologies

In 2022, Kong surveyed 500 senior technology decision-makers in the U.S., Europe (France, Germany, UK, and Sweden), and APJ (Australia, Japan, and Singapore). The survey included CIOs, CTOs, VPs of IT, IT directors, architects, and software engineers from organizations across various industries.

According to the survey results, the complexity of using multiple technologies is the biggest obstacle to speedy innovation, cited by 38% of respondents. A “reliance on legacy technologies/monolithic architectures” and “lack of automation in IT that ties us up on trivial manual tasks” are the next most common challenges, each mentioned by 37% of respondents.

Lack of Skills and Inability to Recruit the Right Talent

Another key obstacle in the post-pandemic era according to the survey is a “lack of skills/inability to recruit the right talent” (34%). As the Great Resignation continues, this obstacle may grow larger in the coming years. Notably, these challenges are also the exact reasons why organizations are transitioning to APIs and microservices.

The factors holding back innovation varied slightly from region to region. For example, in the United States, the “lack of automation in IT” was the number-one impediment to innovation, listed by 43% of respondents. The “reliance on legacy technologies/monolithic architectures” is next (37%), followed by the “complexity of using multiple technologies” (34%).

The APJ region mirrors the United States, with the “lack of automation in IT,” coming in at number one with 41%, the “reliance on legacy technologies/monolithic architectures” (40%), and the “lack of skills/inability to recruit the right talent” (39%) rounding up the top three.

In Europe, “complexity of using multiple technologies” and “reliance on legacy technologies/monolithic architectures” are the top two obstacles to innovation, listed by 41% and 35%, respectively.

Obstaсles to Digital Innovation

Source

Difficulties in Managing Inter-Component Communication

Getting containerized services to discover and interact with one another without manual configuration or setup is another obstacle associated with microservices architectures. However, most major public cloud providers already offer out-of-the-box solutions that address this issue. For example, the Elastic Container Service now includes an integrated service discovery feature that makes it easy for services to find and connect.

In addition, REST APIs are commonly used for inter-component communication in distributed systems. However, different API versions must be backward compatible when implemented.

Difficulties in Maintaining APIs

After an API is published, the next task is to manage and monitor the ecosystem of third-party developers. The developers may also need to know how to throttle requests to secure the backend services, cache API responses, handle the request and response transformations, and create API definitions. The good news is that this challenge can be successfully met. Many popular cloud providers such as Amazon Web Services, Microsoft Azure, and Google Cloud Platform provide API Gateway solutions to mitigate these problems and help make developing and maintaining RESTful APIs easier.

Lack of Optimization

In addition, microservices are small and self-contained, but their connections create a complex distributed system that requires optimization. As deployment becomes more complex (the number of moving parts in the system increases exponentially), providing automated pipelines becomes essential to ensure a secure and consistent deployment.

Here are other challenges that should be taken into account when using microservices.

• Locating exact points of failure without having a comprehensive stack trace in a distributed architecture (and across distributed teams) is time-consuming and with little chance of success. A microservice-oriented architecture typically requires writing multiple mock services to simulate external components and communication channels.
• Designing a system with multiple small, finely-grained services demands special attention to avoid intermittent network system failures caused by compounded latency issues across individual components.
• Having the flexibility to write microservices components using a programming language that best fits the purpose also increases the likelihood of runtime failures when mixing these technologies.
• A distributed system requires that all components manage eventual data consistency, which may not be a good approach for many applications. Also, with microservices each component is responsible for managing its own data, which increases the chances of the system containing duplicated data and the challenges of maintaining data consistency across numerous services.
• Data used by microservices can be distributed across a multi-cloud environment and on-premises (or even a mix of both). This result is a significantly larger attack surface. Protecting user data, ensuring data privacy, and maintaining confidentiality become significant challenges due to the loss of visibility across a distributed component landscape. In addition, establishing access controls and policies for authentication and authorization across services becomes more challenging.

Summary

Using any technology properly requires understanding its intended use, application, and limitations. In this article, I intend to present microservices as objectively as possible. For all of its challenges and perceived “blockers,” microservices provide a way for businesses to meet their most important goals: agility, flexibility, security, and most importantly — improved customer experience.

In our next article, we take a tour of the current microservices technology landscape and better understand some of the key underlying concepts that make microservices possible.

 Stay tuned for part 4.