Scaling Microservices Environment

Microservices Series Part 3: Scaling the Microservices Environment

To coincide with our Docker blog series, we are continuing on with our Microservices Foundations for the Enterprise Series. Our first blog in this series discussed Microservices funding issues, while our second blog discussed the challenges of managing borders and disparate teams. In this particular blog, we will be discussing the challenge for enterprises to scale microservices and how to ensure the investment made is fit-for-purpose and fit-for-growth.

But first, lets have a quick re-cap of what microservices is:

To fully begin to understand the benefits of microservices, it helps to consider the opposite – a monolithic architectural style. As we all know, a monolithic application is built, deployed, scaled as a single  unit. The service-side application is a monolith that handles all of the HTTP requests, processes a multitude of tasks or business processes and retrieves data from a back-end (and often shared) database. The problem with monolithic architectures is in the name; the sheer size makes change and evolution a challenge as all of the cycles end up being tied to one another. 

A small modification to a small section of the application may require building and deploying a whole new version, with an expensive team of resources dedicated to managing change over a lengthy period. If you’d like to scale your monolith application, you may need to scale the entire application as opposed to the one component.  Significant changes to monoliths can take 18 months or more as the strategy team, project management and development teams coordinate initiatives to execute. So although they are sturdy and reliable, they’re not necessarily built for digital evolution at the speeds of technological change that we are seeing today.

In comparison to this, the microservices architecture offers many benefits. With microservices, enterprises are able to flexibly scale at speed as well as support a growing range of platforms and touchpoints – including web, mobile, API, IoT and wearables. The development cycle allows for componentization of business services that are organized around business capabilities, allowing for business innovation that is less hindered by technical inflexibility.

Now, everything worth doing is not easy right? The benefits of microservices notwithstanding, there is a level of complexity when it comes to scalability. Instead of dealing with a single application running on a single server—or load-balanced across a few servers—you might have elements of an application written in different programming languages, loaded on different hardware, running on different virtualization hypervisors, and deployed across disparate cloud and on-premise locations. When demand increases for the app all the underlying components have to be coordinated to scale or you have to be able to identify which individual elements need to scale to address the surge in demand.

With the legacy approach to IT infrastructure and app deployment the whole app had to be addressed as a monolithic entity. If demand spiked, the whole application had to be multiplied to accommodate the load, which meant multiplying the servers or virtual server instances on which the application was running.  Scaling a monolith often requires a large project in itself - potentially including significant planning, procurement, architecture, design, development, testing, and operationalisation activities. 

With microservices, everything is more granular, including scalability and managing spikes in demand. Demand may surge for one component of an app or a certain subset of data and a microservices architecture enables you to scale only the app components impacted, rather than the entire application and underlying infrastructure.  In a microservices architecture, there are requirements to scale dynamically ie. to scale in microseconds or minutes rather than days or months.  The relevance of those requirements is related to a number of factors, such as load variation, and they are also more relevant where there are more microservices interacting. This results in more load variation and more variation in runtime instances of each service.  

Planning any dynamic scaling system needs plenty of forethought.   When it comes to scaling microservices, these are some of the key questions to ask:

  • Auto or manual?
    Do you need or want the ability to scale automatically (e.g. based on current load or based on business events, which are precursors to load) or just the need to scale fast (but where ultra agile manual processes are sufficient)? 
  • Does your infrastructure provide dynamic scaling or otherwise support microservices in doing so? 
    Try not to reinvent wheels here.  There are a number of virtualisation and container-based scaling tools and frameworks.  Even if you use a nice elastic IaaS or Kubernetes, there's a lot of planning and configuration to be carried out to enable your microservice to scale effectively.  
  • How will you manage complex dependencies and data flows in realtime?
    Far beyond a single microservice, you need technical visibility and business outcome visibility end to end.  There are a mass of well trusted and emergent tools and technologies in this space including Application Performance Management, log/data analytics, Business Events Processing, and monitoring/alerting tools.  As networks of microservices grow more complex, you not only need a great view of relationships, data flows, load and errors but you also need more scientific ways of knowing what's normal and how to react to realtime abnormalities (which may be a precursor for future issues).     
  • How will you update or create new operational processes?
    Managing deployments, availability, monitoring/alerting, performance management, and problem determination require new processes for systems which scale themselves.  
  • How will you manage a new set of security threats?
    Security is a fundamental concern for any system.  In the microservices landscape, where there is a proliferation of new code and where applications and components can autoscale and dynamically connect, there are new set of security threats to think about.  This will requires a set of security solutions to manage various stages of the lifecycle and they need to be deployed and utilised in such a way that are always up to date with the latest safeguards against the latest threats.
  • What are the adoption and runtime costs of scaling solutions?
    Many technology vendors have been slow to support dynamic scaling in their licensing.  Where you don't have flexibility to change vendor you can be left picking up a large bill for unused capacity.   Implementing all the facets of a great microservice scaling architecture costs in human resources as new ways of working and new technologies are designed, deployed, and embedded.  There is a price for the flexibility that microservices provides, and understanding licensing and adoptions costs is a first step in controlling those costs.  
  • How will you scale down?
    What goes up must come down.  So as well as auto-scaling up, your microservice should auto-scale down.  But there are gotchas with taking instances offline, so make sure to do your research before slowing down the systems
  • How will you test?
    We won't dwell on this - because hopefully it is obvious to all readers.  But your scaling systems and tools need to be part of your pre-production environments and test plans too.
  • What are your scaling constraints and controls?
    You can't scale limitlessly.  Your key limiting factor may be licensing cost related to a particular component, or it may be something else, like the load on a downstream monolith, data store, or partner system.  You also need to consider safeguards and controls against auto-scaling issues.  For instance, where systems are scaled beyond the business needs and beyond financial or other limits.  
  • How will you adapt as recently established patterns and capabilities change and new ones emerge?
    This is a fast paced environment.  You may be implementing a new circuit breaker framework today but it's highly likely that the framework was developed recently and that changes, improvements, and fixes and a whole new framework are fast on your heels.  

Here at IntegrationWorks, we’re incredibly passionate about microservices (and many other architectures and practices) while focusing on our clients and what is right for them. Our microservices offerings include executive briefings, defining microservices in the context of your business (and how it fits with SOA, ESB, data architecture, API First and Continuous Integration), identifying costs and trade-offs for your business, road mapping and POC’s.

Some of our clients are new on the microservices journey, and others already are making great progress. If you or your team would like an initial chat to discuss a high-level microservices overview, feel free to call us on +64 4 499 0930 or make contact with our Chief Technology Officer, Ian Vanstone via LinkedIn.