As we turn to part 3 of this series on microservices monitoring, let’s look at where we’ve been. In part 1, I explained microservices architecture. In part 2, I discussed what development teams must consider when deploying microservices on infrastructure-as-a-service (IaaS) platforms. In this third and final installment, I’ll examine some of the considerations associated with deploying a microservices application using a database-as-a-service (DBaaS) model.
DBaaS is a managed database offering where the vendor takes care of many of the traditional tasks associated with managing a database. For example, the service provider will upgrade and patch the database, implement security measures, maintain backup copies, and enable high availability and disaster recovery across multiple availability zones. There are many DBaaS offerings in the marketplace, including AWS RDS, AWS Aurora, Azure SQL DB, MongoDB Atlas, GCP Cloud SQL, and many more. A DBaaS solution can be advantageous for development teams because it essentially eliminates administrative overhead and lets developers focus more efficiently on building microservices applications. This allows development teams to focus on application development without worrying about the infrastructure, the database and any associated routine maintenance.
However, application performance is still crucial within DBaaS. Although microservices applications are built smaller in scope, as application usage and data consumption grow for any of the individual microservices or for the application as a whole, operational management, microservices monitoring and database performance visibility become much more critical. Challenges of running microservices applications on DBaaS can manifest themselves in multiple ways.
Data growth and consumption
Over time and with use, it is logical to expect there will be data growth in the application. Depending on your DBaaS contract and service, data growth can create a significant cost from a storage perspective. With this in mind, DBAs or DevOps engineers must make a concerted effort to perform microservices monitoring for data growth. They should consider less expensive storage options, like cold storage or purging unused or unneeded records (accounting for PII data compliance). The cost of data growth is usually negligible on-premises, but it’s something to keep a watchful eye on in DBaaS.
Data growth can also create challenges from a querying perspective and can be exacerbated by a microservices architecture. As applications execute queries that pull from multiple services, the queries will likely take longer to run as they scan all the different data elements from each database from each microservice. This can have an impact on application performance as data grows within each microservice over time. Typically, this kind of a problem can be solved with more VCPU or VCores added to the DBaaS, but again, there can be a significant associated cost. To address this challenge, it’s important to have microservices monitoring tools that continuously examine data growth using baselines and patterns to provide advanced visibility into when you should act to avoid performance impacts.
Growth in users
If an application is successful, it’s natural to expect that more people will use it. This brings its own set of challenges, the first being concurrency. If too many users are trying to transact at the same time, regardless of the microservices architecture you are still at risk of locking transactions and blocks. This problem does not go away in DBaaS and it does not go away with a microservices architecture. Microservices monitoring to gain visibility into concurrency can help remediate locking and blocking-related issues before they become a bigger problem.
As more people use the application, it would also not be a surprise for them to be spending longer periods of time within the overall application and thereby impacting each microservice. It also makes sense that as users spend more time in the application, they’re also using new and different features or conducting searches. This can increase transaction complexity and thereby increase query complexity. This means that more microservices, more databases and more tables are being accessed. The microservices architecture can insulate you somewhat from query complexity, but only so much. The easy solution is to add more computing power and memory, but that can be expensive. An alternative approach might be to store commonly queried data that is fairly static in a cache. Another option might be to create replica sets of the data across geographical locations that might be closer to the users to thereby improving retrieval speeds via proximity. This is one of the advantages of DBaaS where replication across multiple availability zones is frequently offered, and can improve application performance.
Growth in functionality
If an application is popular, you will most likely build new features and functions and new microservices on your DBaaS platform. This often means that new functionality will be built on a separate database, but will involve data that lives in other microservices. While the goal of microservices is to eliminate as many dependencies as possible, if a service requires data from another, dependencies are unavoidable. If you take the approach of completely eliminating dependencies, then you run the risk of creating the same data in multiple services. This leads to its own set of problems as it obscures which one is the single source of truth for that data.
A best practice is to create a master microservices data model that is kept up-to-date and shared across the microservice development teams. This allows the entire team to know which data elements exist in the services and to avoid creating redundant data in another microservice. While a data model for a microservices application is helpful, it only gets you so far. How do you create a model where data is shared between microservices without creating dependencies and do so without affecting performance?
A common practice in traditional architectures is to use database replication for purposes such as horizontal scaling and load balancing. Replication is often used as a means to offload data for reporting and querying. A replication strategy will transfer the load of report queries away from the transactional database, thereby improving database performance while still retaining data integrity. This same methodology can be used for microservices, especially those deployed in DBaaS. As mentioned, many DBaaS offerings now provide replication across multiple availability zones as part of their service. In order to share data while minimizing dependency between microservices, developers can utilize selective data replication for data that might be needed by another microservice. This is an efficient and performant way to share data.
One common methodology is an event-driven model – when a data change is made, a microservice publishes that change and other services that are interested in it can subscribe to that event. All of these are important considerations when it comes to maintaining the loosely coupled nature of microservices while taking advantage of DBaaS capabilities to maximize application performance.
The need for microservices monitoring
When administering a microservices application in DBaaS, DBAs and DevOps engineers need microservice monitoring tools that help them see problems before they happen and react quickly. Monitoring workload performance across your entire DBaaS microservices estate can help you gain visibility into storage and compute utilization, which are important not only for database performance but for managing the cost of your DBaaS platform. Having baseline performance data helps predict resourcing needs that can be spun up or down to manage both cost and performance. As your application usage and user count grow, having visibility into concurrency issues early enhances your ability to resolve potential locking and blocking issues so that you don’t weigh down performance for your customers.
For a single microservices monitoring solution to gain visibility into your microservices applications built on DBaaS, consider Foglight® by Quest®. Foglight is an enterprise performance monitoring and optimization solution for the hybrid enterprise. It features deep diagnostics and cost optimization for modern development and microservices architectures. Foglight is cross platform and can support a homogeneous or heterogeneous microservices estate.
