Performance Testing in Development
Performance monitoring is key to good DevOps principles--to understand how to improve system performance, one must first understand how the system is performing. Engineering teams need performance analytics in order to do fact-based evaluations, improve code quality, and ensure system stability. By putting performance analytics in the hands of developers, Etsy grants them access to the information they need for problem-solving; with database monitoring in particular, members of Etsy's engineering team can fix issues independently, with confidence that they have access to a complete picture. These are a few of the applications where Etsy found value in using a performance management solution.- Database version upgrades. When Etsy upgraded their databases to a new version of MySQL, they used granular database monitoring to help them analyze changes in system performance. They were able to watch their systems performing on separate servers, each with a different version of MySQL, one for production and the other a copy. Using SolarWinds® Database Performance Monitor (DPM), with side-by-side tabs, they could sort and compare metrics like average latency, errors, and total time. Looking at the differences between the servers, the analysis built strong confidence that the switch would afford a real improvement in speed. At the same time, it showed that there was no regression, no query errors. When it came time to push to production, Etsy was confident that all would go smoothly.
- Database schema changes. For schema migrations, Etsy uses performance analytics for real-time feedback. The team looks at historical performance metrics and trends to ensure that there are no resulting spikes in workload metrics or server faults. Engineers and DBAs drill into the data and share what they find by using permanent links into the SolarWinds DPM UI to quickly collaborate with a colleague on issue identification and resolution. A SaaS performance monitoring platform with deep linking and sharing capabilities gives a team confidence that when one person looks at a chart and sends it to a colleague, that colleague will see exactly the same thing.
- Hardware upgrades. When Etsy evaluated a hardware change—switching from traditional spinning-disk storage to SSDs—the DBAs used SolarWinds DPM to A/B test the change and analyze exactly what system performance would be like with the new hardware. Once again, real-time feedback afforded the team confidence in exactly how the hardware change would affect system performance in turn. Rather than make the switch blindly, hoping that the system would continue running well, the A/B testing allowed the team to proceed methodically and insightfully.
- Code changes. Etsy’s developers deploy code frequently—often dozens of times every day. One issue they consistently faced was MySQL’s internal measurements of replication latency made it difficult to understand the impact of a code change. In the past, when Etsy pushed a code change into production, the team couldn’t see the effects on replication delay until it was too late and the replicated data had fallen behind. The only way to get visibility into such complex system behavior is a monitoring product that is specifically designed with knowledge of the database’s intricate inner workings. Using SolarWinds DPM they can see immediately when replication lags, even though MySQL’s own measurements of delay show nothing.
