By: Satyan Prakash

A mobile-first approach is imperative for successful businesses given the huge proliferation of mobile devices, yet many companies are still behind in adopting the DevOps practices for mobile app development. In this blog, I am going to walk you through a case study of one of the leading global providers of vehicle rental services, CarRent (name replaced with the fictitious name due to confidentiality reasons) which wanted to transform itself into a mobility solution provider but was struggling with legacy mobile app development practices. 

Business Need/Pain Points:

• Development Team of CarRent was spending huge amount of time doing regression testing as part of each sprint rather than focusing on new development
• Development, regression testing and go live were spread over two to three sprints rather than happening as part of one sprint. This resulted in about 6-9 weeks of SDLC cycle before a feature can be released to app stores
• Mobile build generation was an onerous, time consuming and error prone task
• In the current Phonegap mobile app framework which is based on HTML and Javascript technologies it was difficult to plan automated unit testing resulting in more testing efforts in later phases
• Dependence on manual testing late in the cycle led to late discovery of defects and defect leakages to staging and live environments
• It was not feasible to manually test all combinations of mobile devices leading to frequent device specific live issues 
• It was quite cumbersome to diagnose any live issues due to the complexity of the mobile app platform
• CarRent was struggling with time to market for mobile releases and losing market share to their competitors. It was very evident that they needed to completely overhaul their development lifecycle and adopt DevOps to accelerate the time to market if they had to survive. 

The Three Ways of DevOps

CarRent used The Three Ways of DevOps as the basic guiding principle to transform their development lifecycle. These three principles offered a valuable focus for evolving the culture change requirements that were the core challenge for embedding a DevOps value system. Just to jog our memories, ‘The Phoenix Project’ book presented the Three Ways of DevOps as the set of underpinning principles from which all the observed DevOps behaviors and patterns are derived. 

• First Way: Work always flows in one direction – downstream
• Second Way: Create, shorten and amplify feedback loops
• Third Way: Continued experimentation, in order to learn from mistakes, and achieve mastery

People/Process/Tools

CarRent used the three pillar view of People, Process and Tools to make sure all aspects of improvement opportunities were considered.

People/Process – Using TDD to change the development approach

• Test Driven Development (TDD) – TDD helped CarRent develop a robust test suite to catch bugs, as well as guiding them to write a more modular, flexible code. This was a paradigm shift for working ways of the team and gave them benefits in terms of improved test coverage & fewer bugs in the build.
• Acceptance Test Driven Development (A-TDD) – is an extension of TDD which really turns the concept of testing upside down where requirements are iteratively refined by creating automatable acceptance tests, or in other words executable specifications. The focus is on prevention rather than detection. Tests and requirements become in-distinguishable. This was planned as part of each sprint by automating the acceptance test cases (based on acceptance criteria) and making it available to the developers to run it iteratively till all the cases pass.
  • By using A-TDD the purpose of the tests changes from finding defects to preventing them by writing the specifications as tests. This is a fundamentally different approach which led to developers and testers working together rather than working against each other.
  • A-TDD helped CarRent solve a major challenge. When they were doing deliveries in 3 week sprints the amount of regression testing needed in each iteration made manual regression testing nearly impossible. Hence they planned for 100% automation though some manual testing may still be needed for example usability testing, tests requiring physical movement, expensive or tricky tests etc.
• Cultural Shift – It was not easy for CarRent to make their development teams adapt to this fundamentally different way of software development. They inducted some TDD experts which acted as change agents to groom the team in TDD techniques. It was a slow learning curve initially but the team gathered momentum and were able to reap the benefits of this superior way of development.

Process/Tools – Improving Test Automation

When the existing test automation flow was analysed as part of value stream analysis it was identified as one of the major factors delaying the release cycle as you can see from the diagram below. Instead of working as a cross-functional team the development and test team used to work in silos delaying the test automation. 

Test automation workflow was completely revamped (diagram below) with better collaboration between development and test teams to remove the handoffs and allow the TDD and A-TDD to work seamlessly. 

Mobile App Development CI/CD 

And here is the fully automated CI/CD pipeline for CarRent’s mobile app SDLC. Let me walk you through some of the main pillars of this solution. 

• Distributed Version Control with Git – Team moved from SVN to Git making it easier for distributed team to carry on parallel development. Git’s easy integration with collaboration tools like Slack allow CarRent to automate the development workflows. Development team was spread over UK, Romania, Bangalore and Hyderabad so this helped the developers working in parallel on their feature branches and then merging to central branch 
• Better code coverage and unit testing – Flutter – In line with the third way principle a brand new mobile app UI framework was selected and proposed by the development team after analysing three potential candidates. Flutter was the preferred choice which is Google’s App Framework with rich native experience, cross platform development in Dart (similar to Swift/Kotlin), strong unit testing and TDD capabilities. Flutter Static Code Analyzer allowed the team to find problems before executing a single line of code. Flutter Cover was the easiest way to see overall test coverage was using the Flutter page on Coveralls. It highlights important functionality in the code that has not been tested. It also identifies code that is not testable or not used which indicates the code should be refactored.
• Mobile Test automation – Appium Flutter Driver – Even though Flutter comes with superb integration test support, Appium Flutter Driver was used as the test automation framework due to better overall capability and also stronger integration with mobile clouds.
• Mobile Cloud for better device coverage – BrowserStack – Mobile apps need to be tested on a wide range of mobile devices of different configurations which keep on getting refreshed every few months. It was not feasible to procure all these devices for testing. Browser Stack was a cloud web and mobile testing platform that enabled the testing of CarRent Mobile apps using real mobile devices, without requiring the team to install or maintain an internal lab of virtual machines or emulators. 
• Continuous integration and deployment with Microsoft App Center – It allowed CarRent to automate and manage the lifecycle of iOS and Android mobile apps. It also allowed them to connect their repositories within minutes and automate their builds, test on real devices in the cloud, distribute apps to beta testers, and monitor real-world usage with crash and analytics data. This was a key enabler of the second way of DevOps to accelerate the feedback loop. 
• Integrated SDLC management – TestRail/JIRA – Fully integrated JIRA and TestRail allowed CarRent to track complete SDLC in real time starting with user stories to automated and manual testing and even go live.  
• Powerful real time crash reporting – CrashLytics – helped CarRent track, prioritize, and fix stability issues that erode app quality, in real time. It helped to pinpoint the root cause of crashes and also helped prioritise the issue to tackle first ensuring quick feedback is available in line with the second way of DevOps. 
• Slack – instant messaging and collaboration system on steroids – Slack’s integration with other DevOps tools allows CarRent to create a strong communication workflow to ensure better collaboration.

Time to Reap Benefits 

With all the aspects of the DevOps solution seamlessly working with each other and cultural changes successfully implemented, CarRent has been able to drastically increase their speed to market and is in a strong position to beat their competition. But this is just the beginning for CarRent as they know that their competitors are also busy doing similar things. The three ways of DevOps are still in motion at CarRent which will make sure that they are always ahead of the curve.