Due to the growing demand for tests automation, the IS subsidiary ICDC - in charge of the tests repository of the French group Caisse des Dépôts - has been initiating a first work to translate a predefined scope of tests repository into automating testing.
To answer the regular need to quickly and effectively produce new versions of an application, test automation has become the main solution for validation process and non-regression testing.
Even more so, the growing demand for projects using agile methods and digital projects strengthens its relevance.
Sébastien GONTRAN is giving us a feedback on his experience and the overcome challenges.
According to you, what are the main advantages to test automation?
Test automation allows improving work productivity and quality for the team in charge of the conception and the maintenance of the applications. Nowadays, test automation is a prerequisite, the basis to software integration and development.
What is the part of Henix in this mission?
Henix brought us its expertise for tooling in test management through Squash TM and Squash TA, thus supporting our projects.
How did you proceed to set up test automation?
The first step to introduce test automation, which began in 2009 and lasted for about 4 years, has raised awareness and educated people to the good practices. Despite a strong interest for automation, the team faced many difficulties:
Collecting the existing needs for automation and defining the suitable perimeter to start building the first projects.At a technical level, defining a methodology to develop the different automated scripts. The lack of standardization for tests run lead to a feeling of poor reliable “handmade” solutions.At a management level, as the result of the absence of internal experts and the lack of interest in skills improvement in test automation, the intervention of external experts appeared to be necessary but made test maintenance more difficult.Finally, dedicating a budget for automation remains tricky and it is not easy to invest such important amount of money in a project when the return on investment is so uncertain. The implementation of automation was perceived as an adjustment variable in the process of projects’ development.
How did you make your offer evolve?
"We can learn valuable lessons from our own mistakes" and, as the Deming wheel, the path towards improvement is endless. To answer the recurrent difficulties for test automation, methodology and tooling were both being improved.
Foremost, we capitalized on the previous years of expertise in projects’ improvement from CMMI and applied our knowledge on test automation: particularly for the management (collect and capitalize) of requirements.
Thus, the need for automated test is collected and supported by shared (swap) files (Excel format). These files are used as documentation but also for technical purpose since it allows to early spot errors in scripts of tests scenario. These files are also a mean to control projects for automation, by measuring the requirements cover, need for test automation to implement.
At a technical level, we now rely on MAKO : It is an automation framework using keywords that looks into the shared files used to collect expressed needs. This method is a standardization process for the description of requirements. From the Excel file, the tool generates the scripts in programming language. It is now the tool which exploits the good development practices and not the developer. The developer is in charge of specific functions and for the introduction of new keywords up to 30%. In case of any change or new needs, the Excel file will be used to regenerate the code, thus avoiding recurrent tedious tasks and facilitating the maintenance and a better capitalization.
Still on the technical axis, we have set up an automated testing platform composed of a tool (Jenkins) for scheduling the tasks, interacting with a test repository management application (Squash TM).
On the organizational axis, the Framework has come to "change the deal" of the automation project. Prior to its use, automation was a purely technical exercise, requiring the presence of experts during all phases of conception and development. The tool allows for the functional “know-how” to re-appropriate much of the approach. This approach makes it easy and quick for them to "script" their test cases using keywords.
Another “catalyst” was the migration of our test assets from HP QC to Squash TM. Squash TM allows to run and centralize the reporting of automated tests of different technologies. Combined with the test execution platform based on Squash TA, it allows to implement this re-appropriation of the automated test by the functional teams which launch the tests on demand.
What were the main impacts on the restructuring of the teams as a result of these evolutions/advancements?
One of the objectives was, and still is, to allow pooling of test efforts between teams and test layers (functional, performance, security).
The introduction of the swap file is leading to the right direction as it allows the functional teams to specify their needs in a directly operational format.
Nowadays, exchanges between the different parties are therefore more efficient and the effort is partly transferred to the functional teams that are more suited to test automation.
On the other hand, we would like to increase the involvement of the project software developers as for them to be fully involved in the process of maintaining of the automated test repository, with the objective of ensuring reactivity in the maintenance of the tests.
What about the impacts on a technical level?
Nowadays, 70% of the testing steps in the yielded scripts are directly generated by preexisting keywords. We estimate a productivity gain multiplied by 4 thanks to the new methodology.
This gain is important for inserting automation in Agile projects that are currently becoming a standard.
Development cycles are shorter and the need for non-regression testing is more important. In such a context, automation becomes an indispensable practice.
If you were to indicate a major development axis for the sequel, what would it be?
One of the objectives is to continue the development of our methodology in order to be able to formulate the tests using a "user-story" syntax.
We also wish to extend our MAKO dictionary towards supporting the automation of mobile applications that is an emerging demand.
Test automation is indeed "the future"!
We acknowledge Sébastien GONTRAN for this enriching feedback and a special thank you for the trust granted to Henix.