The "quality gate", as defined in Squash DEVOPS, is a mechanism for configuring if a CI/CD pipeline should continue or should stop depending on the results of the automated tests. Such a quality gate can be used, for example, to run lengthy UI tests only if all API tests are successful, to deploy a release candidate on an acceptance test environment only if there are few failing UI tests…
In previous release, Squash Orchestrator was supporting only two modes when configuring a quality gate:
a strict one which considers the quality gate as successful only if all tests are successful;
a passing one for which the quality gate is always successful whatever the statuses of the tests.
It is now possible to define a quality gate with some thresholds: the quality gate is configured by providing a list of criteria, each criterion being defined by a minimal success percentage that must be reached for a given type of test (e.g. Postman tests must be 100% successful and at least 90% of the Cypress tests must be successful).
In order to simplify the writing of the criteria, some new functions have been added in the orchestrator syntax: contains, startsWith, endsWith, toJSON, and fromJSON.
This mechanism will be enhanced in the 2023-07 delivery by enabling the use of Squash TM data (CUFs, nature or importance of the test case…) in the definition of the type of tests to be considered for a given threshold.
Hooks can now be defined to be applied just before or just after the setup (i.e. the creation of the execution workspace) and the teardown (i.e. the closing of the workspace) of a job. These hooks will allow, for example, to:
use a given workspace, instead of creating a new one as it was always the case in previous Squash Orchestrator releases, by specifying this workspace in the "before" hook of a job setup;
reserve the workspace, instead of deleting it, by specifying to keep it in the "before" hook of a job teardown.
In order to further reduce memory consumption and to facilitate the future support of new test technologies, the parsers used to interpret Surefire reports for each technology (JUnit, Cucumber, Cypress…) have been replaced by a single configurable parser. This one is configured for the current test technologies supported by Squash Orchestrator, but that configuration can easily be extended for another new test technology generating Surefire reports. Users who want to add the support of their own technology do not need any more to write a parser microservice.
As a reminder, new versions of Squash AUTOM and DEVOPS are delivered monthly (except in August).
2023-07 will allow the use of Squash TM data in the configuration of a quality gate, will provide further reduction of the memory consumption, and will continue improving the logs.
Every AUTOM/DEVOPS delivery contains several components having each one its own version number. You can view them in the Release Notes by version.
Comments