Installing and executing the BDD tests¶
To execute the tests, first you install the necessary packages and set the environment variables.
install necessary packages
cd utilities/identity-manager-utils-keycloak npm install cd ../package-manager-utils-helm npm install cd ../resource-inventory-utils-kubernetes npm install cd ../observability-utils-kubernetes npm install cd ../.. npm install
create a
.envfile and setKEYCLOAK_USER,KEYCLOAK_PASSWORD,KEYCLOAK_BASE_URLandKEYCLOAK_REALM, - or use another option to define the variablesKEYCLOAK_USER=admin KEYCLOAK_PASSWORD=adpass KEYCLOAK_BASE_URL=http://keycloack-ip:8083/auth/ KEYCLOAK_REALM=odari RESOURCE_INVENTORY_BASE_URL=http://localhost:8639/tmf-api/resourceInventoryManagement/v5
How to run the tests¶
Run the test in the command line using the following command:
npm start
All the tests should run and display the results in the command line.
If you only want to run a single test, you can use the following command:
npm start -- features/UC003-F001-Expose-APIs-Create-API-Resource.feature
The use cases and features are tagged. You can run the tests for a given use case with the following command:
npm run start:tags -- "@UC003"
Or run the tests for a single feature with the following command:
npm run start:tags -- "@UC003-F001"
Selecting the tests for a change¶
test-selection.yaml maps the paths a change touches to the features it should run. The selector reads it and prints the features in scope, the tag expression for each run, and the reason for a full run or a skip:
npm run select-features -- --base origin/main
You can also pass the changed paths yourself, one per line (- reads them from stdin):
git diff --name-only origin/main | npm run select-features -- --files -
Add --format json for machine-readable output.
A change to charts/canvas-oda/values.yaml or charts/canvas-oda/Chart.yaml is mapped by the sections it changes, so bumping one operator’s version runs only that operator’s features. The selector needs the diff for this. It gets it from git with --base, or from a saved GitHub PR files API response with --pr-files. A plain --files list has no diffs, so a change to either file in it runs the full suite.
The Kong and APISIX features need a canvas installed with that gateway instead of Istio, so they get a run of their own. The selector prints the --set values to install the canvas for that run.
When you add a feature or a test chart, or rename a path that the map lists, run the check and the unit tests:
npm run check:selection
npm run test:selection
The check fails when a feature has no rule that selects it, when a feature’s tags don’t match its file name, when a test chart isn’t used by any feature, or when a path in the map matches no file.
In CI¶
The trigger_test workflow uses the selector on every pull request to main:
check_skip_tests_joblists the files the PR changes and selects the features, using theselect-bdd-testsaction. The job summary shows the selection.run_tests_jobruns the default selection on a kind cluster with Istio. It’s skipped when nothing is selected, for example for a documentation-only PR.run_gateway_tests_jobruns the Kong and APISIX features, when they’re selected, each on its own kind cluster with that gateway.The BDD badges are updated only by a full run started from the Actions tab on
main, the regression test before a general release. Pull request runs never update them.
[skip tests] in the PR title or the last commit message, or a last commit that only removes prerelease suffixes, still skips the tests.
A couple of minutes after each push, a BDD test selection comment on the PR shows what will run. It lists each changed path and the map rule it matched, the tag expression and features of each run, and the reason for a full run or a skip. The BDD test selection check workflow selects the features, and bdd-selection-comment.yml posts the comment and updates it on later pushes. The comment workflow runs from main with write access, so it works for PRs from forks too. It never runs code from the PR.
To run the full suite, including both gateway runs, open Actions > trigger_test > Run workflow, then pick the branch or enter a tag or commit.
Updating the master test status¶
To update the master test status that appears in the top-level README.md, set the following environment variable:
set or export
CUCUMBER_PUBLISH_TOKENCUCUMBER_PUBLISH_TOKEN=9afda79b-9ea0-44ff-8359-7f381ade4bb6
run ‘npm start’ command to execute all the tests