Run BDD Feature Tests¶
Table of Contents¶
What Do BDD Tests Validate?¶
BDD (Behaviour-Driven Development) features verify that the Canvas operators correctly handle the ODA Custom Resource lifecycle. Each use case maps to specific CRDs and operators:
Use Case |
What it validates |
CRDs / Operators tested |
|---|---|---|
UC002 Manage Components |
Component creation, update, deletion lifecycle |
|
UC003 Configure Exposed APIs |
API Gateway route creation, readiness |
|
UC005 Configure Clients and Roles |
Identity management, role assignment |
|
UC007 Configure Dependent APIs |
Cross-component dependency resolution |
|
UC010 External Authentication |
Authentication flow logging |
|
UC012 Configure Observability |
Metrics scraping, trace collection |
|
UC013 Seamless Upgrades |
Component version upgrades |
|
UC015 API Gateway Management |
Gateway-specific features (Istio/Kong/APISIX) |
|
UC016 Component Registry |
Component registration |
|
UC017 PDB Management |
Pod Disruption Budget policies |
|
The tests themselves deploy a test component (release name ctk) from local test data in feature-definition-and-test-kit/testData/ and then verify that the Canvas processes it correctly. This means:
The Component CRD schema is validated end-to-end
Each operator’s reconciliation logic is tested against real Kubernetes resources
The test component YAML in
testData/productcatalog-v1/serves as a reference implementation of the Component specification
Step 1: Auto-Setup Dependencies¶
Check if node_modules/ exists in each directory and install if needed:
python <scripts>/check_bdd_deps.py <path-to-feature-definition-and-test-kit>
To install manually:
cd feature-definition-and-test-kit
cd utilities/identity-manager-utils-keycloak && npm install && cd ../..
cd utilities/package-manager-utils-helm && npm install && cd ../..
cd utilities/resource-inventory-utils-kubernetes && npm install && cd ../..
cd utilities/observability-utils-kubernetes && npm install && cd ../..
cd utilities/component-utils && npm install && cd ../..
npm install
Only run npm install where node_modules/ is missing. On PowerShell: Test-Path node_modules; on bash: [ -d node_modules ].
Step 2: Browse and Select Tests¶
Present use cases as a menu:
Use Case |
Title |
Features |
|---|---|---|
UC002 |
Manage Components |
6 features |
UC003 |
Configure Exposed APIs |
6 features |
UC005 |
Configure Clients and Roles |
3 features |
UC007 |
Configure Dependent APIs |
2 features |
UC010 |
External Authentication |
1 feature |
UC012 |
Configure Observability |
2 features |
UC013 |
Seamless Upgrades |
2 features |
UC015 |
API Gateway Management |
4 features |
UC016 |
Component Registry Management |
1 feature |
UC017 |
PDB Management |
4 features |
After the user selects a use case, list the specific features and let them pick one or run all.
Step 3: Check Environment Variables¶
Some tests require specific environment variables:
Tests |
Required Variables |
Purpose |
|---|---|---|
UC005-* |
|
Keycloak identity management |
UC010-* |
|
Authentication logging |
UC012-F001 |
OpenTelemetry collector must be port-forwarded to port 8888 |
Traces collection |
If required variables are missing, check for .env in feature-definition-and-test-kit/. If not found, prompt the user.
Step 4: Run Tests¶
Execute from the feature-definition-and-test-kit/ directory:
# All features for a use case
npm run start:tags -- "@UC003"
# Single feature
npm run start:tags -- "@UC003-F001"
# Specific feature file
npm start -- features/UC003-F001-Expose-APIs-Create-API-Resource.feature
Step 5: Report Results¶
Stream terminal output. After completion, summarize:
Total scenarios run
Passed / Failed / Pending counts
For failures: failing scenario name and error message
Cucumber Report URL: shareable link valid for 24 hours — highlight to user
JSON results are written to /tmp/cucumber.json.
After reporting results, explain what the tests validated in ODA Canvas terms:
Passed scenarios confirm that the corresponding operator(s) correctly processed the CRD lifecycle. For example, if UC003 tests pass, the API Operator successfully created gateway routes for ExposedAPIs and reported them as ready.
Failed scenarios indicate an operator behavior gap. Reference the use case mapping table above to identify which operator and CRD are involved, then suggest checking the operator logs:
# Example: UC003 failure → check API Operator kubectl logs -l app=oda-controller-apioperator -n canvas --tail=50 # UC005 failure → check Identity Operator kubectl logs -l app=oda-controller-identityoperator -n canvas --tail=50 # UC007 failure → check Dependency Operator kubectl logs -l app=oda-controller-dependentapi -n canvas --tail=50
Important: BDD Tests Deploy Their Own Components¶
Tests install a test component from local
testData/using release namectk(default)A new component
ctk-productcatalogmanagementwill appear incomponentsnamespaceTests use
helm installfirst run,helm upgradefor subsequent runsAfter tests complete, the
ctkrelease remains — offer cleanup withhelm delete ctk -n componentsTest charts in
testData/may differ from remoteoda-componentsrepo charts
Tag Reference¶
Tag |
Scope |
|---|---|
|
All features in a use case |
|
Single feature |
|
Apisix gateway tests only |
|
Kong gateway tests only |
|
OpenTelemetry tests (requires collector) |
|
OpenMetrics tests (requires ServiceMonitor support) |
|
Skipped tests |
Contextual Next Steps¶
Present these options using ask_questions:
# |
Next Step |
|---|---|
1 |
View components deployed by tests — Inspect the |
2 |
Run another test suite — Select a different use case to validate |
3 |
View ExposedAPIs — See APIs created by the test component |
4 |
Clean up test components — Remove the |
5 |
Return to main menu |