OAuth2 EnvoyFilter Operator

Overview

The OAuth2 EnvoyFilter Operator is a component of the ODA Canvas that enforces token-based authentication at the Istio service mesh level. It is built using the Kopf Kubernetes operator framework to manage EnvoyFilter and related Istio resources, ensuring that only properly authorised callers can communicate between ODA Components — without requiring individual components to implement their own OAuth2 authentication logic.

Key Features

  • Istio Integration: Dynamically creates EnvoyFilter, ServiceEntry, and DestinationRule resources to inject OAuth2 token verification into the Istio service mesh.

  • Lifecycle Management: Automates creation, update, and deletion of authentication configuration in response to DependentAPI resource events.

  • Credentials Management: Works alongside the Credentials Management Operator to obtain and manage OAuth2 client credentials.

Build and Release

The build and release process for docker images is described in work-with-dockerimages.md.

Testing KOPF module from workstation

It uses the config file present in $HOME/.kube/config to run from a local workstation.

Run: kopf run docker/src/oauth2EnvoyfilterOperator.py

Notes

Prerequisites

Credentials-Manager-Operator has to be deployed.

helm upgrade --install canvas-credman-op -n canvas charts/credentialsmanagement-operator --set=credentials.client_secret=IDH9****eIqD

set default namespace to “components”

kubectl config set-context --current --namespace=components

Installation

helm upgrade --install canvas-oauth2-op -n canvas charts/oauth2-envoyfilter-operator

alte objekt löschen

kubectl get serviceentry,envoyfilter,destinationrule,secret -n components
kubectl delete exposedapi externalapi-productcatalogmanagement
kubectl delete serviceentry add-https
kubectl delete destinationrule demo-b-productcatalogmanagement-downstreamproductcatalog-add-https
kubectl delete secret envoy-oauth2-secrets 
kubectl delete envoyfilter demo-b-productcatalogmanagement-envoyfilter-oauth2 

Deployment of ExposedAPI

helm upgrade --install demo-a -n components feature-definition-and-test-kit/testData/productcatalog-v1

Alternative: external Wrapper

kubectl apply -f source/operators/TMFOP008-OAuth2-Config/envoy-filter/docker/manual_test/testdata/yamls/externalapi-productcatalogmanagement-echoservice.yaml
kubectl apply -f source/operators/TMFOP008-OAuth2-Config/envoy-filter/docker/manual_test/testdata/yamls/externalapi-productcatalogmanagement-echo-beepceptor.yaml

Testdeployment of component with dependency

helm upgrade --install demo-b -n components feature-definition-and-test-kit/testData/productcatalog-v1 --set component.dependentAPIs.enabled=true  --set=add_dependency_envvars=true

check if dependencies are injected (requires waiting and restart)

kubectl exec -it demo-b-prodcatapi-XXXXXXXXX -- /bin/sh -c "echo $DEPENDENCY_URL_DOWNSTREAMPRODUCTCATALOG"

test oauth2 injection

curl -XPOST http://echoservice.ihc-dt.cluster-1.de/echo
{"echo_body":"","echo_header":{"Accept":"*/*","Authorization":"Bearer eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJ0c2dUWVVVeFBSSkVSN3g0VHdHb1N5YzdISzNZWl9JWG13YXVrY2RkZkpzIn0.eyJleHAiOjE3NDE2MDM3MTIsImlhdCI6MTc0MTYwMzQxMiwianRpIjoiODlkNzI2ZjktOGUwOC00OTdhLWFhYjItN2ZjYmNlZGQ5Njg1IiwiaXNzIjoiaHR0cDovL2NhbnZhcy1rZXljbG9hay5jYW52YXMuc3ZjLmNsdXN0ZXIubG9jYWw6ODA4My9hdXRoL3JlYWxtcy9vZGFyaSIsImF1ZCI6ImFjY291bnQiLCJzdWIiOiI3ZjM5ZmQ1NS03M2JkLTQwOWItOTUzMy01NDg4NmU1YmUxZDUiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJkZW1vLWItcHJvZHVjdGNhdGFsb2dtYW5hZ2VtZW50IiwiYWNyIjoiMSIsInJlYWxtX2FjY2VzcyI6eyJyb2xlcyI6WyJvZmZsaW5lX2FjY2VzcyIsInVtYV9hdXRob3JpemF0aW9uIiwiZGVmYXVsdC1yb2xlcy1vZGFyaSJdfSwicmVzb3VyY2VfYWNjZXNzIjp7ImFjY291bnQiOnsicm9sZXMiOlsibWFuYWdlLWFjY291bnQiLCJtYW5hZ2UtYWNjb3VudC1saW5rcyIsInZpZXctcHJvZmlsZSJdfX0sInNjb3BlIjoiZW1haWwgcHJvZmlsZSIsImNsaWVudEhvc3QiOiIxMC45Mi4xLjExOSIsImVtYWlsX3ZlcmlmaWVkIjpmYWxzZSwiY2xpZW50SWQiOiJkZW1vLWItcHJvZHVjdGNhdGFsb2dtYW5hZ2VtZW50IiwicHJlZmVycmVkX3VzZXJuYW1lIjoic2VydmljZS1hY2NvdW50LWRlbW8tYi1wcm9kdWN0Y2F0YWxvZ21hbmFnZW1lbnQiLCJjbGllbnRBZGRyZXNzIjoiMTAuOTIuMS4xMTkifQ.lD_2q8WZ2yuUTVi_mlRAxlmiSsRbbNLWiCRT1lx81_Zvcr4B98i8sAnWaHZupgkj3YGyQJl4fg6AapD1Xbeq0RUStCtkRXPMzxmCD__2deO7VP1o2s8leZosFGXBDVVliRYYbhcEDfDL8EqqBWc6sa6qvoZ4AselCOZoaWXI-kCkX5ejWCt6zQdzUs66u7ImwLOG2lEBcNMnfL0_NfYmOfGZF-zVYDM3QehdidfXDIvW6Rb9usIs_IrpF2-v-qDePW2vj-JPyGeJ1lmD98Sye2391Bwn4GNh5xPPs301ajG3RNRewj5KI6AeHVS3iJOwQZBqOk-bHJzPfzHg9PXBJw","Content-Length":"0","Host":"echoservice.ihc-dt.cluster-1.de","User-Agent":"curl/7.64.0","X-Envoy-Attempt-Count":"1","X-Envoy-Internal":"true","X-Forwarded-Client-Cert":"By=spiffe://cluster.local/ns/echoservice/sa/default;Hash=640a4234426843626e304d5e5238a328766999d3767a6c8c6eaf6c08ec314954;Subject=\"\";URI=spiffe://cluster.local/ns/istio-ingress/sa/istio-ingress","X-Forwarded-For":"10.92.1.1","X-Forwarded-Proto":"http","X-Request-Id":"bdc84a0f-0f23-4911-be16-41dc83b590f0"},"timestamp":"2025-03-10T10:43:56.561749"}

inject error at token generation:

root@demo-b-prodcatapi-6675466b75-nrgtl:/src# curl http://echo.free.beeceptor.com/echo
Failed to inject credential.root@demo-b-prodcatapi-6675466b75-nrgtl:/src# curl http://echo.free.beeceptor.com/echo

error analysis

log istiod

kubectl logs -n istio-system deployment/istiod
2025-03-04T14:59:32.399396Z     info    delta   ADS: new delta connection for node:demo-a-prodcatapi-5988ff6d7-q8rr2.components-22
2025-03-04T14:59:32.400608Z     info    delta   CDS: PUSH request for node:demo-a-prodcatapi-5988ff6d7-q8rr2.components resources:69 removed:0 size:68.4kB cached:64/65 filtered:0
2025-03-04T14:59:32.401170Z     info    delta   EDS: PUSH request for node:demo-a-prodcatapi-5988ff6d7-q8rr2.components resources:48 removed:0 size:11.3kB empty:0 cached:47/48 filtered:0
2025-03-04T14:59:32.404255Z     info    delta   LDS: PUSH request for node:demo-a-prodcatapi-5988ff6d7-q8rr2.components resources:49 removed:0 size:96.6kB filtered:0
2025-03-04T14:59:32.405000Z     info    delta   RDS: PUSH request for node:demo-a-prodcatapi-5988ff6d7-q8rr2.components resources:23 removed:0 size:26.7kB cached:22/23 filtered:0
2025-03-04T14:59:32.434877Z     warn    delta   ADS:LDS: ACK ERROR demo-a-prodcatapi-5988ff6d7-q8rr2.components-22 Internal:Error adding/updating listener(s) 0.0.0.0_10259: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
0.0.0.0_9100: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
0.0.0.0_10255: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
0.0.0.0_80: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
0.0.0.0_15010: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
0.0.0.0_9093: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
34.118.239.83_8443: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
34.118.239.83_443: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
34.118.235.169_443: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
0.0.0.0_10249: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
0.0.0.0_8080: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
0.0.0.0_8083: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
34.118.233.239_80: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
0.0.0.0_9153: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
0.0.0.0_10257: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
0.0.0.0_2381: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
34.118.236.245_5000: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
0.0.0.0_15014: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
34.118.227.205_15021: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
10.164.15.198_4194: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
34.118.228.114_443: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
0.0.0.0_9090: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist
0.0.0.0_4000: paths must refer to an existing path in the system: '/envoy_secrets/oauth2secs/demo-a-productcatalogmanagement.yaml' does not exist

2025-03-04T15:01:45.404272Z     info    delta   ADS: "10.92.1.125:45926" demo-b-prodcatapi-76c69fbd6f-lwxsw.components-12 terminated
2025-03-04T15:01:45.589392Z     info    delta   ADS: new delta connection for node:demo-b-prodcatapi-76c69fbd6f-lwxsw.components-23
2025-03-04T15:01:45.590569Z     info    delta   CDS: PUSH request for node:demo-b-prodcatapi-76c69fbd6f-lwxsw.components resources:69 removed:0 size:68.4kB cached:64/65 filtered:0
2025-03-04T15:01:45.591012Z     info    delta   EDS: PUSH request for node:demo-b-prodcatapi-76c69fbd6f-lwxsw.components resources:48 removed:0 size:11.3kB empty:0 cached:47/48 filtered:0
2025-03-04T15:01:45.592895Z     info    delta   LDS: PUSH request for node:demo-b-prodcatapi-76c69fbd6f-lwxsw.components resources:49 removed:0 size:83.0kB filtered:0
2025-03-04T15:01:45.593584Z     info    delta   RDS: PUSH request for node:demo-b-prodcatapi-76c69fbd6f-lwxsw.components resources:23 removed:0 size:26.7kB cached:22/23 filtered:0

log sidecar

kubectl logs -n components demo-b-prodcatapi-7d8c788c77-v7b9h istio-proxy
...
2025-03-04T15:35:56.042168Z     error   envoy credential_injector external/envoy/source/extensions/http/injected_credentials/oauth2/token_provider.cc:111       onGetAccessTokenFailure: Failed to get access token      thread=12
2025-03-04T15:35:56.042254Z     error   envoy credential_injector external/envoy/source/extensions/http/injected_credentials/oauth2/oauth_client.cc:74  Oauth response code: 404        thread=12
2025-03-04T15:35:56.042263Z     error   envoy credential_injector external/envoy/source/extensions/http/injected_credentials/oauth2/oauth_client.cc:75  Oauth response body: {"error":"Realm does not exist"}   thread=12
2025-03-04T15:35:56.042266Z     error   envoy credential_injector external/envoy/source/extensions/http/injected_credentials/oauth2/token_provider.cc:111       onGetAccessTokenFailure: Failed to get access token      thread=12
2025-03-04T15:35:56.042356Z     error   envoy credential_injector external/envoy/source/extensions/http/injected_credentials/oauth2/oauth_client.cc:74  Oauth response code: 404        thread=12
2025-03-04T15:35:56.042370Z     error   envoy credential_injector external/envoy/source/extensions/http/injected_credentials/oauth2/oauth_client.cc:75  Oauth response body: {"error":"Realm does not exist"}   thread=12
2025-03-04T15:35:56.042372Z     error   envoy credential_injector external/envoy/source/extensions/http/injected_credentials/oauth2/token_provider.cc:111       onGetAccessTokenFailure: Failed to get access token      thread=12

deploy

The build and release process for docker images is described here: docs/developer/work-with-dockerimages.md