Job/Pod name

Tasks Description

Input Needed

Frequent issues

sdp-log-explorer-service-setup-tf-apply

• Check Opensearch Availability

• Create if not exists OpenSearch configuration (ISM Policy and Indexes)

• Opensearch started

• opensearch-provider secret

• Network to opensearch open

• User and credentials set in Opensearch

• Infinite loop if Opensearch never starts or is not reachable

• Opensearch is protected with self-signed certificate : Connection Refused

• Opensearch is not reachable : Unknow Host / Connection Refused

• Error 403 when permissions or credentials are not correct

• Terraform download can be long

sdp-semarchy-iam-setup-tf-apply

• Seed IAM prerequisites in the cluster

• Creates secrets keycloak-kafka and keycloak-config 

• kafka-keycloak secret 

• Keycloak POD up and running

• Terraform download can be long

• Secret missing or wrong values.yaml configuration

sdp-semarchy-iam-apply-config-1

• wait for keycloak to be started and sdp-semarchy-iam-create-perm-admin-job to be finished

•     kafka-keycloak secret
•     Keycloak POD up and running

• Infinite loop if Keycloak never starts or is not reachable

sdp-semarchy-iam-create-perm-admin-job-1

• Infinite loop if Keycloak never starts or is not reachable

• kafka-keycloak secret

• Infinite loop if Keycloak never starts or is not reachable

sdp-keycloak-wait-service

• Wait for keycloak to starts

• sdp-self-hosted-keycloak-0 pod up and running

• Ingress controller up and running

• Infinite loop if Keycloak never starts or is not reachable

• Keycloak is protected with self-signed certificate: Connection Refused

• Keycloak is not reachable: Unknow Host / Connection Refused

sdp-self-hosted-keycloak-0

• Start Keycloak pod

• PostgreSQL and Kafka started

• Network to Kafka and PostgreSQL open

• User and credentials already set in Kafka and PostgreSQL

• keycloak-postgres and kafka-keycloak secret

• PostgreSQL is not accessible

• Kafka is not accessible

• Semarchy registry is not accessible

• Wrong values.yaml according to secret (Unexpected handshake request with client mechanism SCRAM-SHA-512, enabled mechanisms are …)

• Ensure Kafka is configured with SCRAM, the secret is using the right port, the security protocol is present in the secret, JAAS is configured for broker/listener communication and IP address is accessible from inside the cluster

Job/Pod name

Tasks Description

Input Needed

Frequent issues

sdp-semarchy-iam-finalize-tf-apply

• Finalize IAM wiring for the management plane

• Creates secret keycloak-provider with informations to connect to keycloak (url, username, login, port…)

• sdp-self-hosted-keycloak-0 pod up and running

• keycloak secrets

• Secret missing or wrong values.yaml configuration

sdp-billing-service-setup-tf-apply

• Set up access to billing and keycloak.

• Generates a keycloak client secret.

• Creates kubernetes_secret billing-keycloak.

• Creates Keycloak OpenID client billing_service.

• Creates a billing_admin role and maps it to the service account

• sdp-self-hosted-keycloak-0 pod up and running

• Keycloak secrets

- Keycloak is not started job fails

- Secret missing or wrong values.yaml configuration

sdp-self-hosted-billing-service-xxxx

• Start Billing Services Pod

• Keycloak up and running

• Opensearch up and running

• Keycloak and OpenSearch secrets

• Wrong values.yaml according to secret 

Job/Pod name

Tasks Description

Input Needed

Frequent issues

sdp-tenant-settings-tf-apply

Apply tenant settings infrastructure/config (base layer)

• Keycloak up and running

• keycloak-provider and mail secret

• Secret missing or wrong values.yaml configuration

• Secret key missing

sdp-tenant-settings-finalize-keycloak-realms

Attaches tenant-settings scope to multiple clients: billing-service, sem-admin-ui, sem-di-migration, sem-xdg-datahub, sem-xdg-fe.

• Keycloak up and running

• curl to keycloak fail (host not reachable or wrong credentials)

sdp-self-hosted-log-explorer-xxx

• Start Log Explorer POD

• Keycloak up and running

• Ingress controller up and running

Job/Pod name

Tasks Description

Input Needed

Frequent issues

sdp-self-hosted-site-admin

• Start Site Admin POD

• Keycloak up and running

• Ingress controller up and running

sdp-self-hosted-user-profile-

• Start User Profile POD

• Keycloak up and running

• Ingress controller up and running

sdp-self-hosted-welcome-

• Start Welcome POD

• Keycloak up and running

• Ingress controller up and running

Job/Pod name

Tasks Description

Input Needed

Frequent issues

sdp-dm-setup-main-tf-apply

• Stand up Semarchy xDM backend (DB + SSO integration)

• PostgreSQL: creates schemas (repository, extensions), enables extensions (uuid_ossp, fuzzystrmatch), and sets grants.

• Keycloak: creates OpenID clients xdm and xdm_api, many protocol mappers (username, names, email, locale, number/date formats, timezone, company, avatarUrl, realm roles, densityMode, site_id_token, group membership), roles (xdm_admin, xdm_designer, xdm_user), and role/scope mappings.

• Kubernetes: creates secret xdm-keycloak used by xDM to talk to Keycloak.

• Keycloak up and running

• PostgreSQL up and running

• DM access to keycloak from external URL (DNS setup)

• dm-postgres secret

• Any of the prerequisites is not fully installed.

• Missing DNS on Kubernetes Load-Balancer. (UnknowHost Exception, Connection Refused)

sdp-dm-core-active-wait-service

• Waits for the DM pod to be running (readiness gate)

• DM Pod active successfully setup

• Infinite Loop due to dm active pod not OK and restarting

sdp-self-hosted-dm-core-active

• Start DM Active POD

• Keycloak up and running

• PostgreSQL up and running

• Kafka up and running

• DM access to keycloak from external URL (DNS setup)

• dm-postgres secret, dm-kafka and opensearch-provider secret

• Wrong values in PostgreSQL secret

• Access denied or PostgreSQL unreachable

• Topic authorization failed for topics [topic-user] : Apply ACLS command

sdp-self-hosted-dm-setup-dm-auto-provisioning

• Job for provisioning initial datasources for DM

• PostgreSQL up and running

• DM active POD up and running

• dm-postgres-datasource-1 and dm-postgres-datasource-2 secrets

• Wrong values in PostgreSQL secret

• Access denied or PostgreSQL unreachable

sdp-semarchy-data-platform-invite-site-admin

• Sends an invitation email to the site admin account

• Mail external service up and running

• Mail-secret

• Job fail if mail user is not well configured

• SMTP is not well configured

• Failed to send execute actions email: Error when attempting to send the email to the server