Service Catalog & Infrastructure as Code

What Is a Service Catalog?

A service catalog is the platform's menu of pre-approved, pre-configured infrastructure and services that development teams can consume via self-service. Instead of filing tickets for a database or S3 bucket, developers submit a Kubernetes manifest (a claim) and the platform provisions the resource automatically — correctly configured, secured, and tagged.

Service Catalog Layers

Crossplane Architecture

Crossplane is a CNCF graduated project that extends Kubernetes with the ability to manage cloud infrastructure using the K8s control loop. Provider plugins add CRDs for each cloud resource type; the composition engine builds higher-level APIs from those primitives.

Install Crossplane

helm repo add crossplane-stable https://charts.crossplane.io/stable
helm repo update

helm install crossplane crossplane-stable/crossplane \
  --namespace crossplane-system \
  --create-namespace \
  --version 1.17.1 \
  --set args='{--enable-composition-functions,--enable-composition-revisions}' \
  --set resourcesCrossplane.limits.cpu=500m \
  --set resourcesCrossplane.limits.memory=1Gi \
  --set resourcesRBACManager.limits.cpu=100m \
  --set resourcesRBACManager.limits.memory=512MiCopy

Crossplane Providers

Providers are Crossplane plugins that add CRDs for specific cloud or infrastructure targets. Each provider runs as a pod in the cluster and reconciles managed resource objects against the real infrastructure.

Installing AWS Providers (Family Pattern)

# provider-family-aws installs sub-providers on demand
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: upbound-provider-family-aws
spec:
  package: xpkg.upbound.io/upbound/provider-family-aws:v1.14.0
  runtimeConfigRef:
    name: provider-aws-runtime
---
apiVersion: pkg.crossplane.io/v1beta1
kind: DeploymentRuntimeConfig
metadata:
  name: provider-aws-runtime
spec:
  deploymentTemplate:
    spec:
      selector: {}
      template:
        spec:
          serviceAccountName: provider-aws
          containers:
          - name: package-runtime
            resources:
              limits:
                cpu: 500m
                memory: 512Mi
---
# Sub-providers: install only what you need
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: upbound-provider-aws-rds
spec:
  package: xpkg.upbound.io/upbound/provider-aws-rds:v1.14.0
---
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: upbound-provider-aws-s3
spec:
  package: xpkg.upbound.io/upbound/provider-aws-s3:v1.14.0
---
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: upbound-provider-aws-ec2
spec:
  package: xpkg.upbound.io/upbound/provider-aws-ec2:v1.14.0Copy

ProviderConfig with IRSA

# ProviderConfig tells the AWS provider which credentials to use
apiVersion: aws.upbound.io/v1beta1
kind: ProviderConfig
metadata:
  name: default
spec:
  credentials:
    source: IRSA   # uses the provider pod's ServiceAccount token via IRSA
---
# IRSA: annotate the provider's ServiceAccount with the IAM role ARN
# Done during provider installation via DeploymentRuntimeConfig or post-install:
kubectl annotate serviceaccount provider-aws \
  -n crossplane-system \
  eks.amazonaws.com/role-arn=arn:aws:iam::123456789012:role/crossplane-provider-awsCopy

Provider Health Check

# Check provider status
kubectl get providers
# NAME                              INSTALLED   HEALTHY   PACKAGE                                    AGE
# upbound-provider-aws-rds          True        True      xpkg.upbound.io/upbound/provider-aws-rds   2d

# Check provider revision
kubectl get providerrevisions -o wide

# Inspect provider CRDs
kubectl get crds | grep rds.awsCopy

Provider Ecosystem

Managed Resources

Managed resources (MRs) are the lowest-level Crossplane objects — each one maps to exactly one cloud resource. They are rarely used directly by application teams; they are the building blocks that Compositions assemble.

RDS Instance Managed Resource

apiVersion: rds.aws.upbound.io/v1beta1
kind: Instance
metadata:
  name: payments-db
  annotations:
    crossplane.io/external-name: payments-db-prod   # actual RDS identifier
spec:
  forProvider:
    region: us-east-1
    dbInstanceClass: db.t3.medium
    engine: postgres
    engineVersion: "16.2"
    dbName: payments
    username: dbadmin
    manageMasterUserPassword: true   # use RDS-managed secret rotation
    allocatedStorage: 100
    storageType: gp3
    storageEncrypted: true
    multiAz: true
    backupRetentionPeriod: 7
    deletionProtection: true
    autoMinorVersionUpgrade: true
    enabledCloudwatchLogsExports:
    - postgresql
    - upgrade
    tags:
      team: payments
      env: production
      cost-center: CC-4892
  writeConnectionSecretToRef:
    namespace: payments-api-production
    name: payments-db-conn   # Crossplane writes connection details here
  providerConfigRef:
    name: defaultCopy

S3 Bucket Managed Resource

apiVersion: s3.aws.upbound.io/v1beta1
kind: Bucket
metadata:
  name: payments-artifacts
spec:
  forProvider:
    region: us-east-1
    tags:
      team: payments
      env: production
  providerConfigRef:
    name: default
---
apiVersion: s3.aws.upbound.io/v1beta1
kind: BucketVersioning
metadata:
  name: payments-artifacts-versioning
spec:
  forProvider:
    region: us-east-1
    bucketRef:
      name: payments-artifacts
    versioningConfiguration:
    - status: Enabled
---
apiVersion: s3.aws.upbound.io/v1beta1
kind: BucketServerSideEncryptionConfiguration
metadata:
  name: payments-artifacts-sse
spec:
  forProvider:
    region: us-east-1
    bucketRef:
      name: payments-artifacts
    rule:
    - applyServerSideEncryptionByDefault:
      - sseAlgorithm: aws:kmsCopy

Managed Resource Lifecycle

# Check managed resource status
kubectl get instances.rds.aws.upbound.io
# NAME          READY   SYNCED   EXTERNAL-NAME        AGE
# payments-db   True    True     payments-db-prod      1h

# Inspect conditions
kubectl describe instance payments-db | grep -A 20 "Conditions:"

# Deletion protection: by default Crossplane deletes the cloud resource
# when the K8s object is deleted. Add this annotation to keep it:
kubectl annotate instance payments-db \
  crossplane.io/paused=true   # pause reconciliation temporarily

# Or set deletionPolicy: Orphan to detach without deleting cloud resource
spec:
  deletionPolicy: Orphan   # Delete | Orphan (default: Delete)Copy

Compositions & XRDs

The composition layer is where the platform team's engineering effort lives. A CompositeResourceDefinition (XRD) defines the API schema teams interact with. A Composition maps that schema to one or more managed resources, applying platform-standard defaults.

CompositeResourceDefinition: TeamDatabase

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xteamdatabases.platform.example.com
spec:
  group: platform.example.com
  names:
    kind: XTeamDatabase
    plural: xteamdatabases
  # Claim names (namespace-scoped version)
  claimNames:
    kind: TeamDatabase
    plural: teamdatabases
  connectionSecretKeys:
  - username
  - password
  - endpoint
  - port
  - dbname
  versions:
  - name: v1alpha1
    served: true
    referenceable: true
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            required: ["parameters"]
            properties:
              parameters:
                type: object
                required: ["engine","size","name"]
                properties:
                  engine:
                    type: string
                    enum: ["postgres","mysql"]
                    default: postgres
                  engineVersion:
                    type: string
                    default: "16"
                  size:
                    type: string
                    enum: ["small","medium","large"]
                    description: "small=t3.micro, medium=t3.medium, large=r6g.large"
                  name:
                    type: string
                    description: "Database name (alphanumeric, max 63 chars)"
                  multiAz:
                    type: boolean
                    default: false
                  backupRetentionDays:
                    type: integer
                    minimum: 1
                    maximum: 35
                    default: 7
          status:
            type: object
            properties:
              endpoint:
                type: string
              phase:
                type: stringCopy

Composition: TeamDatabase → RDS

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: teamdatabase-aws-postgres
  labels:
    provider: aws
    engine: postgres
spec:
  compositeTypeRef:
    apiVersion: platform.example.com/v1alpha1
    kind: XTeamDatabase
  writeConnectionSecretsToNamespace: crossplane-system
  # Composition Functions pipeline (preferred over patches for complex logic)
  mode: Pipeline
  pipeline:
  - step: patch-and-transform
    functionRef:
      name: function-patch-and-transform
    input:
      apiVersion: pt.fn.crossplane.io/v1beta1
      kind: Resources
      resources:

      # ── RDS Instance ──────────────────────────────────────────
      - name: rds-instance
        base:
          apiVersion: rds.aws.upbound.io/v1beta1
          kind: Instance
          spec:
            forProvider:
              region: us-east-1
              storageEncrypted: true
              backupRetentionPeriod: 7
              deletionProtection: true
              autoMinorVersionUpgrade: true
              enabledCloudwatchLogsExports: ["postgresql"]
            writeConnectionSecretToRef:
              namespace: crossplane-system
            providerConfigRef:
              name: default
        patches:
        - type: FromCompositeFieldPath
          fromFieldPath: spec.parameters.name
          toFieldPath: spec.forProvider.dbName
        - type: FromCompositeFieldPath
          fromFieldPath: spec.parameters.engine
          toFieldPath: spec.forProvider.engine
        - type: FromCompositeFieldPath
          fromFieldPath: spec.parameters.engineVersion
          toFieldPath: spec.forProvider.engineVersion
        - type: FromCompositeFieldPath
          fromFieldPath: spec.parameters.multiAz
          toFieldPath: spec.forProvider.multiAz
        - type: FromCompositeFieldPath
          fromFieldPath: spec.parameters.backupRetentionDays
          toFieldPath: spec.forProvider.backupRetentionPeriod
        - type: FromCompositeFieldPath
          fromFieldPath: spec.parameters.size
          toFieldPath: spec.forProvider.dbInstanceClass
          transforms:
          - type: map
            map:
              small: db.t3.micro
              medium: db.t3.medium
              large: db.r6g.large
        - type: FromCompositeFieldPath
          fromFieldPath: metadata.labels[crossplane.io/claim-namespace]
          toFieldPath: spec.forProvider.tags[team]
        - type: FromCompositeFieldPath
          fromFieldPath: metadata.name
          toFieldPath: spec.writeConnectionSecretToRef.name
          transforms:
          - type: string
            string:
              fmt: "%s-conn"
        connectionDetails:
        - type: FromConnectionSecretKey
          name: username
          fromConnectionSecretKey: username
        - type: FromConnectionSecretKey
          name: password
          fromConnectionSecretKey: password
        - type: FromConnectionSecretKey
          name: endpoint
          fromConnectionSecretKey: endpoint
        - type: FromConnectionSecretKey
          name: port
          fromConnectionSecretKey: port

      # ── Security Group ────────────────────────────────────────
      - name: security-group
        base:
          apiVersion: ec2.aws.upbound.io/v1beta1
          kind: SecurityGroup
          spec:
            forProvider:
              region: us-east-1
              description: "Crossplane-managed RDS security group"
              vpcId: "vpc-0abcdef1234567890"   # platform default VPC
            providerConfigRef:
              name: default
        patches:
        - type: FromCompositeFieldPath
          fromFieldPath: metadata.name
          toFieldPath: spec.forProvider.name
          transforms:
          - type: string
            string:
              fmt: "crossplane-%s-sg"Copy

Self-Service Claims

Claims are the namespace-scoped API that application teams actually use. They expose only the parameters the platform team wants teams to control — the complexity of RDS security groups, subnet groups, and parameter groups is entirely hidden.

TeamDatabase Claim

# Submitted by the payments team — lives in their GitOps repo
apiVersion: platform.example.com/v1alpha1
kind: TeamDatabase
metadata:
  name: payments-db
  namespace: payments-api-production
spec:
  parameters:
    engine: postgres
    engineVersion: "16"
    size: medium
    name: payments
    multiAz: true
    backupRetentionDays: 14
  # Connection secret written to this namespace
  writeConnectionSecretToRef:
    name: payments-db-connCopy

Consuming the Connection Secret

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payments-api
  namespace: payments-api-production
spec:
  template:
    spec:
      containers:
      - name: payments-api
        envFrom:
        - secretRef:
            name: payments-db-conn   # written by Crossplane
        # Secret contains: endpoint, port, username, password, dbnameCopy

Claim Status Tracking

# Watch claim provisioning
kubectl get teamdatabase -n payments-api-production -w
# NAME          READY   CONNECTION-SECRET   AGE
# payments-db   False                       30s
# payments-db   True    payments-db-conn    8m   ← RDS created

# Get composite resource details
kubectl get xteamdatabase -o wide

# Describe claim to see conditions and resource refs
kubectl describe teamdatabase payments-db -n payments-api-productionCopy

Database-as-a-Service Pattern

The full production database-as-a-service pattern includes not just RDS creation but automatic secret rotation via Vault, monitoring dashboards, and backup validation.

Full DBaaS Composition Resources

ElastiCache (Redis) Claim Pattern

# XRD for TeamCache
---
apiVersion: platform.example.com/v1alpha1
kind: TeamCache
metadata:
  name: payments-session-cache
  namespace: payments-api-production
spec:
  parameters:
    engine: redis
    engineVersion: "7.2"
    size: small       # small=cache.t3.micro, medium=cache.r6g.large
    replicas: 1
    transitEncryption: true
    atRestEncryption: true
  writeConnectionSecretToRef:
    name: payments-cache-connCopy

S3 Bucket Claim Pattern

apiVersion: platform.example.com/v1alpha1
kind: TeamBucket
metadata:
  name: payments-documents
  namespace: payments-api-production
spec:
  parameters:
    versioning: true
    lifecycleDays: 90          # transition to Glacier after 90 days
    corsEnabled: false
    publicAccessBlock: true    # platform enforces; teams cannot override
  writeConnectionSecretToRef:
    name: payments-bucket-conn  # contains: bucket-name, region, iam-role-arnCopy

Networking Infrastructure

Platform teams manage shared VPC infrastructure; composition patterns allow teams to request network resources (security group rules, Route53 records, ACM certificates) without direct cloud console access.

ACM Certificate Claim

apiVersion: platform.example.com/v1alpha1
kind: TeamCertificate
metadata:
  name: payments-tls
  namespace: payments-api-production
spec:
  parameters:
    domain: "payments-api.company.com"
    alternativeDomains:
    - "payments.company.com"
    validationMethod: DNS    # platform creates Route53 validation records
  writeConnectionSecretToRef:
    name: payments-tls-cert   # contains: certificate-arnCopy

Composition: VPC Peering Request

apiVersion: platform.example.com/v1alpha1
kind: VpcPeeringRequest
metadata:
  name: payments-to-data-warehouse
  namespace: payments-api-production
spec:
  parameters:
    peerAccountId: "987654321098"
    peerVpcId: "vpc-0xyz9876"
    peerRegion: us-east-1
    # Platform team approves requests; composition creates peer connection
    # and adds routes after approval annotation is setCopy

Terraform Provider for Crossplane

The provider-terraform (Upbound) allows wrapping existing Terraform modules as Crossplane managed resources. This is the migration bridge — teams can continue using proven Terraform modules while the platform team incrementally rewrites them as native Crossplane compositions.

Install provider-terraform

apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: upbound-provider-terraform
spec:
  package: xpkg.upbound.io/upbound/provider-terraform:v0.18.0
---
apiVersion: tf.upbound.io/v1beta1
kind: ProviderConfig
metadata:
  name: default
spec:
  credentials:
  - filename: aws_credentials
    source: InjectedIdentity   # use pod IRSA
  configuration: |
    terraform {
      required_providers {
        aws = {
          source  = "hashicorp/aws"
          version = "~> 5.0"
        }
      }
      backend "kubernetes" {
        secret_suffix = "state"
        namespace     = "crossplane-system"
        in_cluster_config = true
      }
    }
    provider "aws" {
      region = "us-east-1"
    }Copy

Workspace: Wrap a Terraform Module

apiVersion: tf.upbound.io/v1beta1
kind: Workspace
metadata:
  name: payments-sns-topic
spec:
  forProvider:
    source: Remote
    module: git::https://github.com/myorg/terraform-modules.git//sns-topic?ref=v1.2.0
    varFiles:
    - source: SecretKey
      secretKeyRef:
        namespace: crossplane-system
        name: terraform-vars
        key: sns-vars.tfvars
    vars:
    - key: topic_name
      value: payments-events
    - key: environment
      value: production
    - key: kms_key_id
      value: arn:aws:kms:us-east-1:123456789012:key/mrk-abc123
  writeConnectionSecretToRef:
    namespace: payments-api-production
    name: payments-sns-conn
  providerConfigRef:
    name: defaultCopy

Backstage Integration

The service catalog gains maximum value when exposed through Backstage — teams browse available services, provision via templates, and track infrastructure in the software catalog. See 04-developer-portal.html for the Backstage setup foundation.

Crossplane Resource in Backstage Catalog

# catalog-info.yaml — added when TeamDatabase is provisioned
apiVersion: backstage.io/v1alpha1
kind: Resource
metadata:
  name: payments-postgres-db
  namespace: default
  annotations:
    backstage.io/managed-by-location: url:https://github.com/myorg/infra/blob/main/payments/database/catalog-info.yaml
    crossplane.io/claim-name: payments-db
    crossplane.io/claim-namespace: payments-api-production
    grafana/dashboard-selector: "{ \"type\": \"tag\", \"value\": \"rds\" }"
    pagerduty.com/integration-key: "abc123"
  tags:
  - postgres
  - rds
  - production
spec:
  type: database
  lifecycle: production
  owner: group:payments-team
  system: payments-platform
  dependencyOf:
  - component:payments-api
  - component:payments-workerCopy

Backstage Software Template for Database Provisioning

apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
  name: provision-database
  title: Provision Team Database
  description: Self-service PostgreSQL or MySQL on RDS via Crossplane
spec:
  owner: platform-team
  type: infrastructure
  parameters:
  - title: Database Configuration
    required: ["name","owner","namespace","engine","size"]
    properties:
      name:
        title: Database name
        type: string
        pattern: "^[a-z][a-z0-9-]{1,30}$"
      owner:
        title: Owner
        type: string
        ui:field: OwnerPicker
        ui:options:
          catalogFilter:
          - kind: Group
      namespace:
        title: Target namespace
        type: string
        ui:field: EntityPicker
        ui:options:
          catalogFilter:
          - kind: Resource
            spec.type: namespace
      engine:
        title: Engine
        type: string
        enum: ["postgres","mysql"]
        default: postgres
      size:
        title: Instance Size
        type: string
        enum: ["small","medium","large"]
        enumNames: ["Small (dev/staging)","Medium (production)","Large (high-traffic)"]
      multiAz:
        title: Multi-AZ (production recommended)
        type: boolean
        default: false
  steps:
  - id: fetch
    name: Fetch Skeleton
    action: fetch:template
    input:
      url: ./skeleton
      values:
        name: ${{ parameters.name }}
        namespace: ${{ parameters.namespace }}
        engine: ${{ parameters.engine }}
        size: ${{ parameters.size }}
        multiAz: ${{ parameters.multiAz }}
        owner: ${{ parameters.owner }}
  - id: pr
    name: Create PR with TeamDatabase claim
    action: publish:github:pull-request
    input:
      repoUrl: github.com?repo=platform-gitops&owner=myorg
      branchName: "db/${{ parameters.name }}"
      title: "feat: provision database ${{ parameters.name }}"
      description: "Auto-generated TeamDatabase claim by ${{ user.entity.metadata.name }}"
  - id: catalog-register
    name: Register in Catalog
    action: catalog:register
    input:
      repoContentsUrl: ${{ steps.pr.output.remoteUrl }}
      catalogInfoPath: catalog-info.yaml
  output:
    links:
    - title: Pull Request
      url: ${{ steps['pr'].output.remoteUrl }}
    - title: Track in Catalog
      entityRef: ${{ steps['catalog-register'].output.entityRef }}Copy

Observability & Health

Crossplane Prometheus Metrics

# Key Crossplane metrics
crossplane_managed_resource_ready_total      # count of ready MRs by provider
crossplane_managed_resource_synced_total     # count of synced MRs
crossplane_managed_resource_deletion_total   # deletions by provider
crossplane_composition_count                 # active compositions
provider_reconcile_errors_total              # reconciliation errors per provider
provider_sync_duration_seconds               # time to reconcile a MRCopy

PrometheusRule: Crossplane Health Alerts

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: crossplane-alerts
  namespace: monitoring
spec:
  groups:
  - name: crossplane.health
    rules:
    # Managed resource stuck NotReady for > 15 minutes
    - alert: CrossplaneManagedResourceNotReady
      expr: |
        crossplane_managed_resource_ready_total{ready="False"} > 0
      for: 15m
      labels:
        severity: warning
      annotations:
        summary: "Crossplane managed resource not ready: {{ $labels.name }}"
        description: "Check: kubectl describe {{ $labels.kind }} {{ $labels.name }}"

    # Provider reconciliation errors spiking
    - alert: CrossplaneProviderHighErrorRate
      expr: |
        rate(provider_reconcile_errors_total[5m]) > 0.1
      for: 5m
      labels:
        severity: warning
      annotations:
        summary: "High reconciliation error rate for provider {{ $labels.controller }}"

    # Claim stuck pending (RDS not provisioning)
    - alert: CrossplaneClaimNotReady
      expr: |
        kube_customresource_status_condition{
          condition="Ready",
          status="False",
          customresource_group="platform.example.com"
        } > 0
      for: 20m
      labels:
        severity: warning
      annotations:
        summary: "Platform claim not ready after 20 minutes"
        description: "Namespace: {{ $labels.namespace }}, Claim: {{ $labels.customresource_name }}"Copy

Claim Health Summary Query

# All claims across platform — ready/not-ready
kubectl get teamdatabases,teamcaches,teambuckets -A \
  -o custom-columns='NS:.metadata.namespace,NAME:.metadata.name,READY:.status.conditions[?(@.type=="Ready")].status,AGE:.metadata.creationTimestamp'

# Claims not ready for > 10 minutes (potential provision failure)
kubectl get teamdatabases -A -o json | jq '
  .items[] | select(
    (.status.conditions[]? | select(.type=="Ready") | .status) == "False"
  ) | {
    namespace: .metadata.namespace,
    name: .metadata.name,
    message: (.status.conditions[]? | select(.type=="Ready") | .message)
  }'Copy

Best Practices