RotatingSecrets

A library and demo for zero-downtime database credential rotation in Kubernetes environments. The rotating-secrets module is a plain-Java core that reads fresh credentials from Kubernetes-mounted secret files and notifies registered pool updaters when passwords are rotated. The rotating-secrets-hikari and rotating-secrets-ucp modules adapt HikariCP and Oracle Universal Connection Pool (UCP) to it, and rotating-secrets-spring creates and registers the provider as a Spring bean. Each Spring-facing module depends only on its own library plus spring-context. The demo module is a Spring Boot application that exercises the library.

How It Works

When running in Kubernetes with a secrets manager (HashiCorp Vault, OpenBao, External Secrets Operator), database credentials can be automatically rotated. The library uses a publisher-subscriber pattern to notify connection pools when credentials change:

  1. Secret Mounting: A secrets manager mounts credentials as files in a configurable directory (default: /var/run/secrets/database/)

  2. Credential Monitoring: CredentialsProviderService watches the secrets directory for changes using WatchService and re-reads credentials when a change event fires

  3. Pool Notification: When credentials change, all registered UpdatableCredential implementations are notified:

    • HikariCP: Updates credentials and soft-evicts existing connections
    • Oracle UCP: Updates credentials and refreshes the connection pool
  4. Seamless Rotation: New connections automatically use updated credentials while existing connections continue unaffected until returned to the pool

sequenceDiagram participant VA as Vault Agent participant SF as Secret Files participant CPS as CredentialsProviderService participant CU as CredentialsUpdater participant Pool as Connection Pool participant DB as Database VA->>SF: Write new credentials loop On directory event or fallback timeout CPS->>SF: Read credentials SF-->>CPS: Return file contents end CPS->>CPS: Detect credential change CPS->>CU: setCredential(username, password) alt HikariCP CU->>Pool: Update credentials CU->>Pool: softEvictConnections() Pool->>Pool: Mark existing connections<br/>for closure on return else Oracle UCP CU->>Pool: setUser() / setPassword() CU->>Pool: refreshConnectionPool() Pool->>Pool: Gracefully replace<br/>connections end Pool->>DB: New connections use<br/>updated credentials

Library: rotating-secrets

API

classDiagram class UpdatableCredential~T~ { <<interface>> +setCredential(username: String, credential: T) void } class HikariCredentialsProvider { <<interface>> +getCredentials() Credentials } class CredentialsProviderService { #usernamePath: Path #passwordPath: Path -username: String -password: String -updatables: List~UpdatableCredential~ ~refreshCredentials() void ~updateCredentials() void } class HikariCredentialsUpdater { -dataSource: HikariDataSource -credentials: Credentials +setCredential(username, credential) void +getCredentials() Credentials +setDataSource(dataSource) void } class UcpCredentialsUpdater { -poolDataSource: PoolDataSource +setCredential(username, credential) void } class HikariDataSource { +getHikariPoolMXBean() HikariPoolMXBean } class PoolDataSource { +setUser(user) void +setPassword(password) void } UpdatableCredential <|.. HikariCredentialsUpdater : implements UpdatableCredential <|.. UcpCredentialsUpdater : implements HikariCredentialsProvider <|.. HikariCredentialsUpdater : implements CredentialsProviderService --> UpdatableCredential : notifies HikariCredentialsUpdater --> HikariDataSource : manages UcpCredentialsUpdater --> PoolDataSource : manages

Connection Pool Support

Feature HikariCP Oracle UCP
Default for Spring Boot Yes No
Oracle-specific features No Yes
Credential update Via CredentialsProvider interface Direct pool refresh
Connection eviction Soft evict (graceful) Pool refresh
FAN support No Yes
Application Continuity No Yes

Production Considerations

Architecture

Component Overview

flowchart TB subgraph K8s["Kubernetes Environment"] subgraph Secrets["Secrets Manager"] Vault["HashiCorp Vault / OpenBao / ESO"] end subgraph Volume["Mounted Volume"] SecretFiles["/var/run/secrets/database/<br/>├── username<br/>└── password"] end Vault -->|"writes credentials"| SecretFiles subgraph App["Application (demo + rotating-secrets library)"] CPS["CredentialsProviderService<br/><i>Reads secrets, detects changes</i>"] subgraph Updaters["Credential Updaters"] HCU["HikariCredentialsUpdater<br/><i>Stores creds, soft evicts</i>"] UCU["UcpCredentialsUpdater<br/><i>Updates pool, refreshes</i>"] end subgraph Pools["Connection Pools"] HDS["HikariDataSource<br/><i>(Primary)</i>"] PDS["PoolDataSource<br/><i>(Oracle UCP)</i>"] end SecretFiles -->|"watches for changes"| CPS CPS -->|"notifies on change"| HCU CPS -->|"notifies on change"| UCU HCU --> HDS UCU --> PDS end end subgraph DB["Database"] Database[("PostgreSQL /<br/>Oracle RAC")] end HDS --> Database PDS --> Database

Deployment Architecture

flowchart LR subgraph Cluster["Kubernetes Cluster"] subgraph NS["Application Namespace"] subgraph Pod["Application Pod"] Init["Init Container<br/><i>Vault Agent</i>"] App["App Container<br/><i>Spring Boot</i>"] Vol[("Shared Volume<br/>/var/run/secrets")] Init -->|"writes"| Vol Vol -->|"reads"| App end end subgraph Vault["Vault Namespace"] VaultServer["Vault Server"] DBSecrets["Database Secrets<br/>Engine"] end Init <-->|"authenticates &<br/>fetches secrets"| VaultServer VaultServer --> DBSecrets end subgraph External["External"] DB[("Database<br/>Server")] end DBSecrets <-->|"rotates credentials"| DB App -->|"connects with<br/>current credentials"| DB

Demo Application

Project Structure

rotating-secrets/                          # Core library (depends only on SLF4J)
└── src/main/java/
    ├── module-info.java
    └── com/maybeitssquid/rotatingsecrets/
        ├── UpdatableCredential.java           # Interface for credential update notification
        ├── CredentialsProviderService.java    # Reads secrets, notifies registered updaters
        └── CredentialRotationException.java   # Exception for rotation failures

rotating-secrets-hikari/                   # HikariCP adapter (HikariCP, spring-context)
└── src/main/java/
    ├── module-info.java
    └── com/maybeitssquid/rotatingsecrets/hikari/
        ├── HikariCredentialsUpdater.java      # HikariCP credential rotation handler
        └── HikariDataSourceConfig.java        # HikariCP configuration (primary)

rotating-secrets-ucp/                      # Oracle UCP adapter (UCP, spring-context)
└── src/main/java/
    ├── module-info.java
    └── com/maybeitssquid/rotatingsecrets/ucp/
        ├── UcpCredentialsUpdater.java         # Oracle UCP credential rotation handler
        └── UcpDataSourceConfig.java           # Oracle UCP configuration

rotating-secrets-spring/                   # Spring configuration (spring-context)
└── src/main/java/
    ├── module-info.java
    └── com/maybeitssquid/rotatingsecrets/spring/
        └── RotatingSecretsConfig.java         # Creates and starts the provider; registers every updater bean

demo/                                      # Spring Boot demo application
└── src/main/
    ├── java/com/maybeitssquid/rotatingsecrets/
    │   ├── DemoRotatingSecretsApplication.java  # Entry point with scheduling enabled
    │   ├── DemoDatabasePollingService.java      # Exercises the connection pool
    │   └── DemoQueryResult.java                 # Query result model
    └── resources/
        └── application.properties         # Datasource and pool configuration

Prerequisites

Configuration

application.properties

spring.application.name=RotatingSecrets

# Kubernetes secrets path (mounted by Vault Agent or CSI driver)
k8s.secrets.path=/var/run/secrets/database
k8s.secrets.refresh-interval=30s  # fallback timeout if no watch event fires

# Common datasource settings
spring.datasource.url=jdbc:oracle:thin:@//host:1521/service
spring.datasource.driver-class-name=oracle.jdbc.OracleDriver
spring.datasource.username=myuser
spring.datasource.password=mypassword

# HikariCP settings (primary datasource)
spring.datasource.hikari.pool-name=HikariRotatingSecrets
spring.datasource.hikari.maximum-pool-size=10
spring.datasource.hikari.minimum-idle=2
spring.datasource.hikari.idle-timeout=30000
spring.datasource.hikari.connection-timeout=20000
spring.datasource.hikari.max-lifetime=1800000

# Oracle UCP settings
spring.datasource.ucp.pool-name=UCPRotatingSecrets
spring.datasource.ucp.url=${spring.datasource.url}
spring.datasource.ucp.connection-factory-class-name=${spring.datasource.driver-class-name}
spring.datasource.ucp.user=${spring.datasource.username}
spring.datasource.ucp.password=${spring.datasource.password}
spring.datasource.ucp.initial-pool-size=2
spring.datasource.ucp.min-pool-size=2
spring.datasource.ucp.max-pool-size=10
spring.datasource.ucp.connection-wait-timeout=20
spring.datasource.ucp.inactive-connection-timeout=30
spring.datasource.ucp.max-connection-reuse-time=1800

Secret Files

The demo expects these files in the secrets directory:

File Description
username Database username
password Database password

Building

./gradlew build

Running

Local Development

Create the secret files in a local directory:

mkdir -p /tmp/secrets/database
echo "myuser" > /tmp/secrets/database/username
echo "mypassword" > /tmp/secrets/database/password

Run with the custom secrets path:

./gradlew :demo:bootRun --args='--k8s.secrets.path=/tmp/secrets/database'

Kubernetes Deployment

Mount your secrets as a volume at the configured path. Example with Vault Agent:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    vault.hashicorp.com/agent-inject: "true"
    vault.hashicorp.com/agent-inject-secret-username: "database/creds/myapp"
    vault.hashicorp.com/agent-inject-template-username: |
      {{- with secret "database/creds/myapp" -}}
      {{ .Data.username }}
      {{- end }}
    vault.hashicorp.com/agent-inject-secret-password: "database/creds/myapp"
    vault.hashicorp.com/agent-inject-template-password: |
      {{- with secret "database/creds/myapp" -}}
      {{ .Data.password }}
      {{- end }}
spec:
  containers:
    - name: app
      image: rotating-secrets:latest

Testing

./gradlew test

Technologies

Component Version
Java 25 (toolchain; runs on 17+)
Gradle 9.6.1
Spring Boot 4.0.6
Spring Cloud Vault 2025.1.1
HikariCP (via Spring Boot)
Resilience4j (via Spring Cloud)
Oracle UCP (via oracle-jdbc)
Oracle JDBC ojdbc11
JaCoCo 0.8.14

Apache 2.0