Skip to content

[!] add Secret Store machinery - #821

Merged
pashagolub merged 8 commits into
masterfrom
feat/secret-store
Aug 24, 2026
Merged

[!] add Secret Store machinery#821
pashagolub merged 8 commits into
masterfrom
feat/secret-store

Conversation

@pashagolub

@pashagolub pashagolub commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

This PR introduces a Postgres-native, pgcrypto-backed secret store (timetable.secret) with ${secret:name} reference resolution in the scheduler, along with masking/redaction changes, updated samples, and extensive specification + documentation.

@pashagolub pashagolub self-assigned this Aug 19, 2026
@pashagolub pashagolub added the enhancement New feature or request label Aug 19, 2026
@coveralls

coveralls commented Aug 19, 2026

Copy link
Copy Markdown

Coverage Report for CI Build 32732603747

Coverage decreased (-0.3%) to 89.703%

Details

  • Coverage decreased (-0.3%) from the base build.
  • Patch coverage: 32 uncovered changes across 5 files (199 of 231 lines covered, 86.15%).
  • No coverage regressions found.

Uncovered Changes

File Changed Covered %
internal/pgengine/secrets.go 183 159 86.89%
internal/pgengine/transaction.go 18 16 88.89%
internal/scheduler/shell.go 5 3 60.0%
internal/scheduler/tasks.go 7 5 71.43%
main.go 3 1 33.33%
Total (8 files) 231 199 86.15%

Coverage Regressions

No coverage regressions found.


Coverage Stats

Coverage Status
Relevant Lines: 2525
Covered Lines: 2265
Line Coverage: 89.7%
Coverage Strength: 1.02 hits per line

💛 - Coveralls

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a Postgres-native, pgcrypto-backed secret store (timetable.secret) with ${secret:name} reference resolution in the scheduler, along with masking/redaction changes, updated samples, and extensive specification + documentation.

Changes:

  • Adds timetable.secret schema + migration 00820 (fresh-install DDL + upgrade migration) and wires migration registration/versioning.
  • Implements secret reference resolution in Go for JSON parameters and libpq conninfo, integrates it into BUILTIN/SQL/PROGRAM execution paths, and adds pgx tracer arg redaction.
  • Updates samples/tests/docs to demonstrate and validate the feature (including “pg_timetable never installs pgcrypto”).

Reviewed changes

Copilot reviewed 28 out of 29 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
spec/tasks/template.md Adds a general task-list template for feature work.
spec/tasks/tasks-design-secret-store.md Adds the detailed implementation task plan for secret-store work.
spec/spec-design-secret-store.md Adds the full v2.1 secret-store design/specification.
samples/RemoteDB.sql Updates sample to use ${secret:...} and self-install pgcrypto for demo/testability.
samples/Mail.sql Updates sample to use ${secret:...} and self-install pgcrypto for demo/testability.
mkdocs.yml Adds Secret Store doc page to the docs nav.
main.go Bumps dbapi to 00820 and runs CheckSecretConfig at startup.
internal/testutils/testcontainers.go Sets a fixed test SecretEncryptionKey for container-based tests.
internal/scheduler/tasks.go Redacts builtin debug logging + resolves secrets for SendMail params.
internal/scheduler/tasks_test.go Adds tests for secret resolution + builtin debug-log redaction.
internal/scheduler/shell.go Resolves secrets for PROGRAM argv while keeping execution logging masked.
internal/pgengine/transaction.go Resolves secrets for SQL params and remote connstrings; adds tracer redaction context.
internal/pgengine/sql/migrations/00820.sql Adds migration creating timetable.secret and related functions/trigger.
internal/pgengine/sql/init.sql Registers migration 00820 in the init seed.
internal/pgengine/sql/ddl.sql Adds fresh-install DDL for timetable.secret mirroring migration 00820.
internal/pgengine/secrets.go Introduces the resolver implementation, quoting rules, and startup config check.
internal/pgengine/secrets_test.go Adds comprehensive unit/integration tests for schema, resolution, and redaction rules.
internal/pgengine/migration.go Registers migration 00820.
internal/pgengine/migration_test.go Extends migration test coverage to assert the secret store exists post-migrate.
internal/log/log.go Adds WithoutQueryArgs marker and drops args in pgx tracer logs when marked.
internal/log/log_test.go Tests that pgx tracer logging drops args under WithoutQueryArgs.
internal/config/config_test.go Adds tests verifying --secret-key / PGTT_SECRET_KEY binding.
internal/config/cmdparser.go Adds SecretEncryptionKey CLI/env config option.
docs/yaml-usage-guide.md Documents secret-store behavior and YAML limitations.
docs/secret_store.md New concept doc page describing the secret store and trust boundary.
docs/samples.md Documents how samples use ${secret:...} and pgcrypto prerequisites.
docs/opentelemetry.md Reformat-only change (no functional content change).
docs/installation.md Updates installation notes re: extensions and secret-store pgcrypto dependency.
docs/database_schema.md Minor doc cleanup (removes ER diagram caption).
Suppressed comments (1)

internal/pgengine/secrets.go:84

  • This block has indentation that gofmt would change (the comment lines aren’t indented under the if). If golangci-lint enforces gofmt, it will fail; please run gofmt or adjust the indentation here.
		if plaintext == nil {
		// Missing secret (one row containing NULL). Indistinguishable
		// across client scopes.
			return "", append(names, name), fmt.Errorf(
				`secret %q not found for client %q`, name, pge.ClientName)

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread internal/pgengine/transaction.go
Comment thread internal/pgengine/secrets.go
Comment thread docs/samples.md Outdated
Comment thread internal/scheduler/tasks.go
Comment thread docs/secret_store.md Outdated
Comment thread internal/log/log.go Outdated
- transaction.go: use unresolved val in parse-error message to avoid leaking decrypted secret plaintext
- log.go: fix gofmt indentation of type block
- secrets.go: add extractRefNames() to correctly extract \ identifiers instead of mis-using uniqueRefNames on raw input
- tasks_test.go: rewrite TestSendMailResolvesSecret to exercise taskSendMail end-to-end via a mocked dialer, catching regressions where secret resolution is skipped
- docs/samples.md, docs/secret_store.md: stop schema-qualifying pgp_sym_encrypt as pgcrypto.pgp_sym_encrypt
- go fmt cleanup of unrelated pre-existing formatting issues (cmdparser.go, provider_internal_test.go, secrets_test.go, testcontainers.go)
@pashagolub
pashagolub merged commit a0268a1 into master Aug 24, 2026
7 checks passed
@pashagolub
pashagolub deleted the feat/secret-store branch August 24, 2026 14:15
@pashagolub pashagolub linked an issue Aug 25, 2026 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add secret store

3 participants