Migrate configuration from environment variables to a YAML deployment file

.dockerignore

@@ -4,7 +4,9 @@
 CONTRIBUTING.md
 Dockerfile
 *.env
-!.env
+tex-bot-deployment.yaml
+tex-bot-deployment.*.yaml
+tex-bot-deployment.yaml.*.tmp
 uv.lock
 pyproject.toml
 tests

.gitattributes

@@ -7,7 +7,6 @@
 .toml text eol=lf
 .md text eol=lf
 .*ignore text eol=lf
-.env* text eol=lf
 .gitattributes text eol=lf
 .gitlint text eol=lf
 .python-version text eol=lf

.gitignore

@@ -104,6 +104,13 @@ pids
 .dmypy.json
 dmypy.json
 
+# TeX-Bot deployment configuration (contains the bot token & other secrets)
+tex-bot-deployment.yaml
+tex-bot-deployment.*.yaml
+!tex-bot-deployment.example.yaml
+# NOTE: Written alongside the configuration file whilst it is being rewritten in place.
+tex-bot-deployment.yaml.*.tmp
+
 # Environments
 .env
 ._env

.pre-commit-config.yaml

@@ -45,12 +45,6 @@ repos:
         - id: zizmor
           args: [--no-progress, --fix, --persona=auditor]
 
-    - repo: https://github.com/dotenv-linter/dotenv-linter
-      rev: v4.0.0
-      hooks:
-        - id: dotenv-linter
-          args: [fix, --no-backup]
-
     - repo: https://github.com/hadolint/hadolint
       rev: v2.14.0
       hooks:
@@ -114,7 +108,6 @@ repos:
           args: [--fix=lf]
         - id: detect-private-key
         - id: fix-byte-order-marker
-          args: [-h]
 
     - repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks
       rev: v2.16.0
@@ -123,7 +116,7 @@ repos:
           args: [--autofix]
 
     - repo: https://github.com/astral-sh/ruff-pre-commit
-      rev: v0.15.22
+      rev: v0.16.1
       hooks:
         - id: ruff-check
           args: [--fix]

.yamllint.yaml

@@ -1,5 +1,13 @@
 extends: default
 
+# NOTE: The deployment configuration is written by whoever deploys TeX-Bot, rather than
+# being one of this repository's own files. Its settings are ordered to be read from the
+# top down (those that must be filled in first, then each section in turn), which the
+# alphabetical key ordering required below would destroy.
+ignore: |
+    tex-bot-deployment.*.yaml
+    tex-bot-deployment.yaml
+
 locale: en_GB.UTF-8
 
 rules:

CONTRIBUTING.md

@@ -52,16 +52,45 @@ If you are submitting a feature request, please include the steps to implement t
 ### Top level files
 
 * [`main.py`](main.py): is the main entrypoint to instantiate the [`Bot` object](https://docs.pycord.dev/stable/api/clients.html#discord.Bot) & run it
-* [`config.py`](config.py): retrieves the [environment variables](README.md#setting-environment-variables) & populates the correct values into the `settings` object
+* [`tex-bot-deployment.example.yaml`](tex-bot-deployment.example.yaml): documents every [configuration setting](README.md#configuring-tex-bot) that TeX-Bot understands, along with its default value
 
 ### Other significant directories
 
+* [`config/`](config): reads & validates the deployment configuration file, and exposes it as the `settings` object, see [below](#configuration) for more information
 * [`cogs/`](cogs): contains all the [cogs](https://guide.pycord.dev/popular-topics/cogs) within this project, see [below](#cogs) for more information
 * [`exceptions/`](exceptions): contains common [exception](https://docs.python.org/3/tutorial/errors) subclasses that may be raised when certain errors occur
 * [`utils/`](utils): contains common utility classes & functions used by the top-level modules & cogs
 * [`db/core/models/`](db/core/models): contains all the [database ORM models](https://docs.djangoproject.com/en/stable/topics/db/models) to interact with storing information longer-term (between individual command events)
+* [`stubs/`](stubs): contains hand-written [type stubs](https://typing.python.org/en/latest/spec/distributing.html#stub-files) for the third-party packages that do not ship their own
 * [`tests/`](tests): contains the complete test suite for this project, based on the [Pytest framework](https://pytest.org)
 
+### Configuration
+
+TeX-Bot is configured by a single [YAML](https://yaml.org) file, which is read, validated & applied by the modules within [the `config` package](config).
+Each module owns one concern, so that no other part of the project needs to know how the configuration file is stored:
+
+* [`config/_schema.py`](config/_schema.py): the single source of truth for the configuration.
+Declares every setting as a [Pydantic](https://docs.pydantic.dev) model: its type, its constraints, its default, its help text, whether it holds a secret & whether changing it requires a restart.
+**Any new setting is added here, and nowhere else**
+
+* [`config/_document.py`](config/_document.py): owns every interaction with the file itself; locating it, parsing it while retaining its comments & formatting, writing it back atomically, & reporting a validation failure against the line that caused it
+
+* [`config/_accessor.py`](config/_accessor.py): holds the loaded configuration as a single immutable snapshot, replaced wholesale whenever it is reloaded, so that no reader can observe a half-applied configuration
+
+* [`config/_editor.py`](config/_editor.py): changes an individual setting, on behalf of [the `/config` command](README.md#changing-settings-from-within-discord), validating the result before anything is written
+
+* [`config/_logging.py`](config/_logging.py): applies the logging settings, replacing existing handlers rather than adding to them, so that reloading cannot accumulate duplicates
+
+* [`config/_messages.py`](config/_messages.py): loads [the messages file](messages.json), which is held separately from the deployment configuration because it is a body of content rather than a set of settings
+
+Settings are read as attributes of the `settings` object, by the section holding them:
+
+```python
+import config
+
+config.settings.commands.ping.easter_egg_probability
+```
+
 ### Cogs
 
 [Cogs](https://guide.pycord.dev/popular-topics/cogs) are attachable modules that are loaded onto the [`Bot` instance](https://docs.pycord.dev/en/stable/api/clients.html#discord.Bot).
@@ -73,18 +102,34 @@ There are separate cog files for each activity, and one [`__init__.py`](cogs/__i
 
 * [`cogs/__init__.py`](cogs/__init__.py): instantiates all the cog classes within this directory
 
+* [`cogs/add_users_to_threads_and_channels.py`](cogs/add_users_to_threads_and_channels.py): cogs for adding Discord members & roles to a channel or thread
+
+* [`cogs/annual_handover_and_reset.py`](cogs/annual_handover_and_reset.py): cogs for performing your group's annual committee handover & membership reset
+
 * [`cogs/archive.py`](cogs/archive.py): cogs for archiving categories of channels within your group's Discord guild
 
+* [`cogs/check_su_platform_authorisation.py`](cogs/check_su_platform_authorisation.py): cogs for checking whether TeX-Bot is still authenticated to your group's SU platform
+
 * [`cogs/command_error.py`](cogs/command_error.py): cogs for sending error messages when commands fail to complete/execute
 
+* [`cogs/committee_actions_tracking.py`](cogs/committee_actions_tracking.py): cogs for tracking the actions assigned to each committee member
+
+* [`cogs/config.py`](cogs/config.py): cogs for viewing & changing [TeX-Bot's configuration](README.md#changing-settings-from-within-discord) at run-time
+
 * [`cogs/delete_all.py`](cogs/delete_all.py): cogs for deleting all permanent data stored in a specific object's table in the database
 
 * [`cogs/edit_message.py`](cogs/edit_message.py): cogs for editing messages that were previously sent by TeX-Bot
 
+* [`cogs/everest.py`](cogs/everest.py): cogs for calculating how many steps of Mount Everest a university assignment is worth
+
 * [`cogs/induct.py`](cogs/induct.py): cogs for inducting people into your group's Discord guild
 
+* [`cogs/invite_link.py`](cogs/invite_link.py): cogs for sending an invite link to your group's Discord guild
+
 * [`cogs/kill.py`](cogs/kill.py): cogs related to the shutdown of TeX-Bot
 
+* [`cogs/make_applicant.py`](cogs/make_applicant.py): cogs related to making guests into committee applicants
+
 * [`cogs/make_member.py`](cogs/make_member.py): cogs related to making guests into members
 
 * [`cogs/ping.py`](cogs/ping.py): cog to request a [ping](https://wikipedia.org/wiki/Ping-pong_scheme#Internet) response

Dockerfile

@@ -14,12 +14,20 @@ RUN --mount=type=cache,target=/root/.cache/uv \
     uv sync --frozen --no-install-project --no-group dev
 
 COPY LICENSE /app/
-COPY config.py main.py messages.json /app/
+COPY main.py messages.json /app/
+COPY config/ /app/config/
 COPY exceptions/ /app/exceptions/
 COPY utils/ /app/utils/
 COPY db/ /app/db/
 COPY cogs/ /app/cogs/
 
+# NOTE: The deployment configuration is kept in its own directory, separate from any
+# application code, so that it can be mounted as a directory. Mounting the directory
+# (rather than the configuration file individually) allows TeX-Bot to rewrite the file
+# in place when the `/config` command changes a setting: replacing an individually
+# mounted file is rejected, because a rename cannot replace a mount point.
+RUN mkdir --parents /app/data
+
 FROM python:3.13-slim-trixie
 
 RUN groupadd --system --gid 999 nonroot && useradd --system --gid 999 --uid 999 --create-home nonroot
@@ -29,7 +37,8 @@ LABEL org.opencontainers.image.licenses=Apache-2.0
 
 COPY --from=builder --chown=nonroot:nonroot /app /app
 
-ENV LANG=C.UTF-8 PATH="/app/.venv/bin:$PATH"
+ENV LANG=C.UTF-8 PATH="/app/.venv/bin:$PATH" \
+    TEX_BOT_CONFIG_PATH=/app/data/tex-bot-deployment.yaml
 
 WORKDIR /app