Skip to content

ENG-2238 Create a Discourse Graphs skill for security advisory assessment #1410

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mdroidian merged 1 commit into from
Sep 7, 2026
Merged

ENG-2238 Create a Discourse Graphs skill for security advisory assessment #1410

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .agents/skills/dg-assess-security-advisories/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
name: dg-assess-security-advisories
description: Assess Discourse Graphs Dependabot alerts for actual exposure, produce evidence-backed reports, and carry out approved dismissal or remediation decisions. Use for individual advisories or review batches.
---

# Assess security advisories

Explain whether an advisory affects our system, what an attacker would need, and what we should do. Package severity and installation alone do not establish exposure.

Use `$dg-engineering-writing-style` for explanations and `$dg-create-engineering-ticket` when a ticket is requested. Preserve the user's scope: a request for a count, ticket, explanation, or dry run does not authorize a full assessment or GitHub mutations.

## Assess

1. For a batch, snapshot authenticated GitHub alerts and record the selection rule. Exclude prior reviewed alert numbers when the user asks for the next batch. Preserve duplicate advisories across manifests as separate entries.
2. Pin evidence to the reviewed main commit. Follow the vulnerable function from its workspace dependency path to actual callers, inputs, and outputs. Read [assessment.md](references/assessment.md) for the evidence checklist and parent-package investigation.
3. Classify the result as unused vulnerable code, exposure proposed for risk acceptance, routine remediation, product remediation, or unresolved. State confidence and what has not been tested. Do not convert uncertainty into a dismissal.
4. Produce the requested explanation or report. For batches, use [templates.md](references/templates.md) and the [HTML template](assets/report.html). Save artifacts under the repository's `local/` directory, not an operating-system `/local` directory.

## Decide and act

Present a concrete recommendation before requesting a decision. Honor approval already given for the same action and batch; do not ask twice. Do not infer approval for a different alert from a previous dismissal.

Before an approved dismissal, remediation PR, or closure, read [decisions.md](references/decisions.md). It covers rechecking evidence, choosing the dismissal reason, recording comments, verifying results, and closing the assessment without misrepresenting deferred risks as fixed.

The default is an assessment, not an automatic upgrade or dismissal. Michael has accepted some tooling risk, but that historical decision is not blanket approval. A dependency override also has maintenance cost; explain why it is needed and when it can be removed.

## Examples and validation

Read [cases.md](references/cases.md) when an example helps explain a decision or when validating changes to this skill. Its three historical cases cover an unused vulnerable function, accepted tooling risk, and shared content reaching a parser. They are fixtures, not current security determinations.

A dry run produces local recommendations only. It must not post comments, dismiss alerts, change dependencies, or open tickets/PRs. Validate the reasoning against the evidence, not by matching exact wording.
4 changes: 4 additions & 0 deletions .agents/skills/dg-assess-security-advisories/agents/openai.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
interface:
display_name: "Assess DG security advisories"
short_description: "Assess actual exposure and record approved decisions"
default_prompt: "Use $dg-assess-security-advisories to assess the selected Dependabot alerts and recommend actions based on our actual usage."
187 changes: 187 additions & 0 deletions .agents/skills/dg-assess-security-advisories/assets/report.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,187 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Security advisory assessment</title>
<style>
:root {
color-scheme: light dark;
font-family: system-ui, sans-serif;
}
body {
max-width: 72rem;
margin: auto;
padding: 1.5rem;
line-height: 1.6;
}
header,
article,
aside {
border: 1px solid #8886;
border-radius: 0.6rem;
padding: 1rem;
margin-block: 1rem;
}
nav {
display: flex;
flex-wrap: wrap;
align-items: end;
gap: 1rem;
}
label {
display: grid;
gap: 0.3rem;
}
input,
select,
button {
font: inherit;
padding: 0.5rem;
max-width: 100%;
}
article,
pre,
dd,
a {
overflow-wrap: anywhere;
}
pre {
white-space: pre-wrap;
padding: 0.7rem;
background: #8882;
}
dt {
font-weight: 650;
margin-top: 0.6rem;
}
dd {
margin-left: 0;
}
[hidden] {
display: none !important;
}
a {
color: LinkText;
}
:focus-visible {
outline: 3px solid Highlight;
outline-offset: 3px;
}
@media (max-width: 36rem) {
body {
padding: 0.75rem;
}
nav,
label {
width: 100%;
}
input {
min-width: 0;
}
}
</style>
</head>
<body>
<header>
<h1>Security advisory assessment</h1>
<p>Repository: REPLACE_REPOSITORY. Reviewed main: REPLACE_SHA.</p>
<p>Assessment snapshot: REPLACE_TIME. Selection: REPLACE_SELECTION.</p>
<p>
Open repository totals: REPLACE_SEVERITY_COUNTS. Batch counts and
recommendations: REPLACE_BATCH_SUMMARY.
</p>
<p>
Decision record: REPLACE_TICKET_LINK. Last outcome verification:
REPLACE_VERIFICATION_TIME.
</p>
</header>
<aside>
<strong>Assessment status:</strong> REPLACE_STATUS. Recommendations below
are proposals unless their execution fields say otherwise.
</aside>
<nav aria-label="Filter alerts">
<label
>Search
<input
id="search"
type="search"
placeholder="Package, alert, or evidence"
/></label>
<label
>Recommendation
<select id="action">
<option value="all">All</option>
<option value="unused">Unused vulnerable code</option>
<option value="accept">Proposed or approved risk acceptance</option>
<option value="routine">Routine remediation</option>
<option value="fix">Product remediation</option>
<option value="unresolved">Unresolved</option>
</select>
</label>
<button id="reset" type="button">Reset filters</button>
</nav>
<p id="count" role="status" aria-live="polite"></p>
<main>
<!-- Duplicate and populate this card for each selected alert. Insert escaped text only. -->
<article id="alert-example" data-action="unresolved">
<h2>REPLACE_ALERT_NUMBER: REPLACE_PACKAGE</h2>
<p>REPLACE_PLAIN_LANGUAGE_IMPACT</p>
<dl>
<dt>Advisory and snapshot state</dt>
<dd>REPLACE_GHSA_SEVERITY_STATE_AND_HTTPS_LINK</dd>
<dt>Versions and manifest</dt>
<dd>REPLACE_INSTALLED_AFFECTED_PATCHED_MANIFEST</dd>
<dt>Dependency path and environment</dt>
<dd>REPLACE_WORKSPACE_PARENTS_AND_USAGE</dd>
<dt>Call, input, and attacker prerequisites</dt>
<dd>REPLACE_SOURCE_LINKS_ARGUMENTS_AND_USER_ACTION</dd>
<dt>Evidence and limitations</dt>
<dd>REPLACE_OBSERVATION_CONFIDENCE_AND_NOT_TESTED</dd>
<dt>Parent packages and options</dt>
<dd>REPLACE_LATEST_PARENT_PIN_RANGE_AND_OPTIONS</dd>
<dt>Recommendation</dt>
<dd>REPLACE_ACTION_AND_REASON</dd>
<dt>Proposed dismissal</dt>
<dd>
REPLACE_REASON_OR_NOT_APPLICABLE
<pre>REPLACE_PROPOSED_COMMENT</pre>
</dd>
<dt>Owner decision and execution</dt>
<dd>
REPLACE_PENDING_OR_APPROVED_DECISION_ACTUAL_STATE_AND_VERIFICATION
</dd>
<dt>Follow-up and revisit condition</dt>
<dd>REPLACE_LINK_AND_CONDITION</dd>
</dl>
<a href="#">Back to top</a>
</article>
</main>
<script>
const cards = [...document.querySelectorAll("article[data-action]")];
const search = document.getElementById("search");
const action = document.getElementById("action");
const count = document.getElementById("count");
const filter = () => {
const query = search.value.trim().toLowerCase();
let visible = 0;
for (const card of cards) {
card.hidden = !(
card.textContent.toLowerCase().includes(query) &&
(action.value === "all" || action.value === card.dataset.action)
);
if (!card.hidden) visible++;
}
count.textContent = `${visible} of ${cards.length} alerts shown`;
};
search.addEventListener("input", filter);
action.addEventListener("change", filter);
document.getElementById("reset").addEventListener("click", () => {
search.value = "";
action.value = "all";
filter();
});
filter();
</script>
</body>
</html>
68 changes: 68 additions & 0 deletions .agents/skills/dg-assess-security-advisories/references/assessment.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Assessment procedure

## Snapshot and selection

Use an authenticated GitHub connector or CLI for the requested repository. Paginate all results; do not count only the first page. Record open totals by severity, snapshot time, and exact selected IDs.

```sh
gh api --paginate --slurp 'repos/DiscourseGraphs/discourse-graph/dependabot/alerts?state=open&per_page=100'
```

`--slurp` produces an array of pages. Flatten before filtering. Some gh versions do not combine `--slurp` with `--jq`; sort locally when needed. On Windows, use explicit UTF-8 when saving JSON rather than relying on shell redirection defaults.

Use the user's requested ordering. If none is supplied, the previous batch convention is descending alert number within severity. Say this is a batch order, not an exploitability ranking. Exclude prior reviewed numbers, including reviewed alerts still open, when selecting the next batch.

Record each alert's number, URL, state, GHSA, package, manifest, vulnerable range, and first patched version. Resolve installed versions from the lockfile. Preserve duplicate entries across manifests and major versions, while sharing evidence where justified.

## Reachability evidence

Record the main commit SHA and source links pinned to it. Distinguish the checked-out branch from main and report relevant differences. Never silently reuse a stale local installation as evidence of main's dependency graph.

For each alert, answer:

| Question | Evidence to collect |
| ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Why is the package installed? | Workspace root, direct parent, complete transitive path, dependency category, and resolved versions. Include auto-installed peers; `pnpm why` alone may miss them. |
| What is vulnerable? | Advisory mechanism, affected function/options, input requirements, and first patched version. Read the affected implementation where needed. |
| Do we execute that function? | App/build entry points, callers and arguments, imports, feature configuration, and relevant dependency source. An import or devDependency label alone is insufficient. |
| Who controls the input? | Customer, group member, repository contributor, build configuration, downloaded archive source, or fixed code values. Follow input through transformations to the vulnerable call and relevant output sink. |
| What could happen? | Concrete confidentiality, integrity, or availability impact in that process. Do not infer data theft from a CPU-exhaustion advisory. |
| What remains uncertain? | Missing source, unavailable environment, assumptions, confidence, and the smallest verification step that would change the decision. |

Check host-provided globals and build externalization. For example, upgrading a package does not replace `window.CryptoJS` if the Roam build maps imports to that global. Distinguish package code from a host-loaded implementation.

Separate product runtime, developer/build/install tooling, and unused paths. Tooling can still process hostile input; it is not automatically safe. For archive extraction, identify the archive source and extraction method. For glob denial of service, distinguish a malicious pattern from a filename matched against a fixed pattern.

For shared content, record membership controls and recipient actions. Admin-approved membership narrows access but does not prevent a malicious or compromised member from submitting executable content. Keep a local parser reproduction separate from a complete cross-user exploit in the host app.

Use harmless, bounded probes in a local scratch directory. Avoid exhaustion payloads, production mutations, or testing against other users. State exactly what the probe proves.

## Parent versions and alternatives

Inspect both the installed parent's package manifest and current published metadata. Check whether the latest parent release actually changes or removes the affected dependency; a newer parent is not proof of a fix.

```sh
npm view <parent>@latest version dependencies optionalDependencies engines --json
```

Follow optional/platform packages when a packaging change may move the dependency. Determine whether we use the parent itself, just a different feature of the parent, or neither.

Compare these options:

1. Refresh targeted lockfile resolutions if existing parent ranges allow the patched version.
2. Upgrade a used parent, with validation of its affected workflows.
3. Remove an unused direct dependency if its consumers and scope justify that change.
4. Use a version-scoped override only when necessary and authorized.
5. Recommend no action or explicit acceptance of residual risk with evidence.

Exact pins and compatible ranges require different explanations. An override installs a published replacement package; it is not a source patch. Scope it to affected versions within an existing major when compatible. Record the parent, reason, and removal check: remove the override, refresh relevant lockfile entries, confirm affected versions do not return, and validate. Broad parent ranges can keep matching an override and hold the selected version; it does not clean itself up.

Avoid global major overrides, speculative parent upgrades, and unrelated lockfile churn. A grouped tooling PR is reasonable only when the changes are compatible, useful, and approved together.

## Report checks

For batch reports, include counts, the reviewed SHA, all selected alerts, source evidence, limitations, options, proposed comments, and separate final decision/state fields. A recommendation is not an executed action.

Copy `assets/report.html` to repository `local/`, replace its example card, and create one card per selected alert. Escape untrusted advisory/source text and URLs before insertion. Use only intended HTTPS links and escaped text; never embed executable input from an advisory or note. The report itself must remain self-contained.

Verify counts against the snapshot, search/filter/reset behavior, internal links, HTTPS link targets, browser errors, and mobile overflow. Keep raw authenticated API responses local. Put the durable rationale in the ticket so future work does not depend on ignored local files.
Loading