Skip to content

feat(dashboard): make missing required evidence explicit - #642

Merged
seonghobae merged 2 commits into
feat/dashboard-case-metricsfrom
feat/dashboard-missing-facts
Aug 25, 2026
Merged

feat(dashboard): make missing required evidence explicit#642
seonghobae merged 2 commits into
feat/dashboard-case-metricsfrom
feat/dashboard-missing-facts

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • require every case-specific question to be either supported by an exact cited span or explicitly missing
  • persist unsupported answers in a normalized relation without invented value or evidence text
  • render an actionable collection instruction in Dashboard cards
  • add a Storybook missing-evidence scene and real migration coverage

Verification

  • uv run --extra backend --extra dev pytest -q tests/test_operations_case_analysis.py tests/test_operations_case_ingestion.py tests/test_operations_dashboard.py tests/test_schema.py::test_migration_applies_cleanly tests/test_public_docstrings.py tests/test_documentation_hygiene.py (20 passed)
  • corepack pnpm exec vitest run src/components/OperationsDashboard.test.tsx (5 passed)
  • corepack pnpm lint
  • corepack pnpm build
  • corepack pnpm build-storybook
  • Storybook RequiredFactMissing desktop screenshot inspected

Stacked on #640. No unsupported fact receives a fabricated value, evidence span, keyword rule, threshold, or weight.


Open in Devin Review

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitai Bot commented Aug 25, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 097e5f64-1d20-418f-b16e-719d09c36cf8

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae
seonghobae merged commit f9a7a78 into feat/dashboard-case-metrics Aug 25, 2026
3 of 4 checks passed
@seonghobae

Copy link
Copy Markdown
Contributor Author

Resolved the stacked-base conflict by merging the current feat/dashboard-case-metrics base into the PR branch at 1f3568e9, retaining both ExternalInformationEmpty and RequiredFactMissing Storybook scenes. The branch is no longer dirty against its current stack base.

Validation: backend operations-case/dashboard focused tests 12 passed; frontend Vitest 41 files / 383 tests passed. Hosted checks remain authoritative; this stacked PR still waits for its parent #640 and protected approvals.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

Open in Devin Review

Comment on lines +154 to +161
supported_types = {fact.fact_type_code for fact in parsed_facts}
if (
any(not isinstance(code, str) or code not in FACT_TYPES for code in missing_fact_types)
or len(set(missing_fact_types)) != len(missing_fact_types)
or supported_types.intersection(missing_fact_types)
or supported_types.union(missing_fact_types) != REQUIRED_FACT_TYPES[item["case_kind_code"]]
):
return None

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Completeness gate rejects the whole response

The gate at operations_case_analysis.py requires each case's supported plus missing fact types to exactly equal that kind's required set. Any case that fails makes parse_operations_case_response return None, which raises ValueError and fails the whole analysis job. This also forbids a case from carrying a supported fact of a type belonging to another kind, unlike the prior facts loop.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant