Skip to content

Commit 6f57af2

Browse files
sidmohan0sidmohan0
committed
docs: make v4.5 core story obvious
Refs DFPY-72
1 parent 90c426d commit 6f57af2

4 files changed

Lines changed: 181 additions & 89 deletions

File tree

‎README.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,14 @@ It provides:
99
- A simple agent-oriented API for LLM applications
1010
- Backward-compatible `DataFog` and `TextService` classes
1111

12+
## 4.5 Focus
13+
14+
DataFog 4.5 is focused on lightweight text PII screening: a small core install,
15+
fast regex-based scan/redact helpers, explicit optional extras, and a clearer
16+
path toward future middleware use cases. Dedicated Sentry, OpenTelemetry,
17+
logging-framework, and cloud DLP adapters are future-facing work and are not
18+
part of the 4.5 release.
19+
1220
## Installation
1321

1422
```bash

‎docs/getting-started.rst‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
================================
2+
Getting Started With DataFog 4.5
3+
================================
4+
5+
DataFog 4.5 focuses on lightweight text PII screening. A core install should
6+
let you scan and redact common structured PII without installing OCR, Spark,
7+
large NLP models, or middleware integrations.
8+
9+
Install Profiles
10+
================
11+
12+
Core text screening:
13+
14+
.. code-block:: bash
15+
16+
pip install datafog
17+
18+
Optional extras are explicit:
19+
20+
.. list-table::
21+
:header-rows: 1
22+
23+
* - Profile
24+
- Install command
25+
- Use when
26+
* - Core
27+
- ``pip install datafog``
28+
- You need regex-based text scanning, redaction, and guardrail helpers.
29+
* - NLP
30+
- ``pip install "datafog[nlp]"``
31+
- You need spaCy-backed named entity recognition.
32+
* - Advanced NLP
33+
- ``pip install "datafog[nlp-advanced]"``
34+
- You need GLiNER-backed named entity recognition.
35+
* - OCR
36+
- ``pip install "datafog[ocr]"``
37+
- You need local image text extraction before PII scanning.
38+
* - OCR from URLs
39+
- ``pip install "datafog[web,ocr]"``
40+
- You need DataFog to download image inputs before OCR.
41+
* - Spark
42+
- ``pip install "datafog[distributed]"``
43+
- You need the optional ``SparkService`` surface.
44+
* - Everything
45+
- ``pip install "datafog[all]"``
46+
- You are developing or deliberately want every optional surface.
47+
48+
Python Usage
49+
============
50+
51+
Use the top-level helpers for the 4.5 core path:
52+
53+
.. code-block:: python
54+
55+
import datafog
56+
57+
text = "Contact jane@example.com or call 415-555-1212"
58+
59+
scan_result = datafog.scan(text, engine="regex")
60+
print(scan_result.entities)
61+
62+
redact_result = datafog.redact(text, engine="regex")
63+
print(redact_result.redacted_text)
64+
65+
print(datafog.sanitize("Card: 4111-1111-1111-1111"))
66+
67+
Agent-oriented helpers use the same lightweight text path:
68+
69+
.. code-block:: python
70+
71+
import datafog
72+
73+
prompt = "My SSN is 123-45-6789"
74+
scan_result = datafog.scan_prompt(prompt, engine="regex")
75+
76+
if scan_result.entities:
77+
print("PII detected before sending the prompt")
78+
79+
output = "Email me at jane.doe@example.com"
80+
safe_output = datafog.filter_output(output, engine="regex")
81+
print(safe_output.redacted_text)
82+
83+
CLI Usage
84+
=========
85+
86+
The CLI core path is text-first:
87+
88+
.. code-block:: bash
89+
90+
datafog scan-text "Contact jane@example.com"
91+
datafog redact-text "Contact jane@example.com"
92+
datafog replace-text "Contact jane@example.com"
93+
datafog hash-text "Contact jane@example.com"
94+
95+
Image commands are optional. Install ``datafog[ocr]`` for local OCR and
96+
``datafog[web,ocr]`` when the CLI needs to download image inputs.
97+
98+
What 4.5 Is Not
99+
===============
100+
101+
DataFog 4.5 prepares the package for future middleware use cases, but it does
102+
not ship dedicated Sentry, OpenTelemetry, logging-framework, or cloud DLP
103+
adapters. Those integrations are future-facing work built on the same core
104+
text screening path.
105+
106+
Next Pages
107+
==========
108+
109+
* :doc:`python-sdk` documents the Python API surface.
110+
* :doc:`cli` documents command-line usage.
111+
* :doc:`optional-surfaces` documents OCR and Spark install notes.
112+
* :doc:`roadmap` explains how 4.5 leads toward later middleware work.

‎docs/index.rst‎

Lines changed: 31 additions & 89 deletions
Original file line numberDiff line numberDiff line change
@@ -2,105 +2,47 @@
22
DataFog Documentation
33
=====================
44

5-
DataFog is an open-source tool for lightweight text PII detection and
6-
anonymization. The core install focuses on fast regex-based scanning and
7-
redaction, with optional extras for NLP, OCR, and Spark-style workflows.
5+
DataFog 4.5 is a lightweight text PII screening package for Python. The
6+
primary path is a small core install, fast regex-based scanning and redaction,
7+
agent-friendly guardrail helpers, and explicit optional extras when you need
8+
NLP, OCR, Spark, or web inputs.
9+
10+
Start with :doc:`getting-started` if you want the shortest route from install
11+
to scanning text. The roadmap and historical planning pages remain available,
12+
but the live user docs are the first path for 4.5.
13+
14+
Use DataFog 4.5
15+
===============
816

917
.. toctree::
1018
:maxdepth: 2
19+
:caption: Use DataFog 4.5
1120

12-
important-concepts
13-
cli
21+
getting-started
1422
python-sdk
23+
cli
1524
optional-surfaces
16-
definitions
17-
roadmap
18-
v44-bridge-release
19-
v5-product-brief
20-
v5-compatibility-matrix
21-
v5-cut-line
22-
23-
=====================
24-
Getting Started
25-
=====================
26-
27-
Installation
28-
------------
29-
30-
Install the lightweight text screening core via pip:
31-
32-
.. code-block:: bash
33-
34-
pip install datafog
35-
36-
Optional extras such as ``nlp``, ``nlp-advanced``, ``ocr``, ``distributed``,
37-
and ``web`` are installed only when you need those surfaces.
38-
39-
---------------------
40-
CLI Usage
41-
---------------------
42-
43-
For a list of available operations, run:
44-
45-
.. code-block:: bash
46-
47-
datafog --help
48-
49-
Scan text for PII:
50-
51-
.. code-block:: bash
52-
53-
datafog scan-text "Your text here"
54-
55-
Image/OCR commands are optional. Local OCR requires ``datafog[ocr]``; URL-based
56-
image downloading requires ``datafog[web,ocr]``.
57-
58-
.. code-block:: bash
59-
60-
datafog scan-image "path/to/image.png" --operations extract
61-
62-
Scan for PII in image text:
63-
64-
.. code-block:: bash
65-
66-
datafog scan-image "path/to/image.png" --operations scan
67-
68-
For more information on optional OCR and Spark surfaces, see
69-
:doc:`optional-surfaces`.
70-
71-
---------------------
72-
Python SDK Usage
73-
---------------------
74-
75-
Scan text for PII:
76-
77-
.. code-block:: python
78-
79-
80-
import datafog
81-
82-
text = "Contact jane@example.com or call 415-555-1212"
83-
result = datafog.scan(text, engine="regex")
84-
print(result.entities)
85-
print(datafog.redact(text, engine="regex").redacted_text)
25+
important-concepts
8626

87-
Run OCR and then scan extracted text only when the OCR extra is installed:
27+
Reference
28+
=========
8829

89-
.. code-block:: python
30+
.. toctree::
31+
:maxdepth: 2
32+
:caption: Reference
9033

91-
92-
import asyncio
93-
from datafog import DataFog
34+
definitions
9435

95-
# For OCR and PII annotation
96-
ocr_client = DataFog(operations="extract,scan")
36+
Planning And History
37+
====================
9738

98-
async def run_ocr_pipeline_demo():
99-
image_url = "https://s3.amazonaws.com/thumbnails.venngage.com/template/dc377004-1c2d-49f2-8ddf-d63f11c8d9c2.png"
100-
results = await ocr_client.run_ocr_pipeline(image_urls=[image_url])
101-
print("OCR Pipeline Results:", results)
39+
The pages below document release planning, migration history, and future
40+
direction. They are useful context, but they are secondary to the live 4.5
41+
usage path above.
10242

103-
# Run the async function
104-
asyncio.run(run_ocr_pipeline_demo())
43+
.. toctree::
44+
:maxdepth: 1
45+
:caption: Planning and history
10546

106-
For detailed information on the Python SDK, see :doc:`python-sdk`.
47+
roadmap
48+
planning-history

‎docs/planning-history.rst‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
====================
2+
Planning And History
3+
====================
4+
5+
These pages and artifacts are preserved for context, but they are not the
6+
first path for using DataFog 4.5. Start with :doc:`getting-started` for live
7+
user docs.
8+
9+
Release Planning
10+
================
11+
12+
.. toctree::
13+
:maxdepth: 1
14+
15+
v44-bridge-release
16+
v5-product-brief
17+
v5-compatibility-matrix
18+
v5-cut-line
19+
20+
Audit Artifacts
21+
===============
22+
23+
Historical audit notes remain available in the repository for maintainers who
24+
need the detailed background:
25+
26+
* :download:`Reconnaissance notes <audit/00-reconnaissance.md>`
27+
* :download:`Coverage baseline <audit/01-coverage-baseline.md>`
28+
* :download:`Detection accuracy review <audit/02-detection-accuracy.md>`
29+
* :download:`Architecture review <audit/03-architecture-review.md>`
30+
* :download:`Final coverage notes <audit/06-final-coverage.md>`

0 commit comments

Comments
 (0)