Readwise Recommendation Engine

A Node.js system that surfaces relevant articles from your Readwise library using semantic search. It scores your "Later" queue (and optionally a tagged collection) against a personalized interest profile, generates a daily HTML recommendations page, and optionally publishes it to GitHub Pages or your own server.

No LLM required at runtime. Queries are static in your interest profile.

Prerequisites

Node.js 18+
Readwise account with API token
Readwise CLI — required for the scoring engine

Quick Start

1. Clone and Install

git clone https://github.com/derekvan/Readwise-recommender.git
cd Readwise-recommender
npm install

2. Install and Authenticate the Readwise CLI

This is what the pipeline authenticates with. Every Readwise call in the scoring, analytics, and generation steps shells out to this CLI, which stores its own credential in ~/.readwise-cli.json.

npm install -g @readwise/cli
readwise login                          # OAuth, recommended — refreshes itself
# or, for scripts/CI:
readwise login-with-token your-readwise-token

That file holds a live access and refresh token, so lock it down — the CLI writes it without an explicit mode, and a fresh login recreates it world-readable:

chmod 600 ~/.readwise-cli.json

Verify it works:

readwise reader-search-documents --query "productivity" --location-in later --limit 5 --json

3. Store Your Readwise API Token (optional)

Only needed for the Readwise MCP server and the legacy scoring engines — the default readwise/hybrid pipeline never reads it. Keep it in the macOS keychain rather than in a dotfile, a crontab, or a launchd plist:

security add-generic-password -a "$USER" -s readwise-token -w

Then derive the env var in ~/.zshrc so there is only ever one copy to rotate:

export READWISE_TOKEN="$(security find-generic-password -a "$USER" -s readwise-token -w)"

Code reads it through utils/readwise_token.js, which falls back to the keychain when the env var is unset.

4. Create Your Interest Profile

Your interest profile defines the topics you care about. The scoring engine uses this to query Readwise's hybrid search (full-text + semantic) and rank results with Reciprocal Rank Fusion.

See Interest Profile for the full format.

Option A: Use Claude Code (Recommended)

Claude Code can analyze your Readwise highlights and generate a high-quality semantic interest profile:

Step 1: Build your reader persona

Use the build-persona skill to analyze my reading history and build a profile

Step 2: Generate the semantic profile

Based on my reader persona, create data/interest_profile_semantic.json with 20-25 clusters.
Each cluster should have keyword, nl, question, hyde, and highlight query types.

Option B: Create Manually

Create data/interest_profile_semantic.json — see Interest Profile for the schema.

5. Configure Settings

cp utils/settings.example.json utils/settings.json

Edit utils/settings.json and set the engine:

{
  "scoring": {
    "engine": "readwise"
  }
}

See Configuration for all options.

6. Run

node daily_recs.js

This scores your entire library against your interest profile (takes ~2-5 minutes on first run, faster on subsequent runs), generates an HTML file at output/recommendations/YYYY-MM-DD.html, and uploads it based on your settings.

Daily Usage

node daily_recs.js

What happens each run:

Scores your library — Runs all interest profile queries against Readwise's hybrid search. Uses Reciprocal Rank Fusion to combine results and rank your library by relevance.
Updates analytics — Compares previous recommendations against today's results to detect which articles you archived or removed.
Change gate — If none of your previously-recommended articles have moved (archived, deleted, or tag removed), the run exits early:
```
⏸️  No changes detected — previously recommended docs are still in your library.
   Archive a recommendation to surface new ones.
```
This prevents surfacing the same articles repeatedly. Archive at least one to generate new recommendations.
Generates HTML — Selects top-scoring articles across multiple batches, fetches summaries, attaches pregenerated pitches, and builds a self-contained HTML page.
Uploads — Copies to GitHub Pages, uploads via SCP, or saves locally depending on your settings.

The generated page includes:

Multiple recommendation batches with a "Get More Recommendations" button
Score, matched themes, reading time, and a personalized "why this matters" pitch per article
Interactive actions (archive, remove tag, delete) that call the Readwise API directly
Mobile-friendly, bookmarkable design

How Scoring Works

The Readwise engine uses the Readwise CLI's hybrid full-text + semantic search to discover relevant documents, then applies Reciprocal Rank Fusion (RRF) to combine results across all queries.

Per-Cluster Scoring

For each interest cluster, the engine runs all queries in parallel batches of 5. For each query result set, documents are scored by their rank using RRF:

rrf_score = queryWeight × (1 / (60 + rank + 1)) × (1 + 0.1 × chunks)

60 is the standard RRF constant (reduces sensitivity to rank)
chunks is a boost for documents that appear in multiple search result pages (indicates higher relevance)
Scores are summed across all queries to get each cluster's total RRF score for each document

Final Score

Once all clusters are scored, documents receive a final score using a hybrid formula with diminishing returns for overlapping clusters:

Sort a document's cluster scores, highest first
Apply 1/√N decay to the Nth matched cluster (reduces the benefit of matching many clusters weakly)
Final score = (strongest_cluster_score × strongestThemeMultiplier) + sum(decayed_other_clusters) × rawScoreMultiplier, capped at 10

Why this formula? It prioritizes documents with a strong single-theme match over documents that weakly match many themes. A deeply relevant article on your strongest interest will beat a superficially broad article every time.

Interest Profile

The scoring engine uses data/interest_profile_semantic.json (v2.0). If that file doesn't exist, it falls back to data/interest_profile.json.

Schema

{
  "version": "2.0",
  "metadata": {
    "created_at": "2026-01-01T00:00:00.000Z",
    "notes": "Optional notes about this profile"
  },
  "clusters": [
    {
      "id": "cluster_001",
      "theme": "Decision Making & Cognitive Frameworks",
      "weight": 0.07,
      "keywords": ["decision", "intuition", "cognitive biases", "judgment"],
      "queries": [
        "intuition versus rational thinking",
        "how to make better decisions under uncertainty",
        "When we face an important decision, the pull toward certainty can itself become a bias.",
        "What is the relationship between fear and poor decision-making?",
        "the interplay of intuition and reason in real-world choices"
      ]
    }
  ]
}

Query Types

Mix these in each cluster's queries array for broad coverage:

Type	Description	Example
keyword	Exact domain terms	`"GTD getting things done methodology"`
nl	Natural language description	`"how vulnerability creates intimacy"`
question	A question the ideal article would answer	`"What keeps desire alive in long-term relationships?"`
hyde	A paragraph that sounds like the ideal article	`"Intimacy requires two separate people who choose to come together..."`
highlight	A passage you've highlighted that represents this interest	Copy a memorable highlight from Readwise

Tips:

5-10 queries per cluster is a good range — more queries improve coverage but increase run time
weight (0.0–1.0) affects how strongly this cluster influences the final score
The quality of your queries directly determines recommendation quality — refine them over time
Queries can optionally be objects with a weight field: {"query": "...", "weight": 1.5} to boost specific queries

Rebuilding Your Profile

After accumulating new highlights, ask Claude Code to refine your profile:

My reading has evolved. Update data/interest_profile_semantic.json based on
my recent Readwise highlights. Focus especially on [topics you want to add/remove].

Analytics Feedback Loop

After each scoring run, the engine automatically tracks which recommendations you acted on by comparing yesterday's recommendation log against today's scored results. If a previously-recommended article is no longer in your library, it was archived, deleted, or had its tag removed.

Analytics are saved to data/recommendation_analytics.json and a human-readable report is prepended to data/analytics_report.md after each run. The report shows:

Cluster hit rates — which themes are leading to articles you act on
Score bucket effectiveness — whether high-scored (8-10), mid-scored (5-8), or low-scored (1-5) articles get the most engagement
Underperforming clusters — themes with <20% hit rate (candidates for query refinement)
Suggested weight adjustments — clusters to promote or demote

Use this report to refine your interest profile over time.

Configuration

Copy utils/settings.example.json to utils/settings.json and edit as needed. Settings are deep-merged with defaults, so you only need to specify what you want to change.

See CONFIGURATION.md for the full reference. Key settings for the Readwise engine:

{
  "scoring": {
    "engine": "readwise",
    "strongestThemeMultiplier": 2,
    "rawScoreMultiplier": 6,
    "maxMatchedClusters": 5
  },
  "laterBucket": {
    "label": "Top Picks",
    "emoji": "✨",
    "cooldownMonths": 2,
    "count": 5
  },
  "tagBucket": {
    "enabled": false
  },
  "batchCount": 3,
  "excludedCategories": ["pdf", "epub"],
  "upload": {
    "method": "local"
  }
}

Tag bucket: Enable a second recommendation section drawn from a specific Readwise tag (e.g., a reading list for a project or research topic):

{
  "tagBucket": {
    "enabled": true,
    "tag": "my-project-2026",
    "label": "Project Readings",
    "emoji": "📚",
    "cooldownMonths": 6,
    "count": 3
  }
}

Note: utils/settings.json is gitignored — your settings stay private.

Hosting Options

Option 1: Local File (Default)

{ "upload": { "method": "local" } }

The run prints the local file path. Open it directly:

open output/recommendations/$(date +%Y-%m-%d).html

Pros: Zero setup, complete privacy, works offline.

Option 2: GitHub Pages

Host on GitHub Pages for a consistent, bookmarkable URL.

Setup:

Create a GitHub repository (e.g., my-recommendations) and enable GitHub Pages from the docs/ folder in Settings → Pages.

Configure settings:

{
  "upload": {
    "method": "github-pages",
    "pagesRepoPath": "/absolute/path/to/my-recommendations",
    "pagesUrl": "https://yourusername.github.io/my-recommendations/"
  }
}

node daily_recs.js automatically copies the HTML to docs/index.html, commits, and pushes.

Pros: Free hosting, bookmarkable URL, archive buttons work from any browser. Note: The repository must be public — article titles will be visible.

Option 3: Personal Web Server (SCP)

Configure SSH credentials in utils/config.json (see utils/config.example.json):

{
  "ssh": {
    "host": "your.server.com",
    "user": "username",
    "path": "/var/www/html/recommendations/",
    "keyPath": "~/.ssh/id_rsa"
  }
}

Then set "upload.method": "scp" in settings.json. Requires key-based SSH authentication (no password prompts).

Automating with Cron

Setup

Find your node path: which node
Edit crontab: crontab -e
Add a daily job (replace paths with your own):

0 6 * * * cd /Users/yourusername/Code/Readwise-recommender && /usr/local/bin/node daily_recs.js >> /tmp/daily_recs.log 2>&1

Verify: crontab -l
Check logs: tail -f /tmp/daily_recs.log

Common issues:

Cron runs with a minimal environment — it needs readwise on PATH and a valid ~/.readwise-cli.json. It does not need READWISE_TOKEN; never paste the token into your crontab, where crontab -l will leak it and rotations will strand it
Use the full path from which node — cron may not find nvm/asdf-managed node
Test the exact cron command manually before relying on it, with the same bare environment cron will use: env -i PATH=/opt/homebrew/bin:/usr/bin:/bin HOME=$HOME /opt/homebrew/bin/node daily_recs.js
Step 0 verifies CLI auth before the ~90-minute scoring run, so a broken credential fails in seconds with a clear message rather than silently scoring zero documents

Alternative: launchd (macOS)

For more reliable scheduling on macOS, create ~/Library/LaunchAgents/com.user.readwise-recs.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.user.readwise-recs</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/node</string>
        <string>/Users/yourusername/Code/Readwise-recommender/daily_recs.js</string>
    </array>
    <key>StartCalendarInterval</key>
    <dict>
        <key>Hour</key>
        <integer>6</integer>
        <key>Minute</key>
        <integer>0</integer>
    </dict>
    <key>StandardOutPath</key>
    <string>/tmp/readwise-recs.log</string>
    <key>StandardErrorPath</key>
    <string>/tmp/readwise-recs-error.log</string>
</dict>
</plist>

launchctl load ~/Library/LaunchAgents/com.user.readwise-recs.plist
launchctl list | grep readwise-recs

Project Structure

Readwise-recommender/
├── data/                                    # Runtime data (gitignored)
│   ├── interest_profile_semantic.json       # Your semantic interest profile (v2.0)
│   ├── interest_profile.json                # Fallback keyword profile (v1.1)
│   ├── documents_later_scored.json          # Scored "later" documents cache
│   ├── recommendation_log.json              # History of recommended items
│   ├── recommendation_analytics.json        # Analytics data (cluster hit rates, etc.)
│   ├── analytics_report.md                  # Human-readable analytics report
│   ├── pitches_pregenerated.json            # Pregenerated "why this matters" pitches
│   └── qmd-docs/                            # Markdown exports for legacy QMD engine
│
├── output/
│   └── recommendations/                     # Daily HTML files
│       └── YYYY-MM-DD.html
│
├── utils/
│   ├── readwise_scoring_engine.js           # Readwise CLI semantic engine (active)
│   ├── generate_recommendations.js          # HTML generation and bucket selection
│   ├── update_analytics.js                  # Analytics data collection
│   ├── analytics_report.js                  # Analytics report generation
│   ├── generate_pitches.js                  # Pregenerate document pitches
│   ├── settings.js                          # Settings loader (deep-merges with defaults)
│   ├── settings.example.json                # Configuration template
│   ├── upload_github_pages.js               # GitHub Pages upload
│   ├── upload_recommendations.js            # SCP upload
│   ├── scoring_engine.js                    # Legacy keyword engine
│   ├── qmd_scoring_engine.js                # Legacy QMD engine
│   ├── scoring_engine_selector.js           # Engine routing
│   ├── build_scored_cache.js                # Legacy cache builder
│   ├── fetch_later_incremental.js           # Legacy: fetch via REST API
│   ├── export_docs_to_qmd.js                # Legacy: export for QMD
│   └── update_cache_from_yesterday.js       # Legacy: incremental cache update
│
├── .agents/skills/                          # Claude Code agent skills
│   ├── build-persona.md                     # Build reader persona from highlights
│   ├── reader-recap.md                      # Weekly reading recap
│   ├── triage.md                            # Inbox triage
│   └── ...                                  # Other skills
│
├── daily_recs.js                            # Main entry point
├── CONFIGURATION.md                         # Full configuration reference
└── README.md                                # This file

Troubleshooting

"No changes detected" every day? Archive or delete at least one article from your recommendations. The change gate requires you to act on at least one previously-recommended item before generating new recommendations.

No recommendations matching your interests?

Review data/analytics_report.md — it shows cluster hit rates and underperforming themes
Add more specific queries to underperforming clusters in your interest profile
Try different query types (hyde paragraphs and highlight excerpts often perform better than simple keywords)

Readwise CLI not found?

npm install -g @readwise/cli
readwise login

If readwise isn't on your PATH after install, check npm bin -g is on your PATH.

Runs fail immediately at Step 0? That's the auth preflight. Confirm the credential exists and is valid:

ls -l ~/.readwise-cli.json && readwise reader-list-tags --json | head -c 80

Re-run readwise login if it reports "Not logged in". Note that rotating your Readwise access token does not invalidate an OAuth CLI login, and vice versa — they are separate credentials.

Scores are all low?

Check your query quality — vague queries return broad results with low RRF scores
Check data/analytics_report.md for cluster effectiveness
Rebuild your interest profile with more specific, personal queries

Archive buttons not working in browser? Archive buttons call the Readwise API using a token stored in browser localStorage. Open the browser console and run:

localStorage.setItem('readwise_token', 'your-token-here')

This is per-origin — GitHub Pages users need to set it once per browser.

Run is slow? The Readwise engine runs all cluster queries in parallel batches of 5. Run time scales with total query count across all clusters. Reducing queries per cluster or reducing cluster count will speed it up. Typical time: 1-3 minutes for 22 clusters with 6-10 queries each.

Legacy Scoring Engines

Two additional engines are available for reference or migration, selected via "scoring.engine" in settings.

Keyword Engine (`"engine": "keyword"`)

Scores documents via substring keyword matching against title, summary, and article body. Requires a manual data-fetch pipeline:

node utils/fetch_later_incremental.js   # fetch full HTML content
node utils/merge_chunks.js              # strip HTML → plain text
node utils/build_scored_cache.js        # score and cache

Then run node daily_recs.js. Cache refresh is incremental by default.

QMD Engine (`"engine": "qmd"`)

Scores documents using BM25 search against a local qmd index of full markdown documents. Requires qmd installed and configured, plus an initial export:

node utils/export_docs_to_qmd.js        # export Later docs as markdown, index via qmd
node utils/build_scored_cache.js        # score via BM25 search

When the QMD engine is set, daily_recs.js runs an incremental export automatically at startup.

Questions or feedback? Open an issue at https://github.com/derekvan/Readwise-recommender/issues

Name		Name	Last commit message	Last commit date
docs		docs
tests		tests
utils		utils
.gitignore		.gitignore
CLAUDE.md		CLAUDE.md
CONFIGURATION.md		CONFIGURATION.md
Claude.pdf		Claude.pdf
PITCH_REFRESH.md		PITCH_REFRESH.md
README.md		README.md
daily_recs.js		daily_recs.js
refactor-plan.md		refactor-plan.md

Navigation Menu

Repository files navigation

Readwise Recommendation Engine

Prerequisites

Quick Start

1. Clone and Install

2. Install and Authenticate the Readwise CLI

3. Store Your Readwise API Token (optional)

4. Create Your Interest Profile

Option A: Use Claude Code (Recommended)

Option B: Create Manually

5. Configure Settings

6. Run

Daily Usage

How Scoring Works

Per-Cluster Scoring

Final Score

Interest Profile

Schema

Query Types

Rebuilding Your Profile

Analytics Feedback Loop

Configuration

Hosting Options

Option 1: Local File (Default)

Option 2: GitHub Pages

Option 3: Personal Web Server (SCP)

Automating with Cron

Setup

Alternative: launchd (macOS)

Project Structure

Troubleshooting

Legacy Scoring Engines

Keyword Engine (`"engine": "keyword"`)

QMD Engine (`"engine": "qmd"`)

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

Readwise Recommendation Engine

Prerequisites

Quick Start

1. Clone and Install

2. Install and Authenticate the Readwise CLI

3. Store Your Readwise API Token (optional)

4. Create Your Interest Profile

Option A: Use Claude Code (Recommended)

Option B: Create Manually

5. Configure Settings

6. Run

Daily Usage

How Scoring Works

Per-Cluster Scoring

Final Score

Interest Profile

Schema

Query Types

Rebuilding Your Profile

Analytics Feedback Loop

Configuration

Hosting Options

Option 1: Local File (Default)

Option 2: GitHub Pages

Option 3: Personal Web Server (SCP)

Automating with Cron

Setup

Alternative: launchd (macOS)

Project Structure

Troubleshooting

Legacy Scoring Engines

Keyword Engine ("engine": "keyword")

QMD Engine ("engine": "qmd")

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

Keyword Engine (`"engine": "keyword"`)

QMD Engine (`"engine": "qmd"`)