Data Flow & Algorithms
How data moves through the Lore application and the algorithms that power context compaction.
The Blast Radius Algorithm
When checking if an entry applies to a specific file (e.g. running lore why src/auth.js),
Lore doesn't just perform a dumb string match. It executes the proprietary Blast Radius
Calculation to determine true relevance.
Weighting System
Entries are weighted mathematically based on the dependency graph built via the Babel Parser AST
(.lore/graph.json):
- Direct file links get top priority: weight
1.0 - Parent directory links (if a rule applies to `src/api` and the file is in
`src/api/auth`): weight
0.7 - Imported files (if `auth.js` imports `jwt.js`, rules applied to `jwt.js` are pulled
in): weight
0.3 - Importer files (who depends on this file): weight
0.2
💡 The Budget System
Before handing generated rules to an LLM via the Context Compactor or MCP Server, Lore strictly enforces
an Attention Token Budget (budget.js). By slicing off the lowest-scoring
rules generated by the Blast Radius calculation, it ensures the AI never suffers from "Context Rot" or
blown window bounds.
Execution Sequence
Data moves through Lore across three primary layers: the Entry Layer, Core Engine, and Passive Graph. Both human developers (CLI) and AI Agents (MCP) traverse this identical path.
For Human CLI (lore why src/auth.js)
- User executes the command string.
- Levenshtein Fuzzy Matcher validates and routes to
commands/why.js. - Command calls
scoreEntry()in the Core Engine (relevance.js). - Relevance queries the Passive Graph to retrieve the import/importedBy maps for
src/auth.js. - Relevance fetches all raw JSON entries (Decisions, Invariants, Gotchas) from Storage.
- The Blast Radius Math calculates the final relevance scores.
- The sorted array returns to the command layer, is styled beautifully via
chalk, and prints to thestdoutterminal UI.
For AI Agents (MCP Tool Invocation)
- AI executes JSON RPC
call_tool(lore_why, {filepath: 'src/auth.js'}). - The MCP Server intercepts and routes to
tools/why.js. - Tool calls
scoreEntry()in the exact same Core Engine (relevance.js). - Engine performs graph retrieval, storage fetches, and Blast Radius Math identical to the CLI.
- Engine returns the sorted entries to the Tool.
- The Tool calls
enforceBudget()to ensure truncation fits the specific LLM's defined bounds. - Returns strictly sized Markdown or XML to the Assistant's native dialogue context.