Open in AthenaHQ
app.athenahq.ai/oraclePurpose
This page analyzes AI model responses to surface discrepancies between what AI says about your brand and your verified facts. It allows you to review and resolve each finding, ensuring that the AI represents your company accurately. By highlighting where AI models contradict your Knowledge Base, you can take action to correct the narrative. It also provides tools to verify your Knowledge Base claims against external sources, ensuring your internal facts remain accurate.What’s on the page
Empty / First-Run State Shown before any analysis has been run. It prompts the user to kick off their first analysis with a “Run first analysis” button and a shield icon. First Run In-Progress State Shown while the very first analysis is running. It displays a spinning loader and estimated time (5–15 minutes) with a note to track progress in the bottom-right overlay. Overview Tab The main dashboard shown after at least one completed run. It contains the brand search, metric cards, the high-impact discrepancies list, and the coverage section. Brand Search (What AI is saying) A free-text search box that synthesizes AI claims about your brand across models. Results include inline citations and an overall sentiment badge (Positive, Negative, Mixed, Neutral). Discrepancy Rate Metric Card Shows the percentage of analyzed AI responses containing at least one claim that doesn’t match your verified facts. Includes a sparkline chart showing the trend across recent analysis runs. Tooltip: “Share of analyzed AI responses with at least one claim that doesn’t match your verified facts. Lower is better.” Open Discrepancies Metric Card Shows the total count of unresolved discrepancies and a sub-label indicating how many of those are considered high-impact. High-Impact Discrepancies List A compact list of the top unresolved high-impact discrepancies.- Index: Sequential number of the discrepancy in the list (01, 02, …).
- Severity: Impact level of the discrepancy: High (critical), Medium (major), or Low (minor).
- Claim: The incorrect or conflicting claim text as stated by AI.
- Truth: The correct claim text from your Knowledge Base.
- Resolution: If resolved, shows the resolution label (A correct, B correct, Both valid, Dismissed), who resolved it, and when.
- Accuracy by Model Chart: A bar chart showing how accurately each AI model represents the brand, and how that accuracy has changed over time. Tooltip: “Click a model to drill into its accuracy over recent runs. Accuracy = 1 − (high-confidence critical wrong claims ÷ total response claims). Variants of the same model (e.g. Claude) are merged.”
- Topics AI Gets Wrong Chart: A horizontal bar chart surfacing which Knowledge Base topic categories have the most or highest-rate wrong claims in AI responses. Tooltip: “Topic categories where AI gets claims wrong. Toggle Count vs % wrong: Count surfaces topics with the most wrong claims overall; % wrong surfaces topics where the error rate is highest regardless of volume. Click a bar to see the actual wrong claims for that topic.”
- Domain: Root domain of the contradicting source, or ‘Knowledge base’ for internal KB inconsistencies.
- Contradictions: Number of contradictions found from this source.
- Arrow: Click affordance to drill into the contradictions for this source.
- Date: Date and time the run was created.
- Type: Whether the run was a ‘Response check’ (accuracy analysis) or ‘Fact verify’ (KB validation).
- Responses / claims: Number of AI responses scanned (for accuracy runs) or KB claims checked (for fact-verify runs).
- Findings: Number of discrepancies or contradictions found.
- Discrepancy rate: Percentage of responses with at least one discrepancy (accuracy runs only; ’—’ for fact-verify).
- Status: Current run status: Complete, Failed, Cancelled, or Running.
What you can do here
- Run first analysis: Opens the Check Responses dialog to configure and start the first accuracy analysis run.
- Run analysis (dropdown trigger): Opens a dropdown menu in the page header with two run options.
- Check responses (dropdown item): Opens the Check Responses dialog to scan AI responses for discrepancies.
- Verify facts (dropdown item): Opens the Verify Facts dialog to run a fact-verification analysis against external sources.
- Start (Check Responses dialog): Starts a new accuracy analysis run for the selected date range.
- Cancel (Check Responses dialog, while running): Dismisses the dialog; the run continues in the background.
- Dismiss (Check Responses dialog, while running): Closes the dialog without stopping the run.
- Review discrepancies (Check Responses dialog: post-run): Closes the dialog and opens the Oracle Review Modal showing unresolved discrepancies.
- Close (Check Responses dialog: post-run): Closes the dialog and marks the completion as acknowledged.
- Cancel run (Run Progress Overlay): Destructive. Cancels the in-flight accuracy analysis run immediately. (No confirmation dialog).
- Hide (Run Progress Overlay ×): Hides the floating progress overlay for the current session without cancelling the run.
- Cancel run (Fact-verify Run Progress Overlay): Destructive. Cancels the in-flight fact-verification run immediately.
- See all (High-Impact Discrepancies): Opens the Oracle Review Modal showing all unresolved discrepancies.
- Discrepancy row click: Opens the Oracle Review Modal focused on the clicked discrepancy.
- Verify your facts (HeroValidationCard: no run yet): Opens the Verify Facts dialog.
- Knowledge Base link (HeroValidationCard): Navigates to the
/knowledge-basepage. - View sources (HeroValidationCard): Opens the Validation Drawer for the latest fact-verification run.
- Verify facts (HeroValidationCard: run exists): Opens the Verify Facts dialog.
- View past runs → (HeroValidationCard): Opens the Run History drawer.
- Source row click (Source Rank Table): Opens the KB Source Contradictions Drawer for the selected domain.
- Previous / Next (KB Source Contradictions Drawer): Navigates between contradiction cards for the selected source.
- View response (Discrepancy Card): Opens the Response Viewer Drawer for the linked AI response.
- Open site (KB Source Contradictions Drawer): Opens the source domain in a new browser tab.
- Claim A / Claim B selection (Oracle Review Modal): Selects which claim is correct in preparation for resolving the discrepancy.
- Mark as resolved / Submit resolution (Oracle Review Modal): Saves the resolution (Claim A correct, Claim B correct, Both valid, or Dismissed) with optional notes.
- Edit (Oracle Review Modal: locked resolution): Unlocks a previously resolved discrepancy for re-evaluation.
- + Add a note (Oracle Review Modal): Expands a text area to add a resolution note.
- Dismiss (Oracle Review Modal: dropdown): Marks the discrepancy as dismissed (available in the overflow menu).
- Mark both valid (Oracle Review Modal: dropdown): Marks both claims as valid (no error) (available in the overflow menu).
- Previous / Next (Oracle Review Modal): Navigates to the previous or next discrepancy in the list.
- Start verification (Verify Facts dialog): Starts a new fact-verification run for the selected pillars.
- Cancel (Verify Facts dialog): Closes the dialog without starting a run.
- Run analysis (Run History drawer): Opens the run-type dropdown to start a new run.
- Accuracy run row click (Run History): Closes the drawer and opens the Oracle Review Modal for that run’s discrepancies.
- Fact-verify run row click (Run History): Closes the drawer and opens the Validation Drawer for that run’s sources.
- Research verdict expand (Oracle Review Modal): Expands the research verdict callout to show the summary and supporting sources.
- Source chip click (Oracle Review Modal): Opens the Response Viewer Drawer for the linked AI response.
- Source chip link (Oracle Review Modal): Opens the source URL in a new browser tab.
Data shown
- AI responses: The text collected across supported AI models (ChatGPT, Claude, Gemini, Perplexity, Copilot, Grok, DeepSeek, AI Overview, AI Mode) for your brand.
- Verified facts: Claims stored in your Knowledge Base pillars.
- External sources: Web pages found by the fact-verification workflow that contradict your Knowledge Base claims.
- Historical accuracy analysis runs: Details and discrepancy counts for past analyses.
- Hallucination rate trend: The discrepancy rate tracking across recent runs.
Common workflows
Run first accuracy analysis- Navigate to the Oracle page.
- Click the “Run first analysis” button.
- Select a date range in the Check Responses dialog (or use the default 7 days).
- Click “Start”.
- Monitor progress in the floating bottom-right overlay (usually takes 5–15 minutes).
- When complete, the dialog reopens showing the number of discrepancies found.
- Click “Review discrepancies” to open the review modal.
- From the Overview tab, click a discrepancy row or “See all”.
- In the Oracle Review Modal, read Claim A and Claim B.
- Review the “Why this is flagged” explanation and any research verdict.
- Select the correct claim (A or B), or choose “Both valid” / “Dismiss” from the overflow menu if neither is wrong.
- Optionally add a note explaining the decision.
- The resolution is saved automatically when a claim is selected and confirmed.
- Click “Run analysis” in the page header.
- Select “Verify facts” from the dropdown.
- In the Verify Facts dialog, select one or more Knowledge Base pillars.
- Click “Start verification”.
- Monitor progress in the floating bottom-right overlay.
- When complete, click “View sources” on the Fact Verification card to see contradicting sources.
- Open the Validation Drawer via “View sources” on the Fact Verification card.
- Click a source domain row in the ranked table.
- Browse contradiction cards in the KB Source Contradictions Drawer using Previous/Next.
- Click “View response” on a card to read the full AI response that contained the contradicting claim.
- On the Overview tab, locate the “What AI is saying about your brand” search box.
- Type a topic (e.g. “pricing”, “competitors”, “founders”) and press Enter.
- Read the AI-synthesized summary with inline citation pills.
- Click a citation pill to open the full AI response in the Response Viewer Drawer.
Empty, loading, and error states
- Empty: A “No analysis run yet” state with a shield icon and a “Run first analysis” button is shown when no completed run exists and no run is in progress.
- Loading: A centered spinning loader is shown while the latest run status is being fetched on initial page load.
- Error: If the subscription/plan check fails, an error screen with a “Retry” button is shown. If the feature is not included in the current plan, a lock screen is shown instead of the page content.
Linked from / links to
Linked from:- Main sidebar navigation (Oracle / Recommendations item).
- Email or notification deep-links with
?analysisRunId=,?reviewRunId=, or?validationRunId=query params. - Legacy URLs with parameters like
?tab=runs,?tab=kb-analysis,?tab=fact-bank,?tab=search,?tab=facts,?tab=topics(these are automatically redirected to the correct view).
/knowledge-base: Opens the Knowledge Base./knowledge-base/review: Opens Knowledge Base claims awaiting review./content: Opens the Content action center./content?contentId=<id>: Opens a specific drafted content piece./content?published=in-progress: Opens content drafts.