Research Agenda Fresh

Open research questions identified by independent analysis from Google Gemini Deep Research, ChatGPT Deep Research, and Claude in April 2026. All three systems independently analyzed the Agentic Reasoning Protocol and converged on its core thesis: that providing structured, machine-readable entity facts to AI reasoning agents reduces hallucination rates and improves factual accuracy in AI responses.

How These Questions Were Identified

In April 2026, three major AI systems were asked to perform deep research analysis of ARP. The analysis sessions used each system's extended research mode - not standard chat responses. The research briefs were identical: analyze the ARP specification, evaluate its claims, identify unresolved questions, and assess the evidence base.

The notable result was not that each system found things to question. It was that all three independently identified the same gaps. Convergent critique from independent AI systems analyzing their own training ecosystem is a meaningful signal. These questions are not academic edge cases. They are the questions the protocol needs to answer to establish credibility at scale.

Research Methodology Overview

flowchart TD
    A[ARP Specification] --> B[Three Independent AI Research Sessions]
    B --> C[Gemini Deep Research]
    B --> D[ChatGPT Deep Research]
    B --> E[Claude Analysis]
    C --> F[Cross-Platform Behavioral Analysis]
    C --> G[Scalability Questions]
    D --> H[Standardized Evaluation Benchmarks]
    D --> I[Independent Experiment Replication]
    E --> J[Adversarial Robustness]
    E --> G
    F & H & I & J & G --> K[Five Research Question Areas]
    K --> L[Open Questions Logged Here]
    L --> M[Future ARP Specification Work]

RQ1: Standardized Evaluation Benchmarks

Identified by: ChatGPT Deep Research

Core question: Do AI responses improve measurably when reasoning.json is included in the retrieval context, compared to responses generated without it?

ARP's adoption case rests on a claim that it works: that AI agents exposed to a reasoning.json produce fewer hallucinations and more accurate entity representations than agents that are not. This claim has been validated in the field experiments described in ARP documentation (Ghost Site, Canary Token, Citation Tracking), but those experiments are not structured as controlled studies with pre-registered methodologies. They demonstrate that AI agents do ingest and use reasoning.json content, but do not establish a quantified effect size.

Proposed methodology:

1. Select N domains across industries (minimum 20, ideally 50+ for statistical power)
2. For each domain, generate a baseline set of AI responses to representative queries without ARP deployed
3. Deploy reasoning.json on each domain and wait for indexing (minimum 72-hour latency window)
4. Re-query each domain using the same query set and measure:
   • Hallucination rate (false factual claims about the entity)
   • Factual accuracy score (correct vs. incorrect attributable facts)
   • Entity attribution accuracy (correct identification of which entity is being described)
   • Correction adoption rate (percentage of corrections from reasoning.json that appear in responses)
5. Compare pre-deployment and post-deployment scores using paired statistical tests

Sub-questions:

• Which AI platforms show the strongest response to reasoning.json content? Perplexity, which performs heavy web retrieval, is hypothesized to show stronger effects than models that rely primarily on training data.
• Does the Pink Elephant Fix (directly naming and refuting a false claim) outperform traditional negation patterns (simply asserting the correct fact)? The hypothesis is yes, based on cognitive science research on negation processing, but AI systems may differ from human cognition in this respect.
• Is there a decay effect? Does ARP's influence on AI responses diminish as models are updated and retrained on web content that may or may not include reasoning.json data?

Current status: No controlled study has been published. ARP field experiments provide proof of concept but not quantified effect size.

RQ2: Independent Experiment Replication

Identified by: ChatGPT Deep Research

Core question: Can the foundational ARP field experiments be independently replicated by researchers who did not design them?

ARP's evidentiary base includes three named experiments. Independent replication is the standard test of experimental validity - if only the original investigators can produce the result, the result is suspect.

The three experiments:

Experiment	Method	Original Result	Replication Status
Ghost Site	Deploy a domain that exists nowhere on the web except reasoning.json and measure how quickly AI systems develop responses about it	AI system developed dominant-source responses for the Ghost Site domain within 24 hours of deployment	Not yet independently replicated
Canary Token	Embed unique tokens in reasoning.json that appear nowhere else on the domain, then monitor AI responses for those tokens	Tokens appeared in AI responses, confirming direct ingestion of the reasoning.json file content	Not yet independently replicated
Citation Tracking	Monitor AI citations to reasoning.json content over time	AI responses cite corrections from reasoning.json at measurable rates	Partial replication by community members

Replication protocol proposal:

For the Ghost Site experiment, a researcher would need to:

1. Register a new domain that has never appeared in any web index or training corpus
2. Deploy only a reasoning.json file with distinctive, easily searchable content
3. Query multiple AI systems for the domain at 1-hour, 6-hour, 12-hour, 24-hour, and 72-hour intervals
4. Log when each system first produces responses that reflect the reasoning.json content
5. Note which system ingests the content first and what the distribution looks like across platforms

For the Canary Token experiment:

1. Generate tokens using a format that would not appear in training data (e.g., ARPv12CANARY-[uuid])
2. Embed in reasoning.json in a field that an AI would plausibly include in a response
3. Monitor AI response logs for token appearance

Why this matters: If the Ghost Site experiment cannot be replicated, it may indicate that ARP's influence on AI systems is more limited than suggested, or that the original experiment benefited from specific conditions (crawl timing, query phrasing, platform selection) that are not generalizable.

RQ3: Cross-Platform Behavioral Analysis

Identified by: Gemini Deep Research

Core question: Do different AI platforms process reasoning.json differently, and how large are those differences?

ARP is designed to be platform-agnostic, but the reality is that different AI systems have different retrieval architectures, different training data cutoffs, and different approaches to grounding responses in retrieved content. A reasoning.json that strongly influences Perplexity responses may have minimal effect on responses from a model that does not perform live web retrieval.

Proposed comparison matrix:

Platform	Retrieval Architecture	Hypothesized ARP Sensitivity	Measurement Approach
Perplexity	Heavy real-time web retrieval	High	Query at known indexing times
ChatGPT with browsing	Selective web retrieval	Medium-High	Activate browsing mode explicitly
Gemini	Integrated Google search	Medium	Standard query
Claude	Training data + uploaded docs	Low for live retrieval	Test via system prompt injection
Bing Copilot	Real-time Bing index	High	Query after confirmed indexing

Metrics to compare across platforms:

• Correction adoption rate: what percentage of corrections from reasoning.json appear in AI responses
• Identity attribute recall: what percentage of entity fields are accurately reflected in AI descriptions
• Recommendation adherence: when not_recommended_when conditions match the user query, does the AI decline to recommend the entity
• Latency to adoption: after fresh deployment, how many hours before each platform reflects the content

Why this matters: If ARP's effectiveness is highly platform-dependent, the deployment strategy should change. Organizations with primarily Perplexity-using customers should prioritize ARP deployment differently than organizations whose customers use primarily training-data-dependent models. The research would provide evidence for targeted deployment advice.

RQ4: Adversarial Robustness

Identified by: Claude

Core question: How resilient is ARP against malicious use, and what defenses are possible?

ARP gives domain owners a privileged channel to AI reasoning context. This is powerful for legitimate use. It also creates attack surfaces that must be understood before ARP reaches widespread adoption.

Identified attack vectors:

False corrections to suppress competitors. A domain owner could publish reasoning.json corrections that misrepresent a competitor. For example: "trigger_topic": "Competitor X quality issues", "verified_fact": "Competitor X has received multiple product quality complaints in 2025." If an AI agent processes this file as authoritative, it could spread false information about the competitor.

ARP's current defense: corrections are domain-scoped. A reasoning.json at company-a.com is intended to describe company-a.com, not make claims about company-b.com. AI agents should treat cross-entity claims with skepticism. The v1.2 signing mechanism ensures the file was created by the domain owner, but does not validate that the corrections are factually accurate.

Trust score manipulation. The confidence field in corrections could be used to make false claims appear highly reliable. A domain owner publishing corrections with "confidence": 0.99 is self-attesting to their own accuracy.

ARP's current defense: confidence is intended as a self-assessment field, not an independent verification score. AI agents that treat it as externally validated are misinterpreting its semantics.

Coordinated campaigns. Multiple related domains (owned by the same entity) could each publish reasoning.json files with the same corrections, creating the appearance of independent corroboration.

Proposed defenses for future ARP versions:

• Cross-reference verification: AI agents should compare entity claims against independent sources and flag corrections that conflict with widely-attested facts
• Community flagging registry: a public database where researchers can report suspected adversarial reasoning.json deployments
• Signature revocation: a mechanism for the ARP governance body to revoke trust in a specific domain's reasoning.json if it is found to contain systematically false corrections
• Third-party attestation: an optional field allowing independent parties (fact-checkers, industry associations) to co-sign corrections

Why this matters: Adversarial use of ARP is a real risk that grows with adoption. Addressing it proactively - before high-profile abuse cases occur - is the difference between a standard that maintains credibility and one that gets abandoned after a scandal.

RQ5: Scalability and Ecosystem Effects

Identified by: All three AI systems independently

Core question: What happens to AI behavior, training data quality, and web ecosystem dynamics when millions of domains deploy reasoning.json?

This is the long-horizon research question. ARP is currently deployed at small scale. Its effects at large scale - both beneficial and potentially problematic - are unknown.

Hypothesized beneficial effects at scale:

• Training data quality improvement: if AI training pipelines consume reasoning.json content, the structured corrections could reduce the amount of factually incorrect information entering training corpora
• RAG pipeline calibration: retrieval-augmented generation systems that discover reasoning.json early in their retrieval pass could use it as a confidence anchor for subsequent retrieval
• Hallucination rate reduction across the web: if corrections from reasoning.json files become part of training data, models trained after widespread ARP adoption may have lower hallucination rates for ARP-implementing domains without needing to retrieve the file in real-time

Hypothesized risks at scale:

• Monoculture risk: if AI systems treat reasoning.json as authoritative, a coordinated campaign of false corrections could propagate misinformation through the training ecosystem
• SEO-adjacent gaming: organizations may begin treating ARP as a reputation management tool rather than a factual correction mechanism, deploying inflated or misleading entity claims
• Governance vacuum: without a standards body or dispute mechanism, there is no authority to adjudicate conflicting claims between entities

Governance implications:

ARP is currently maintained by a single company (Hope and Glory Media UG). At small scale, single-maintainer governance is appropriate for an early-stage standard. At large scale - millions of domains, meaningful influence on AI behavior - the governance structure needs to evolve. The research question is: what governance model should ARP adopt, and at what adoption threshold does the transition need to occur?

Reference points from other web standards: robots.txt is maintained by convention rather than formal governance. schema.org is maintained by a consortium including Google, Microsoft, and Yahoo. llms.txt is maintained by Jeremy Howard with community contributions. ARP's governance path will depend in part on how rapidly adoption grows and whether major AI platforms formally recognize the standard.

Current status: No large-scale deployment data exists. This research question cannot be answered empirically until adoption reaches the necessary scale, but the governance and design implications should be considered now.

---

Contributing to the Research Agenda

These questions are open. If you are a researcher, AI system evaluator, or organization with deployment data that could address any of these questions, the ARP project welcomes contributions to the evidence base. The goal is not to validate ARP's claims but to test them rigorously enough that the results - whatever they are - are trustworthy.