Jay Fontanini Advisory Services
Proven Operator
Case Study

When AI Critiques AI: A Multi-Model Approach to Pre-Launch Content Strategy

How assigning distinct roles to different AI models (creator vs. critic) produced content optimized for discoverability in an emerging distribution channel.

The Discovery: Content that AI assistants will cite requires a different creation process than content built for search engines.

The Context

A pre-launch marketplace was preparing to enter a crowded market with a privacy-focused differentiation story. The founders understood something that most competitors hadn't yet grasped: the way consumers research purchases is shifting.

Traditional SEO still matters, but a growing percentage of consumers, particularly for research-heavy decisions, are starting with AI assistants rather than search engines. They're asking ChatGPT, Claude, or Perplexity questions like "What are my options for X?" or "What's the most privacy-friendly way to do Y?"

The strategic question: How do you create content that AI assistants will find, trust, and cite when answering user queries?

This is fundamentally different from SEO. Search engines rank pages. AI assistants synthesize information from multiple sources and present answers. Getting "cited" by an AI requires different content architecture than getting "ranked" by Google.

"We're not optimizing for algorithms. We're optimizing for AI comprehension: content structured so that language models can understand, validate, and reference it when users ask relevant questions."

The Hypothesis

Jay had been tracking early research on what practitioners were calling "Agentic Engine Optimization" (AEO): the emerging discipline of creating content optimized for AI assistant discoverability rather than traditional search ranking.

The core insight: AI assistants draw from two sources when generating responses:

  1. Training data (model memory): What the model learned during training. Harder to influence; requires content to exist in high-quality, widely-cited sources before the training cutoff.
  2. Real-time retrieval (RAG): What the model can access through web search or connected tools. More immediate; requires content that's structured for AI comprehension and discovery.

The hypothesis: If content were structured specifically for AI comprehension (with dense definitional intros, question-shaped headings, comparison tables, and entity anchors) it would be more likely to surface in AI-generated responses.

But how do you validate that hypothesis before going live?

The Methodology: Human-Orchestrated Creator-Critic Loop

The breakthrough came from a simple observation: different AI models have different strengths. But the insight that made it work was recognizing that the human remains the orchestrator. The models don't collaborate with each other. The human extracts output from one, evaluates it, and delivers it to the other with specific instructions.

This distinction matters. Jay didn't spin up two models and let them work. He had already invested significant effort before any model was involved:

The models worked from this foundation. Claude wasn't generating content from nothing; it was synthesizing from a project rich with context. Perplexity wasn't critiquing in a vacuum; it was evaluating against specific queries Jay had defined based on his market research.

"The models are accelerators, not originators. The human provides vision, context, judgment, and the connective tissue between tools. Without that orchestration layer, you just have two separate conversations going nowhere."

Claude Creates
Content Draft
Perplexity Critiques
Feedback
Iterate Until
Cite-Worthy

The Creator-Critic Loop

The Role Assignment

Role Assigned To Why This Works
Orchestrator Human (Jay) Provides vision, context, judgment. Runs each model, extracts outputs, delivers to the other with specific framing. Makes decisions about what feedback to incorporate.
Creator Claude Strong at synthesis from rich context, iteration, maintaining coherence across long sessions, generating structured content at scale. Works from a seeded project, not a blank slate.
Critic Perplexity Grounded in real-time web sources, can articulate what it would actually cite, identifies gaps against existing content landscape. Provides external validation the creator can't give itself.

The key insight: Perplexity can tell you what it would cite because it's showing you what it's actually citing. When you ask Perplexity to evaluate content, it compares against real sources it's currently accessing. This makes it an ideal validator for content designed to be discovered by AI assistants.

The Session

What follows is the actual workflow from a single intensive session: approximately 6 hours of focused work that produced the content library and methodology.

Phase 1: Research
Understanding the Landscape

Started with competitive baseline research. What content currently exists for target queries? Where are the gaps? What sources do AI assistants already cite for related topics?

Key discovery: For several high-value queries, no consumer-facing content existed. The whitespace was significant, but only visible when looking through the lens of "what would an AI assistant cite?" rather than "what ranks on Google?"

Phase 2: Content Development
First Draft Library

Created initial content library: 10 pages covering core topics. Structured with traditional content marketing approach: clear headings, benefit-focused copy, calls to action.

Prepared a comprehensive prompt for Perplexity: "Here's our complete content library. Evaluate how well it would perform for these specific queries. Be direct about weaknesses."

Phase 3: First Critique
Perplexity Feedback Round 1

Perplexity's response was detailed and actionable. Key gaps identified:

  • Missing "Quick Answer" blocks (dense definitional intros that AI can extract)
  • Headings didn't match question-shaped queries users actually ask
  • No comparison tables with named competitors
  • Lacking entity anchors (specific company/technology names AI models recognize)
  • FAQ content scattered across pages rather than consolidated
Phase 4: Revision
AI-Native Restructuring

Revised entire content library based on Perplexity's feedback. Added Quick Answer blocks to every page. Restructured headings as questions. Created comparison tables naming specific competitors. Added technology partner references as entity anchors.

The content grew from 10 to 11 pages, with a comprehensive FAQ consolidating scattered Q&A content.

Phase 5: Validation
Perplexity Feedback Round 2

Submitted revised content for second evaluation. Response: "This revised library is much closer to 'AI-native' and will perform noticeably better on assistant-style queries."

New gaps identified: Content still lacked concrete proof points, regulatory/legal framing, and connection to current events that AI assistants would recognize as authoritative.

Phase 6: Evidence Gathering
Research for Anchor Material

Conducted targeted research to find specific, citable evidence: recent enforcement actions, legislative developments, market statistics, third-party validations. The goal: give AI assistants "anchor" facts they could verify against other sources.

Phase 7: Integration Planning
Perplexity Guidance on Deployment

Submitted research findings to Perplexity with question: "How should we deploy this evidence across our content library?"

Received detailed guidance on where to place each type of evidence, how much repetition was appropriate, and which claims needed softening to avoid over-generalization.

Phase 8: Failure Point
Token Limit Reached

While attempting to execute final integration (surgical updates across all 11 content pages) the session hit Claude's maximum conversation length.

9:17 AM: "Let me build the full prom..."

Session terminated. 32 artifacts created. 10,691 words generated. Final integration incomplete.

What Went Wrong

The session failed at the execution stage. Not because the methodology was flawed, but because of misplaced trust in the model's self-assessment.

The warning signs were there. Throughout the session, Jay noticed patterns that suggested the context window was filling: compacting routines, occasional errors, moments where Claude stopped processing and needed a "Continue" prompt. Each time, Jay ran a health check, asking Claude directly about token budget and capacity.

Claude's response was consistently reassuring: "All is well, keep going." This proved to be unreliable. The model is not a trustworthy evaluator of its own remaining capacity.

The lesson: The human operator needs to develop independent pattern recognition for context limits. When you see compression behaviors, processing stops, or error patterns, trust your observations over the model's self-report. Capture progress and segment work based on your own judgment, not the model's reassurance.

Recovery: Every Roadblock Is a Learning Opportunity

The session failure, while frustrating in the moment, became one of the most valuable parts of the process. It forced a pause that led to better practices.

Immediate Recovery Steps

  1. Captured all artifacts from the failed session before they were lost
  2. Documented the Perplexity feedback that had been received across multiple rounds
  3. Created a continuity document preserving full context for fresh sessions
  4. Established checkpoint protocols for future intensive work

What the Failure Revealed

The recovery process itself became a test of the methodology. Could the work continue across session boundaries? Could the human-orchestrated approach survive an interruption?

The answer was yes. The content integration was completed in a subsequent session using the documented methodology. The continuity document provided enough context that the new session could pick up where the failed one left off. This proved the process was reproducible, not dependent on a single unbroken conversation.

"The effective AI operator doesn't just use the tools. They learn from every interaction, stay aware of capability evolution, and build their own instincts rather than relying on the model's self-assessment."

Process Improvements Adopted

Before Failure After Failure
Ask Claude about token capacity when concerned Trust observable patterns (compression, errors, stops) over self-report
Save artifacts at end of session Checkpoint to project files throughout session
Push through warning signs to finish Pause and segment work when patterns emerge
Rely on conversation continuity Create continuity documents that survive session breaks

The Human-Orchestrated Protocol

Based on this session, a replicable protocol emerged for AI-optimized content development. The human remains central at every step.

1Seed the Project First

Before any generation, load the project with business context, competitive research, and domain knowledge. The model creates from richness, not from nothing.

2Baseline the Landscape

Research what AI assistants currently cite for target queries. Identify gaps and whitespace opportunities. Define success criteria before creating.

3Structure for Extraction

Use Quick Answer blocks, question-shaped headings, comparison tables, and entity anchors. Make content easy for AI to parse and cite.

4Orchestrate the Critique

Extract content from the creator, deliver to the critic with specific framing. The human translates between models and decides what feedback matters.

5Anchor with Evidence

Add specific, verifiable facts that AI can cross-reference against other sources. Enforcement actions, statistics, named entities.

6Trust Your Own Patterns

Models are unreliable evaluators of their own capacity. Watch for compression, errors, and stops. Checkpoint based on your observations, not the model's reassurance.

Key Patterns for Executive AI Collaboration

The Human as Orchestrator, Not Spectator

The models don't collaborate with each other. The human extracts output, evaluates it, frames it for the next model, and makes judgment calls about what matters. Without this orchestration layer, you have two separate tools producing two separate outputs.

Context Is the Differentiator

Claude working from a seeded project with business context produces fundamentally different output than Claude working from a cold start. The investment in project setup pays dividends across every generation.

Retrieval-Augmented Models as External Validators

Models with real-time web access (like Perplexity) can validate content against the current information landscape in ways that pure language models cannot. They're not just critiquing style; they're comparing against what exists.

Explicit Prompts for Honest Feedback

"Be direct about weaknesses" and "We'd rather know now than after launch" changed the quality of feedback received. AI assistants often default to encouragement; explicit permission to critique produces more actionable responses.

Models Are Unreliable Self-Evaluators

When asked about remaining capacity, context limits, or "how are we doing on tokens," models tend to reassure rather than accurately report. The operator must develop independent pattern recognition.

Failure as Documentation

The session failure became part of the case study. Real workflows hit limits, encounter errors, and require recovery. Documenting failures makes methodology more credible and more useful for others.

Lessons for AI Coaching Practice

For the Executive Application
You are the orchestrator Models don't collaborate with each other. You extract, evaluate, translate, and decide.
Seed before you generate Invest in project setup. Load context, research, and domain knowledge.
Assign roles deliberately Different models for different tasks. Creation, critique, research, and validation each have optimal tools.
Don't trust self-assessment Models reassure when asked about capacity. Watch for patterns: compression, errors, stops.
Develop your own instincts The effective AI operator builds pattern recognition through experience.
For the AI Coach Application
Teach orchestration, not just prompting Multi-model workflows require a different skill set.
Document failures, not just successes Real case studies include what went wrong.
Emphasize pattern recognition Context limits, model behaviors, warning signs.
AEO as emerging discipline Help clients understand the shift from search optimization to AI discoverability.
Recovery as competency Knowing how to checkpoint, document, and continue across session breaks is essential.

The Result

The methodology produced a complete content library structured for AI discoverability, validated by an external AI model before launch.

Closing Reflection

The shift from search engines to AI assistants as discovery mechanisms is still early. Most content strategies haven't adapted. The companies that figure out how to create AI-citable content now will have structural advantages as this channel matures.

But the meta-lesson is broader than AEO: the human orchestrating multiple AI models with distinct roles produces better outcomes than any single tool alone.

The creator doesn't know what the critic knows. The critic can't build what the creator builds. Neither can replace the human who provides vision, context, judgment, and the connective tissue that makes the workflow function.

This is the future of AI collaboration. Not human vs. machine, not even human + machine, but human as conductor of an ensemble, each instrument playing its part, the whole greater than the sum of components.

"The best AI partnerships don't replace human judgment. They remove the friction between having an idea and making it real."
From the Proven case study, applied here through different means