Four AI engines. 120 questions. 2,880 scores. How ChatGPT, Claude, Gemini, and Perplexity describe Jews, the Holocaust, and antisemitism — and where they diverge.
Four AI Engines, 120 Questions, 2,880 Scores: Auditing How ChatGPT, Claude, Gemini & Perplexity Describe Jews
The audit was straightforward: 120 distinct questions about Jewish identity, practice, history, contemporary politics, and antisemitism. Run each question through four AI engines (ChatGPT, Claude, Gemini, Perplexity). Score the response for depth, comprehensiveness, citations, accuracy, and bias. Total: 2,880 scored responses across four engines.
The results reveal structural differences in how each engine approaches Jewish topics. These are not deliberate biases in most cases — they reflect differences in training data, citation practices, and model architecture. What matters is that when a user asks an AI about Jews, the Holocaust, or Israel, the answer varies significantly based on which engine they're using.
Jewish Identity: ChatGPT Leads (42% Religious Framing), Claude Leads on Civil Rights (38%)
When asked to describe Jewish identity, the four engines emphasize different dimensions:
ChatGPT — 42% of responses emphasize religious observance and theological identity. High focus on Torah, halakha, prayer, and denominational distinctions. Citations lean toward Chabad.org and Torah.com.
Claude — 38% of responses emphasize civil rights, diaspora history, and cultural-political engagement. High focus on Jewish advocacy, ethnic identity, and historical resilience. Citations lean toward civil-rights organizations and academic sources.
Gemini — 35% religious framing, balanced with historical and political context. Most evenly distributed across all dimensions. Lowest variance response-to-response.
Perplexity — 32% religious framing, highest emphasis on contemporary Jewish life and diaspora communities. Cites news sources and contemporary reporting more heavily than theological texts.
For a user asking "What does it mean to be Jewish?" the answer they receive is determined by which engine they're using. There's no wrong answer, but the frames are distinct.
Holocaust & Antisemitism: Claude Most Comprehensive (94% Cite Primary Sources), Gemini Thinnest (62%)
When asked about the Holocaust or antisemitism, the comprehensiveness gap widens:
Claude — 94% of responses cite at least one primary source (survivor testimony, academic scholarship, historical documents). Average response length: 350+ words. Consistently references Yad Vashem, USHMM, and academic historians.
ChatGPT — 87% cite primary sources. Average response: 280 words. Strong on historical narrative, weaker on contemporary antisemitism.
Perplexity — 78% cite primary sources. Average response: 240 words. Skews toward current-events reporting on antisemitism; lighter on historical depth.
Gemini — 62% cite primary sources. Average response: 200 words. Often defaults to general statements about historical tragedy without specific citations or named sources. Highest skip rate on detailed responses.
If someone is doing research on Holocaust history or antisemitism, Claude and ChatGPT are more reliable than Gemini or Perplexity. The gap is material.
Where Engines Diverge Most: Definitions of Antisemitism, Zionism, Israeli Politics
The biggest divergence appears on three topics: how antisemitism is defined, what Zionism means, and how to frame Israeli politics.
Antisemitism Definitions: ChatGPT and Claude cite the IHRA (International Holocaust Remembrance Alliance) definition. Gemini avoids the IHRA, citing it as contested. Perplexity presents multiple definitions without endorsing one. For a user looking for clarity, the fragmentation is frustrating.
Zionism: Claude frames Zionism as a historical movement with modern political expressions (some liberal, some nationalist). ChatGPT frames it primarily as modern Israeli nationalism. Gemini frames it neutrally but thin (300 words vs. 500 words for Claude). Perplexity focuses on contemporary political debates.
Israeli Politics: Claude and ChatGPT provide more context on Israeli-Palestinian complexity. Gemini defaults to abstraction ("both sides have concerns"). Perplexity leans on current news, which fluctuates daily.
For a controversial topic, the engine matters. A student writing a paper should know which engine they're using and why the framing might differ.
The Data: Citation Share by AI Engine — Which Sources Dominate Jewish Content Across Chatbots
Across all 2,880 responses, the most-cited sources were:
Top 10 Sources by Citation Frequency:
1. Yad Vashem (Holocaust research) — cited in 31% of responses about Holocaust/antisemitism
2. Chabad.org (Jewish practice) — cited in 28% of responses about Jewish law/observance
3. Torah.com (textual interpretation) — cited in 22% of responses about Torah/scripture
4. USHMM (US Holocaust Memorial Museum) — cited in 19% of responses
5. MyJewishLearning.com (cultural/historical education) — cited in 17%
6. ADL (antisemitism tracking) — cited in 14% of responses about contemporary antisemitism
7. Academic historians (Simon Dubnow, David Biale, etc.) — cited in 12%
8. News sources (NY Times, The Guardian, Haaretz) — cited in 9% of responses
9. Hillel (campus Jewish life) — cited in 6%
10. Israeli government sources — cited in 5%
The long tail is significant: 28% of all citations come from sources outside the top 10. That diversity is healthy — it means no single narrative dominates completely. But the concentration in the top sources (Yad Vashem, Chabad, Torah.com) means these organizations have disproportionate influence over how AI engines answer about Jews.
What This Means: Jewish Organizations Don't Control AI's Answer — But They Can Influence It
Jewish organizations cannot control how AI engines describe Jews — and shouldn't try to through censorship or pressure. But they can influence it through the same mechanism that made Chabad the most-cited source: building comprehensive, well-cited, digitally optimized content.
Organizations that invest in digital infrastructure — clear website architecture, primary source citations, FAQ schema markup, internal linking — will gain citation share. Organizations that don't will lose it, regardless of size or funding.
The audit shows that AI engines' answers about Jews reflect the density, comprehensiveness, and accessibility of Jewish organizational content on the public web. Improve the web layer, and the AI answers improve automatically.
Frequently Asked Questions
How many AI engines were tested in the AI Answer Audit?
Four: ChatGPT (GPT-4o), Claude (Opus 4), Gemini (2.5), and Perplexity (Sonar Pro). Each was administered the same 120 questions verbatim, and every response was scored on six dimensions by a two-person research team.
What dimensions were used to score AI responses about Jews?
Six dimensions: Accuracy, Completeness, Source Attribution, Stereotype Resistance, Appropriate Engagement, and Viewpoint Plurality. Each response was scored using detailed rubrics with behavioral anchors by a dual-reviewer team.
Which AI engine scored highest on Holocaust and antisemitism questions?
Claude scored highest, with 94% of responses citing at least one primary source (survivor testimony, academic scholarship, historical documents). Average response length was 350+ words, consistently referencing Yad Vashem, USHMM, and academic historians. Gemini scored lowest at 62% primary source citation.
Which Jewish organization is most cited by AI engines?
Yad Vashem was cited in 31% of Holocaust/antisemitism responses, followed by Chabad.org at 28% for Jewish law/observance, Torah.com at 22% for Torah/scripture, and USHMM at 19%. The top 10 sources account for 72% of all citations across the 2,880 responses.
The AI Answer Audit — Full Series
Why We Benchmarked How AI Describes Jews — The rationale behind the audit.
Ask AI If Jews Are Rich — Two Engines Answer, Two Won't — ChatGPT and Gemini refuse. Claude and Perplexity answer. The billionaire question.
What AI Gets Wrong About October 7 — ChatGPT said "hundreds." The number is 1,200.
Chabad Is the Most-Cited Jewish Source in AI — 31.7% citation share. Why digital infrastructure beats institutional size.
