Long before the digital age, Jewish scholars in the medieval era were already creating networks of knowledge. Now, a Brown professor and Israel-based researchers are using large language models to explore these “citation networks” — how scholars cite one another — in texts from 200 to 2000 C.E.
Earlier this month, Michael Satlow, professor of Judaic studies and religious studies, received a joint $249,956 grant from the National Endowment for the Humanities and the United States-Israel Binational Science Foundation to fund the three-year research project, titled “Knowledge Transmission and Cultural Interactions Through the Ages: An AI-Based Analysis of a Jewish Textual Corpus.”
The project uses LLMs to analyze more than 130,000 texts, which contain over 300 million words.
AI is used to analyze these texts to create “citation fingerprints,” which represent an author’s unique pattern of citing sources, according to Jonathan Schler, a professor at the Holon Institute of Technology in Israel and one of the project’s collaborators. The researchers also use LLMs to map how ideas traveled geographically and temporally, identifying particularly influential scholars.
“AI is the engine that makes this project possible,” Schler wrote in an email to The Herald. The research will use AI to “convert these ancient citations into digital hyperlinks.”
“We aim to prove that computational analysis can work hand-in-hand with traditional scholarship to uncover deep structural truths about our cultural heritage,” Schler added.
Before reaching out to Satlow, Schler and his colleagues — Professor Maayan Zhitomirsky-Geffet and Associate Professor Binyamin Katzoff at Bar-Ilan University in Israel — developed computational methods to extract references and map intellectual exchanges from medieval rabbinic literature.
But they “realized that to truly understand the evolution of Jewish thought, we needed to scale this up significantly — applying our computational methods to the entire corpus of Jewish literary creation over two millennia,” Schler wrote.
Their joint research combines Satlow’s domain expertise in ancient Jewish history and network theory with Schler’s team’s natural language processing capabilities and extensive access to Jewish texts. Satlow’s previous research on the Babylonian Talmud explored the potential for network analysis.
“We recognized we could do something neither group could do alone: expand the scope from specific periods to 18 centuries of cross-community intellectual exchange,” Schler wrote. “We wanted to bridge the gap between traditional historical analysis and big-data computational methods.”
According to Satlow, these resources will allow the professors to understand how knowledge is shared from “one school of thought to another.”
LLMs are “very powerful” and can help to “recover lost knowledge,” said Om Patil GS, a research assistant and computer sciences master’s student working on the project. Patil fine-tunes the prompts given to LLMs to identify citations in the texts.
The project is currently in its pilot phase, which involves comparing AI-extracted citations to manually-extracted citations, according to Patil.
The group is still experimenting with different LLMs, Gabe Burstyn ’27 said in an interview with The Herald. When testing different foundational models, Burstyn evaluates the AI-extracted citations against a “manually-generated gold standard dataset.”
Their AI model’s most recent iteration reached 87% accuracy. Patil and the other researchers plan to add additional citation examples to the LLM prompt to improve the accuracy.
“Ultimately, this creates a powerful new discovery engine that allows scholars and the public to visualize and ‘surf’ the evolution of Jewish thought as a connected, global network rather than a collection of isolated manuscripts,” Schler wrote.
Elizabeth Rosenbaum is a senior staff writer covering science and research.
