Inside the AI Competition That Decoded an Ancient Herculaneum Scroll

On a warm Saturday night at the end of August last year, Luke Farritor, an undergraduate at the University of Nebraska, was sitting alone in a corner at a house party in Omaha when his iPhone pinged. The music was booming, and Farritor, 21 years old at the time with a boyish face and black rectangular glasses, was surrounded by other students drinking and mingling. He opened the message. It was from Ben Kyles, a 45-year-old computer scientist and pianist from British Columbia, known to Farritor as “Hari Seldon”—Kyles’s online avatar, named for a character in Isaac Asimov’s Foundation series. Kyles had some news to share. He had just finished digitally unrolling some high-resolution scans of carbonized papyrus. He’d uploaded the images, he said, to a shared server. “Dude,” Farritor replied, “this is awesome. I’ll run it very soon.”

Kyles’s papyrus was from Herculaneum, an ancient Roman town on the Bay of Naples, at the base of Mount Vesuvius, that is home to the only preserved library from classical antiquity. The collection of papyri—from which about 1,800 mostly unreadable scrolls and fragments have so far been extracted—was interred under 60 feet of material deposited by pyroclastic flows, at temperatures greater than 900 degrees Fahrenheit, during the same eruption that destroyed Pompeii in C.E. 79. Without enough oxygen to burn, the scrolls were baked into charcoal—a blessing because it allowed them to join the small trove of papyri that has managed to endure since antiquity, all of it protected from humidity in some way or another, whether inhumed in Egyptian sands or singed by fire. But it also means they cannot be unrolled without turning to dust.

Using his phone, Farritor, who had been working late nights attempting to decipher the scrolls for most of the preceding six months, remotely dialed in to his desktop computer in his dorm room in Lincoln, an hour’s drive away. He located Kyles’s new papyrus segment on the server and immediately fed it into the artificial-intelligence-powered detector he had been building over the past few weeks. The detector was programmed to find ink, and therefore letters, and therefore words. He booted up the program to let it run, and he put his phone away. As the designated driver, he waited for the party to finish so he could deliver his friends back to their dorms.

On supporting science journalism

If you’re enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.

For four centuries monks and princes, papyrologists and archaeologists, classicists and computer scientists have struggled to little avail to detect any letters or words inside the scrolls—which resemble saggy little brown burritos—without destroying them in the process. If we could read them, as classicists and papyrologists have long hoped to do, we might discover lost works of classical literature or philosophy or records of history and science. Perhaps they contain tragedies by Sophocles or Aeschylus or the lost writings of Livy. “The possibilities,” says David Blank, a professor of classics at the University of California, Los Angeles, “are enormous.”

Almost all classical literature has come down to us from medieval monks, choosy in what they resolved to copy. As a result, relatively little “original” writing from antiquity exists, which means classical literature, from our point of view, is a panorama seen through a pinhole. We have seven plays by Aeschylus, but we know the tragedian wrote at least 10 times more than that. The preserved papyri in Herculaneum are our best shot at rescuing lost works, and some classicists suspect that even more texts could remain in areas of the villa yet to be excavated. “Who knows what’s there?” says Annalisa Marzano, a professor of classical archaeology at the University of Bologna and a trustee of the Friends of Herculaneum Society. In addition to works by the big hitters such as the poets Virgil and Horace, there’s also the tantalizing possibility of finding writing from authors “we know absolutely nothing about,” Marzano says.

After dropping off his friends, Farritor parked outside his dorm and, walking toward his building, took his phone out of his pocket. He unlocked the screen. He halted. “Holy cow,” he said. The AI had outputted something.

On the phone’s screen, set against the grayscale crisscross texture of woven papyrus, were three black lowercase Greek characters arranged in clear sequence. They were fuzzy but unmistakable: pi (π), omicron (ο), rho (ρ)—“πορ.”

The first person in almost 2,000 years to see those letters glimpsed them late on a summer night in a parking lot in Lincoln, having salvaged them from an ancient eruption. “I freaked,” he told me when we spoke last November. He sent a screenshot to his mom. It was a stunning beginning. But what, Farritor wondered, was the word containing those letters? And which lost book housed that word?

As the AI industry exploded last spring with the release of GPT-4, so did anxiety about superintelligent AI. Many “doomer” developers and thinkers in Silicon Valley and beyond warn that a system could someday choose to reduce our civilization to dust and rubble with the force of numerous Vesuviuses. This concern makes it all the more poignant that, over the past several months, hundreds of amateur decipherers have decided to spend their time building AI in the hopes of glimpsing never before seen writings found in a city that went extinct in the wake of a terrible catastrophe. The Vesuvius Challenge is a competition to rescue the knowledge in these scrolls. For Ahron Wayne, a Michigan-based industrial CT engineer participating in the challenge, the mission, he says, is about “as close as you can get to a video game.”

None of the strongest contenders have any expertise in classics. Most have only a passing interest in, and very little knowledge of, the ancient world. Most speak no Latin, no Greek. It is the technical problem that interests them—that and the Vesuvius Challenge’s collective prize of more than $1 million, which has been donated by some of Silicon Valley’s most powerful players.

The competition was the brain wave of Nat Friedman, a 46-year-old Bay Area investor. Until 2021, Friedman was CEO of Git-Hub, Microsoft’s open-source software-development platform. Together with Daniel Gross, his longtime investment partner, he was an early financier of today’s AI windfall. In the 2010s Friedman and Gross wrote checks to machine-learning researchers, and later, when the field exploded, they started funding AI companies. Today, to train the AI models they’ve bet on, the pair possess more Nvidia AI chips than most countries. Perhaps you’ve heard that tech billionaires are planning to build a utopian city from scratch on farmland north of San Francisco; Friedman has plowed his money into that project, too. In the spring of 2020, though, with much of the world in shutdown, Friedman was hoping simply to take his mind off the plague.

Quarantined at home in San Francisco and newly fascinated by ancient Rome, Friedman was reading Wikipedia articles about ancient disasters and calamities. In 1709, he learned, workers in the town of Resina near Naples were digging a well. At around 60 feet down their spades hit the remnants of an enormous theater. The building, which seated 2,500 people, was filled with statues of horses and noblemen. They must have been stunned. The existence of Herculaneum, the ancient town underneath their feet, had long ago slipped from collective memory.

Over the next few decades a succession of military engineers, desperate for ancient artworks to decorate their own villas, gave orders to dig underground tunnels fanning out from the theater. This activity caused severe damage. Conservation-oriented archaeological methods had yet to be developed, and one Spanish military engineer, in the words of 18th-century art historian Johann Joachim Winckelmann, “knew as much of antiquities as the moon knew of lobsters.”

In 1750 Karl Weber, a Swiss engineer, discovered a lavish villa by following an underground wall. The oceanfront property probably once belonged to the father-in-law of Julius Caesar, Lucius Calpurnius Piso Caesoninus. In a corner of the building, workers discovered a pile of black, misshapen cylinders a few inches high. The objects were initially thought to be carbonized wood, and some were thrown away—until Weber realized the room was a library. The workers removed more than 1,000 papyrus rolls and fragments, which were placed in a local museum.

The prospect of finding an unknown work of literature in the stash captivated much of Europe, and experimentalists tried various approaches to reading the papyri. A museum curator vertically sliced through a few scrolls with a knife, scraping off layers. This brutal method uncovered some legible text, but it ruined the scrolls. An Italian prince who had recently invented a waterproof cape for the future king of Spain immersed a few scrolls in mercury, hoping the liquid metal would separate the pages. It destroyed them. Others tried exposing them to a foul-smelling “vegetable gas” or slathering the rolls in rosewater.

In 1753 Antonio Piaggio, a monk who oversaw the ancient manuscripts in the Vatican Library, was summoned from Rome. Once in Naples, he invented a machine for slowly unrolling the papyrus, attaching silk threads to the edge of the sheets and gently prizing the layers apart at a rate of maybe a tenth of an inch a day. Piaggio had some success with this method and was able to reveal works by Philodemus, who taught Virgil and was one of the Greek Epicurean philosophers who held that atoms swerving and colliding in a void created the universe. But Piaggio’s approach was as destructive as it was slow. It seemed impossible to read the more than 330 still unopened scrolls without damaging them.

A few centuries later Friedman read about some recent breakthroughs. A group at the University of Kentucky, led by Brent Seales, a professor of computer science, seemed to be on the edge of success. In 2019 Seales’s team had arranged for two complete scrolls to be transported in custom-made cases, along with four broken-off fragments, to the Diamond Light Source, a synchrotron particle accelerator in Oxfordshire, England. Using the synchrotron’s high-energy photons, Seales and his team took micro-CT scans of the papyri at a resolution of eight microns, about the diameter of a red blood cell.

Seales’s plan was to import the synchrotron scans into a custom-designed computer program to virtually unwrap each layer of papyrus, in the hopes of revealing ink on the rendered surfaces. The carbon-based ink used on the scrolls, though, had a radiodensity similar to that of the papyrus. That meant there wasn’t enough contrast for the ink to show up in the scans.

To get around the problem, Seales’s team built a machine-learning model trained on manuscripts written in carbon ink. A successful ink-detection AI model could perhaps then be applied to the virtually unwrapped surfaces of the scrolls.

When Friedman read about this effort, he got an idea: perhaps Silicon Valley’s AI community could help, either by investing in the project or by offering its expertise. In 2022 Friedman invited Seales to Frontier Camp, an exclusive, furtive gathering (it has no online presence) that Friedman co-organizes and that takes place in the remote woods of northern California, where a handpicked elect of technologists—about 200 founders and CEOs—camp out in the cold for a few nights every year to throw ideas around.

Seales ignored the e-mail. He’d heard of Friedman but didn’t believe the correspondence was real. Friedman, though, was tenacious—and in October 2022 Seales arrived at the spartan summer-camp venue in the redwood forest of Sonoma County. That night, in one of the camp’s wood outbuildings, Seales gave a talk to a group of machine-learning engineers. “We’re going to solve this in the next hour,” Friedman whispered to Gross while Seales was still speaking. But they didn’t—and when the event ended, Friedman and Gross were worried that Seales would return to Kentucky empty-handed. So that evening, over bourbon in the bar of Seales’s Palo Alto hotel, Friedman suggested they organize an open contest instead. “We’ll put up some money for it,” he told Seales.

Seales flew home to discuss the idea with the other members of his laboratory. “We don’t want to be stupid and just give away all this work we’ve done,” said Stephen Parsons, a digital-restoration researcher who was finishing up a Ph.D. thesis on the lab’s work. At the same time, they had more ideas than they could reasonably try out by themselves. More people grappling with the problem would make it more likely the scrolls would be read, which was their ultimate goal. It was settled. They would open the project to the world. Friedman struggled to contain his enthusiasm. “Working on a very exciting and weird new side project,” he wrote on Twitter (now X). “Kind of a lifelong dream type thing.”

With Friedman, the group devised the competition’s structure. There would be stages of freestanding prizes for different challenges, including detecting ink, finding the first letters in the scrolls and building useful open-source software. The deadline for entries for the grand prize—for identifying four separate passages of at least 140 characters—would be December 31, 2023. “How would the ancient Romans feel,” Friedman wrote on X on the Ides of March, the day the competition launched (which was also the day after the release of GPT-4), “if they knew that 2000 years later, we would be using particle accelerators and supercomputers to read their words, preserve them for eternity, and whisper them into the ear of a baby god?”

Shortly after the competition opened, I joined the contest’s server on Discord, a message-board platform built for gamers. I was one of around 400 people who also signed up in the first few weeks, and by fall the board was bustling with 1,428 members. The competitors had downloaded the 5.5 terabytes of scan images of two scrolls, which Seales had nicknamed Banana Boy and Fat Bastard (their real names are PHerc_Paris_3 and PHerc_Paris_4), and were discussing what to do with them. For contestants who knew serious prize money was at stake, people were tossing ideas around with surprising candor.

There were two main tasks: segmentation and ink detection. To find letters, you need clean surfaces, or segments, of papyrus. Along the z axis of each scroll, from top to bottom, the researchers had taken thousands of cross-sectional x-rays. Each cross section reveals rolled-up papyrus sheets like rings of a tree, wispy white lines spiraling against a dark background. Unwrapping the rolls and extracting a flat surface takes forever. It requires using mouse clicks to mark the changing position of a sheet in every cross section. Then, bespoke algorithms stitch the individual cross sections together into a single sheet. But this is stymied by the fact that the carbonization melded some of the sheets together. Sometimes the papyrus folds back on itself, or it becomes unstuck, with one sheet becoming several and no way of knowing which surface holds the writing.

“It’s fused, mushy chunks of coal,” says Kyles, who was hired by Friedman to segment full-time and share the results with the community. With the help of open-source software developed by other participants, Kyles and his team of nine-to-five segmenters were able to churn out about 0.2 square inch of papyrus surface per hour. (The length of a Herculaneum scroll can be more than 32 feet.)

Spotting any ink on those surfaces, though, was a different matter entirely. To boost progress in ink detection, the organizers launched a machine-learning contest on Kaggle, an online platform for data-science competitions, with prizes totaling $100,000. The task was relatively straightforward: build a machine-learning model to detect ink in the CT scans of broken-off papyrus fragments on which writing was already clearly visible (an approach the Kentucky team had already tried with some success). An ink-detection model trained on the fragments, the organizers hoped, could then be applied to segments of virtually unwrapped papyrus.

A total of 2,763 competitors and teams—including a pair of students at China’s Harbin Institute of Technology, a team of archaeologists from Kyiv, a medical-imaging research group in Germany, and machine-learning engineers in Japan and South Korea—signed up. They built AIs to predict the presence or absence of ink in each voxel (the three-dimensional equivalent of a pixel) of the scanned fragments and uploaded their results. Their entries were verified against data from infrared photographs of the fragments.

In the weeks leading up to the close of the Kaggle competition on June 14, 2023, a team in San Diego found itself toward the bottom of the leaderboard. Tinkering with their model in cafés across the city, the teammates noticed that the ink had saturated the papyrus more deeply in certain areas. Their model was learning to place importance on the ink’s depth—but this approach was confusing it. The researchers tweaked the model to ignore the depth of the ink, which led to better results, propelling the group to the top of the leaderboard. The depth-invariant model went on to win the ink-detection competition. But something was wrong.

When Ryan Chesler, one of the team members, applied his winning model to a large section of Kyles’s virtually unrolled papyrus, known as the Monster Segment, he was disappointed. The AI was not detecting any ink at all. A model trained on the fragments, it seemed, would not work on the full scrolls. Wayne, the CT engineer in Michigan, thought he knew why. The ink-detection prizewinners “are a bunch of freaking brainiacs,” he says. But they were thinking of the conundrum as a math problem. “The real world is a little bit messier,” Wayne explains. The ink-detection models might have failed to work on the scrolls because the AI was not able to learn what carbonized papyrus looked like. Without more scans of carbonized papyrus, even the most sophisticated algorithms would struggle.

More familiar with scanning objects such as rocket motors and bassoons, Wayne convinced his employer to let him use one of their state-of-the-art CT scanners in his free time. (“If someone were to win the grand prize,” he says, “they could afford almost half of one of these.”) He drew sophisticated Grecian profiles on papyrus, which he then carbonized and scanned. He shared the results with others, creating a rich store of training data.

Others believed the contestants were too focused on AI. Casey Handmer, an Australian physicist in his 30s, decided to visually inspect the scans instead. “If a machine can see it, a human can see it,” he says. Most machine-learning algorithms for detecting visual features, Handmer explains, are built on human detection, and for good reason: our visual cortex is highly adept at identifying subtle patterns and textures.

Handmer is the founder and CEO of Terraform Industries, a California start-up that produces carbon-neutral natural gas from sun and air, with headquarters in a mock medieval castle in Burbank. He spent hours inspecting the images, to the irritation of Friedman, an investor in Terraform, who did not approve of the CEO’s distraction. After a break to 3-D print a copy of his skeleton for his own amusement, Handmer found he was becoming increasingly familiar with the visual features of the burned papyrus fibers. And in May, while inspecting Kyles’s Monster Segment, he noticed something remarkable. He kept seeing a recurring texture that looked like cracked mud baked on the surface of the papyrus. After an hour or so of intense staring, he noticed an upside-down π. The cracked texture had to be ink.

Handmer found more of that texture in the shape of other letters and even believed he had discovered the word “Calliope” (Καλλιόπη), the name of the Muse of epic poetry. His findings, though, failed to persuade the six papyrologists who evaluated his entry in the first letters prize in June—a $40,000 contest for the first person to find 10 letters in an area of 0.6 square inch. But Handmer had discovered the first ink, for which he was awarded a prize of $10,000. By sharing his breakthrough with the community in almost real time, he prepared the way for the next big advance.

Farritor, the college student in Nebraska, was interning at SpaceX in Texas when he learned about Handmer’s discovery of the cracked texture. He spent his days working on the launchpad-software team for Starship—the most powerful rocket ever made. After work, though, he was up much of the night building an AI to find more of the cracked texture. Meanwhile Youssef Nader, an Egyptian data-science student at the Free University of Berlin, was working on a system that he had adapted from a successful Kaggle competition model. Both Farritor and Nader succeeded in finding letter sequences. Nader’s results were cleaner, but Farritor was faster. After finding the πορ the night of the August party, Farritor continued refining his model until it spat out a couple dozen fuzzy shapes surrounding the πορ that looked like they might also be Greek letters.

In September the papyrologists inspected Farritor’s results. They realized Farritor’s πορ was the beginning of the word πορφύραc, or porphoras, the ancient Greek word for “purple.” The term is rare, says Federica Nicolardi, a papyrologist at the University of Naples who helped to confirm the word. And it is likely to be from a new text.

A few weeks later the competition flew Farritor out to Kentucky for a symposium organized around the breakthrough. Afterward JP Posma, one of the organizers, handed Farritor an oversize $40,000 check, and Nader, who found the same word just a few days later, was awarded $10,000 for second place. But just as the papyrologists were arriving for the symposium, Nader executed the biggest leap forward so far: he released an image showing πορφύραcin the context of four full columns of text—a sight papyrologists had not expected to see in their lifetimes.

In the columns there were other identifiable words, including the possible phrase κατάμουσικήν (kata mousikēn), meaning something like “relating to music”—which makes the scroll, according to Nicolardi, most likely a work of philosophy. One night during the symposium Farritor was seen hauling Diet Cokes up to his hotel room to help him stay awake to find other long passages. In Berlin, Nader wasn’t sleeping much, either. The $700,000 grand prize seemed closer than ever.

At the end of 2023, Nader teamed up with Farritor and Julian Schilliger, a robotics student in Switzerland whose software to accelerate segmentation and map the papyrus in three dimensions had won him an earlier prize. In December, having combined their approaches, the team of three produced something astounding. Building on the work each had done individually, their AI models revealed 2,000 characters in four full columns—far outstripping the Grand Prize’s criterion of four passages of 140 characters. In early February the Vesuvius Challenge awarded them the $700,000 Grand Prize.

The readable text comprises around 5 percent of the first scroll, and it is from the same text as the earlier discoveries. It is a previously unread tract, probably by Philodemus, about pleasure. Are good things in small quantities more delightful than copious good things? Not at all, the author concludes. “As, too, in the case of food,” writes the author, “we do not right away believe things that are scarce to be absolutely more pleasant than those which are abundant.”

The fields of papyrology and classics are changed forever. Thanks in large part to a group of amateur AI builders, we now have tools for reading the unopened Herculaneum papyri. If the technological advances continue and can be rolled out to the many unopened scrolls, says Tobias Reinhardt, a classics scholar at the University of Oxford who helped to confirm the winning entries, “we could see a recovery of ancient texts at a volume not seen since the Renaissance.”

Friedman wants more—and he isn’t stopping. His goal for 2024 is to build on the winning team’s approach to read 90 percent of the four scrolls that have now been scanned using high-energy physics. If successful, this will unlock the hundreds of unopened Herculaneum scrolls. But those are only the ones we know about. He ultimately wishes to persuade the Italian authorities to allow new excavations of the villa, in the hopes of digging out even more material.

There are still a few wrinkles to be ironed out before the process can scale. Researchers will need to find a way to automate the time-consuming and expensive manual-segmentation process. Plus, hiring a particle accelerator to scan hundreds of scrolls is too expensive, and cheaper workarounds for producing high-resolution scans will have to be found.

What is most startling for papyrologists, though, is the speed at which the AI pipeline is now finding identifiable letters. Going from three letters to entire words and phrases and then columns of text, which took Farritor, Nader and Schilliger a month, usually takes papyrologists 20 years of intense study, says Gianluca Del Mastro, Nicolardi’s colleague and a professor of papyrology at the University of Campania Luigi Vanvitelli. When Del Mastro looked at Nader’s columns, Friedman noticed, he had tears in his eyes.

The technology that the Vesuvius Challenge helped to develop could be adapted for deciphering other lost texts beyond the Bay of Naples, Seales says. There are plenty of enticing contenders. In 1993, 140 carbonized papyrus scrolls dating from the sixth century C.E. were discovered in a Byzantine church in Petra, Jordan. Blackened and fragile, they were considered unreadable. And tens of thousands of fragments of the Dead Sea Scrolls have never been read because so many are stuck together. They are now prime candidates for the Vesuvius Challenge’s virtual-unwrapping-to-AI-ink-detection pipeline.

Ancient Egyptian mummy masks were also made of papyrus, arranged in layers coated with plaster—a material called cartonnage, essentially a kind of papier-mâché. That papyrus often contained writing, which has been difficult to decipher without destroying the plaster. Those papyri may now have an afterlife, too.

In the fourth century B.C.E. Greek historian Xenophon noted on his return from Mesopotamia that there was a bustling trade in scrolls across the Black Sea. This means there are almost certainly sunken ships on the seafloor that contain boxes of papyrus rolls, according to Richard Janko, a classics professor and papyrologist at the University of Michigan. These scrolls are probably still preserved in this sea, which has exceptionally low oxygen and salinity for a marine environment. So far AI excitement has revolved mainly around neural networks learning how to chat. Yet more compelling is how they will get silent things to speak.