From caws to code: AI helps decrypt animal communication
Scientists are increasingly using artificial intelligence models to decode the communications of other species.
The Earth Species Project has built a generalizable model that could be used across species; the team also works with scientists around the world to develop custom models for specific species.
In northern Spain, ESP’s AI tools are helping scientists understand how a population of cooperative-breeding crows communicate with one another.
The technology is also being deployed to understand how orcas communicate with each other, and how underwater noise affects their communication.
This article was originally written by Abhishyant Kidangoor appeared on Mongabay, an independent media organization bringing you news and inspiration from nature’s frontline.
What are carrion crows saying to each other?
Answering that question has been Vittorio Baglione and Daniela Canestrari’s mission for decades. Carrion crows (Corvus corone) are especially interesting because they engage in cooperative breeding, where entire families, not just the mother and father, are involved in raising chicks and protecting nests. That requires intricate and nuanced communication between individuals.
“They have a very complex society, and they do very complex things together,” Baglione, a professor at the University of León in Spain, told Mongabay in a video interview. “It’s really coordinated behavior and we have answered why they do it, but we want to know how they coordinate and exchange information.”
The duo have deployed audio recorders and biologgers in northern Spain to decode crow calls and monitor corresponding behavioral patterns. But as the data piled up, they came across a massive hurdle. “Each microphone lasted for six to seven days,” Canestrari, also a professor at the same university, told Mongabay in a video interview. “We realized we actually have too much data to analyze.”
Since 2024, the scientists have collaborated with the Earth Species Project (ESP). A team at the U.S.-based nonprofit has helped them develop artificial intelligence models to categorize crow calls and start building a data set of different call types.
How different animals communicate with each other has been a topic of human fascination for generations. The advent of AI in conservation has sped up the process of identifying patterns and structures in huge data sets, taking us one small step closer to understanding how animals communicate with one another. ESP has built a model that could be used to analyze communications across various species. The group also works with scientists around the world, like Baglione and Canestrari, to build custom AI tools specific to the animals they’re working with.
“If you look at the biodiversity crisis, there is a huge disconnect from nature,” David Robinson, senior AI research scientist at ESP, told Mongabay in a video interview. “Studying communication can be a window into the inner lives and complexity of other species and can help us gain insight into their behavior.”
For example, with the carrion crows, the technology has helped scientists detect more than 127,000 vocalizations. It has also helped differentiate between the calls of adults and babies, as well as other species of birds. When multiple crows are calling, the model also helps synchronize data from different loggers. “In many territories, we had more than one individual with dialogue at the same time,” Canestrari said. “They needed to be synchronized to see what exactly a bird was doing while the other was calling.”
While data analysis is still ongoing, the scientists have found that most crow vocalizations are soft, low-amplitude murmurs. This, they say, potentially indicates that a big chunk of communication happens at close range rather than over long distances. The scientists are continuing to collaborate with the team at ESP to produce a data set, a semantic map of sorts, that would bring together audio data with video as well as data from accelerometers, devices that are used to measure movement and speed. “We could match behaviors and calls to give indication about the functions of those calls,” Baglione said.
The technology is not restricted to terrestrial species.
At the Raincoast Conservation Foundation in Canada, scientists are collaborating with ESP to understand the communication and behavioral patterns of orcas (Orcinus orca). The species live in tight-knit groups and are known to use their calls to move and hunt together. Decoding their calls would help scientists understand the species’ behavioral patterns. It would also shed insight into “the conservation issue of underwater noise and how it impacts the ability of these animals to chat with each other,” Valeria Vergara, co-director of the Raincoast Conservation Foundation’s cetacean conservation research program, told Mongabay in a video interview.
As part of a pilot project, ESP and Raincoast Conservation are using data from drones as well as acoustic recorders to develop a data set that can link orca vocalizations to their behaviors and environmental factors. Even as this work is underway, the team has already used AI models to speed up the process of identifying orcas from vast amounts of ocean audio data. “Instead of having to listen to every minute of a recording, this tool can just pick out the vocalizations so that we can then look at the vocalizations and label them,” Vergara said.
The technology has also helped synchronize the voice notes that scientists record in the field with the corresponding whale audio or drone clip. “Synthesizing all these data sets would have taken us months in the past, but the tool can do it in an hour,” Vergara said. “What it does is it gets the data ready to analyze and quantifies our very qualitative voice notes.”
Vergara said the eventual goal is to use the technology to move beyond identifying dialects and call types to understanding the meaning behind those calls. However, she calls for caution over the growing public fascination with whether AI can enable humans to communicate with other animals.
“I would say let’s just be happy understanding whales on their own terms,” she said, “without inserting ourselves in the conversation.”
Banner image: Carrion crows engage in cooperative breeding which requires intricate communication between entire families that are involved in raising chicks. Image courtesy of Vittorio Baglione and Daniela Canestrari.
Here is the question: which is more important, survival of human communities or scientific research?