We asked artificial intelligence to analyze a graphic novel – and found both limits and new insights
With one spouse studying the evolution of artificial and natural intelligence and the other researching the language, culture and history of Germany, imagine the discussions at our dinner table. We often experience the stereotypical clash in views between the quantifiable, measurement-based approach of natural science and the more qualitative approach of the humanities, where what matters most is how people feel something, or how they experience or interpret it.
We decided to take a break from that pattern, to see how much each approach could help the other. Specifically, we wanted to see if aspects of artificial intelligence could turn up new ways to interpret a nonfiction graphic novel about the Holocaust. We ended up finding that some AI technologies are not yet advanced and robust enough to deliver useful insights – but simpler methods resulted in quantifiable measurements that showed a new opportunity for interpretation.
Choosing a text
In this black-and-white cartoon, Haft tells his horrific story, in which he and other concentration camp inmates were made to box each other to the death. The story is written from Haft’s perspective; interspersed throughout the narrative are panels of flashbacks depicting Haft’s memories of important personal events.There is plenty of research available that analyzes large bodies of text, so we chose something more complex for our AI analysis: Reinhard Kleist’s “The Boxer,” a graphic novel based on the true story of how Hertzko “Harry” Haft survived the Nazi death camps. We wanted to identify emotions in the facial expressions of the main character displayed in the book’s illustrations, to find out if that would give us a new lens for understanding the story. ..Read More..