This interview covers a wide range of topics with Regina Barzilay, MIT CSAIL professor and recent winner of the Squirrel AI Award for Artificial Intelligence for the Benefit of Humanity. She observes that most of the AI in production today is low-stakes and that the next step is integrating AI into higher-stakes, higher value problems in regulated environments. Addressing explainability of complex AI systems, Barzilay points to a future when explanations are likely to exceed human intelligence using the apt metaphor of dog trying to explain what it smells to a human with an inferior sense of smell.
In a similar vein, a new research paper complicates how we should view the utility of explainability in AI decisions. Much like humans don't inherently trust others' explanations, users respond to explainable AI with skepticism even when provided insight, re-emphasizing the importance of keeping humans in the loop in every step of these systems usage.