PhD · he/him
I practice AI risk management inside a large bank. Everything else I do, from research to assurance to policy, comes back to improving the practice itself.
Speaking
Scalable Oversight of Agents in Regulated Industries
Xoogler community panel · panelist with Jason Stanley and Mike Hsu
What Agents Reveal About Our Assurance Tools
Mastercard × All Tech Is Human Workshop on Inter-Party Trust for AI · invited speaker
Should AGI Really Be the Goal of Artificial Intelligence Research?
The Tech Policy Press Podcast · guest with Eryk Salvaggio and Margaret Mitchell
Red Teaming Generative AI Harm
Data & Society Databite No. 161 · moderator
Experimental Publics: Democracy and the Role of Publics in GenAI Evaluation
Knight First Amendment Institute Symposium on AI and Democratic Freedoms · panelist/coauthor
Rethinking Intelligence in the Age of AI
The Only Constant, Episode 27 · guest
Responsible AI: Discussing The Governance Maturity Model
All Tech Is Human report-release event · panelist
Working Through Dissonance: Towards Red-Teaming as Sociotechnical Critical Thinking
Google Responsible AI Talk · invited speaker
Making Intelligence: Ethical Values in IQ and ML Benchmarks
ACM FAccT · paper presenter
Mapping the Risk Surface of Text-to-Image AI: A Participatory, Cross-Disciplinary Workshop
ACM FAccT · organizer/facilitator
AVID: Empowering Communities to Put AI Risk Management into Practice Through Collaborative Open Knowledge
Mozilla Festival · facilitator
BABL AI: AI and Research Ethics
Lunchtime BABLing podcast · guest
Making Intelligence: Ethics, IQ, and ML Benchmarks
Queer in AI at NeurIPS · poster presenter
Twitter's Bias Bounty Program and the Ethics of Research
DigEthix, Episode 23 · guest
Taking Algorithms to Court: Empowering Communities to Enact Legal Accountability
RightsCon · speaker/cohost
Writing
Consistency in Language Models: Current Landscape, Challenges, and Future Directions
ICML Workshop on Reliable and Responsible Foundation Models
Red-Teaming in the Public Interest
Data & Society Research Institute
Experimental Publics: Democracy and the Role of Publics in GenAI Evaluation
Knight First Amendment Institute
Is ETHICS about ethics? Evaluating the ETHICS benchmark
NeurIPS Evaluating Evaluations workshop
A Flexible Maturity Model for AI Governance Based on the NIST AI Risk Management Framework
IEEE-USA AI Policy Committee
Responsible AI Governance Maturity Model: 2024 Hackathon Report
All Tech Is Human
AI Red-Teaming Is Not a One-Stop Solution to AI Harms: Recommendations for Using Red-Teaming for AI Accountability
Data & Society Research Institute
Can We Red Team Our Way to AI Accountability?
Tech Policy Press
Press
After a year building infrastructure for responsible disclosure of AI vulnerabilities at ARVA, I told The Washington Post that shaming shouldn't be the only way independent researchers get heard. That's bad for the public, and bad for companies too:
“We have a broken oversight ecosystem. Sure, people find problems. But the only channel to have an impact is these ‘gotcha’ moments where you have caught the company with its pants down.”
— The Washington Post , on the open letter calling for a researcher safe harbor · March 2024
About
Today, I'm an applied machine learning scientist at TD Bank, where I conduct technical reviews of the bank's AI systems and contribute to the consumer protection requirements they're held to.
Previously, I audited AI systems for harm as a senior consultant at BABL AI. I also helped lead the AI Risk and Vulnerability Alliance (ARVA), where I directed its research partnership with Data & Society. That work was funded through a Magic Grant I received from the Brown Institute for Media Innovation (Columbia × Stanford), with our red-teaming research supported by Omidyar Network.
My research examines the building blocks of AI risk management, from how AI is evaluated and red-teamed to the goals of AI research itself. My work has appeared through ICML, FAccT, AIES, Data & Society, and IEEE-USA. I hold a PhD in philosophy from Columbia University.
In my spare time, I love making musical instruments misbehave.