Andrew Critch

Basic Info

Current affiliations:
Encultured AI / HealthcareAgents — CEO & Cofounder (full-time)
UC Berkeley / Center for Human-Compatible AI (CHAI) — Research Scientist (~1 day/week)
Berkeley Existential Risk Initiative (BERI) — Founder and President (volunteer; ~1hr/week)
Survival and Flourishing Fund (SFF) — Fund Advisor (volunteer; ~1 hr/week)
Survival and Flourishing .Com (SFC) — Director (~1 hr/week)
OpenLetter.net — Cofounder (~1 hr/week)
Survival and Flourishing Projects (SFP) — Project Director (volunteer; ~1hr/month)

What I’m up to

… mostly, a bunch of projects intended to help humanity navigate the development of increasingly advanced AI technology. As of spring 2022, I’m working full-time as CEO and Cofounder of Encultured AI, which is now doing business as HealthcareAgents. I also work for ~1 day/week as a research scientist at UC Berkeley, within the Center for Human-compatible AI (CHAI) in the Department of Electrical Engineering and Computer Sciences. Prior to that, I worked full-time at CHAI from 2017-2021. I also volunteer for about an hour per week at the Berkeley Existential Risk Initiative, a nonprofit I founded to help research groups streamline operations for projects relevant to humanity’s long-term survival and flourishing. I’m an organizer for and advisor to the Survival and Flourishing Fund (SFF), where I designed a utility-based system for distributing philanthropic donations through the assessments of a team of “grant recommenders”, now also supported by Survival and Flourishing .Com. I also co-founded OpenLetter.net, a small web service for hosting important public statements that is still very much under development. Finally, I volunteer for about 1 hour per month as project director of Survival and Flourishing Projects, a fiscally sponsored project of Social and Environmental Entrepreneurs.

My research aims to identify and address societal-scale safety issues with future applications of artificial intelligence… sort of like cybersecurity, but for future, non-human agents that may also be able to take actions in the physical world. I’m interested in collaborating with students and faculty on projects relevant to AI safety, AI ethics, AI transparency, and other topics that could become relevant to how to structure a society with powerful AI systems in it. See, e.g., CS 294-149: Safety and Control for Artificial General Intelligence (Fall 2018),

My motivation for working in this area is that I think the impact of AI on the future of civilization is likely to be huge, and I think working to make that impact as positive as possible is a huge opportunity to do good for humanity. Many researchers are relatively shy to think and talk about extreme risks from AI (relative to extreme benefits, modest benefits, or modest risks). As a result, extreme risks from AI seem to be relatively neglected relative to their potential future importance, which is why I’ve chosen to focus on this area, and why I co-authored AI Research Considerations for Human Existential Safety (ARCHES). If it starts to look like extreme risks are getting enough research attention, I might someday switch back to thinking more about benefits.

Contact info

Since 2017 I have been using the following two email addresses to receive work and personal communications separately, as I have separate systems for managing each type of mail. Please use the appropriate address:

Press — “Press Inquiries (reply-all) for Andrew Critch” <press@acritch.com>. This address is for press inquiries, media appearances, interview invitations, or other requests for public comment. Unless otherwise indicated, please reply-all on threads with this address.

Work — “Andrew Critch (work-related email)” <work@acritch.com>. This address forwards to wherever I work. Please use it for professional, academic, or career-related matters.

Personal — “Andrew Critch (personal email)” <personal@acritch.com>. Please use this address for social/fun/personal stuff only (e.g., “want to have a picnic?”); not for work.

Other email addresses (private Google doc) — I maintain a few other private email addresses for projects/topics that I want to pay attention to in batches, and keep them listed in this document for a small number of correspondents.

In general I tend to avoid email and prefer telephone for most communications, especially scheduling decisions. See Phone calls for avoiding Byzantine failures below.

Instant messaging policy:
I generally find that instant messaging services (such as Facebook Messenger, Slack, WhatsApp, or mobile SMS services) are poorly used relative to their latency and attentional costs. So, by default I am not very responsive to instant messaging, except from close friends and co-workers who already have a good understanding of when I think instant messaging is useful and appropriate. If you send me an instant message that feels like a “to-do” item that I think would be better sent to one of my emails instead, or try striking up a distracting conversation with me in the middle of the workday, I might send you a link to this paragraph to summarize why I’m not responding 🙂

Calling to schedule things
Sometimes I say “Please call me at [my number] if you can schedule a time to talk about X” or something similar in an email or message. I then turn my attention to other activities and stop reading the thread, trusting that you will just call me or not accordingly. If you respond with a written message rather than a call and I don’t see your message, I’m sorry if that was confusing! Sometimes people think “call me” means “text me to discuss possible times for a call”, but for me “call me” literally means “call me”. Here’s a doc explaining a bit more: Why I often prefer using phone calls to schedule things

Video and phone calls for relatively “deep” conversations
I find video calls, and sometimes phone calls, are much better than emails for “deep” conversations. I consider a conversation “deep” (as opposed to “broad”) if we make a large number of serially dependent agreements about what we believe and what we want to discuss next. In other words, the conversation goes deep into the decision tree of what we’ll discuss and how we’ll discuss it. Phone and video calls are better than email for this because they have low latency and hence allow a larger number of joint decisions to be made in a given amount of time. Video is particularly nice because, e.g., if I’m pausing to thinking deeply about something then you can probably see that on my face, and vice versa. I usually prefer Zoom because it has the most reliable and flexible screen sharing functionalities.

Reverse bio

Prior to my work at CHAI, from 2015 to 2017 I was a research fellow at the Machine Intelligence Research Institute in Berkeley, CA. Before that, I worked as a quant / stock trader / data scientist for a financial trading firm in New York City. Before that, in September 2013 I took leave from a mathematics postdoc at the NSF Mathematical Biosciences Institute in Columbus, Ohio, to build and expand the Center for Applied Rationality (CFAR) in Berkeley. And just before that, in May 2013 I finished my PhD in mathematics under  Bernd Sturmfels at UC Berkeley, with mentor  Shaowei Lin. My thesis research was in algebraic statistics, with applications of  algebraic geometry to the study of  causal inference and machine learning models. I switched from pure algebraic geometry to algebraic statistics in fall 2011 because machine learning became so incredibly cool that I just can’t help thinking about it.

Also during my PhD between 2011 and 2012, I cofounded the Center for Applied Rationality in Berkeley, which is now a registered 501(c)(3) non-profit with 7 full-time employees and many volunteers. CFAR runs workshops teaching techniques for rational decision-making and action-taking, on the basis of empirical findings in cognitive science, and of course, data collected from previous workshops. It’s the best education per-dollar I’ve ever seen available, anywhere. Check it out.

Where I’m from and where I’ve been

I was organically grown in a tiny fishing village of about 250 people called Hillview in Newfoundland, Canada, where I learned how to be happy, and I proudly consider Newfoundland English to be my native language. I attended high school in Clarenville, Newfoundland (class of 2004), and went to Memorial University of Newfoundland for my undergrad (class of 2006). Then I started my PhD studying analytic geometry at the University of Toronto, but soon decided that I liked algebraic geometry more, so I got an MSc from Toronto instead (class of 2008), and started a new PhD in algebraic geometry at Berkeley. I moved to Rome in January, 2010, where I was a visiting scholar at Università degli Studi Roma Tre. I moved back to Berkeley in August, 2010, to New York City in April, 2014, and back to Berkeley for a third time in September, 2015… if you noticed a pattern here, you’re right: the Bay Area is definitely my favorite place to live so far 🙂