CSER at Catastrophic And Existential Risk Colloquium

The UCLA’s Garrick Institute held its first colloquium on Catastrophic And Existential Risk from March 27-29, with the opening keynote being delivered by CSER’s Executive Director Seán Ó hÉigeartaigh, and a lecture being given by CSER’s Catherine Rhodes. Ó hÉigeartaigh gave an overview of recent global progress in the field of existential risk, and presented some of the challenges for traditional risk analysis posed by risks of global catastrophic magnitude. Dr Rhodes spoke on risk management in systems of international governance, in particular outlining challenges associate with the governance of biological weapons. CSER adviser Professor Jonathan Wiener also spoke on psychological, political and policy challenges associated with rare or unprecedented risks of global catastrophic scale.

The Colloquium represented a welcome coming-together of communities doing leading work on quantitative risk assessment across risks of different scales, leaders in global risk areas such as nuclear war and bioterrorism, and the growing community of academic and research specialists in global catastrophic and existential risk as a class of risk. Alongside CSER, Oxford’s Future of Humanity Institute, the Machine Intelligence Research Institute, the Foresight Institute, and the Global Catastrophic Risk Institute were also represented We thank the Garrick Institute for organising and leading on the colloquium. A report or edited volume may be produced on the basis of the talks and workshop sessions.

New Research Affiliates

The Centre for the Study of Existential Risk is happy to welcome six new Research Affiliates. This affiliation recognises the contribution they have made to the success of the Centre. We are proud to have them onboard!

Research Affiliates

Seth Baum
Research Affiliate

View Profile.

Martina Kunz
Research Affiliate

View Profile.

Ryan Carey
Research Affiliate

View Profile.

William MacAskill
Research Affiliate

View Profile.

Nikita Chiu
Research Affiliate

View Profile.

Ellen Quigley
Research Affiliate

View Profile.


Dr Simon Beard – a New Generation Thinker

On the 16th of March 2017 the AHRC and BBC Radio 3 announced the ten academics at the start of their career whose research will be made into radio and television programmes for the BBC – the New Generation Thinkers.

CSER is proud to announce that Dr Simon Beard, one of our philosophers, has been selected to participate among hundreds of applications from early career academics. The final 10 ‘New Generation Thinkers’ were selected following a four-month process involving a series of day-long workshops at the BBC in Salford and London.

Prof Andrew Thompson, Chief Executive of the AHRC, says:” This scheme is all about helping the next generation of academics to find new and wider audiences for the research by giving them a platform to share ideas and allowing them to have the space to challenge our thinking. More than ever we need the new insights and knowledge that come from arts and humanities researchers to help us to navigate through the complexities of our globalized world and address the moral and ethical challenges of today and tomorrow.”

Dr Simon Beard will explore the ethical challenge of ensuring the long-term future of humanity. He has written on topics including population ethics, disability rights, assisted dying, imprisoned mothers, equal marriage, global justice and the meaning of life. He is currently fascinated by the question of when does morality require us to act straight away and when are we justified in waiting until tomorrow.

Read the full press release from BBC

Envisioning and Addressing Adverse AI Outcomes

On the weekend of 24-26 February CSER executive director Seán Ó Heigeartaigh and researcher Shahar Avin participated and chaired sessions in the Origins workshop “Envisioning and Addressing Adverse AI Outcomes” hosted at Arizona State University. The workshop was organised by CSER co-founder Jaan Tallinn, Microsoft technical fellow and research managing director Eric Horvitz and Origins director and theoretical physicist Lawrence Krauss. 

At the workshop participants explored adverse AI outcomes in a red team/blue team adversarial format, exploring scenarios contributed by the participants, including several contributions from CSER (for which we thank inputs from FHI and the broader AI safety community). The workshop was also attended by researchers from related organisations exploring existential risk, including FHI, FLI and MIRI. 

There were numerous actionable take-home messages, including further exploration of several intersections between AI safety and cybersecurity, and scope for novel regulation policies in specific domains such as automated finance and healthcare.

Bloomberg covered the workshop – read the article here

Biological Extinction Workshop at the Vatican

Sir Partha Dasgupta, Chair of CSER, is one of the organisers of the Vatican workshop on Biological Extinction. He is presenting on the workshop’s ‘Goals and Objectives’ and the ‘Summary and Conclusions’. He is also speaking about Why We Are in the Sixth Extinction and What It Means to Humanity, while our co-founder Lord Martin Rees is speaking about ‘Extinction: What it Means to Us’.

This workshop follows a previous Vatican workshop and report, 2014’s Sustainable Humanity Sustainable Nature: Our Responsibility, with which Sir Partha and Lord Rees were heavily involved.

The Guardian article about the 2017 workshop (Biologists say half of all species could be extinct by end of century) quotes Sir Partha Dasgupta as saying:

“We need to unravel the processes that led to the ills we are now facing. That is why the Vatican symposia involve natural and social scientists, as well as scholars from the humanities. That the symposia are being held at the Papal Academy is also symbolic. It shows that the ancient hostility between science and the church, at least on the issue of preserving Earth’s services, has been quelled.”

The crucial point is to put the problem of biological extinctions in a social context, he said. “That gives us a far better opportunity of working out what we need to in the near future. We have to act quickly, however.”

Bad Actors and AI – workshop

Photo credit: Future of Humanity Institute

On the 19th and 20th of February, the Future of Humanity Institute (FHI) hosted a workshop on the potential risks posed by the malicious misuse of emerging technologies in machine learning and artificial intelligence. The workshop, co-chaired by Miles Brundage at FHI and Shahar Avin of the Centre for the Study of Existential Risk, invited experts in cybersecurity, AI governance, AI safety, counter-terrorism and law enforcement. The workshop was jointly organised by the Future of Humanity Institute, the Centre for the Study of Existential Risk, and the Centre for the Future of Intelligence.

The attendees were invited to consider a range of risks from emerging technologies included automated hacking, the use of AI for targeted propaganda, the role of autonomous and semi-autonomous weapons systems, and the political challenges posed by the ownership and regulation of advanced AI systems.

The outputs of the workshop will be consolidated into a research agenda for the field over the coming months and made available to the research and policy communities to inform their future work prioritisation.

If you are a researcher interested in contacting the researchers regarding this project, you can email miles dot brundage at philosophy dot ox dot ac dot uk. (Media inquiries should be directed here.)

Dealing with Extremism – Professor David Runciman

14 February 2017

On Friday 3 February, Professor David Runciman gave a talk on “Dealing with Extremisim” the third of the popular Darwin Lecture Series, which this year is co-convened by CSER Research Associate Julius Weitzdoerfer. Professor Runciman argued that not all conspiracy theorists are extremists; but that almost all extremists are conspiracy theorists.

The talk can be viewed on the Darwin College Lectures website.

The ‘Extremes’ lecture series take place every Friday during Lent term (January to March). The lectures are given at 5.30 p.m. in The Lady Mitchell Hall, Sidgwick Avenue, with an adjacent overflow theatre with live TV coverage. Each lecture is typically attended by 600 people so you must arrive early to ensure a place.

The next lectures are:


Extreme Rowing – Roz Savage MBE, Ocean Rower, Yale University.

Friday 10 February 2017

Extremes of the Universe – Professor Andy Fabian, University of Cambridge.

Friday 17 February 2017

Extreme Politics – Professor Matthew Goodwin, University of Kent.

Friday 24 February 2017

Extreme Ageing – Professor Sarah Harper, University of Oxford.

Friday 03 March 2017

Reporting from Extreme Environments – Lyse Doucet, BBC.

Friday 10 March 2017

Call for Papers and Responders: Risk, Uncertainty and Catastrophe Scenarios

14 January 2017

Call for Papers and Responders: Risk, Uncertainty and Catastrophe Scenarios

Workshop on Climate Ethics and Climate Economics

May 9th & 10th, Centre for the Study of Existential Risk

Scholars have warned that there is an uncertain chance of runaway climate change that could devastate the planet. At least since Hans Jonas’s The Imperative of Responsibility, some have argued that even low-probability existential risks should be treated in a fundamentally different way. How should we act when we believe that there is a chance of a catastrophe, but cannot make reliable probability estimates? How much should we worry about worst-case scenarios? What should we do when experts disagree about whether catastrophe is possible?

These are some of the questions we will be posing at the fifth of six ESRC-funded workshops exploring issues where the ethics and economics of climate change intersect. It will be held at the University of Cambridge’s Centre for the Study of Existential Risk.

We are seeking both paper givers and discussants from philosophy, economics and other disciplines. Funds are available to cover accommodation and internal travel expenses for up to three research students and early-career researchers. Papers, where available, will be circulated before the workshop.

Those wishing to present a paper should submit a 500-word abstract by 24th March to Simon Beard (sjb316@cam.ac.uk). Anyone interested in serving as a discussant should send an expression of interest by the same date. If applying for funding, please indicate that you are a student, or the year that you received the PhD.

Wired UK article

13 February 2017

Wired Front Cover

The Centre for the Study of Existential Risk is featured in a piece on existential risk in Wired UK, alongside the Future of Humanity Institute, the Global Catastrophic Risk Institute, and the Centre for the Future of Intelligence.

Wired UK Existential Risk article