REAL ML 2019 workshop

Date

October 2, 2019

Location

Berlin, Germany

Anna Bacciarelli

Researcher and advocate

REAL ML, Human Rights Watch

William Isaac

Principal Scientist and Head of Ethics Research

Google DeepMind

Seda Gürses

Researcher

TU Delft

Showing 3 of 11 ()

Abstract/Summary

A residential workshop for public interest researchers investigating the social impacts of algorithmic systems

Researching Algorithmic Decision-Making Systems

A collaborative three-day workshop for 30 public interest researchers investigating the social impacts of algorithmic systems

Our first REAL ML workshop came about through a shared curiosity about and frustration with understanding the impacts of new algorithmic and automated technologies on the societies and communities they were being deployed in. We wanted to explore how established methodologies – from qualitative data science and statistical approaches to human rights documentation – could help us adequately capture the scale and impact of these technologies in the wild.

With leadership from our Co-Chairs and Steering Committee, and support from MacArthur Foundation and Open Society Foundations, we held a three-day residential workshop in Berlin, Germany, for researchers from a range of disciplines and skillsets to explore how we could pool resources and share feedback to help us collectively advance our research methodology toolkit.

Our key aims? To collaboratively advance the standard of critical research and advocacy on algorithmic decision-making systems by:

Accelerating and expanding the development of promising new multidisciplinary methodologies

Fostering a community of like-minded public interest researchers and practitioners

Supporting promising research projects that can contribute to critical public discourse about the development and use of algorithms.

Program

Berlin, 2-4 October 2019 at the Michelberger Hotel

No panels! This was a space for everyone to learn from one another in a non-hierarchical structure. We split our time between plenary sessions discussing The workshop will debate and explore methods for researching the human rights and public interest impacts of algorithmic decision-making systems, with most of the time devoted to working on live projects.

The aim of this event is to bring together experts and share our problems and potential solutions. Many of us are struggling with similar issues, from hurdles to accessing data, to ensuring our work is truly participatory and inclusive.

We want this to be an open and collaborative space to share and discuss approaches, looking at what has worked for each of us – and what has not. Over the course of the three days, we will foster a trusting, problem-solving environment where delegates can contribute to one another’s research. The workshop is inspired by the Citizen Lab’s Summer Institute model and has benefited from advice provided by Citizen Lab staff.

This is a small pilot event for around 30 experts. All participants will be expected to be present for the full three days. If the workshop is successful, we hope to expand and build on it in the future.

Workshop structure

Day one (whole group)

Introductions
Learnings from other disciplines and fields: tools and methods we can apply from other domains
Standards for ethical research: how do we ensure research is truly ethical?
Divide into small groups to begin collaborative discussions and work on live projects
Dinner with the whole group

Day two (small groups)

Whole day devoted to project work within small groups
Lunch and break at your leisure

Day three (small groups/whole group)

Continue project work within small groups
Present back to the wider group
What we have vs. what we need: building a wishlist of accessible info, tools and standards

Is it for me?

Are you currently working on an active public interest research project that is grappling with the impact of algorithms on society? If yes, we want to hear from you! Whether you are an academic, advocate or an investigative journalist – this will be an interdisciplinary event, and your expertise and skills will contribute to a lively group from a range of backgrounds. You will find this event valuable if you are interested in both learning and actively contributing to solving challenges other participants are confronting with their research.

One of the primary aims of this conference is to counter existing norms within industry and academic conferences to minimize the lived experiences and interdisciplinary research by practitioners from historically marginalized groups. To this end, we seek to invert existing power structures by especially welcoming applications from individuals from historically marginalized groups or those investigating the impact of algorithmic systems on marginalized or vulnerable people. As we are hosting this pilot workshop in Europe, we also especially welcome applications for projects that are examining unique social, technical or legal challenges around critical research of these issues in the region.

Research focused on the impacts of algorithmic decision-making systems often involves teams of people. However, given that this is a small event, we are allowing up to two delegates to apply to participate under the same project. There is a place to indicate this on the application. Applicants will be considered on an individual basis, and each applicant should indicate their unique role to play in the project on the application form (we have limited space so want to avoid duplication of roles).

People

Co-Chairs

Anna Bacciarelli, Amnesty International (Co-Chair)
William Isaac, Open Society Foundation Fellow (Co-Chair)

Steering Committee

Frederike Kaltheuner, Privacy International
Jeff Kao, ProPublica
Kristian Lum, Human Rights Data Analysis Group
Irene Poetranto, Citizen Lab, University of Toronto
Rashida Richardson, AI Now Institute, New York University
Ashkan Soltani, independent researcher and technologist
Michael Veale, University College London

Funders

MacArthur Foundation, Technology in the Public Interest Program (Eric Sears)

Open Society Foundations, Information Program (Becky Hogge)

All Events