Building a responsible approach to data collection with the Partnership on AI
At DeepMind, our commitment to safety and ethics is unwavering. Everything we do is aligned with our Operating Principles, ensuring the highest standards are met. When it comes to data collection, we understand the importance of responsible practices. That’s why, in collaboration with the Partnership on AI (PAI), we have developed standardized best practices and processes for human data collection that prioritize ethics and integrity.
Human data collection
Three years ago, we established the Human Behavioural Research Ethics Committee (HuBREC) to oversee research involving human participants. This committee ensures the dignity, rights, and welfare of individuals involved in studies, particularly in the context of human-AI interactions. Additionally, as the AI community explores data enrichment tasks like data labeling and model evaluation, we recognize the need for clear guidelines and governance to uphold ethical standards.
Data enrichment tasks often involve paid contributors on crowdsourcing platforms. Addressing concerns related to worker pay, welfare, and equity is crucial as the demand for data enrichment grows. This evolution necessitates stronger guidance to protect the interests of all stakeholders.
To fulfill our commitment to AI safety and ethics, we are dedicated to upholding best practices and contributing to advancements in fairness, privacy, and responsible data collection.
The best practices
Following PAI’s recent white paper on Responsible Sourcing of Data Enrichment Services, we partnered to develop practical guidelines for data enrichment practices. These guidelines encompass five key steps for AI practitioners to enhance working conditions for individuals involved in data enrichment tasks.
- Select an appropriate payment model and ensure all workers are paid above the local living wage.
- Design and execute a pilot before initiating a data enrichment project.
- Identify suitable workers for specific tasks.
- Provide clear instructions and training materials for workers.
- Establish effective communication channels with workers.
Our collaborative efforts have resulted in refined policies and resources, with input from various internal teams. Piloting these practices has demonstrated their effectiveness in improving study design and execution, benefiting both researchers and participants in data enrichment tasks.
For more insights on responsible data enrichment practices, refer to PAI’s case study on how we implemented these guidelines at DeepMind. Additionally, PAI offers resources for AI practitioners looking to enhance their processes.
Looking forward
While best practices guide our work, we acknowledge the need for ongoing vigilance to ensure participant and worker welfare in research. Our human data review process enables us to continually assess and mitigate risks on a project-by-project basis, reinforcing our commitment to ethical standards.
By sharing our experiences and lessons learned, we aim to inspire dialogue and collaboration across industries to elevate standards for responsible data collection. Together, we can build a more ethical and transparent future for AI development.
Learn more about our Operating Principles and our dedication to ethical AI practices.