# Data protection ## Data protection ### Why data protection is important in GenAI Generative AI applications with potential for development impact typically deal with sensitive topics. Livelihoods, vulnerabilities, health-related questions, children's data in education and beyond: the list is long and substantial. Where users reveal details of their situation in these fields, registration data and chat logs record and process sensitive personal data. As responsible stewards, the providers of GenAI systems need to safeguard and protect that data carefully. But instead of being a burden, responsible data handling is an opportunity to build the trust necessary for open and honest user evaluation. ### Requirements depend on local context Compliance with local legal requirements is the foundation for processing personal information. These requirements differ substantially and it is thus impossible to list needs that are relevant in any setting. Instead, this section gives a brief overview of basic considerations for professional practice in handling sensitive data. It is neither legal advice nor a complete listing of advisable practices, but simply a starting point for assessing what responsible data protection means in a particular context. Since human subjects research is integral to evaluating generative AI applications, the need to involve institutional review boards and similar bodies should also be considered where appropriate. ### Minimum Data Protection Practices In addition to locally relevant data protection requirements – which often include standard principles such as data minimization, purpose and storage limitations – foundational steps towards responsible and trustworthy handling of sensitive personal information should include: | Practice | Description | | -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Data protection impact assessment (DPIA)** | A structured review of privacy risks should be conducted before deployment, and also whenever the system changes substantially. A DPIA includes documentation of data flows, the legal basis or other authorization for processing, potential harms such as re-identification or model memorisation, and the controls that mitigate them. | | **User rights** | Where required by regulations and perhaps also as a matter of transparent and fair practice, users should have access to the data held on them, the right to correct mistakes, and the option of ending their participation in consent-based activities (including data deletion). User-friendly processes that implement these rights need to be integrated into system design. | | **ICT security standards** | Although safe practices are discussed at various points of the Playbook, a general security posture that secures data and systems against internal and external threats is an essential part of data protection. Established frameworks such as the ISO 27k family or the NIST Cybersecurity Framework set out the core requirements. | | **Legal advice** | Jurisdiction-specific legal counsel should be sought before data collection even begins. The requirements vary strongly across regimes and cross-border transfers to upstream LLM providers introduce a second layer of complexity. The fast evolution of data protection and AI regulation suggests that compliance may need to be treated as ongoing rather than one-off. | ### Consent is not a tick-box exercise A cautionary note is due for situations where consent to personal data processing – including items such as chat logs and registration information – is taken to be the legal basis. Development-focused AI applications often aim to support people in considerable need who may not have the ability to withhold or withdraw consent, or who may not be in a position to assess the processing fully. The economically vulnerable, illiterate, children, or older adults can be examples of such groups. In such situations, the AI provider should recognise that it is their duty to mitigate risks and take protective measures, foregoing performative consent in favour of user-centric intervention design. ### Cross-border transfer and third-party access The concentration of GenAI model providers and processing infrastructure in a few countries often implies the transfer of sensitive data across borders. Apart from verifying that this is permissible under local regulations, it also entails a duty to verify that the data will be processed in a compliant manner abroad. A similar need arises where third parties access, store and process personal data. Providers of GenAI systems must verify that they will uphold the same or higher custodial standards than the own organisation. Although verification may be burdensome, sensitive user data is as worthy of protection abroad and by others than by the direct system provider. ### Selected data protection considerations by evaluation level Data protection considerations reveal themselves in different ways across the Playbook's evaluation levels, including but not limited to: | Evaluation level | Key considerations | | --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Level 1 – Model (System) Evaluation** | Training data may itself be sensitive and require filtering or scrubbing to avoid leakage risks. Bias audits reliant on access to protected characteristics such as ethnic background pose additional governance challenges. | | **Level 2 – Product evaluation** | Usage tracking data can be sensitive, and the same is true for session-level analytics and potential metadata, such as location. | | **Level 3 – User evaluation** | Chat logs reveal thoughts, misconceptions, personal circumstances, representing the most sensitive interaction data. PII scrubbers for LLM inputs/outputs can partly mitigate resulting risks, but not eliminate them. Responsible data practices enable product optimisation as they can support the trust that is required to evaluate some L3 aspects adequately and accurately. | | **Level 4 – Impact evaluation** | Administrative data linkages, such as health records, exam scores, and social registries, represent data risks, including re-identification risk for de-identified or pseudonymized data. Primary data collection for evaluation may be held separately, but needs to be treated as carefully. | --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://eval.playbook.org.ai/level-linkages/linkage-across-levels/data-protection.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.