The - legal and general - debate around artificial intelligence systems is more intense than ever at a global level. Especially in the E.U., the published Regulation proposal on artificial intelligence has created a clearer picture of how to deal with AI systems at a European level, by categorizing them based on their potential risk for fundamental rights and freedoms, although its finalization and mandatory application is not expected before 2026.

The AI Act will be applied in parallel with the General Data Protection Regulation (EU) 2016/679, with regard to AI systems collecting and processing personal data. However, the very operational nature of AI systems presents a significant challenge in obtaining the consent of data subjects.

According to the General Data Protection Regulation (GDPR), the processing of personal data, as long as there is no other legal basis for such processing, is lawful only if and as long as the data subject has consented to the processing of their personal data for one or more certain purposes.

Consent should be obtained before the processing by the Controller begins, in an understandable and easily accessible form, with clear and simple wording, without abusive clauses. For the consent to be considered asked lawfully, the data subject should be informed at least of Controller’s identity and of the purposes of the processing for which the personal data are requested. Consent should not be considered as lawfully provided if the data subject does not have a genuine or free choice or is unable to refuse or withdraw consent without prejudice.

The key issue identified is whether the Controller can fully inform the data subject of the purposes of the processing, given that AI systems have the ability –to an important extent- to proceed autonomously in the establishment of "invisible" correlations between given personal data and even in creating new data concerning the data subject. At the same time, AI systems continuously reuse given data, redefining the purposes of the processing, making the challenge of properly informing the subject and obtaining consent even greater. Especially with respect to machine learning systems, the involvement of the Controller in the formation of new correlations and ultimately in the creation of new data is considered minimal to non-existent. This reduces the importance of consent, given also the unpredictability of ​​ AI systems’ outcomes.

Another important issue is also the case where the data subject withdraws their consent. Such withdrawal of consent does not have a retroactive effect; however, it entails the obligation to the Controller to completely delete the subjects. Exercising this right, however, may pose a threat to the development of AI, which actually relies on data to train itself. In addition, there may be a distortion in the representativeness of data, which could lead to a “malicious” training of the AI ​​system and therefore to biased decision-making. For this reason, it is important to technically ensure that data deletion does not affect the representativeness of data. In any case, the complete deletion of a subject's data is not possible, since, as mentioned above, their data is reused constantly, producing new data related in whole or in part to the subject.

In conclusion, the intended transparency provided for in the General Data Protection Regulation appears to be the biggest challenge in terms of the collection and processing of personal data by AI systems. Especially the lawful reception of consent is a major challenge, which seems hard to be achieved by Data Controllers, posing a significantly high risk of affecting the proper training of AI systems.

As published on athensvoice.gr