The EU AI Act enumerates six types of prohibited practices when developing, deploying or using AI systems. These are - (i) manipulative or deceptive practices, (ii) exploitation of user’s vulnerability, (iii) unrestricted use of social scores to classify users, (iv) predictive policing only using AI, (v) scraping facial recognition data indiscriminately, (vi) inferring emotions, (vii) categorising users based on biometrics , and (viii) real-time use of biometrics in public.
-
Manipulative, subliminal and deceptive practices
=> Practices that employ manipulative or subliminally deceptive practices that intend to affect decision making of a person are prohibited.
=> These practices must be deployed with the intent or effect of imparting an individual’s decision making ability without their awareness.
=> The practices must distort the person or a group of persons to make decisions that they would not have taken otherwise.
=> Autonomy, decision-making ability and free choice are relevant factors to determine whether a person is impaired.
Permitted practices: Transparent practices that are consciously perceived by the user; Practices that do not cause harm to users.
-
Exploiting vulnerability of users
=> Practices that exploit the vulnerabilities of users such as age, disability or a specific social or economic situation.
=> The effect of such exploitation should be to materially distort the user’s behaviour or that is likely to cause significant harm to the users.
=> An example of such a practice is AI models exploiting the decision making ability of elderly users by influencing their financial decisions.
No permitted practices. Applicable to both providers and deployers of AI systems.
-
Classifying users or groups of users based on a social score
Practices that evaluate a social score of users based on their social behaviour over time or certain known, provided or inferred personality characteristics.
The social score must be used to (i) detrimental or unfavorable treatment of users in context other than for which the score was determined or (ii) detrimental or unfavorable treatment of users disproportionate to their behaviour or actions.
Permitted practices: Lawful evaluation of users that is carried out for a specific purpose.
-
Predictive policing based solely on personality traits or characteristics
Deployment or use of AI systems for risk assessment of natural persons to assess or predict risk of a person committing crimes solely based on profiling or personality traits or characteristics is prohibited.
Policing solely based on following characteristics without human assessment of objective facts indicating involvement in an offence is prohibited: profiling, personality traits or characteristics, nationality, place of birth, place of residence, number of children, level of debt or type of car.
Permitted practices: AI systems used to assess financial fraud, risk analytic tools based on known trafficking routes
-
Creating or expanding facial recognition databases
AI systems that scrape facial images of individuals from the internet or CCTV cameras in an untargeted manner to create or expand facial recognition databases are prohibited.
Such untargeted scraping may lead to surveillance and gross violation of fundamental rights including the right to privacy of the individuals.
Permitted practices: scraping facial images for training models, labelling biometric datasets to ensure adequate representation and prevent discrimination.
-
Inferring emotions of individuals at workplace or educational institutions
AI systems that infer emotions of individuals at workplace or at educational institutions are prohibited. Deploying AI systems for the purpose of inferring emotions of individuals in a similar manner is also prohibited.
Prohibition stems from limited reliability, lack of specificity and limited generalisability of the models. Further, at work or in educational institutions, the imbalance of powers can lead to detrimental or unfavorable treatment of individuals.
Permitted practices: AI systems that are used for medical or safety reasons including therapeutic uses.
-
Categorising natural persons based on biometric data
AI systems that deduce or infer the race, political opinions, trade union membership, religious or philosophical beliefs, sex life or sexual orientation of an individual from their biometric information are prohibited.
Biometric information includes data regarding fingerprints or face of the individual.
Permitted practices: Labelling or filtering of lawfully acquired biometric datasets, categorising biometric data in the field of law enforcement such as sorting images based on eye color or hair color.
-
Real-time biometric identification systems in publicly accessible spaces
Use of real-time biometrics for law enforcement in publicly accessible spaces is prohibited subject to exceptions.
The practices are prohibited as real-time biometric monitoring is intrusive and evokes a feeling of constant surveillance. This may indirectly affect the exercise of freedom of assembly and other fundamental rights. Further, technical inaccuracies can lead to biased results leading to discrimination of individuals.
Permitted practices: Real-time biometrics can be used for policing if it is for one of the following purposes: . => Targeted search for specific victims - abduction, trafficking, sexual exploitation or missing persons => Preventing specific and genuine threat to an individual or a terrorist attack => Localisation of accused for offences punishable with imprisonment of 4 years or more or as specified in Annex II. The processing is subject to General Data Protection Regulations.