1. Introduction: artificial intelligence in law enforcement
1.1. Scope and research questions
Artificial intelligence (AI) seems to be disrupting most fields of life in one way or the other. For the field of law enforcement, as in many other sectors, it is nothing new to use the potentials of electronic computation, data analysis and – more generally – integrating scientific advancements into police investigations (Fontes & Perrone, Reference Fontes and Perrone2021; Navarro, Reference Navarro2023). From data scraping on the internet and lie detectors to bulk telecommunication surveillance, biometric identification or instant translation (Donohue, Reference Donohue2014; Edri, 2020; Khder, Reference Khder2021): what is possible with AI is a next step in a long-lasting technological development in computer science rather than the quantum leap, or even an “AI revolution” (Awwad, Reference Awwad2024; Chataut et al., Reference Chataut, Nankya and Akl2024; Makridakis, Reference Makridakis2017), that stakeholders in industry and media seem to suggest at times (Cebral-Loureda et al., Reference Cebral-Loureda, Rincón-Flores and Sanchez-Ante2023; Chowdhury & Sadek, Reference Chowdhury and Sadek2012). Yet, many new AI tools are currently flooding the market, and some have the potential to modernize and improve the work of police and other security forces, while others allow surveillance in a dimension that might come in conflict with digital human rights and the rule of law (Berk, Reference Berk2021; Casaburo & Marsh, Reference Casaburo and Marsh2024; Cataleta, Reference Cataleta2021; Muller, Reference Muller2020; Završnik, Reference Završnik2020).
This article takes a closer look at the instrument of regulatory sandboxing in AI regulation (Charisi and Dignum, Reference Charisi, Dignum, Régis, Denis, Axente and Kishimoto2024; Ruschemeier, Reference Ruschemeier and Steffen2025) – the central measure in support of innovation laid down in Chapter VI of the AIA. The legal instrument aims to “ensure a legal framework that promotes innovation, is future-proof and is resilient to disruption” by allowing the “development and testing of innovative AI systems under strict regulatory oversight before the systems are placed on the market” (Recital 138 AIA). The focus of this article lies on the potential of regulatory sandboxes to develop legally compliant, technologically sovereign and democratically controllable AI systems for law enforcement agencies. Recitals 138, 139 AIA give the first impression that regulatory sandboxes are mainly drafted as a tool to test innovative products of SMEs and start-ups in an early stage (Boura, Reference Boura2024; Yordanova, Reference Yordanova2022) and reduce their administrative and regulatory burden. Yet, the legislator has foreseen the potential of this “limited regulatory space” (DeMeola, Reference DeMeola2021) as a protected environment – a “balanced solution” (Christoph, Reference Christoph2023) – for law enforcement agencies to co-create technical tools, potentially alongside precise legal provisions, including necessary safeguards and technical as well as organizational measures of data protection (Ballestrem et al., Reference Gausling, Ballestrem, Bär, Gausling, Hack and von Oelffen2020; Helminger, Reference Helminger2022; Jandt, Reference Jandt2017; Müller-Quade & Houdeau, Reference Müller-Quade and Houdeau2023) and supervisory structures. By analyzing the provisions of the AIA, I want to find out whether the instrument of regulatory sandboxing – as Christoph (Reference Christoph2023) has put it the context of US law – can contribute to bringing “coherence and discipline to the use of new technology in law enforcement and criminal proceedings” where the status quo of procuring private sector solutions “has led to uncertainty, inconsistency, and the danger of manifest injustice within local, state and federal justice systems” (Christoph, 2023). This article analyzes the hypothesis that regulatory sandboxing can be an effective way out of the conflicting paths of (quickly) modernizing policing by allowing the use of AI systems on the one hand and drafting legal provisions and implementing AI systems that comply with high fundamental rights standards on the other. It takes the discussion around biometric internet scans, as introduced as part of a federal “security package” in the Federal Republic of Germany (section 1.3. of this article), as a case study to outline the advantages of the instrument of regulatory sandboxing in the field of law enforcement compared to hasty legislative processes.
1.2. AI systems in law enforcement under rule of law and the AIA
In the future, those who provide (Art. 3[3] AIA) or deploy (Art. 3[4] AIA) AI systems in the EU will have to comply with product safety provisions of the AI Act. This “uniform legal framework (…) for the development, the placing on the market, the putting into service and the use of artificial intelligence systems” ultimately aims at “ensuring a high level of protection of (…) safety, fundamental rights (…), including democracy, the rule of law” (Recital 1[1] AIA). That the use of technology needs to be supported by a legal basis is nothing new in the field of public administration. Already under the rule of law, law enforcement agencies are not free to integrate technology in their investigative toolbox (Greenstein, Reference Greenstein2022; Hesse, Reference Hesse1999; Kommers and Miller, Reference Kommers and Miller2012;Weber, Reference Weber2008). The AI Act follows this basic understanding of fundamental rights doctrine.
The AIA follows a risk-based approach and densely regulates the category of “high-risk AI systems” (Art. 6 AIA). According to Recital 46 AIA, such systems should only be placed on the Union market, put into service or used if they comply with certain mandatory requirements in order to ensure that they “do not pose unacceptable risks to important Union public interests.” Certain AI systems used by law enforcement agencies are considered high-risk AI systems (Art. 6[2], Annex III [6][a-e] AIA) due to their potential impact on fundamental rights of citizens. Also, the field of “migration, asylum and border control management” widely falls into the high-risk category (Art. 6[2], Annex III[7] AIA). The lists of Annex III, Nr. 6 and 7 show different forms of AI systems that are considered high-risk: from assessing “the risk of a natural person becoming the victim of criminal offences” (Nr. 6[a], polygraphs (Nr. 6[b]), “assess personality traits and characteristics or past criminal behaviour “(Nr. 6[d]) to AI systems in migration meant for “detecting, recognising or identifying natural persons” (Nr. 7[d]).
Apart from whether an AI system is banned or considered high-risk according to the AIA, the limits of the national constitution must be respected (Almada and Petit, Reference Almada and Petit2025; Masala, Reference Masala2024; Micklitz et al., Reference Micklitz, Pollicino, Reichman, Simoncini, Sartor and De Gregorio2021), especially if the use of technology has an impact on fundamental rights of citizens. Every encroachment of fundamental rights must be based on a legally certain provision in national law that is proportionate. The AIA acknowledges the necessity of a legal basis by clarifying in Annex III(6) that their use must additionally be “permitted under relevant Union or national law.” Against this background, it becomes apparent that the use of AI systems by law enforcement agencies will be strictly regulated under the AIA based on a complex interplay of national and EU legislation in the field of AI, data protection and national security law.
1.3 The German “security package” as an example for political and legal debates around the legality and constitutionality of high-risk AI systems in the realm of law enforcement
As an example, I will introduce a case from Germany that has attracted significant public attention. For decades, police forces were not able to track down the alleged leftist terrorist Daniela Klette, who had gone underground in the 1990s and is suspected of having committed a series of robberies afterwards (The Times, 2023). At the end of 2023, a team of journalists was able to find new evidence about Klette’s whereabouts in the course of their research for a podcast: They fed the official mugshot being displayed at airports and train stations across the country into the AI-Tool PimsEyeFootnote 1 and found her face on pictures of a Capoeira group in Berlin on Facebook (Bovermann et al., Reference Bovermann, Fink and Mutter2024; Großekemper et al., Reference Großekemper, Höfner, Rosenbach and Wiedmann-Schmidt2024). Police forces followed the lead of those journalists and were able to arrest Klette a few weeks later. It turned out she had lived and freely moved in the heart of Berlin for years without ever being recognized. Security agencies felt embarrassed and got the impression that the success of the journalists was dancing on their noses. As a result, a public debate sparked: How can it be possible that a small group of journalists can track down an alleged terrorist gone underground with a freely available software application, while police have been unable – and not allowed – to do so?
And: What are the limits of security agencies in the EU to use AI systems that are available on global markets? Legal questions surrounding the use of AI software, especially by some non-EU companies, have arisen in manifold ways. In the case of Palantir, a series of judgments of the German constitutional court (Bundesverfassungsgericht) have circled around the question if and under which circumstances law enforcement authorities in the states of Hessen, Bayern and Hamburg (Bäuerle, Reference Bäuerle2024) are allowed to automatically analyse large databases of state and federal police authorities; with Palantir also being used in Nordrhein-Westfalen. The law enforcement authorities had procured a software application from Palantir and tried to adjust it to the procurement conditions. So far, however, no constitutional possibility has been found, and the Bundesverfassungsgericht asked for adjustments (see Judgment of the First Senate of 16 February 2023, file numbers: 1 BvR 1547/19, 1 BvR 2634/20). Similarly, in the case of Daniela Klette, law enforcement authorities must have felt inclined to procure Clearview AI or PimsEyes. However, Clearview AI had already been heavily fined by Italian and Dutch data protection authorities for stark infringements of data protection law (Der Hamburgische Beauftragte für Datenschutz und Informationsfreiheit, 2020; Martini & Kemper, Reference Martini and Kemper2023a; Martini & Kemper, Reference Martini and Kemper2023b; Rezende, Reference Rezende2020; Jung and Kwon, Reference Jung and Kwon2024).
Working with selected SME or larger companies in the protected experimental environment of a regulatory sandbox would allow for a path that follows the political goal of “digital sovereignty”, now often used as a shorthand for an ordered, value-driven, regulated and therefore reasonable and secure digital sphere (Bellanova et al., Reference Bellanova, Carrapico and Duez2022; European Commission, 2023; European Parliamentary Research Service (EPRS), 2020; Floridi, Reference Floridi2020; Heidebrecht, Reference Heidebrecht2024; Pohle and Thiel, Reference Pohle and Thiel2020). Both the market surveillance authorities (Art. 70 AIA), authorities for the protection of fundamental rights (Art. 77 AIA), and the law enforcement authorities would gain insights into the technical design, configuration and main components of the used AI systems – instead of having to deal with proprietary systems of large tech companies which have been developed in a different regulatory environment. For a regulatory sandbox for AI systems in the field of law enforcement, the federal and state levels could encourage certain companies – especially SME and European-based companies – to figuratively throw their hat in the ring – with the additional (implicit) prospect to later have a top-runner position in procurement procedures to acquire suitable AI systems for the police.
Interestingly, the arrest of Klette coincided with the final negotiations of the European Union’s landmark AI Act. In the final trilogue, Members of the European Parliament had pushed for a ban on biometric internet scans by PimEyes, Clearview AI and the likes by implementing a ban in Art. 5 lit. e AIA (accompanied by Recital 43), forbidding the “placing on the market, the putting into service for this specific purpose, or the use of AI systems that create or expand facial recognition databases through the untargeted scraping of facial images from the internet or CCTV footage.” Nonetheless, the German Federal Government – under the influence after a terrorist attack in the city of Solingen (Euractiv, 2024; Kurmayer, Reference Kurmayer2024) – pushed for a “security package” (Sicherheitspaket) to react to threats of supposed Islamist terrorism. The ministerial draft included the legal competence of migration authorities to compare a photograph taken of foreign nationals upon entry biometrically with publicly accessible personal data from the internet by means of an automated data processing application in case there are no other means of identification.Footnote 2 Similar provisions implemented for the Federal Police (Bundeskriminalamt and Bundespolizei) to use the same technology under certain additional requirements – expanding the possibility to not only the face, but also the voice of a person.Footnote 3
The draft law sparked a public debate, including an outcry of civil society organizations who had been campaigning against all forms of biometric identification in public space for years (Reclaim Your Face, n.d.; Reuter, Reference Reuter2024). Under the pressure of the federal government, which wanted to show a swift reaction to the terrorist attack, and after intense negotiations in parliament that added additional safeguards, the Bundestag finally approved the two laws of the Sicherheitspaket.Footnote 4 An expert hearing on the first draft had previously led to media coverage, after several experts held the law in breach of German constitutional law, GDPR and also the AIA. The Federal Commissioner for Data Protection and Freedom of Information, Prof. Dr. Louisa Specht-Riemenschneider, criticized that the legal provisions did not even cursorily sketch the technical functionality and implementation of biometrical internet scans, thus not setting up specific safeguards and limitations, opening law enforcement a too wide scope that can lead to unproportionate infringements of the right to informational self-determination (Bundesbeauftragte für den Datenschutz und die Informationsfreiheit, 2024). While the competence of migration authorities to carry out biometric scans of asylum seekers – particularly in relation to internet data – came into force (with substantial changes during the parliamentary process), the legal provisions concerning the federal police were ultimately rejected by the second chamber of federal legislation, the Bundesrat. The opposing conservative party declared that – in their opinion – the legislative changes in the Bundestag, the Sicherheitspaket had been “softened and watered down” (Welt, 2024).
These developments in recent legislation in Germany show: if law enforcement agencies want to implement novel AI tools in areas sensitive to digital human rights, it is vital not to observe the drafting of legal provisions and their technical implementation as something separate. A provision that is intended to permit the use of an AI system that is not precise enough to give legal clarity, but rather leaves the details of implementation largely to the law enforcement agencies themselves, will necessarily get in conflict with the principle of statutory reservation.
The new provision of Section 15b Asylgesetz – with the competence of migration authorities to conduct biometric scans with internet data – will most likely end up in front of the Federal Constitutional Court (Bundesverfassungsgericht) or even the European Court of Justice (for infringement on the ban in Art. 5 lit. e AIA, see for more details section 5 of this article). In the worst-case scenario (from the perspective of law enforcement agencies), they will have by then acquired an (expensive) AI system infringing upon digital human rights, and would need to restart from scratch.
The case study raises the question: Could the instrument of a regulatory sandbox prevent the possibility of implementing digital technologies in the field of law enforcement that bear the risk of falling short of the requirements laid down in the AIA and fundamental rights doctrine?
2. Regulatory sandboxes in the AIA and their potential in the field of law enforcement
Creating AI systems in the field of law enforcement that are in accordance with fundamental rights is admittedly not the main field of application that the European legislator had in mind whilst drafting “Chapter VI: Measures in Support of Innovation.” Rather, a different intention and target group was the driving force (as Art. 57[9] AIA clearly points out): Regulatory sandboxes are mainly designed for private sector organizations and shall serve both as a fastlane towards a successful conformity assessment, and thus legal compliance, for certain products as well as an instrument to support SMEs and start-ups (“removing barriers for SMEs, including start-ups,” Recital 139 S. 1 AIA) to better deal with the regulatory burden of the AIA.Footnote 5 Since high-risk-AI-systems (Art. 6 AIA) need to be in compliance with the complex requirements of Chapter III – among them Data Governance (Art. 10 AIA), Technical Documentation (Art. 11 AIA) or Human Oversight (Art. 14 AIA) – regulatory sandboxes can reduce the regulatory burden specifically for their providers and deployers. Many of the fields of application in Annex III ultimately focus on public sector deployment of high-risk AI systems. The field of law enforcement is thus certainly not a “blind spot” in the provisions for regulatory sandboxes – yet rather one partly foreseen sector of potential deployment of AI-systems that have gone through the process of a regulatory sandbox.
2.1. Joint regulatory sandboxes in law enforcement and the overall intention of chapter VI
As far as regulatory sandboxes are supposed to open a controlled field of experimentation and testing under the supervision of competent authorities in the “pre-marketing phase” (Recital 139 S. 1 AIA), they can also be interpreted as a vehicle that widens the playbook for public administration on procuring market-ready products that are designed alongside with European values and principles as laid out in the European Charter of Fundamental Rights.Footnote 6 Before this background, regulatory sandboxes can open a sweet spot from both the perspective of digital sovereignty and digital human rights: Instead of being inclined to procure “quick fixes” for the intended investigation methods from other regulatory environments (e.g., biometrical scans of suspected persons with internet data with Clearview AI or PimEyes – or cross data-basis analysis and visualization with Palantir products), a regulatory sandbox could bring law enforcement authorities and European SMUs together to develop a tailored solution.
A secondary goal of regulatory sandboxes has a rather introspective angle for institutions of AI governance and legislation: With the instrument, a gap in the policy circle can be closed as the close ties between providers and supervisory authorities can “facilitate regulatory learning for authorities and undertakings, including with a view to future adaptations of the legal framework” (Recital 139 AIA). This intention can be further made fruitful in the context of AI systems for law enforcement agencies. There is a chance to combine the instrument of regulatory sandboxing with funding schemes of research and development projects, with the process of drafting new legislation in police law. Recital 142 AIA points out the “principle of interdisciplinary cooperation between AI developers, experts on inequality and nondiscrimination (…) and digital rights, as well as academics.” This regulatory idea can be realized in the process of shaping AI tools for police agencies that will comply with constitutional law. If the federal and state levels decide to set up a joint regulatory sandbox in the field of law enforcement, they can potentially also include ministries and parliamentarians who draft and decide on legal provisions allowing such AI systems for police forces – regulatory learning could take place on the spot. The legal provision, a suitable AI system fulfilling the intended purpose under the sole control of public authorities, and its factual implementation in police IT infrastructure and investigative methods could be tailored altogether – reducing the risk of lawsuits, disproportionate infringements of digital human rights and legal uncertainty.
2.2. Further processing of personal data acquired by law enforcement authorities (art. 59[2] AIA)
According to Article 57(1) of the AI Act, each Member State must establish at least one regulatory sandbox by August 2nd 2026. This may involve setting up a new sandbox or significantly adapting an existing one to support the development, testing and validation of AI systems. Yet, the regulation is open for all kinds of sectoral (Moraes, Reference Moraes2023) and particularly for more specific regulatory sandboxes on different state levels (Art. 57[2] AIA). Especially, in federal states such as Germany, where law enforcement and policing are largely regulated and carried out on the state and not the federal level, the option of “AI regulatory sandboxes at regional or local level, or established jointly with the competent authorities of other Member States” might open room for specialized and joint regulatory sandboxes in the field of law enforcement. By setting up such regulatory sandboxes, the goal of “evidence-based regulatory learning” (Art. 57[9] lit. d) could be expanded to comparative legal insights as well.
Regulatory sandboxes can be more than a support mechanism for SMEs and a fast lane to successful conformity assessments. This is underlined by a closer look at Art. 59 AIA. Even though the provision raises manifold legal questions on further processing of personal data in regulatory sandboxes, the article focuses only on those parts that are relevant for the field of law enforcement : The provision shows that the EU legislator had law enforcement in mind when drafting Chapter VI – even if only as a subordinate goal. It allows for the further processing of “personal data collected for other purposes for developing certain AI systems in the public interest” (Recital 140[1] AIA). What comes across solely as an authorization for further processing of personal data under data protection law at first sight in Art. 6 (4) GDPR and Art. 5(1)(b) GDPR, opens a door in Art. 59(2) for further processing of personal data for law enforcement authorities that has been acquired for “the prevention, investigation, detection or prosecution of criminal offenses or the execution of criminal penalties, including safeguarding against and preventing threats to public security” in regulatory sandboxes. So, the provision points to law enforcement agencies using their data pools to develop AI systems together with potential providers – from the private or public sector – to enhance their digital capacities and ultimately enable them as deployers. Recital 140(4) AIA underlines that (prospective) providers “should ensure appropriate safeguards and cooperate with the competent authorities, including by following their guidance and acting expeditiously and in good faith to adequately mitigate any identified significant risks to safety, health and fundamental rights.”
Interestingly, though, the corresponding recitals do not specify the scope and effects of Art. 59(2) AIA in the field of law enforcement. Recital 142 rather revolves around other fields of application of Art. 59(1) AIA and explicitly points out “socially and environmentally beneficial outcomes” (Recital 142 p. 1) as use-cases. There is no recital that can help specify the normative scope of Art. 59(2) AIA. A deeper look at legislative history shows that the whole provision was neither mentioned in the AIA’s draft of the European Commission nor in the European Parliaments Mandate, yet was only part of the EU Council’s position. The provision – that most likely goes back to ministries of the interior of national governments – seems to have been added without changes to the final text of the AIA in the – historically long (Bertuzzi, Reference Bertuzzi2023; Waem and Demircan, Reference Waem and Demircan2023) – political trilogue without any further explanation in recitals (den Heijer et al., Reference den Heijer, Abeelen and Maslyka2019). However, the clear structure of the provision makes it possible to interpret it without any further explanation of legislative intent.
Overall, the provision puts a high threshold: the processing of personal data must (1) stay “under the control and responsibility of law enforcement authorities,” (2) “be based on a specific Union or national law” and (3) “subject to the same cumulative conditions as referred to in paragraph 1” which constitutes a total of 10 conditions that must be fulfilled aimed to ensure an alignment with legitimate public interest (see for a full overview Textbox 1 below).
2.3. Can public (law enforcement) authorities play an active part in regulatory sandboxes?
A combined look at Art. 57(2) and Art. 59(2) AIA shows that the EU legislator has foreseen and intended the possibility of setting up joint regulatory sandboxes that involve different state levels (e.g., federal and state level in Germany) where also data sets from law enforcement authorities can be further processed. This opens a wide regulatory space for federal collaboration – to develop, modify or update AI systems in the field of policing, from threat prevention over law enforcement to the execution of criminal offenses.
It is clarified in Art. 58(2)(b) AIA that “providers may also submit applications in partnerships with deployers and other relevant third parties.” In practice, law enforcement agencies can play an active role as (future) deployers of a high-risk AI system (Art. 3[4] AIA). They can also participate as “relevant third party” in cases where the only or most likely deployer of a specific AI system is a police agency. It can be in the interest of the provider of a high-risk AI system to file an application together with a law enforcement agency, without it being clear if a future deployment will take place. Furthermore, Art. 59(1) clearly shows that also “a public authority” can be an active participant in a regulatory sandbox where personal data are processed to “safeguard substantial public interest.” This is underlined by Art. 59(3) AIA which states that the personal data must stay “under the control and responsibility of law enforcement authorities” – how would this be possible without them being involved in the experimental setting and the oversight mechanism of a regulatory sandbox?
Recital 139 S. 5 underlines that “where appropriate,” competent authorities “should cooperate with other relevant authorities, including those supervising the protection of fundamental rights, and could allow for the involvement of other actors within the AI ecosystem,” thus allowing a broad participation of actors who are not directly involved in the product design and marketing of an AI system. However, law enforcement authorities cannot get involved in regulatory sandboxes as authorities supervising the protection of human rights. As deployers of AI-driven surveillance software, they are the object of supervisory activities – and fall short of the necessity to “exercise their duties independently” (Art. 70[1] AIA). According to Art. 74(8) AIA, it is the data protection authorities, “or any other authority designated pursuant to the same conditions laid down in Articles 41 to 44 of Directive (EU) 2016/680” who carry out the task of market surveillance. A police authority cannot be appointed as a national competent authority; it is rather supervised by them while using high-risk AI systems in the field of law enforcement.
According to their respective role, data protection authorities will focus on fundamental rights questions, suggesting privacy-preserving technologies and legal safeguards, while law enforcement agencies will focus on usability and effectiveness of AI systems for carrying out their duties. In an ideal scenario, these opposed views can be mitigated within a regulatory sandbox through dialogue, technical and organizational means and legal safeguards – and lead to AI systems that suit the needs and technical infrastructure of police authority while respecting fundamental rights.
3. Interplay between regulatory sandboxes and testing in real world conditions
According to Art. 57(5) S. 2 AIA, regulatory sandboxes can “include testing in real-world conditions supervised therein.” The instrument framed by Art. 60 AIA differs from the experimental space of a regulatory sandbox by allowing providers to work with “real” data and outside of the box of a mere experimental scenario. Testing in the real world is not limited to nonpersonal data, but allows for the inclusion of information about real people under certain conditions, such as “informed consent by the participants” (Art. 61 AIA; Buocz et al., Reference Buocz, Pfotenhauer and Eisenberger2023). The instrument was proposed by the EU Council and – against the position of the EP – adopted in the final version of the AI in the political trilogue.
For law enforcement authorities who are already engaged in a regulatory sandbox, processing of sensitive data from police databases according to Art. 59(2) AIA might open an interesting enlargement of preparations to incorporate state-of-the-art AI systems into their investigative portfolio: Testing in real-world conditions can acquire new personal data from people interacting with AI systems in an experimental, yet real-life setting and feed it into the process of their development and implementation. Not only that, but especially in the field of criminal investigations, the “real-world test” of digital tools can turn out valuable for several reasons. What works in laboratory conditions might fail “out on the streets,” especially in the field of crime prevention (Gerchick & Cagle, Reference Gerchick and Cagle2024; Schreiner, Reference Schreiner2020). Testing under real-world conditions can thus ultimately contribute to preventing “unnecessary or inappropriate (…) criminal justice tech” (DeMeola, 2023). Ultimately, error-prone technology with which state authority is exercised and fundamental rights are impaired implies that the executive’s legitimate claim to power is at stake.
Pre-testing and evaluation can take place in manifold ways. Volunteers could participate in the experimental space of the regulatory sandbox by providing their personal data, which is not available in police databases (and without the restrictions for further processing, e.g., prior anonymization or synthetization Art. 59[1] lit. b AIA) in a specific app they use in their everyday life – or by partaking in an experimental testing setup during the pre-market phase. In the case of AI systems that rarely translate dialects, real interviews could be used for testing (Hamidov, Reference Hamidov2025; Zhang and Feng, Reference Zhang and Feng2023). In more human rights sensitive scenarios, volunteers might be moving around a protected space, testing the capabilities of gait recognition (Mandlik et al., Reference Mandlik, Labade, Chaudhari and Agarkar2025; Shen et al., Reference Shen, Yu, Wang, Huang and Wang2024) – or providing their social media profiles for testing a scraping tool.Footnote 7 A special focus must in any case lay on the intentional implementation of privacy-enhancing technologies (Ajala et al., Reference Ajala, Arinze, Ofodile, Okoye and Daraojimba2024; Lemieux and Werner, Reference Lemieux and Werner2024; Melzi et al., Reference Melzi, Rathgeb, Tolosana, Vera-Rodriguez and Busch2024; O’Hara, Reference O’Hara2022; Seamons, Reference Seamons2022; Shehu and Shehu, Reference Shehu and Shehu2023) into AI tools, that do not follow the paradigm of massive data collection, centralization and intense extradition of information. An example is the work of Koch (Reference Koch2020) on a “video surveillance framework which maintains the advantages of face recognition whilst prohibiting mass-surveillance.”
Admittedly, the legislator does not see regulatory sandboxing and testing in real-world conditions as Siamese twins. Both instruments of innovation do not necessarily need to be taking place together in the same space – the AIA rather even suggests the opposite: Art. 60 AIA explicitly sets up “appropriate and sufficient guarantees and conditions” (Recital 141, p. 2 AIA) for testing in real-world conditions outside of a regulatory sandbox. The fact that the legislator solely briefly mentions testing in real-world conditions within regulatory sandboxes in Art. 57(5) p. 2 AIA whilst Art. 60 focuses on stand-alone testing, seems to follow the assumption that the safeguards of a sandbox are sufficient and do not necessarily need further legal clarity when combined with real-world testing. This is supported by Art. 58(4) p. 1 AIA, which gives competent authorities the responsibility to “specifically agree on the terms and conditions of such testing, and particularly the appropriate safeguards with the participants, with a view to protecting fundamental rights, health and safety.”
But wouldn’t it be possible for law enforcement authorities to contribute their data to stand-alone testing in real-world conditions and combine it with personal data of real people – outside of a regulatory sandbox? With regard to the text of Art. 59(2) AIA, there seems to be at least no explicit permission in the AIA to further process such data in a real-world testing environment outside of a regulatory sandbox.
It thus seems coherent that personal data can only be used in testing under real-world conditions with explicit consent of data subjects “participating in such testing” (Art. 61[1] S. 1 AIA), who have received information on “their rights (…) in particular their right to refuse to participate” and be handed “dated and documented (…) copy.” However, the legislator mentions exceptions from informed consent in the case of law enforcement, both in Recital 141(3) (“with the exception of law enforcement where the seeking of informed consent would prevent the AI system from being tested”) and in Art. 60(4)(i) AIA. It remains unclear when seeking consent of a data subject would “prevent the AI system from being tested” while “testing in the real-world conditions” would “not have any negative effect on the subjects” (Art. 60[4][i] AIA). How could such personal data even be acquired from a data subject in a stand-alone test under real-world conditions, circumventing the necessity of the data subject to actively participate by giving their consent? In the scope of this article, I will have to leave the specifics as an open question for future research.
4. Retrospective biometric identification with scraped data from the internet
The case of the German “security package” shows (see section 1.3): Where government and legislators want to move fast to integrate AI software that has been made available on global markets into their investigative toolbox, questions of constitutionality, compliance with EU law and technical implementation arise. A clear advantage of the pathway of regulatory sandboxing AI systems before drafting a legal basis or even starting to procure AI products that are in line with a legal basis is. Different perspectives – reaching from effective law enforcement and fundamental rights protection to the technical design of such systems – can be brought together at an early stage and mitigated effectively. Open legal questions, such as if § 15b Asylgesetz allows migration authorities to use an AI system that is banned under Art. 5(1) lit. e AIA – or whether it needs to be merely considered high-risk according to Art. 6(2), Annex III Nr. 7 lit. d,Footnote 8 could at least be discussed and clarified through the direct involvement of different competent authorities and perspectives. To show how legal and technical questions are intertwined, and where further clarification is needed, I will carry out a brief legal analysis around the question: Is § 15b Asylgesetz in breach of Art. 5(1) lit. e – or would an AI system that migration authorities use to put the legal basis into practice solely fall into the high-risk category?
Art. 5(1) lit. e AIA puts a ban on
“the placing on the market, the putting into service for this specific purpose, or the use of AI systems that create or expand facial recognition databases through the untargeted scraping of facial images from the internet or CCTV footage”
The corresponding Recital 43 reads as follows:
“The placing on the market, the putting into service for that specific purpose, or the use of AI systems that create or expand facial recognition databases through the untargeted scraping of facial images from the internet or CCTV footage should be prohibited because that practice adds to the feeling of mass surveillance and can lead to gross violations of fundamental rights, including the right to privacy.”
During parliamentary negotiations and final drafting of the “security package” in the Bundestag, major changes have been made to § 15b AsylG, including several additional safeguards.Footnote 9 In order to stay in the scope of this article, and within the question whether the provision is in breach of EU law, I will focus mainly on the legal basis to use a tool for biometric internet scans, which reads:
“Das (…) biometrische Lichtbild des Ausländers darf mit allgemein öffentlich zugänglichen personenbezogenen Daten aus dem Internet mittels einer automatisierten Anwendung zur Datenverarbeitung biometrisch abgeglichen werden.”
It translates as follows:
“The (…) biometric photograph of the foreigner may be biometrically compared with generally publicly accessible personal data from the Internet by means of an automated application for data processing.”
Since the bans of the AI Act have only come into force on 2nd February 2025, there is little legal literature and no relevant jurisdiction on the individual bans. In the case of Art. 5(1) lit. e, there is also no literature since it has been proposed in the EP mandate, which came last. On the 4th of February 2025, the EC has, however, published “Guidelines on prohibited artificial intelligence practices” (C[2025] 884 final ANNEX) following its obligation in Art. 96(1) lit. b. Guidelines are soft law instruments and nonbinding for competent authorities. However, they might be of help to interpret the provisions and, after all, represent the EU Commission’s perspective, which can influence judicial and administrative decision-making (Craig and de Búrca, Reference Craig and de Búrca2021; Klamert, Reference Klamert2014).
The national provision in § 15b AsylG solely describes the investigation method that a migration authority is allowed to take: to use software (“automated data processing”) in order to compare a biometric photograph taken of an asylum-seeker “biometrically” with “generally publicly accessible personal data from the internet.” It neither mentions if an AI system (Art. 3[1] AIA) nor if a “facial recognition database” is intended or necessary to be used to fulfill this goal. At first sight, these discrepancies might come with no surprise: The AIA is a product safety law that regulates not only the “use of AI systems” by deployers but mostly the “placing on the market” of a provider and the “putting into service for this specific purpose” (Art. 5[1] lit. e AIA), while § 15 AsylG is a legal basis for a migration authority in the process of identifying asylum-seekers. So while the AIA bans certain products, the legal basis in § 15 AsylG does not specify the product used or its exact way of functioning but outlines rather what is permissible to be done by migration authorities in an abstract manner.Footnote 10
In any case, the purpose of § 15d AsylG is the identification of a specific person (“Feststellung der Identität,” engl. “determination of identity”), which falls inside the scope of Art. 6(2), Annex III Nr. 7 lit. d (“for the purpose of […] recognizing or identifying natural persons”) – and thus surely falls in the tightly regulated high-risk category. But does it also allow migration authorities to use a banned AI system here?
Regarding the explicit wording of Art. 5(1)(e) AIA, § 15b AsylG theoretically leaves room for forms of “automated data processing” for biometric scans using “generally publicly available data from the internet” (§ 15[1] AsylG), provided that the system used is not clearly classified as an AI system banned for scraping internet data to build facial recognition databases. But are there technical solutions that allow a biometric photograph as input and matching pictures of that same person from “the internet” as an output which are not an AI system after all?
Clearview AI has served as the blueprint for the ban in Art. 5(1) lit. e AIA – a provision introduced by the EP Mandate: the software available only to public authorities scrapes the internet using techniques of machine learningFootnote 11 and feeds them into a database for the purpose of facial recognition, which users can check against facial images they use as input. In order to set up the database, Clearview seems to also scrape the internet in an “untargeted” way, collecting every facial image it can find, thus not being limited to a certain group or cohort of people.Footnote 12 Ultimately, a database of all pictures of human faces on the internet is being created (Martini and Kemper, Reference Martini and Kemper2023a). One might argue, though, that the ban does not apply in cases where public authorities use such tools with the purpose of identifying a specific person – the scraping then being “targeted” instead of “untargeted.” However, the legislator has used “untargeted” as an adjective to describe the technical process of “scraping of facial images from the internet” in creating or expanding the database – not specifying the purpose of the intended use of the system. As a result, Clearview AI is banned; the German migration authorities will not be allowed to procure it to carry out the identification of asylum-seekers.
In its guidelines, the EC defines the term “untargeted” as follows:
“If a scraping tool is instructed to collect images or video containing human faces only of specific individuals or a pre-defined group of persons, then the scraping becomes targeted, for example to find one specific criminal or to identify a group of victims.”
This would open room for a permissible use of targeted scraping tools that are used to clarify the identity of asylum-seekers. Yet, from a technical perspective, the relation between the “facial recognition database” and the “targeted scraping” would then become blurry. Either there is a facial image database that has been built up by scraping facial images from the internet, which is then used for a targeted search for a specific person – or a “targeted” scraping of the internet takes place where each image is analyzed in two steps: (1) is it a facial image, and (2) does it match the biometrical image of the targeted person. In the second case, there would be no specific “facial recognition database,”Footnote 13 but only the output of a targeted internet scraping. This interpretation at least suggests that the provision has inconsistencies when thinking about its practical application outside of the business models of Clearview AI, for which the provision has been tailored.Footnote 14 According to the EC guidelines, however, forms of “reverse engineers image search engines” shall be permissible – which would suggest that PimEyes, which adds a component of facial recognition to reverse image search,Footnote 15 shall be considered as a form of a – not banned – targeted scraping. So is the question whether a form of scanning for biometrical matches in facial images on the internet is banned or not, depending on the technical method? The provision in § 15b AsylG abstains from any specification of the functionality of the technical system.
It turns out to be not solely a legal but an inherently interdisciplinary question if biometric scans with internet data can be carried out with software that works either (1) without a “facial recognition database”Footnote 16 in useFootnote 17 or based on (2) “untargeted” (or targeted) scraping of the internet.Footnote 18 The Federal Commissioner for Data Protection and Freedom of Information, Prof. Dr. Louisa Specht-Riemenschneider, has stated in the expert hearing in the German parliament that targeted scraping of the internet for biometric matches without using a forbidden database “is unrealistic under today’s technical conditions.”Footnote 19 On the other hand, the EC considers “reverse engineering image search engines” as a permitted targeted scraping in its guidelines (EC guidelines p. 79). So, what the EC guidelines seem to imply is: Clearview AI – which has set up a database for its clients for facial recognition – is banned, while PimEyes – which reverse engineers image search engines combined with biometric face recognition – is permitted under the AI Act.
Considering the legislative intent to ban a “practice” that creates “the feeling of mass surveillance and can lead to gross violations of fundamental rights, including the right to privacy” (Recital 43 AIA), it, however, seems questionable to draw the line between banned and permitted alongside merely technical facts. The ban would only apply when a facial recognition database similar to that of Clearview AI is used as a buffer – and not when the internet is scraped to find individual matches via reverse engineering image search engines. Apart from the question whether such search engines deploy a database in the background to make search results available quickly: From the citizens’ perspective, all personal data ever publicly published – intentionally or unintentionallyFootnote 20 – on the internet would be under scrutiny to be collected by law enforcement agencies. Would it make a difference if facial images are ultimately stored in a facial recognition database or gained through reverse engineering image searches with a biometrical matching option? One could argue that data subjects have more influence on whether one of their pictures is findable in the open web through search engines compared to a database that cannot be accessed publicly. Also for courts and supervisory authorities, it might be more difficult to control the content and scope of such a database, which could also be filled with images acquired through social media platforms, data brokers or nonpublic parts of the internet, than a browser-based application that builds on existing search engines of the public internet.
Even though there is no explicit violation of the text of Art. 5(1) lit. e AIA, since it is ultimately unclear and vague how the German migration authorities will make use of the legal basis in § 15(1) AsylG, it seems at least doubtful that there is a possible way to use the legal basis without violating Art. 5(1) lit. e AIA at this point. The German legislator has, anyway, decided to adopt the law. Other than proposed by the government draft, the Bundestag has, however, added a provision to the law that the German Federal Government has to first pass a subordinate statutory regulation (Rechtsverordnung) that specifies technical details and safeguards (§ 15[11] AsylG) before migration authorities can ultimately use such software (see footnote 18 above). There is thus still room for interdisciplinary efforts to decide whether there is a technical and legal way to implement § 15 AsylG without violating Art. 5(1) lit. e AIA while preparing the statutory regulation in consultation with the national data protection authority. Overall, the analysis of § 15 AsylG in light of the bans of the AIA shows that there is and will be immense legal and technical uncertainty in carrying out such biometric internet scans.
I argueFootnote 21 that the described legal and technical specifics could best be clarified in the realm of a regulatory sandbox in close regard to the actual AI system in development and in dialogue with the competent authorities for AI regulation, as well as authorities for the protection of fundamental rights. Taking a regulatory sandbox will surely take more time than just drafting a legal basis for biometric internet scans. Yet, as a thorough first step, it can potentially prevent a maldevelopment towards unlawful infringements of fundamental rights, politically unpleasant judgments of high courts, an inefficient allocation of public funds, and a loss of trust in law enforcement authorities. The outcome in the case of biometric internet scans would in my opinion however be: Such systems are banned under Art. 5(1) lit. and cannot be implemented in a compliant manner.
5. Outlook: A sketch for a joint regulatory sandbox of different state levels and law enforcement agencies
The analysis of the provisions of the AIA has shown that different state levels can set up joint regulatory sandboxes in which not only potential providers but also law enforcement agencies as potential deployers or relevant third parties can work together in a protected and experimental environment. In a regulatory sandbox, personal data from data pools of law enforcement agencies can be further processed (Art. 59[2] AIA) and testing under real-world conditions with personal data of data subjects who give their informed consent can take place (Art. 57[5] S. 2 AIA). It remains unclear when personal data can be used without informed consent in testing under real-world conditions outside of a regulatory sandbox according to Art. 60(4)(i).
Through the active participation of national competent authorities for the AIA, public bodies for the protection of fundamental rights (e.g., data protection authorities), civil society organizations, academic experts (e.g., from the scientific panel of independent experts, Art. 68 AIA) or even civil servants from ministries who draft laws, different goals can be reached simultaneously:
• It can be avoided that systems are developed that are banned under Art. 5 AIA or that high-risk AI systems are (unwantedly) used in breach of the provisions of Chapter III.
• AI systems can be designed and developed in a manner that suits the needs and preconditions of the relevant law enforcement agencies and their existing IT infrastructure.
• National particularities in the regulation of law enforcement can be considered, be it in the realms of constitutional law, police law or shared/separated responsibility on federal levels.
• A high level of transparency on how the relevant AI systems work, including data flows and data analysis mechanisms, can be reached – potentially leading to AI software that is effectively open-source.
• Paving a way to effective public procurement of AI systems for law enforcement authorities that follows the overall political goal of digital or technological sovereignty (European Commission, 2023; Floridi, Reference Floridi2020; Pohle and Thiel, Reference Pohle and Thiel2020).
• Efficient use of public fundsFootnote 22 if law enforcement agencies of different state levels cooperate, rather than operate separately, in procuring AI systems.
• SMU and start-ups that are currently not the key accounts of law enforcement authorities in the EU might get access to a new market segment of so-called GovTech (Bharosa, Reference Bharosa2022; Kuziemski et al., Reference Kuziemski, Mergel, Ulrich and Martinez2022).
In the preparation of joint regulatory sandboxes in law enforcement, international best practices should be taken in close consideration, e.g., the “criminal justice tech sandbox” in Utah (Christoph 2023), especially in order to “ensure informed, scientifically rigorous and objective evaluations.” This would contribute to the goal of “evidence-based regulatory learning” (Art. 57[9] lit. d AIA) in an optimal way.
So far, there is no specific legal basis yet in either national or Union law that allows further processing in a regulatory sandbox. But, as this article has shown, the door is open for national legislator(s) to take the opportunity to tailor a national provision – on the federal and/or state level – that meets the standard of Art. 59(2) AIA, and later even include testing under real-world conditions in the specific joint regulatory sandbox. In order to set up a joint regulatory sandbox for AI systems that can contribute to the overall goal of developing legally compliant and technically sovereign AI systems for law enforcement agencies, the national legislator(s) need to adopt a legal basis that meets the criteria summarized in Textbox 1.
Personal data lawfully collected by law enforcement agencies can be used in a regulatory sandbox according to Art 59 (2) AIA if
they were originally collected for the purposes of the prevention, investigation, detection or prosecution of criminal offenses or the execution of criminal penalties, including safeguarding against and preventing threats to public security.
They can only be further stay “under the control and responsibility of law enforcement authorities.”
The processing in the regulatory sandbox “is based on a specific Union or national law” – and
the conditions in Art. 59(1) AIA cumulatively met; this means
• the AI system is developed to protect substantial public interest by a public authority or another natural or legal person in the area of public safety (lit. a [i] 1st alternative)
• the data processed are necessary for complying with one or more of the requirements referred to in Chapter III, Section 2, where those requirements cannot effectively be fulfilled by processing anonymised, synthetic or other nonpersonal data;
• there are effective monitoring mechanisms to identify if any high risks to the rights and freedoms of the data subjects, as referred to in Article 35 of Regulation (EU) 2016/679 and in Article 39 of Regulation (EU) 2018/1725, may arise during the sandbox experimentation, as well as response mechanisms to promptly mitigate those risks and, where necessary, stop the processing;
• Any personal data to be processed in the context of the sandbox is in a functionally separate, isolated and protected data processing environment under the control of the prospective provider, and only authorized persons have access to that data.
• Providers can further share the originally collected data only in accordance with Union data protection law; any personal data created in the sandbox cannot be shared outside the sandbox.
• Any processing of personal data in the context of the sandbox neither leads to measures or decisions affecting the data subjects nor does it affect the application of their rights laid down in Union law on the protection of personal data.
• Any personal data processed in the context of the sandbox is protected by means of appropriate technical and organisational measures and deleted once the participation in the sandbox has terminated or the personal data has reached the end of its retention period.
• The logs of the processing of personal data in the context of the sandbox are kept for the duration of the participation in the sandbox, unless provided otherwise by Union or national law.
• A complete and detailed description of the process and rationale behind the training, testing and validation of the AI system is kept together with the testing results as part of the technical documentation referred to in Annex IV.
• A short summary of the AI project developed in the sandbox, its objectives and expected results is published on the website of the competent authorities; this obligation shall not cover sensitive operational data in relation to the activities of law enforcement, border control, immigration or asylum authorities.
A joint regulatory sandbox for law enforcement purposes could be permanently set up, thus being able to move fast when new technological trends arise or the state-of-the-art is further developed. The focus of joint regulatory sandboxes in the field of law enforcement would – and should – not only focus on surveillance tools that are highly sensitive to fundamental rights. Also, AI systems in the fields of translation, text or image generation, trace analysis or data visualization can be tailored for the specific needs of law enforcement authorities – which tend to differ from those of commercial enterprises (Kemper and Kolain, Reference Kemper and Kolain2025).
If sandboxes will turn out as “an easier sell politically” (Christoph Reference Christoph2023) remains to be seen. In the case of the “security package” in Germany, this was not the case – maybe due to the novelty of the provisions of the AIA as well as the fact that the mandatory regulatory sandbox (Art. 57[1] AIA) has not been set up yet. But this might change in the future. In a collaborative effort, legislators, competent supervisory authorities and law enforcement authorities of different state levels, potential providers of AI systems and research institutions can walk through the door Chapter VI has opened.
Especially in times of political tensions, polarization and a rise of digital authoritarianism (Dragu and Lupu, Reference Dragu and Lupu2021; Hellmeier, Reference Hellmeier2016; Pearson, Reference Pearson2024; Taylor, Reference Taylor2022; Turner, Reference Turner2019; Wilson, Reference Wilson2022), new surveillance laws that disproportionately interfere with fundamental digital rights and are later stopped by constitutional courts as being contrary to fundamental rights, or turn out as nonfunctional, can destroy trust in the liberal constitutional state and democratic law enforcement authorities. Against this backdrop, it is worth modernizing the police thoroughly and with respect to a high standard of digital human rights by testing new police technologies in regulatory sandboxes first – and then follow up with a legal basis in conformity with constitutional law and a joint procurement plan for technologically sovereign AI systems that align with European values.
Acknowledgements
A special thanks goes to Konstantin von Notz and Dietrich Haußecker for inspiring and discussing the regulatory idea of regulatory sandboxes in law enforcement early on, to Hannah Ruschemeier for inviting me to write this paper, to Katharina Buchsbaum for assisting me in researching for and finalizing the manuscript and to Cigdem Caglayan Kolain for her emotional support.
Funding statement
There are no funders to report for this submission.
Competing interests
The author was involved as a digital policy advisor in der Federal German Parliament (Bundestag) in the final steps of the legislative process around the “Sicherheitspaket,” which is used as a case study in this article.
Michael Kolain is a legal scholar and conducts research at the interface between law, technology and public policy. He has a research focus in state of law and digital human rights, the regulation of emergent digital technologies, digital statehood, and data (protection) law.
Open access
