CELIA – Common European Language Indication and Analysis
| Please find our English flyer on Celia. For more information: please contact us at: CELIA@ind.nl |
Language assessments are used in the asylum procedure to investigate the origin claimed by asylum seekers. The EU project CELIA aspires innovation and optimisation of language assessment methods so that authorities of the EU+ countries (the 27 countries of the EU plus Norway and Switzerland) can use them more often and more effectively in migration procedures. In the ongoing first phase (1 January 2024 to 31 December 2027 inclusive), the development of methods is at the forefront
Why the CELIA project?
It will benefit Europe to be able to use available instruments – such as language assessments – uniformly as much as possible, also at Europe’s external borders. Migration authorities are spending a lot of time and energy on investigating asylum seekers’ identities and countries of origin. Innovation and optimisation of language assessment methods can improve and accelerate the identification process.
What does CELIA do?
The project focuses on:
1. development of new language assessment methods using artificial intelligence, observing the AI Act;
2. optimisation of language assessments by human experts.
In addition, work is being done to prepare for implementation in the migration procedure in the Netherlands, and subsequently in other EU+ countries.
What is the objective of CELIA?
The objective of CELIA is to be able to offer reliable and validated language assessment methods to migration authorities within the EU. The methodological development and validation takes place in cooperation with external experts and is being recorded meticulously.
CELIA must make it possible to use combinations of different language assessment methods to support the assessment of asylum applications. A fast-track indication of an asylum seeker’s language variant (by a human and/or machine) can, for example, be carried out on a large scale. Where necessary, it can be followed up by a full-track analysis by an expert.
What is being done to achieve the objective?
- Methodological development through innovation: development and validation of automatic and semi-automatic language indications for Arabic language variants
- Optimisation of existing human language assessment methods: further development of existing standards, and training and information material
- Certification of language experts: standardisation of the selection and training procedure for human language analysts, including peer review by external language experts
- Preconditions for European roll-out: preparations for application in other EU+ countries
Frequently Asked Questions
-
What do the terms ‘language indication’, ‘language analysis’ and ‘language assessment’ mean?
Language indication: ‘fast-track’ analysis of an applicant’s speech, indicating whether the claimed origin is supported or not. Performed by an AI-based tool and/or a human listener. Not a full analysis by an expert.
Language analysis: ‘full-track’ analysis of an applicant’s speech by a human language expert, resulting in an expert report that can also be used as partial motivation for a negative decision, as it will meet the requirements of forensic expert evidence.
Language assessment: cover term for both language indications and language analyses. -
How do language assessments help in the decision making process of asylum applications?
Assessment of an asylum applicant’s language skills provides clarity on the linguistic community in which they were raised. A strength is that language skills are acquired throughout a lifetime and cannot be stolen, lost, left behind or acquired through short-term learning. A limitation is that the linguistic origin of someone does not always equal to their country of nationality, for example if the person moved away from their country of nationality at a young age.
Language assessments offer a solution for bona fide asylum applicants who cannot provide substantive proof of their origin by other means, and they can offer concrete evidence against unsubstantiated claims by non-bona fide applicants. Language assessments are one of the tools that support the identification of asylum applicants. They support the national migration and asylum authorities to reach fair decisions on applications for international protection and other migration procedures.
If language assessments are to be a reliable source of information, they should be performed in controlled conditions. Assessments by interpreters during interviews are not sufficiently controlled: there is no mechanism in place for correction of errors (known to occur) or detection of biases (known to exist). CELIA aims to offer controlled and reliable tools for language assessments to all EU+ countries. -
When will an AI tool be ready for implementation in the asylum process?
CELIA’s first priority is quality. An AI tool will not be ready for implementation until it has been rigorously evaluated in various use case conditions. Within CELIA phase 1 we aim to establish if an AI tool can deliver reliable results, and if so, in which cases. An AI tool may become ready for implementation in CELIA phase 2, that is, not before 2028.
-
Why does an AI tool for language assessment constitute a 'high-risk application'?
AI applications used by immigration and asylum authorities to assist the assessment of eligibility for a status are explicitly mentioned to fall in the high-risk category (AI Act, Annex III 7c). Article 6(3) lists possible exceptions to this statement, e.g. when the AI system is only intended to perform a preparatory task or when its output will not materially influence the outcome of decision making. Given the intended use of the tool as a preselection or indicative tool, these exceptions do not apply.
Moreover, we principally agree with the criteria for high-risk applications set by the AI Act. We feel it is imperative to understand the basic workings and limitations of such an AI tool and to implement meaningful human oversight, in order to ensure ethical and responsible use. -
How do you position language assessments in the asylum process?
In principle each authority can determine for themselves how they want to position language assessments in their asylum process, but some best practices can be mentioned.
- Recording a speech sample from an asylum applicant is best done early in the process, when their language use is least contaminated.
- Background information on the applicant’s claimed residential history and claimed level of education should be available in order to assess expected language proficiency.
- A ‘fast-track’ language indication should only be used by an asylum officer to determine if the claimed origin may be considered credible or should be investigated further. An indication should not be used or presented as expert evidence.
- If a language assessment is necessary to motivate a negative decision, a ‘full-track’ language analysis should be done.
-
What kind of tools are being developed in the CELIA project?
- Automatic Dialect Identification (ADI), a fully automatic AI-based tool.
- Semi-automatic dialect identification, an AI-based tool that requires not only an audio file but also a transcription, so that the tool can analyze the pronunciation of specific sounds.
- Human analysis, requiring (near-)native competence in the languages, linguistic expertise and forensic training.
-
Who is leading the CELIA project and how is it financed?
The CELIA project is carried out by the IND (Immigration and Naturalisation Service, NL). It is co-funded by the European Commission (as a Specific Action within the Asylum, Migration and Integration Fund, i.e. AMIF) and the IND.
-
What is the role of EUAA in the CELIA project?
While the CELIA project is led by the IND, the EUAA has launched the Working Group on Language Assessment for Determination of Origin (LADO WG). This WG aims to (1) coordinate CELIA’s contacts with interested EU+ countries, and (2) coordinate preparations to roll-out the envisaged language assessment tools to all EU+ countries. These objectives are incorporated as products of the CELIA project.
Furthermore, EUAA will support the CELIA team in determining the legal and technical conditions under which the sharing of language recordings from asylum applicants for the purpose of a language assessment is allowed considering EU legislation.
EUAA is not a project partner as such, but the support provided by EUAA and the members of the Working Group are of paramount importance to achieve the vision of one EU+ language assessment service.
Sources of knowledge
- Years of experience in carrying out language assessments in the asylum procedure
- Contact with other language assessment agencies across the globe
- Contact with migration authorities and EUAA (European Union Agency for Asylum)
- Academic research in the field of automatic dialect recognition
- Network of experts and forensic speech investigators
- Standards applicable to forensic expert evidence
| Would you like to contribute via the International Expert Group (IEG)? You can! One of the overarching objectives of this project is to open up the development of methods to external peer review and academic input. The IEG was set up for this, and is open to academics and other professionals who are able and would like to contribute to the objectives of this project. If you are interested, please send a message to celia@ind.nl. |
Would you like to find out more about the project or get in touch with the CELIA project group within the IND? Then send us a message at celia@ind.nl.
