AI in the TCF
What do we know about the artificial intelligence used in TCF?
Since May 2024 — a year ago already — an automated scoring system called FIDELIA has gradually been introduced into the computer-based TCF writing section. It doesn’t replace human assessment; it supplements the usual double marking. One of the two scores can now be assigned by artificial intelligence.
What is FIDELIA?
FIDELIA is an automated verification assistance system developed by France Éducation international in collaboration with the scientific laboratory CENTAL (Université de Louvain-la-Neuve, Belgium). The system belongs exclusively to FEI.
FIDELIA uses a directed AI model, not a generative one. This means, as stated by the compilers, that for the same work it will always give the same result, without random deviations. The model is stable, reproducible and not affected by fatigue or mood. We hope 😂
How does the new assessment scheme work?
📍The principle of double-blind testing, which has been in effect since 2014, remains.
📍If the work is submitted on a computer, then it will be checked by one human proofreader and the FIDELIA system.
📍The final assessment is based on two results.
📍In case of discrepancies between assessments, a third human check is carried out.
Thus, the assessment is based on the cooperation of man and machine.
The website says that
📌in 70% of cases, the assessments of FIDELIA and the human corrector coincide;
📌however, the system may find it difficult to evaluate atypical texts that were not in its database;
📌In such situations, the role of a human proofreader is especially important, because only a person is able to flexibly interpret unfamiliar language structures.
FIDELIA, in turn, is always stable, does not get tired and does not make mistakes due to inattention. We've already heard this somewhere. It ensures a uniform check - as if the work was being checked by a whole group of proofreaders.
How FIDELIA was created: main stages
The development of FIDELIA is the result of many years of work. Here are the key steps the developers went through:
1 ️⃣ Even 10 years ago, double checking of written work was introduced - it became a necessary precondition for the emergence of an automatic assessment system.
2️⃣France Éducation international has created a large database of written works TCF, already verified by examiners. From this database, 27,000 texts were selected, with care taken to ensure that the corpus was diverse and did not favor any one group of candidates.
3️⃣All these texts were re-checked by 55 certified TCF proofreaders to clarify and consolidate the levels.
4️⃣Then experts identified 48 language features - non-duplicate, clearly measurable - that really help distinguish one level of language proficiency from another.
5️⃣The model was trained to assess a text’s level using the level already assigned to it and these 48 parameters.
If anyone is interested in my opinion, for me, the introduction of artificial intelligence into the TCF assessment system is a logical and logical step. Personally, this innovation only once again convinced me of a thought that I have long shared: passing an exam is not the same as “learning all French.”
This is why in my work I focus on the corpus approach, I just didn’t know it was called that, and I teach how to meet specific expectations of the format and the required level. Apparently, I will continue to do this in the future - now with an eye on cars. My immediate plans are to write approximately +100500 letters to FEI to get to the bottom of some issues. I'll keep you posted.