recent
Latest Articles

Assessing AI Literacy: What Learners Should Be Able to Demonstrate: Second in a three-part series on AI literacy in Sweden and the EU.

Assessing AI Literacy: What Learners Should Be Able to Demonstrate

Second in a three-part series on AI literacy in Sweden and the EU

The first piece in this series looked at why AI literacy is becoming everyone's problem, not just schools'. Several serious frameworks now exist for it: UNESCO's competency frameworks for teachers and students, the OECD and European Commission's framework for primary and secondary education, TeachAI's implementation resources, and the legal obligation created by Article 4 of the EU AI Act.

A horizontal, realistic photograph of a diverse group of adult students—including young adults, seniors, and a woman wearing a hijab—gathered around a wooden table in a brightly lit modern Swedish classroom. They are working on laptops during an AI literacy workshop, while an instructor stands presenting at the end of the table. Large windows in the background overlook a European cityscape, and a clean banner overhead reads "Adult Education: The Frontline for AI Literacy in Sweden & the EU."

That is progress. But none of it teaches a student anything by itself. A framework has to be converted into tasks, rubrics, feedback and assessment before it changes what happens in a classroom. The practical question for schools, adult education and higher education is simpler than the frameworks suggest: what should an AI-literate learner actually be able to do?

The answer should not begin with prompt engineering. Prompting is a visible skill, but it is only one part of the competence. A learner may write a good prompt and still accept a false answer, expose personal data or misunderstand how the output was produced. AI literacy has to include understanding, evaluation, responsible use and governance. Figure 1 shows this shift from simple AI use toward assessable evidence of learner judgement.

AI use

Prompts, tools and generated outputs

Learner process

Checking, revising and documenting

Evidence of judgement

Sources verified, errors found and risks considered

Assessment of AI literacy

Understanding, evaluation, responsible use and accountability

Figure 1. Moving from simple AI use to assessable evidence of AI literacy.

Four Observable Abilities

A simple outcomes model can help teachers avoid both extremes — vague ethics talk on one side, tool tutorials on the other. The model in Figure 2 translates the international frameworks into four observable abilities. Table 1 shows example evidence that the learner has achieved the desired ability.

An infographic diagram illustrating 'What an AI-literate learner should be able to do', structured as a four-quadrant map centered on 'AI literacy'. The top-left quadrant, 'Understand AI', states learners should: 'Explain pattern-based systems, limits, uncertainty and automation.' The top-right quadrant, 'Evaluate outputs', states learners should: 'Check sources, compare answers, detect hallucination, bias and missing context.' The bottom-left quadrant, 'Use with judgement', states learners should: 'Choose suitable tasks, document assistance and protect sensitive data.' The bottom-right quadrant, 'Shape and govern', states learners should: 'Ask who is affected, what evidence is needed and who remains accountable.' The caption notes the map is 'derived from UNESCO, EC/OECD and TeachAI frameworks.'
Figure 2. Four observable abilities of an AI-literate learner.

Ability

What learners should demonstrate

Example evidence

Understand AI

Explain that AI systems infer patterns from data and can produce plausible errors

Short explanation of why a chatbot answer may sound certain but still be wrong

Evaluate outputs

Check sources, compare outputs and identify missing context

Annotated AI answer showing which claims were verified, corrected or rejected

Use with judgement

Choose appropriate AI use, protect sensitive data and document assistance

Process log describing what the learner used AI for and what remained human work

Shape and govern

Discuss affected people, accountability and limits

Decision note explaining when AI should not be used in a learning or work task

Table 1 explains the four qualities demonstrated by an AI-literate person.

This model is not a replacement for UNESCO or EC/OECD frameworks. It is a practical draft proposal, shaped by teaching experience in this subject area, and intended as a bridge between policy language and classroom practice.

Why Assessment Is the Hard Part

Many teachers are already adjusting assignments because AI can generate essays, summaries and code. The Swedish department of education Skolverket's 2026 follow-up found that nearly eight out of ten surveyed teachers had used AI in some part of their work, and four out of ten had initiated or approved student use in teaching. 

The same report flags ongoing concerns: cheating, unreflective use, stalled writing development, and a need for more support and professional development. Skolverket is careful to note that the results come from a teacher panel and are indicative rather than nationally representative.

The pattern is still important: AI has entered teaching practice before assessment systems have caught up.

If assessment remains focused only on final written products, teachers will be forced into a weak policing role. They will try to detect AI text even though Skolverket's guidance notes that there are no secure ways to determine whether a text was written by AI. That is a dead end.

A better route is to assess process and judgement. Learners can be asked to submit AI interaction logs, source checks, oral explanations, draft histories, reflection notes and comparisons between AI suggestions and verified sources. The question changes from “Did the learner use AI?” to “Can the learner use AI in a way that preserves understanding and accountability?”

A Practical Assessment Matrix

Task type

Weak AI use

Strong AI-literate use

Assessment focus

Information search

Copies AI answer without sources

Uses AI to generate search terms, then verifies through primary sources

Source quality and correction of errors

Writing support

Submits polished text without understanding

Uses AI for language feedback and explains the revisions

Ownership of argument and vocabulary

Mathematics/STEM

Accepts solution steps without checking

Compares AI solution with manual reasoning, units and graph/experiment

Reasoning, not only final answer

Public-policy analysis

Summarises policy without reading source

Uses AI to map questions, then cites official documents

Link between claim and source

Workplace scenario

Uploads sensitive material to a public tool

Redacts data, chooses low-risk tasks and records AI use

Privacy, risk judgement and documentation

Table 2. Practical Assessment Matrix for Evaluating AI Literacy Across Task Types

This type of matrix can be adapted to upper secondary school, adult education, vocational education and university courses. It also fits workplace training. The underlying competence is the same: learners must show that they can keep control of the task.

The Adult Education Challenge

Sweden's policy signals- covered in the first piece in this series- point in the right direction: AI in school and higher education, lifelong learning, more public AI education, Skolverket steering educators toward the EC/OECD and UNESCO frameworks. None of that solves the assessment problem by itself, in Sweden or anywhere else building on the same frameworks.

What is missing everywhere is a practical outcomes layer: examples by subject and level; mathematics, language instruction, civics, vocational health care, engineering, administration; that show how to assess understanding, not just how to use the tool. Sweden is further along than most in having the policy layer in place; the outcomes layer is still being built.

Adult education is where this gap shows up fastest. A learner in a Swedish Komvux class, or an adult learner anywhere retraining for work, may use AI for language support, mathematics explanations, job applications and public-service communication in the same week. A rigid ban would be unrealistic. Unstructured use would be irresponsible. The middle path is documented, guided use with clear criteria, the same criteria set out above.

Learner Reflection as Evidence

One useful test is whether learners can explain their own AI use in plain language.

Prompt for learner reflection

What it reveals

What did you ask the AI system to do?

Task clarity

Which parts of the answer did you verify?

Source judgement

What did the AI get wrong or leave out?

Critical reading

What personal or sensitive data did you avoid sharing?

Privacy awareness

What part of the final work is your own reasoning?

Ownership and learning

Would AI use be acceptable in this context? Why or why not?

Ethical and institutional judgement

Table 3. Learner Reflection Prompts and Evidenced AI Competencies

These questions are simple. That is their strength. They force the learner to move from output to responsibility. It is also the test built into the final week of the free AI Literacy Course, a twenty-week open-access program for adult and upper-secondary learners. Its capstone asks learners to assemble a portfolio and action plan documenting what they asked an AI system to do, what they checked, what they rejected, and what remained their own reasoning the same evidence this piece has argued classrooms need.

From National Policy to Transferable Practice

I believe AI literacy will be taken seriously once assessment catches up with use. Frameworks give education systems a vocabulary. Assessment gives them behavior. Sweden's next test is whether it develops subject-specific AI literacy examples, builds assessment design into teacher training, extends adapted guidance to adult education, and lets work such as PISA 2029's Media and Artificial Intelligence Literacy strand inform national evaluation.

The four-ability model and the assessment matrix set out in this piece do not depend on any of that happening first. A university, teacher-training program or adult education provider elsewhere can apply them now, in whatever subjects and languages it teaches.

The most useful AI literacy test may be this: can the learner explain what the machine did, what the human checked, and why the final judgement can be trusted? That question travels well beyond Sweden.

This article is part of Nordic R&D Bridge's ongoing coverage connecting Swedish research and innovation with universities and researchers across the Middle East and Africa. We welcome comments, corrections, and counterarguments.

Are you assessing AI use in a classroom, training program, or institution  in Sweden or elsewhere? What rubric or process evidence has actually worked? Leave a comment below, or write directly to us at Swedish Research.

We are particularly interested in hearing from teachers and assessment designers who have moved beyond detecting AI text toward evaluating learner judgement.

author-img
Saad Muhialdin

Comments

No comments
Post a Comment

    Stay Updated with Nordic R&D Bridge

    google-playkhamsatmostaqltradent