Evaluation Subscribe to Evaluation RSS feed

Introduction In Atomic Tests vs. Holistic Tests: A New Testing Methodology, we discussed balancing Atomic and Holistic tests. Now we need to address how we’ll “assert” and “document” these results. This is where Assertions come in. The scoring and conformance model covered in WCAG 3.0 Conformance Model: Changes After A/AA/AAA also connects with Assertions. This is because they provide a way to supplement areas not covered by quantitative tests with organizational processes and evidence. ...

Introduction When people hear “accessibility testing,” they often think of a checklist: “Does this button have alternative text?” “Is the contrast ratio high enough?” WCAG 2.2 is built around clear pass/fail checks like these. WCAG 3.0 moves toward a broader unit of evaluation, aiming to consider overall user experience quality. That shift naturally changes how we test. We now combine fine-grained checks (Atomic) with real-world contextual evaluation (Holistic). This shift directly connects to the score-based conformance model discussed in WCAG 3.0 Conformance Model: Beyond A/AA/AAA. The weight you give each test type can change the score and the level you reach. ...

Introduction In the previous post, we looked at why WCAG 3.0 reorganized “Success Criteria” into “Outcomes.” This post focuses on changes to the conformance model. WCAG 3.0 aims for a different approach than WCAG 2.x, but it is still in the Editor’s Draft (2026-01-05) stage and is not finalized. Conformance levels, scoring approaches, and evaluation methods are still being explored. Important: This post is based on the WCAG 3.0 Editor’s Draft (2026-01-05). The draft can change at any time, and the document itself is explicitly marked as a work in progress. ...

Introduction “1.3.1 Info and Relationships - Level A” If you’ve worked with WCAG 2.2, you’re familiar with this format of Success Criteria. Numbers, levels, and clear test conditions. This structure has been the standard for web accessibility for over 15 years. However, as we explored in the previous article, this approach had limitations. “Websites that pass checkboxes but are actually unusable” are proof of this. WCAG 3.0 has completely redesigned the structure itself to address this issue. It didn’t just add items—it changed the way we evaluate accessibility. ...

Introduction I just returned from the AI Agent Scenario Competition Citizen Evaluation Panel held in Seoul. The panel consisted of expert judges and citizen evaluators, with the final scores reflecting both expert evaluation and citizen evaluation. Since the evaluation details are confidential, I can’t discuss individual teams or specific results. However, through this experience, I was able to clearly define what standards I use when evaluating AI public services. So in this post, rather than “how this team performed,” I want to record what questions I asked and why I thought those questions mattered. ...

Introduction Let me be honest — this choice is a losing deal on paper. I’m spending a whole day, paying for transportation out of my own pocket, and the compensation is only about $35 (50,000 KRW). Yet, I’m heading to Seoul tomorrow. I wanted to organize my reasons step by step. View from a train window - a journey to somewhere Photo: Dieter K / Unsplash The Evaluation Panel Offer and My Decision One day, while browsing the NIA (National Information Society Agency) website as usual, I discovered the Public Institution Website Citizen Evaluation Panel. Since then, I’ve been grateful to participate in the public web/app citizen evaluation activities for two years straight. The rewards and achievements may seem small, but the process and experience have been building up as personal assets. ...