Questions I Asked at the AI Public Service Evaluation

Introduction

I just returned from the AI Agent Scenario Competition Citizen Evaluation Panel held in Seoul.

The panel consisted of expert judges and citizen evaluators, with the final scores reflecting both expert evaluation and citizen evaluation.

Since the evaluation details are confidential, I can’t discuss individual teams or specific results. However, through this experience, I was able to clearly define what standards I use when evaluating AI public services.

So in this post, rather than “how this team performed,” I want to record what questions I asked and why I thought those questions mattered.

The Perspective I Valued Most in the Evaluation

Today’s evaluation theme was AI-powered public services that help citizens in their daily lives.

When looking at such services, I naturally find myself asking these questions:

• Can this service be connected to real-world operations?
• Can it work equally for all citizens?
• When problems occur, where should users turn?

From this perspective, most of my questions today were closer to operations and user experience rather than ’the technology itself’.

In particular, I tried to carefully examine many aspects from an accessibility perspective. Having accumulated experience through my activities as a public institution web/app citizen evaluator, I wanted to thoroughly check whether the participating teams’ services could actually help diverse users from that perspective. I hoped that the questions I asked would contribute, even if just a little, to improving their services.

Above all, I hope that many services in our country can be provided equally to all users without information discrimination. Being able to access the same information regardless of disability, age, or environment—I believe that is the foundation of true public services.

Question 1: How Will You Handle Integration with Existing Services That Don’t Fully Meet Accessibility Standards?

My first question was about the challenge of integrating with existing web and app services that don’t fully comply with accessibility standards.

It’s true that most government services are making efforts to comply with accessibility guidelines. However, reflecting on my experience as a public institution web/app UI/UX evaluator, I still frequently encounter cases where accessibility falls short in certain services or specific elements.

So when new AI-powered public services integrate with these existing systems, how can they absorb or compensate for those limitations?

This question wasn’t meant to simply point out accessibility compliance issues, but to consider realistic problems that will be faced during actual operations.

Question 2: When Depending on a Single Service, How Will You Handle Failures?

My second question was about dependency on specific services.

Most AI-based public services are designed on top of existing platforms or external services. This naturally leads to a question:

“If this service experiences an outage, what happens to the users?”

When fully dependent on a single service, failure response and user satisfaction must be considered together.

That said, I also considered another point while asking this question. The idea was more of a supplementary convenience service rather than a replacement for existing services, and the platform they depended on was one that has been operated with considerable thought given to accessibility and stability in Korea, which gave me the impression it was a realistic choice.

What I found important at this point wasn’t whether it was completely independent, but how clearly the service’s role was defined.

Question 3: How Is Communication with Users with Disabilities Designed?

The third question was personally the most important part for me today.

In real-world situations, users with disabilities, including those with visual impairments, often experience greater difficulties in the communication process when problems occur than in using the service itself.

• Chat support that doesn’t consider accessibility
• Structures that make immediate response difficult
• Single communication channels with no alternatives

These factors significantly impair user experience regardless of how good the service’s features are.

In response, the participating team proposed solutions using various means including supplementary IoT devices, and the approach of diversifying communication channels was very impressive.

What I realized again through this question is that accessibility is not just about screen layout, but about designing how to communicate with users.

Well-Prepared Teams Become Clearer When Faced with Questions

One thing I felt through today’s evaluation:

Well-prepared teams become clearer in their service direction as more questions are asked.

When preparing for competitions or presentations, simply creating presentation materials isn’t enough. You need to fully understand the content you want to convey and prepare responses to anticipated questions in advance. Most importantly, you need conviction and confidence in your topic.

Teams that don’t avoid questions, acknowledge assumptions and limitations, and can explain next steps will inevitably receive good evaluations.

The teams that received good evaluations today were completely understandable results in this context. They were brimming with confidence, and their thorough preparation was clearly evident in how they answered questions. Well-prepared teams don’t waver in the face of questions—instead, they use those questions as opportunities to explain their service even more clearly.

This Experience Became an Opportunity to Reflect on My Side Projects

This evaluation experience went beyond just observing a competition—it became an opportunity to examine the direction of my current and future side projects.

Until now, I had been focused on questions like:

• Is it technically feasible?
• Is it worth implementing?

But the questions that kept coming to mind at today’s event were a bit different:

• Who will actually use this service?
• Where will things get stuck when problems occur?
• Are accessibility and communication being considered from the start?

These questions felt like ones I should ask before focusing on features in any service I create going forward.

For AI-powered side projects to not remain just technical demos, I reconfirmed that the perspectives required for public services—operations, accessibility, failure response, communication structure—must be considered from the beginning.

What Today’s Experience Left Me

From a financial perspective, today was definitely a minus. Transportation costs and time were all on me.

Nevertheless, this was clearly a meaningful experience:

Above all, I thought that if there are competitions like this, I’d like to participate next time as a competitor, or as an accessibility expert on the judging panel. I hope that someday I can stand in this position as an expert judge, recognized for my expertise in accessibility.

Closing

Today, I was there in a role of asking questions rather than giving answers.

And those questions ultimately came back to myself.

What standards am I using to evaluate services?

And what kind of services do I want to create?

I am currently preparing to obtain IAAP’s CPACC (Certified Professional in Accessibility Core Competencies) and WAS (Web Accessibility Specialist) certifications. After obtaining these certifications, I want to go beyond simply expanding my personal career and contribute to improving the overall accessibility level of services in our country. Today’s evaluation panel activity made that desire even stronger.

I continue to contemplate what my role can be going forward. Whether as an evaluator, developer, or consultant—in whatever form—I want to maintain a perspective that places accessibility at the center when looking at services.

AI is becoming a universal part of society. In this flow, I hope that services that can embrace everyone, including the elderly and people with disabilities, become not something special but something natural. I believe that is an ordinary society—a society for everyone.

I hope that when I look back at this record later, it will tell me “that direction wasn’t wrong.”

Addendum

Since evaluation details are confidential, this post is merely a record of personal perspectives and experiences. It does not represent any judgments or results.

However, I think it would be sufficient if this serves as a small reference for someone interested in the intersection of AI, accessibility, and public services.