Requirements feedback about silver HOT 5 CLOSED

I agree with most of what Wilco has said here.

The problem is that we don't currently have consensus on how to measure such things. That is a VERY different problem. Silver (or maybe WCAG 2.2) may need to develop new metrics. Conformance MUST be a pass or fail statement.

I would rephrase that: The CURRENT concept of conformance is that it is a pass / fail statement. As we all know, the problem is that conformance applies to a page, which often is a container for a lot of things. So there are often situations where the page fails, strictly speaking, because something on the page (possibly something very minor) failed.

If we retain the point of reference of "the page and all its states" (to keep things simple), I think the real issue is to determine tolerances - to find a way to define what kind of defect would be acceptable for something to still pass an SC. This is something which I know every evaluator has to deal with, and practically deals with on a regular basis, today often without making those tolerance choices explicit. The simplest thing would be to introduce a 'non-critical' flag - the issue is then captured for remedies and not swept under the carpet, but does not prevent conformance.

There have been attempts to create more fine-grained instance-based metrics approaches (UWEM) but in practice, these have not caught on. In my opinion, quantitative instance based metrics approaches are just too complex and messy - it might work like a breeze if everything were testable automagically, but we know that the majority of SCs need at least semi-automatic checks.

The alternative is to accept human judgment as the final arbitrator for pass / fail, and define processes where evaluator judgments (and a case base it may rest upon) can get calibrated / challenged / adapted. The advantage is that both qualitative and quantitative issues can enter the judgment - and both are usually needed when assessing actual content. It's these two questions: A. Is this issue non-critical? B. How many of these issues on the page do I accept before PASS flips over into FAIL?

So if we keep a pass/fail based concept of conformance, one useful addition might be to flag whether content conforms with minor issues (within tolerances), or conforms straight away, and possibly make that as transparent and quantifiable as possible. (Re-reading the comment above, I see that Wilco has made a similar point under 1.2.2 Measurable: 100% passing content)

from silver.

I think we do need a way to express conformance in terms of websites and web apps.

I am not sure I understand. Can you elaborate? Do you see a category difference between them? (many websites have traits of applications). Do you envisage a different process for evaluating one or the other? At the moment the main issue in my experience is that you have to choose whether to lump states together and basically discuss them as part of one page (a process with several steps may be treated as one page) or define them as separate pages. Would you make that choice solely based on the URL characteristics? Or what are you after?

from silver.

Partly addressed by #589

from silver.

Several comments are out of date and have been fully or partially addressed. PR #732 should address the remaining comments.

from silver.

@alastc will add a suggestion for accessibility supported.

from silver.