Changes can be tracked here:
https://github.com/KhronosGroup/3DC-Certification/tree/fix/certification

It appears that the evaluation software generates a human-readable, styled report. Is it reasonable to have JSON or other computer-friendly output format(s)? Can the output format and file be specified on the command line to facilitate batch processing?

Fixed here:
#10

Please use https://github.khronos.org/3DC-Sample-Viewer/

This document states, that the submission should be 24 bits:
https://github.com/KhronosGroup/3DC-Certification/blob/main/documents/test-plan.adoc#4-how-to-create-the-test-results
However, the test images itself are 32 bits.
This solves the issues, that both is allowed:
#8

The following asset is a valid glTF asset. However, e.g. for occlusion, normal maps are used. From my perspective, this asset should be enhanced.
https://github.com/KhronosGroup/3DC-Certification/tree/main/models/GreenChair

Reading the evaluation software, it appears that there are some assumptions made about the rendered file names. It looks as if the software processes all rendered images in one go. Is it possible to change this so that the reference and test images are provided from the command line. This also impacts another issue discussing output files and formats.

For certifiaction, it is 100% obvious, that the resolution of the canvas has to be used:
https://github.com/KhronosGroup/glTF/blob/master/specification/2.0/schema/camera.perspective.schema.json

e.g. use SSIM and store a value for each test image, which has to be "beaten".

https://en.wikipedia.org/wiki/Structural_similarity
https://en.m.wikipedia.org/wiki/Structural_similarity

Sometimes when I run the program everything works correctly and runs to completion. Other times I get an error.

C:\...\evaluation>python evaluation.py  --rep .. --name 12 --output ..\Results\12 ..\12  1>..\Results\12.log
Traceback (most recent call last):
  File "evaluation.py", line 148, in <module>
    reference_image, candidate_image = normalize_images(reference_image, candidate_image)
  File "evaluation.py", line 113, in normalize_images
    print(f"??  Resizing {candidate_path} from {candidate.shape[:2]} to {reference.shape[:2]}")
  File "C:\Applications\Python38\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-1: character maps to <undefined>

There is only the "gather_image:..." output line when the code fails. When it succeeds. there are also 3 data lines for each commerce model by camera position.

Change the Certification Program document to reflect current browser landscape. New versions of browsers are coming out about every other week/browser. Web viewers should be referenced to all browsers and the entry should just indicate the OS, browser, and version used for rendering.

Something similar may need to apply to rendering engines and not the full UX skin. This is especially true for tools such as FB's Spark tool. It gets updated about once/week; however, the rendering engine update cycle is about once per quarter.

Please include a README that describes how to use the evaluation software, especially for non-python developers. For example, on my local machine, I get a message "No module names 'skimage'". I am not sure what to do about that.