In the TKML Report, (turnkeyml==1.1.2), benchmark_status column currently shows: <

Benchmark Status Shows Error when a previous Build Stage failed. about turnkeyml HOT 4 OPEN

vgodsoe commented on August 26, 2024

Benchmark Status Shows Error when a previous Build Stage failed.

from turnkeyml.

Comments (4)

jeremyfowers commented on August 26, 2024 1

I just tried to reproduce this on the head of TKML source code, and couldn't. My guess is the bug was in tkml 1.1.2 but has already been fixed. Let's monitor for the bug the next time we make a big report.

On head of tkml:

Info: The stats for ~/.cache/turnkey\linear_selftest_76af2f62\turnkey_stats.yaml are:
author: selftest
builtin_model_script: https://github.com/onnx/turnkeyml/tree/cbd3abc173b4bc93d06131f6199d08ef2f6bc143/models/selftest/linear.py     
class: LinearTestModel
evaluations:
  57651c20:
    benchmark_status: not_started
    build_status: error
    device_type: x86
    error_log: "Exporting PyTorch to ONNX\nTraceback (most recent call last):\n  File\
      \ \"C:\\work\\turnkeyml\\src\\turnkeyml\\build\\stage.py\", line 135, in fire_helper\n\
      \    state = self.fire(state)\n  File \"C:\\work\\turnkeyml\\src\\turnkeyml\\\
      build\\export.py\", line 224, in fire\n    raise ValueError(\"Gotcha\")\nValueError:\
      \ Gotcha"
    iterations: 100
    runtime: ort
    selected_sequence_of_stages:
    - export_pytorch
    - optimize_onnx
    stage_duration:export_pytorch: 0.3145256042480469
    stage_duration:optimize_onnx: '-'
    stage_status:export_pytorch: error
    stage_status:optimize_onnx: not_started

The benchmark_status is correctly set to "not started"

from turnkeyml.

Recommend Projects