pkg/_macros/test/executor/executor_test is failing about sdk HOT 10 OPEN

//cc @jakemac53

from sdk.

area-test, type-bug

The pkg/_macros/test/executor/executor_test tests are failing on Windows ARM64 and x64 configurations, resulting in a RuntimeError instead of the expected Pass.

from sdk.

around area-test might need to be made more explicit that it should prefer the affected functional area, or otherwise weighted down

I agree with that (and thanks for the feedback on the meta / triaging bot issue). Here, is it area-vm tests that are failing? We don't have a specific area for macros.; we're using area-language for non-implementation discussion issues.

We could improve the prompt for these 'test' cases. We're also not using the tuned model yet, just explicit directions in a prompt; switching to the tuned model should also improve accuracy here.

from sdk.

The area should be area-pkg afaik (the tests in question are in pkg/_macros).

from sdk.

@a-siva do you have a failure you can link me? I did recently land a fix for this test, and I am not sure how to validate if it was the same error or something different etc.

from sdk.

Here is the first time the failure showed up logs
(I believe it was a Timeout initially and then turned into a RuntimeError later).

from sdk.

The area should be area-pkg afaik (the tests in question are in pkg/_macros).

Yes, but area-pkg is not a great area- itself. It's not triaged - there's no team that does secondary triage on it or ultimately has responsibility for items in that area.

from sdk.

Ok yeah looking at the log, that is the one that should be fixed. Is this still failing?

from sdk.

It was failing on the windows bots until I created this issue and approved the failure

from sdk.

Moved this to the "no milestone/release" bucket. It isn't directly tied to any release and I haven't discovered any reasonable way to try and investigate the failure. I intend to just leave this as open tech debt for some time. There are no actual affected users, just a failing test in an environment that I can't reproduce.

from sdk.