Hermes Engine takes 2.5 to 19 times longer to process promises than Chakra about hermes-windows HOT 7 OPEN

hermes-promise-test.xlsx

from hermes-windows.

From your steps, you mentioned opening the RNW developer menu. Are you running a debug build of your application?

Chakra exposes it's APIs to react-native-windows in the form of ABI-safe C functions. It lets RNW debug builds use a release version of Chakra. The Hermes boundary is not yet ABI safe, and MSVC has different debug snd release ABIs. This currently forces RNW to use an "optimized debug build" of Hermes, which performs worse than its release optimized build.

This constraint go away I think, with work to move JS engine interface to be ABI safe. But right now Hermes direct debugging, like web debugging, is not representative of release build performance. Debug-time developer experience is still super important though. Did switching to Hermes direct debugging degrade that for your app?

from hermes-windows.

Also from your steps you are not comparing Chakra vs Hermes at all. When you enable remote debugging, you are running the JS inside a webbrowser (either Edge or Chrome probably) - both of which use V8. When running the JS in a browser you are running inside a very different JS environment including a completely different promise implementation. There are so many differences between how things run in "Remote Debugging" env, that it should not be used for any kind of performance testing.

from hermes-windows.

From your steps, you mentioned opening the RNW developer menu. Are you running a debug build of your application?

Chakra exposes it's APIs to react-native-windows in the form of ABI-safe C functions. It lets RNW debug builds use a release version of Chakra. The Hermes boundary is not yet ABI safe, and MSVC has different debug snd release ABIs. This currently forces RNW to use an "optimized debug build" of Hermes, which performs worse than its release optimized build.

This constraint go away I think, with work to move JS engine interface to be ABI safe. But right now Hermes direct debugging, like web debugging, is not representative of release build performance. Debug-time developer experience is still super important though. Did switching to Hermes direct debugging degrade that for your app?

Yes, all tests were performed with a Debug build initially and were essentially live switched using the developer menu.

For our actual application, which my test repository is imitating, we do indeed have about a 20 second load screen time which is similar to the 10K Nested Promise test in the referenced repository.

I have gone back and rebuilt and tested in a Release build. The performance of Hermes in Release was still worse than webdebugging. It's still about 8 seconds for a nested 10K promise test. The others are under 1 second but still have the worst performance. I have updated the application to alert with the measure and have updated the spreadsheet with the data.
hermes-promise-test.xlsx

from hermes-windows.

Also from your steps you are not comparing Chakra vs Hermes at all. When you enable remote debugging, you are running the JS inside a webbrowser (either Edge or Chrome probably) - both of which use V8. When running the JS in a browser you are running inside a very different JS environment including a completely different promise implementation. There are so many differences between how things run in "Remote Debugging" env, that it should not be used for any kind of performance testing.

I can go ahead and provide you with updated results from Chakra in release for the repository in question.

from hermes-windows.

I have tested Chakra with both Debug and Release. Results are as follows:
For Debug, Hermes takes 3.5 to 19 times longer than Chakra.
For Release, Hermes takes 2.5 to 7.7 times longer than Chakra.

Important to note; for the larger Nested Promise tests, Chakra Debug outperforms Hermes in Release.
hermes-promise-test.xlsx

from hermes-windows.

@AlexLablaiksSAP I am reopening the issue.
As we discussed with your team today, I clearly see the same issue playing with your repro.
From what I see in the WPA (Windows Perf Analysis tooling), where are no significant delays in the JS thread. The most of time is spent inside of the Hermes interpreter. The function that seems to be the most expensive is the Array.indexOf().

The next steps:

• Discuss this issue with Meta's Hermes team
• I would like to see the Hermes CPU sampling profiler results. I need to use the latest Hermes bit for it.

from hermes-windows.