[Performance]: dotnet/runtime tests are bottlenecked by MSBuild performance about msbuild HOT 6 OPEN

fyi @rainersigwald

from msbuild.

I thought the North Star here was far fewer assemblies?

from msbuild.

You mean fewer projects? We've already performed a significant amount of consolidation. We can continue to shrink this down over time, but it will still likely be a lot of projects. And it doesn't look like csc has a problem with this scale.

from msbuild.

The other reason this came up is that the 1ES pipeline work revealed how extremely complicated the runtime pipeline is. It's easily the largest pipeline in dotnet, exceeding standard AzDO rate limits by two orders of magnitude. The runtime tests currently represent an unresolvable complication in our AzDO pipelines wherein we would like to remove the extra job + upload + join, but can't do so if it would regress time-to-test-results by 20 min.

from msbuild.

You mean fewer projects? We've already performed a significant amount of consolidation. We can continue to shrink this down over time, but it will still likely be a lot of projects.

Right. I'm not defending MSBuild. But why are more than say O(100) assemblies needed? I'm just curious, not suggesting I have any special insight.

from msbuild.

I believe it's a mix of: Native AOT tests can't be mixed because more code has trimming side effects, some tests contaminate the process such that it can't be reused, some simply have code patterns that can't easily be combined, and finally test consolidation still ends up with a fair amount of manual effort and we can only commit so much per unit time.

from msbuild.