visionOS Build is succeeding but I know it is broken about swiftpackageindex-server HOT 8 CLOSED

Wait a second!!

env DEVELOPER_DIR=/Applications/Xcode-15.3.0-Beta.2.app xcrun xcodebuild -IDEClonedSourcePackagesDirPathOverride=$workDir/.dependencies -skipMacroValidation -skipPackagePluginValidation -derivedDataPath $workDir/.derivedData build -scheme TextStory-Package -destination generic/platform=xros

TextStory-Package

This is a dependency of this package, but not the correct scheme for this package itself.

from swiftpackageindex-server.

Yup that's it. I had copy-pasta'ed the scheme name. This command fails for me locally:

xcodebuild build -scheme TextFormation -destination generic/platform=visionOS

from swiftpackageindex-server.

Huh, that's bizarre. We shouldn't be picking that scheme, I'll need to check what's going on with our scheme heuristics.

Thanks for the report, @mattmassicotte !

from swiftpackageindex-server.

So I know what's happening but I don't know exactly why and how to best address it.

Your package lists four schemes when you run xcodebuild -list -json:

{
  "workspace" : {
    "name" : "TextFormation",
    "schemes" : [
      "TextFormation",
      "TextStory",
      "TextStory-Package",
      "TextStoryTesting"
    ]
  }
}

This is the input to our package scheme detection heuristics. One of them is to filter for *-Package schemes and if there's only one, that's the one we pick. That's clearly wrong in this case, so the first impulse is to reject this heuristic.

If we do that, we probably fail to build some (many?) packages. We could recover some if we only pick *-Package if it's the only scheme.

Looking at the JSON one might think to check if there's a scheme with the same name as the workspace name. The problem with that is that the workspace name is really the checkout directory base name. We check out all repositories into <tempdir>/checkout and when you run xcodebuild -list -json on that you get

{
  "workspace" : {
    "name" : "checkout",
    "schemes" : [
      "TextFormation",
      "TextStory",
      "TextStory-Package",
      "TextStoryTesting"
    ]
  }
}

😖

Of course, we could look at the project URL's last path element ... and we'd be placing another piece on our jenga tower of scheme guesses.

I wanted to get a feel how broken our assumption is that picking the autogenerated *-Package is a good idea. As I recall this scheme isn't / shouldn't be generated for dependencies.

I just picked a random repo that seems similar in that it

• has a dependency
• has no embedded .xcodeproj or .xcodeworkspace
• has no scheme: set in .spi.yml

i.e. a project that should be structurally equivalent to TextFormation. Runestone seems to fit that bill.

So here's xcodebuild -list -json for Runestone:

{
  "workspace" : {
    "name" : "Runestone",
    "schemes" : [
      "Runestone"
    ]
  }
}

None of its dependencies show up. There's no *-Package scheme at all.

I suspected that perhaps the custom target logic in TextFormation's Package.swift

let swiftSettings: [SwiftSetting] = [
	.enableExperimentalFeature("StrictConcurrency")
]

for target in package.targets {
	var settings = target.swiftSettings ?? []
	settings.append(contentsOf: swiftSettings)
	target.swiftSettings = settings
}

file might mess with scheme autogeneration and so I removed it (from a fresh checkout, to avoid caching issues). Still, Xcode lists all four schemes, as above. I have no idea why it's generating these schemes here and not for Runestone.

Moreover, I now wonder and worry what it does for other repos.

I think there are a few things to consider here:

1. Should we deploy the hack of looking for a clone URL base name match with a scheme name? If so, how lenient should be be here? Obviously case insensitive but what about prefixes like swift- or suffixes like -iOS which we do see frequently. That's a nice looking hole and I'm sure there's a rabbit at the end of it.
2. Do we have a (much) bigger problem here where we're building the wrong schemes in some/many cases? (NB: of 6964 packages that have Package.resolved reported in, 3965 (57%) have no dependencies. So a good chunk are safe by default. Runestone indicates that even among those with dependencies there are some/many that will be ok.)
3. How can we get a better view of the problem so we can decide how to fix it?
4. Can we get some info on how Xcode autogenerates schemes?

It's tempting to hack something together along the lines of 1) but our heuristics are getting messier and messier and it might be better to hold off.

An immediate fix for Matt's repo is for it to add a scheme: TextFormation entry to its .spi.yml file to ensure we do the right thing.

In order to figure out how big a problem we've got, we could do the following:

• capture the xcodebuild -list -json output and send it with the build report
• also send the scheme we've picked (via heuristics or an .spi.yml override)

This would allow us to run a full analysis across all packages and see how well this is working.

I'm leaning towards spending the effort on 3. rather than patch this over via 1.

We should also try and get in touch with someone on the Xcode team to find out about scheme autogeneration so there's less guesswork on our part (4.). I vaguely recall talking to a couple of folks very early on when creating the build system but this was never any formal documentation, more along the lines "this should work".

from swiftpackageindex-server.

TextStory itself is special, in that it has multiple targets. I think Xcode will only auto-generate schemes for package dependencies that have multiple targets. Also, I believe the "-Package" scheme is only present in that situation as well. But I'm not 100% sure about either of those things.

I think what you need to do is pull out the package name field from the Package.swift file. From there, you list the schemes. If there is a name-Package you build that. Otherwise, you build name. Could that work?

from swiftpackageindex-server.

I think that might be best, yes. It would mean another dump-package in the builder (which triggers package resolution) but it shouldn't actually be that bad, because xcodebuild -list also triggers a package resolution. The only problem I can see is that they don't share the "resolution cache".

I'll give that a go, that's certainly a more sensible approach that peeking at the URL and feels like it might solve the whole problem without adding more instrumentation. (However, that's perhaps still a good idea to do at some point.)

from swiftpackageindex-server.

Fixed here: https://gitlab.com/finestructure/swiftpackageindex-builder/-/merge_requests/262

• deploy

from swiftpackageindex-server.

This is now deployed. I believe you've actually fixed the visionOS build now, haven't you @mattmassicotte ? Let me know if not and I'll retrigger builds on main to correct the compatibility matrix!

from swiftpackageindex-server.