Giter Club home page Giter Club logo

Comments (2)

artemcm avatar artemcm commented on July 28, 2024 2

Thanks for the investigation @finagolfin.
It seems to me that fixing this ought to be done in the frontend's print-target-info logic, since the driver fully delegates to it and then relies on the resulting target info as the source of truth all over the driver logic.

Moreover, it would be tough to implement this logic in the new driver since the target info is queried when the target triple is not known to the driver yet, relying on the target info query to yield the default host triple when one is not specified to the compilation.

Potentially something like this could work, but I will need to do more testing on it:
swiftlang/swift#72409

from swift-driver.

finagolfin avatar finagolfin commented on July 28, 2024

Alright, spent a couple hours looking into this, looks to me like a long-standing latent bug in the swift-frontend.

Some background first: the legacy C++ Driver has long had a getResourceDirPath() method, which first checks for an explicitly specified -resource-dir, then looks for usr/lib/swift/ in an explicitly specified -sdk, and finally, if neither flag was specified, just looks for one next to the compiler, ie usr/bin/swiftc/../../lib/swift/. All the toolchains in the old C++ driver appear to use this core logic, and that's why the legacy C++ Driver command above works.

This new swift-driver has no such method and simply delegates finding the Swift resource directory to the swift-frontend's -print-target-info option. However, the swift-frontend only checks for an explicit -resource-dir then looks for one next to the compiler, ie it doesn't look for a Swift resource directory in the -sdk like the old C++ Driver does.

I was surprised that the swift-frontend never looked in a passed-in -sdk for Swift resources, but the following commands convinced me that is indeed the case. First, I tried to use the latest trunk toolchain on linux with the legacy C++ Driver to build with an -sdk that contains 5.10 resource files, which should fail when compiling the object file but doesn't:

> ./swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swiftc -v swift/test/Interpreter/hello_toplevel.swift -sdk swift-5.10-RELEASE-ubi9/ -disallow-use-new-driver
<unknown>:0: warning: legacy driver is now deprecated; consider avoiding specifying '-disallow-use-new-driver'
Swift version 6.0-dev (LLVM d1625da873daa4c, Swift bae6450bf96dceb)
Target: x86_64-unknown-linux-gnu
/home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swift-frontend -frontend -c -primary-file swift/test/Interpreter/hello_toplevel.swift -target x86_64-unknown-linux-gnu -disable-objc-interop -sdk swift-5.10-RELEASE-ubi9 -color-diagnostics -plugin-path /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/lib/swift/host/plugins -plugin-path /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/local/lib/swift/host/plugins -module-name hello_toplevel -o /tmp/hello_toplevel-7b72d3.o
<unknown>:0: warning: libc not found for 'x86_64-unknown-linux-gnu'; C stdlib may be unavailable
/home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swift-autolink-extract /tmp/hello_toplevel-7b72d3.o -o /tmp/hello_toplevel-739e5c.autolink
/home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/clang -fuse-ld=gold -pie -Xlinker -rpath -Xlinker swift-5.10-RELEASE-ubi9/usr/lib/swift/linux swift-5.10-RELEASE-ubi9/usr/lib/swift/linux/x86_64/swiftrt.o /tmp/hello_toplevel-7b72d3.o --sysroot swift-5.10-RELEASE-ubi9 @/tmp/hello_toplevel-739e5c.autolink -L swift-5.10-RELEASE-ubi9/usr/lib/swift/linux -lswiftCore --target=x86_64-unknown-linux-gnu -v -o hello_toplevel

It gets to the link step before failing, this suggests the swift-frontend compilation step is not using the Swift 5.10 resource directory. If I specify the 5.10 -resource-dir explicitly, it fails when compiling as expected:

> ./swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swiftc -v swift/test/Interpreter/hello_toplevel.swift -sdk swift-5.10-RELEASE-ubi9/ -disallow-use-new-driver -resource-dir swift-5.10-RELEASE-ubi9/usr/lib/swift
<unknown>:0: warning: legacy driver is now deprecated; consider avoiding specifying '-disallow-use-new-driver'
Swift version 6.0-dev (LLVM d1625da873daa4c, Swift bae6450bf96dceb)
Target: x86_64-unknown-linux-gnu
/home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swift-frontend -frontend -c -primary-file swift/test/Interpreter/hello_toplevel.swift -target x86_64-unknown-linux-gnu -disable-objc-interop -sdk swift-5.10-RELEASE-ubi9 -color-diagnostics -resource-dir swift-5.10-RELEASE-ubi9/usr/lib/swift -plugin-path /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/lib/swift/host/plugins -plugin-path /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/local/lib/swift/host/plugins -module-name hello_toplevel -o /tmp/hello_toplevel-dcf2f5.o
<unknown>:0: warning: libc not found for 'x86_64-unknown-linux-gnu'; C stdlib may be unavailable
<unknown>:0: error: module compiled with Swift 5.10 cannot be imported by the Swift 6.0 compiler: swift-5.10-RELEASE-ubi9/usr/lib/swift/linux/Swift.swiftmodule/x86_64-unknown-linux-gnu.swiftmodule

To hammer the point home, let's use the new swift-driver, which I noted above always passes a -resource-dir to the swift-frontend, regardless if one was passed in to the new swift-driver or not:

> ./swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swiftc -v swift/test/Interpreter/hello_toplevel.swift -sdk swift-5.10-RELEASE-ubi9/            warning: Could not read SDKSettings.json for SDK at: /home/fina/swift-5.10-RELEASE-ubi9
Swift version 6.0-dev (LLVM d1625da873daa4c, Swift bae6450bf96dceb)
Target: x86_64-unknown-linux-gnu
/home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swift-frontend -frontend -c -primary-file swift/test/Interpreter/hello_toplevel.swift -target x86_64-unknown-linux-gnu -disable-objc-interop -sdk /home/fina/swift-5.10-RELEASE-ubi9 -color-diagnostics -empty-abi-descriptor -resource-dir /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/lib/swift -module-name hello_toplevel -plugin-path /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/lib/swift/host/plugins -plugin-path /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/local/lib/swift/host/plugins -o /tmp/TemporaryDirectory.5mibhA/hello_toplevel-1.o
<unknown>:0: warning: libc not found for 'x86_64-unknown-linux-gnu'; C stdlib may be unavailable
/home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swift-autolink-extract /tmp/TemporaryDirectory.5mibhA/hello_toplevel-1.o -o /tmp/TemporaryDirectory.5mibhA/hello_toplevel-2.autolink
/home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/clang -fuse-ld=gold -pie -Xlinker -rpath -Xlinker /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/lib/swift/linux /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/lib/swift/linux/x86_64/swiftrt.o /tmp/TemporaryDirectory.5mibhA/hello_toplevel-1.o @/tmp/TemporaryDirectory.5mibhA/hello_toplevel-2.autolink --sysroot /home/fina/swift-5.10-RELEASE-ubi9 -L /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/lib/swift/linux -lswiftCore --target=x86_64-unknown-linux-gnu -v -o hello_toplevel

This passes the swift-frontend compilation step and we can see it is not looking in the -sdk for the resource directory, passing in -resource-dir /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/lib/swift to the swift-frontend instead. If I specify the 5.10 resource directory explicitly, it fails as expected:

> ./swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swiftc -v swift/test/Interpreter/hello_toplevel.swift -sdk swift-5.10-RELEASE-ubi9/ -resource-dir swift-5.10-RELEASE-ubi9/usr/lib/swift
warning: Could not read SDKSettings.json for SDK at: /home/fina/swift-5.10-RELEASE-ubi9
Swift version 6.0-dev (LLVM d1625da873daa4c, Swift bae6450bf96dceb)
Target: x86_64-unknown-linux-gnu
/home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swift-frontend -frontend -c -primary-file swift/test/Interpreter/hello_toplevel.swift -target x86_64-unknown-linux-gnu -disable-objc-interop -sdk /home/fina/swift-5.10-RELEASE-ubi9 -color-diagnostics -empty-abi-descriptor -resource-dir swift-5.10-RELEASE-ubi9/usr/lib/swift -module-name hello_toplevel -plugin-path /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/lib/swift/host/plugins -plugin-path /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/local/lib/swift/host/plugins -o /tmp/TemporaryDirectory.ZDGmZ2/hello_toplevel-1.o
<unknown>:0: warning: libc not found for 'x86_64-unknown-linux-gnu'; C stdlib may be unavailable
<unknown>:0: error: module compiled with Swift 5.10 cannot be imported by the Swift 6.0 compiler: swift-5.10-RELEASE-ubi9/usr/lib/swift/linux/Swift.swiftmodule/x86_64-unknown-linux-gnu.swiftmodule

To sum up, the old C++ Driver always looked in -sdk for a Swift resource directory, but the swift-frontend and now the new swift-driver don't. That appears to be a bug that hasn't been noticed, perhaps because almost nobody is shipping full platform SDKs that aren't situated next to the compiler, ie a separate platform sysroot containing both the platform C headers/libraries and Swift resource files. I suppose it's also possible we're always explicitly specifying the -resource-dir, so this issue wasn't noticed.

Now that we're starting to ship cross-compilation SDK bundles that may have such full platform SDKs, we should close this hole up. It will need to be fixed in the swift-frontend or at least overridden in this new swift-driver.

Pinging @artemcm and @DougGregor on how I should fix this, since you guys wrote some of the relevant logic, and @compnerd and @MaxDesiatov, since this fix will affect your work on Windows and SDK bundles. Since this bug was never noticed this long, I figure the fix won't break anything, but this will affect all toolchains, so I want to run it by you all.

from swift-driver.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.