Comments (2)
artemcm
commented on July 28, 2024
2
Thanks for the investigation @finagolfin.
It seems to me that fixing this ought to be done in the frontend's print-target-info logic, since the driver fully delegates to it and then relies on the resulting target info as the source of truth all over the driver logic.
Moreover, it would be tough to implement this logic in the new driver since the target info is queried when the target triple is not known to the driver yet, relying on the target info query to yield the default host triple when one is not specified to the compilation.
Potentially something like this could work, but I will need to do more testing on it:
swiftlang/swift#72409
from swift-driver.
finagolfin
commented on July 28, 2024
Alright, spent a couple hours looking into this, looks to me like a long-standing latent bug in the swift-frontend.
Some background first: the legacy C++ Driver has long had a getResourceDirPath() method, which first checks for an explicitly specified -resource-dir, then looks for usr/lib/swift/ in an explicitly specified -sdk, and finally, if neither flag was specified, just looks for one next to the compiler, ie usr/bin/swiftc/../../lib/swift/. All the toolchains in the old C++ driver appear to use this core logic, and that's why the legacy C++ Driver command above works.
This new swift-driver has no such method and simply delegates finding the Swift resource directory to the swift-frontend's -print-target-info option. However, the swift-frontend only checks for an explicit -resource-dir then looks for one next to the compiler, ie it doesn't look for a Swift resource directory in the -sdk like the old C++ Driver does.
I was surprised that the swift-frontend never looked in a passed-in -sdk for Swift resources, but the following commands convinced me that is indeed the case. First, I tried to use the latest trunk toolchain on linux with the legacy C++ Driver to build with an -sdk that contains 5.10 resource files, which should fail when compiling the object file but doesn't:
> ./swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swiftc -v swift/test/Interpreter/hello_toplevel.swift -sdk swift-5.10-RELEASE-ubi9/ -disallow-use-new-driver
<unknown>:0: warning: legacy driver is now deprecated; consider avoiding specifying '-disallow-use-new-driver'
Swift version 6.0-dev (LLVM d1625da873daa4c, Swift bae6450bf96dceb)
Target: x86_64-unknown-linux-gnu
/home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swift-frontend -frontend -c -primary-file swift/test/Interpreter/hello_toplevel.swift -target x86_64-unknown-linux-gnu -disable-objc-interop -sdk swift-5.10-RELEASE-ubi9 -color-diagnostics -plugin-path /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/lib/swift/host/plugins -plugin-path /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/local/lib/swift/host/plugins -module-name hello_toplevel -o /tmp/hello_toplevel-7b72d3.o
<unknown>:0: warning: libc not found for 'x86_64-unknown-linux-gnu'; C stdlib may be unavailable
/home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swift-autolink-extract /tmp/hello_toplevel-7b72d3.o -o /tmp/hello_toplevel-739e5c.autolink
/home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/clang -fuse-ld=gold -pie -Xlinker -rpath -Xlinker swift-5.10-RELEASE-ubi9/usr/lib/swift/linux swift-5.10-RELEASE-ubi9/usr/lib/swift/linux/x86_64/swiftrt.o /tmp/hello_toplevel-7b72d3.o --sysroot swift-5.10-RELEASE-ubi9 @/tmp/hello_toplevel-739e5c.autolink -L swift-5.10-RELEASE-ubi9/usr/lib/swift/linux -lswiftCore --target=x86_64-unknown-linux-gnu -v -o hello_toplevel
It gets to the link step before failing, this suggests the swift-frontend compilation step is not using the Swift 5.10 resource directory. If I specify the 5.10 -resource-dir explicitly, it fails when compiling as expected:
> ./swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swiftc -v swift/test/Interpreter/hello_toplevel.swift -sdk swift-5.10-RELEASE-ubi9/ -disallow-use-new-driver -resource-dir swift-5.10-RELEASE-ubi9/usr/lib/swift
<unknown>:0: warning: legacy driver is now deprecated; consider avoiding specifying '-disallow-use-new-driver'
Swift version 6.0-dev (LLVM d1625da873daa4c, Swift bae6450bf96dceb)
Target: x86_64-unknown-linux-gnu
/home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swift-frontend -frontend -c -primary-file swift/test/Interpreter/hello_toplevel.swift -target x86_64-unknown-linux-gnu -disable-objc-interop -sdk swift-5.10-RELEASE-ubi9 -color-diagnostics -resource-dir swift-5.10-RELEASE-ubi9/usr/lib/swift -plugin-path /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/lib/swift/host/plugins -plugin-path /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/local/lib/swift/host/plugins -module-name hello_toplevel -o /tmp/hello_toplevel-dcf2f5.o
<unknown>:0: warning: libc not found for 'x86_64-unknown-linux-gnu'; C stdlib may be unavailable
<unknown>:0: error: module compiled with Swift 5.10 cannot be imported by the Swift 6.0 compiler: swift-5.10-RELEASE-ubi9/usr/lib/swift/linux/Swift.swiftmodule/x86_64-unknown-linux-gnu.swiftmodule
To hammer the point home, let's use the new swift-driver, which I noted above always passes a -resource-dir to the swift-frontend, regardless if one was passed in to the new swift-driver or not:
> ./swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swiftc -v swift/test/Interpreter/hello_toplevel.swift -sdk swift-5.10-RELEASE-ubi9/ warning: Could not read SDKSettings.json for SDK at: /home/fina/swift-5.10-RELEASE-ubi9
Swift version 6.0-dev (LLVM d1625da873daa4c, Swift bae6450bf96dceb)
Target: x86_64-unknown-linux-gnu
/home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swift-frontend -frontend -c -primary-file swift/test/Interpreter/hello_toplevel.swift -target x86_64-unknown-linux-gnu -disable-objc-interop -sdk /home/fina/swift-5.10-RELEASE-ubi9 -color-diagnostics -empty-abi-descriptor -resource-dir /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/lib/swift -module-name hello_toplevel -plugin-path /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/lib/swift/host/plugins -plugin-path /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/local/lib/swift/host/plugins -o /tmp/TemporaryDirectory.5mibhA/hello_toplevel-1.o
<unknown>:0: warning: libc not found for 'x86_64-unknown-linux-gnu'; C stdlib may be unavailable
/home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swift-autolink-extract /tmp/TemporaryDirectory.5mibhA/hello_toplevel-1.o -o /tmp/TemporaryDirectory.5mibhA/hello_toplevel-2.autolink
/home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/clang -fuse-ld=gold -pie -Xlinker -rpath -Xlinker /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/lib/swift/linux /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/lib/swift/linux/x86_64/swiftrt.o /tmp/TemporaryDirectory.5mibhA/hello_toplevel-1.o @/tmp/TemporaryDirectory.5mibhA/hello_toplevel-2.autolink --sysroot /home/fina/swift-5.10-RELEASE-ubi9 -L /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/lib/swift/linux -lswiftCore --target=x86_64-unknown-linux-gnu -v -o hello_toplevel
This passes the swift-frontend compilation step and we can see it is not looking in the -sdk for the resource directory, passing in -resource-dir /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/lib/swift to the swift-frontend instead. If I specify the 5.10 resource directory explicitly, it fails as expected:
> ./swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swiftc -v swift/test/Interpreter/hello_toplevel.swift -sdk swift-5.10-RELEASE-ubi9/ -resource-dir swift-5.10-RELEASE-ubi9/usr/lib/swift
warning: Could not read SDKSettings.json for SDK at: /home/fina/swift-5.10-RELEASE-ubi9
Swift version 6.0-dev (LLVM d1625da873daa4c, Swift bae6450bf96dceb)
Target: x86_64-unknown-linux-gnu
/home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/bin/swift-frontend -frontend -c -primary-file swift/test/Interpreter/hello_toplevel.swift -target x86_64-unknown-linux-gnu -disable-objc-interop -sdk /home/fina/swift-5.10-RELEASE-ubi9 -color-diagnostics -empty-abi-descriptor -resource-dir swift-5.10-RELEASE-ubi9/usr/lib/swift -module-name hello_toplevel -plugin-path /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/lib/swift/host/plugins -plugin-path /home/fina/swift-DEVELOPMENT-SNAPSHOT-2024-03-13-a-ubi9/usr/local/lib/swift/host/plugins -o /tmp/TemporaryDirectory.ZDGmZ2/hello_toplevel-1.o
<unknown>:0: warning: libc not found for 'x86_64-unknown-linux-gnu'; C stdlib may be unavailable
<unknown>:0: error: module compiled with Swift 5.10 cannot be imported by the Swift 6.0 compiler: swift-5.10-RELEASE-ubi9/usr/lib/swift/linux/Swift.swiftmodule/x86_64-unknown-linux-gnu.swiftmodule
To sum up, the old C++ Driver always looked in -sdk for a Swift resource directory, but the swift-frontend and now the new swift-driver don't. That appears to be a bug that hasn't been noticed, perhaps because almost nobody is shipping full platform SDKs that aren't situated next to the compiler, ie a separate platform sysroot containing both the platform C headers/libraries and Swift resource files. I suppose it's also possible we're always explicitly specifying the -resource-dir, so this issue wasn't noticed.
Now that we're starting to ship cross-compilation SDK bundles that may have such full platform SDKs, we should close this hole up. It will need to be fixed in the swift-frontend or at least overridden in this new swift-driver.
Pinging @artemcm and @DougGregor on how I should fix this, since you guys wrote some of the relevant logic, and @compnerd and @MaxDesiatov, since this fix will affect your work on Windows and SDK bundles. Since this bug was never noticed this long, I figure the fix won't break anything, but this will affect all toolchains, so I want to run it by you all.
from swift-driver.
Related Issues (20)
- Crash due to an assertion in libswiftCore
- Xcode 14.3 RC 2 (14E222b) not pass '-strict-implicit-module-context' flag to swift-frontend
- `Process.findExecutable(arguments[0])` gives an unexpected result HOT 5
- Windows builds broken when querying target-info from libSwiftScan HOT 2
- Swift Potentially Uses the Wrong Swift Driver HOT 15
- "swift build" command duplicates error messages HOT 8
- iOS get error Dependency for P1: Debug:SwiftDriver Compilation Requirements app normal arm64 com.apple.xcode.tools.swift.compiler is not absolute
- swift help build doesn't work when it's documented to
- Test suite broken on Windows due to CAS HOT 1
- SPM doesn't enumerate source files properly in different configurations HOT 6
- tests crash on windows HOT 1
- `swift run` usage suggestion should be fixed HOT 1
- Program is several times slower when compiled with swiftc than when compiled with XCode due to retain&release calls HOT 1
- “swift ./-” does the same as “swift -” HOT 5
- Using -g for non-macho targets can cause driver planning errors
- Leaking temporary directory and file /tmp/TemporaryDirectory.*/dummyInput-*.swift
- Duplicate diagnostics being output in Swift 6 HOT 2
- Batch mode in swift-driver passes wrong object name
- Compiling on Terminal generates a warning
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from swift-driver.