[SR-13605] Build script ‘swift/utils/build-parser-lib’ fails with undefined symbols about swift-syntax HOT 13 CLOSED

cc @akyrtzi

from swift-syntax.

Thanks for reporting this!

The toolchain is not relevant for this, the issue needs to be fixed by changes in the swift repo. The parser library build aims to avoid linking in the libclangAST.a library, since that is not necessary for its purpose, but occasionally symbol references leak out and trigger linker errors.

On another note, given that you want to use the parser library for iOS, you should know that the app store does not accept apps with embedded `.dylibs`, so an open task that we need to make it applicable for iOS apps is to build it as a `.framework`

from swift-syntax.

Comment by Ivan Lugo (JIRA)

Hey there @akyrtzi, thanks much for the info! I grok the issue here - seems like I'll need to continue testing with the current macOS implementation if I want to use this directly. I completely forgot about the .dylib usage limitation on iOS, and I again appreciate your word on it. Definitely puts a bit of a damper on the mobile use case there.

I mentioned my unfamiliarity with the build process, but I'd be happy to assist in either experimenting with or testing out symbol fixes and framework builds. If it's something you think I'd be able to pick up quickly enough to be dangerous and throw up a PR, I'll happily give it some more cycles.

Furthering the tangent, I've been looking around at other solutions for Swift source parsing and analysis, and found open repositories like SourceKitten that use other (what I guess can be called) analysis backends. My goal is using the above would the same: knowing fairly high level information about a source file - declarations and identifiers, file positioning with trivia (for manual rendering of the file), and 'structures' like beginnings and ends of code components (classes, structs, etc). Do you have any thoughts on the tool above with respects to its ability to provide useful source information, remaining compatible with the language, or using it across devices?

EDIT: Did a bit more reading and realized SourceKitten also relies on SourceKit and other mac-only libraries. Drat

from swift-syntax.

> I mentioned my unfamiliarity with the build process, but I'd be happy to assist in either experimenting with or testing out symbol fixes and framework builds. If it's something you think I'd be able to pick up quickly enough to be dangerous and throw up a PR, I'll happily give it some more cycles.

I can look into the linker issue sometime next week. I also have a TODO task to get it built as a framework.

> Did a bit more reading and realized SourceKitten also relies on SourceKit and other mac-only libraries.

SwiftSyntax is way much easier to embed but it only gives you syntactic information. With SourceKit you'd have the full power of the Swift compiler but it would involve essentially building a swift toolchain for use from iOS. I'm not sure if there is a convenient way to do that or not, if this is a direction you want to explore I'd suggest asking in the forums for some direction.

from swift-syntax.

Comment by Ivan Lugo (JIRA)

Lots to learn here. I'll watch for linker news in the mean time. The iOS toolchain rings some bells across various blogs and articles I've seen, so I'll definitely look into it. I've done some cross positing with this as I originally though it was something I was doing wrong, so I'll pivot on those topics.

For now, I'll shift toward making the mobile app a bit more tethered. I've got an idea to throw some of the results from the mac to the mobile device via something like the multipeer kit, and if it can handle the heavy lifting for the time being, it'd be a neat showcase anyway of having devices work together.

Cheers again for all your info and help.

from swift-syntax.

Linker errors are fixed via swiftlang/swift#34187

from swift-syntax.

Comment by Ivan Lugo (JIRA)

Fantastic. I just pulled everything down and within a few minutes I had a fresh iOS-capable library to embed. This simplifies a ton of my current project work.

If I could inquire about one more thing: I've disabled bitcode support for my project target, and it seems to build just fine without it. Is it possible to build these libraries with support for this? I'm done some more grepping about to find bitcode comments in the build scripts, but again I'm a bit out of my depth.

Cheers and many, many thanks for your fix. I'm looking forward to running this tomorrow!

from swift-syntax.

It's a bit unclear to me, are you asking about building the library with bitcode? Not sure about that, there could be some CMake variable to set.

from swift-syntax.

Comment by Ivan Lugo (JIRA)

Yep! (sorry for the late response) I did take a bit of dive through the cmake flags and flipped a few switches without much success. That's totally out of scope, though, and I don't have any real reason to need it so far, but I figured knowing ahead of time would be good.

from swift-syntax.

Comment by Ivan Lugo (JIRA)

Hey again!

I've come back to say that everything was working great for the last month or so, but I seem to have hit the same problem again. It looks like sometime in the last few weeks, more of those symbols became visible to the build script. Taking your lead, I attempted to short-circuit the methods that were found to be problematic and I was able to correctly build the dynamic library as before. I've attached a git patch here to show the changes I made (if this is more appropriate as a pull request, please let me know!)

diff.patch

So there's that, and now I've hit something brand new:

ld: unknown option: -no_adhoc_codesign

The above error occurs when building with the latest trunk toolchain, Development Snapshot 2020-11-08 (a). Some context for it is

-L/usr/lib/swift -Xlinker -add_ast_path -Xlinker /Users/ivan.lugo/Library/Developer/Xcode/DerivedData/LookAtThat_AppKit-ebjswrdgtfgsvwgyyusmcptsrhya/Build/Intermediates.noindex/LookAtThat_AppKit.build/Debug/LookAtThat_AppKit.build/Objects-normal/x86_64/LookAtThat_AppKit.swiftmodule -l_InternalSwiftSyntaxParser -Xlinker -no_adhoc_codesign -Xlinker -dependency_info -Xlinker /Users/ivan.lugo/Library/Developer/Xcode/DerivedData/LookAtThat_AppKit-ebjswrdgtfgsvwgyyusmcptsrhya/Build/Intermediates.noindex/LookAtThat_AppKit.build/Debug/LookAtThat_AppKit.build/Objects-normal/x86_64/LookAtThat_AppKit_dependency_info.dat

My best guess for the above, using https://keith.github.io/xcode-man-pages/ld.1.html#no_adhoc_codesign as a reference, leads me to believe some breaking changes were added to support the new Apple Silicon paradigm. Could be wrong though.

Any advice for this new set of interesting problems?

from swift-syntax.

Comment by Ivan Lugo (JIRA)

Hey again

from swift-syntax.

Thanks for looking into it, I fixed the linker errors here with only the minimum necessary changes: swiftlang/swift#34976

> ld: unknown option: -no_adhoc_codesign

I'm not sure where this flag is coming from, it doesn't look like it's coming from the Swift compiler; are you sure it's not coming from project changes?

Also BTW, why are you "building with the latest trunk toolchain"? The parser library+SwiftSyntax are independent of a particular toolchain, you can use them while building your project with the toolchain from latest Xcode.

from swift-syntax.

Closing this because the linker issues from invoking build-parser-lib seem to be fixed. If you don’t agree that this should be closed, please re-open the issue.

from swift-syntax.