anchore / quill Goto Github PK
View Code? Open in Web Editor NEWSimple mac binary signing from any platform
License: Apache License 2.0
Simple mac binary signing from any platform
License: Apache License 2.0
Issue used to upload assets to github for reference in markdown files. (Do not delete this issue)
Today if you try to pass a universal binary to many commands it wont work:
$ quill describe /bin/ls
[0000] INFO quill version: [not provided]
[0000] ERROR 1 error occurred:
* unable to parse macho formatted file with blacktop: invalid magic number in record at byte 0x0
We need to unwrap each contained binary and sign accordingly (I think... more research needed here). The unwrapping / wrapping could be dealt with by https://github.com/anchore/go-macholibre/ upon detecting a universal binary without changing the overall signing/notarization logic too much.
What would you like to be added:
Please provide binary packages for NetBSD/amd64.
Why is this needed:
I tried building https://github.com/anchore/syft
Its Makefile's bootstrap
target tries installing quill using the curl method, and this failed on my system with:
curl -sSfL https://raw.githubusercontent.com/anchore/quill/main/install.sh | sh -s -- -b ./.tmp/ v0.2.0
[info]\033[0m using release tag='v0.2.0' version='0.2.0' os='netbsd' arch='amd64' \033[0m
\033[0;31m\033[1m[error]\033[0m could not find release asset for os='netbsd' arch='amd64' format='tar.gz' \033[0m
\033[0;31m\033[1m[error]\033[0m failed to install quill \033[0m
Additional context:
Thos seems to only sign the binary.
I sssume I also need to sign the .app.
I also need to include some ffmpeg binaries that he golang binary depends on.
if you can let me know what’s supported currently with quill that would be cool.
I'm trying to work through the signing process with a developer id application cert (p12).
I've followed the directions and am not able to get notarization to correctly work.
The signing continues without error, but upon notarization I get a few errors.
How to reproduce it (as minimally and precisely as possible):
codesign -vvv --deep --strict $artifact
returns CSSMERR_TP_NOT_TRUSTED In architecture: x86_64
.spctl -vvv --assess --type exec $artifact
returns CSSMERR_TP_NOT_TRUSTED
.The notarization attempt responds with 2 errors in the json response:
"issues": [
{
"severity": "error",
"code": null,
"path": "myapp-darwin-amd64",
"message": "The signature of the binary is invalid.",
"docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087735",
"architecture": "x86_64"
},
{
"severity": "error",
"code": null,
"path": "myapp-darwin-amd64",
"message": "The signature does not include a secure timestamp.",
"docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087733",
"architecture": "x86_64"
}
]
quill describe
against my original and -with-chain
p12 file: * unable to parse single-arch binary: unable to parse macho formatted file with blacktop: invalid magic number in record at byte 0x0
.I've gotten quill to mark the file as signed with a non-apple developer cert (pfx/p12), but what I understand is that for mac Gatekeeper to allowlist the app I need to use only an Apple cert. I've iterated on this for days and not sure where else to go with it.
export QUILL_P12_PASSWORD=''
export QUILL_SIGN_PASSWORD=${QUILL_P12_PASSWORD}
export P12_APP=pathtodeveloperid-application-cert.p12
export P12_WITH_CHAIN=pathtodeveloperid-application-cert-with-chain.p12
export QUILL_SIGN_P12=$P12_WITH_CHAIN
# ensure full chain is attached for the sign/notarization process via quill
quill p12 attach-chain $P12_APP
quill sign-and-notarize --p12 $P12_WITH_CHAIN \
--notary-key-id 'notarykeyid' \
--notary-key 'pathtoAuthKey_zzzzzzz.p8' \
--notary-issuer 'guidplacedhere' \
$artifact
# Tried explicit timestamp add and no difference, so removed.
# --timestamp-server 'http://timestamp.apple.com/ts01' \
# other follow-up commands I've used.
security find-identity
codesign -vvv --deep --strict $artifact
spctl -vvv --assess --type exec $artifact
What would you like to be added:
Support the same ability as cosign --entitlements <xml-file>
where the input would be an XML Plist which entitlements the app needs to run.
What happened:
Right now quill will attempt to notarize any valid macho binary:
$ quill notarize bin/syft_test
⠧ Notarizing binary [status "invalid", poll 3] bin/syft_test
error: 1 error occurred:
* submission result is Invalid:
{
"logFormatVersion": 1,
"jobId": "...",
"status": "Invalid",
"statusSummary": "Archive contains critical validation errors",
"statusCode": 4000,
"archiveFilename": "syft_test-...-7bd5e189",
"uploadDate": "2022-10-18T00:49:29.779Z",
"sha256": "...",
"ticketContents": null,
"issues": [
{
"severity": "error",
"code": null,
"path": "syft_test-...-7bd5e189/syft_test",
"message": "The binary is not signed with a valid Developer ID certificate.",
"docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087721",
"architecture": "x86_64"
},
{
"severity": "error",
"code": null,
"path": "syft_test-...-7bd5e189/syft_test",
"message": "The signature does not include a secure timestamp.",
"docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087733",
"architecture": "x86_64"
},
{
"severity": "error",
"code": null,
"path": "syft_test-...-7bd5e189/syft_test",
"message": "The executable does not have the hardened runtime enabled.",
"docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087724",
"architecture": "x86_64"
}
]
}
What you expected to happen:
Something that stops the user from attempting to notarize something that will obviously fail:
$ quill notarize bin/syft_test
⠙ Notarizing binary [initializing client] bin/syft_test
error: 1 error occurred:
* binary file is not a signed darwin macho executable file
Anything else we need to know?:
Could that could fix this:
func IsSigned(path string) (bool, error) {
f, err := os.Open(path)
if err != nil {
return false, err
}
defer f.Close()
if macholibre.IsUniversalMachoBinary(f) {
mf, err := blacktopMacho.NewFatFile(f)
if err != nil {
return false, err
}
defer mf.Close()
for _, arch := range mf.Arches {
sig := arch.CodeSignature()
if sig == nil {
return false, nil
}
return len(sig.CMSSignature) > 0, nil
}
return true, nil
}
mf, err := blacktopMacho.NewFile(f)
if mf != nil || err != nil {
return false, err
}
defer mf.Close()
sig := mf.CodeSignature()
if sig == nil {
return false, nil
}
return len(sig.CMSSignature) > 0, nil
}
However, this produces Unsupported code directory
notices to stdout, which mangles the TUI (track upstream issue here blacktop/go-macho#11)
What happened:
I'm testing quill to implement into our process for code signing osx binaries. I've exported Apple Developer ID as .p12 cert and password as Quill P12 environment variables (as noted in README). While signing does complete, and I can see certificates embedded into Mach-o binaries, our application cannot start because TeamID is not set.
What you expected to happen:
I expected TeamID to be set
How to reproduce it (as minimally and precisely as possible):
Build MacOSX application with xcode and try to sign it with quill.
Anything else we need to know?:
I've taken a peek into codebase, however, I am not a go developer so I might've understood it wrong, but I couldn't find teamid signature in signing go module.
Environment:
Application: quill
Version: 0.4.1
BuildDate: 2023-08-25T19:47:39Z
GitCommit: 8129c5808e3717838dfb5a32f886a48fa24ed8a9
GitDescription: v0.4.1
Platform: darwin/arm64
GoVersion: go1.18.10
Compiler: gc```
- OS:
```[~] ❱❱❱ sw_vers
ProductName: macOS
ProductVersion: 13.4
BuildVersion: 22F66
What would you like to be added:
Would be nice to have an easy way to install quill on a mac via Homebrew
Why is this needed:
Makes install on mac easer, for example on a mac when I try to install per the install instructions I get:
» curl -sSfL https://raw.githubusercontent.com/anchore/quill/main/install.sh | sh -s -- -b /usr/local/bin
[info] using release tag='v0.4.0' version='0.4.0' os='darwin' arch='arm64'
install: /usr/local/bin//quill: Permission denied
[error] failed to install quill
using Zsh
Additional context:
There seems to already be a formula in Homebrew 'main' named quill
:
» brew info quill
==> quill: stable 3.3.1 (bottled), HEAD
C++17 Asynchronous Low Latency Logging Library
https://github.com/odygrd/quill
Not installed
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/q/quill.rb
License: MIT
==> Dependencies
Build: cmake ✔
==> Requirements
Required: macOS >= 10.15 (or Linux) ✔
==> Options
--HEAD
However, I think creating your own tap will 'get around' this so users can do homebrew install anchore-quill/quill
or similar
What happened:
When running unit tests with go 1.20 I see the following failure:
--- FAIL: TestSign (2.92s)
--- FAIL: TestSign/sign_the_syft_binary_(with_a_password) (0.00s)
sign_test.go:197:
Error Trace: /Users/wagoodman/code/quill/quill/sign_test.go:197
Error: Received unexpected error:
unable to parse certificate 1 of 1: x509: certificate contains duplicate extensions
Test: TestSign/sign_the_syft_binary_(with_a_password)
FAIL
We're on go 1.18 now, bumping to 1.20 you will be able to see the same thing locally.
What you expected to happen:
No test failures!
How to reproduce it (as minimally and precisely as possible):
make unit
while using go 1.20.
Anything else we need to know?:
More details on the current change in behavior for x509 certificate verification https://groups.google.com/g/golang-checkins/c/nishT5TtWeo .
What would you like to be added:
Today we support adding a single code directory with sha256 hashes of each page in the binary. By default the codesign
utility attaches two code directories, one for sha256 hashes and another one for sha1 hashes.
Why is this needed:
This can help with backwards compatibility for older macs (more details needed).
What happened:
I'm getting this error code signing command is not the last loader command, so cannot remove it (easily) without corrupting the binary
quite often with binaries built with golang 1.21. Looking at them with machoview
, there's a LC_SEGMENT_64 (__DWARF)
command after the LC_CODE_SIGNATURE
command, so it's expected that isSigningCommandLastLoader
reports false
What you expected to happen:
No error, and a signed binary :)
How to reproduce it (as minimally and precisely as possible):
I can reproduce this with the binaries from https://github.com/crc-org/crc or https://github.com/crc-org/vfkit
Anything else we need to know?:
Looking at the code quill/macho/file.go and at the outptu of machoview , it looks like it should be doable to either reorder the load commands (they are only pointers to other parts of the file), or to reuse the signature command instead of first removing it, and then recreating it. The signature command, while not last in the load commands array, contains an offset + len which correspond to the end of the file (ie offset + len == total filesize) so we could replace this signature with the newly computed one.
Environment:
I've been seeing this with golang 1.21 on an m1 laptop.
$ sw_vers
ProductName: macOS
ProductVersion: 14.3.1
BuildVersion: 23D60
We need to use app specific passwords with Team ID (https://developer.apple.com/documentation/technotes/tn3147-migrating-to-the-latest-notarization-tool#App-specific-password). We do not have App Store applications to notarize and just notarize binaries.
Is this supported? I see TeamID is not according to #147.
If not, this is extremely important as almost every one of our projects are just golang binaries that need signing and notarization to be distributed on github for example, Quill is the perfect solution for this if it supports app specific password/team ID.
Our current scripts (which run on mac) are as follows:
codesign --sign "Developer ID Application" --force -o runtime --timestamp "$BINARY"
notarytool submit
if ! INFO=$(xcrun notarytool submit --team-id TTXXXXXX --apple-id "${NOTARIZE_USERNAME}" --password "${NOTARIZE_PASSWORD}" --wait "$TEMP"); then
echo "problem with notarization command -- run manually to determine failure reason"
exit 3
fi
What would you like to be added:
Command Line Flag for allowing to set a different timeout, seems like to be at 15 minutes at default.
Why is this needed:
Don't know why but the notarization takes longer then 15 minutes in our case.
$ quill submission list
┌──────────────────────────────────────┬──────────────────────────────────────────────────────────────────────────────────┬─────────────┬──────────────────────────┐
│ ID │ NAME │ STATUS │ CREATED │
├──────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────┼─────────────┼──────────────────────────┤
│ 0c510253-2a03-472b-ac8f-c0da3ac0f6a7 │ app-edca7e2199c8e982935560ebf5d3071dcf7b85c6e7f87cfd10b110f969ca9a7b-cb544cda │ In Progress │ 2024-04-11T12:58:17.718Z │
│ 4a787033-b066-437b-ac57-dd759e845b63 │ app-ce72c46413e669f30a0797937f9367305c2af7cd25b34a35613fa7dff36f41b5-377f754a │ In Progress │ 2024-04-11T12:58:10.674Z │
└──────────────────────────────────────┴──────────────────────────────────────────────────────────────────────────────────┴─────────────┴──────────────────────────┘
Apple says 98% of the apps should be notarized within 15 minutes. Seems not to be valid in our case. 🤔
Additional context:
I am using goreleaser
in my CI pipeline which fails entirely when timing out. Would be awesome to increase the timoeut a little bit further, even if the entire GitHub action runtime increases.
Using quill
0.2.0 to sign and notarize an arm64 binary and I've been getting HTTP 400 error from the notarization server. Signing and notarization with quill
has worked without hiccups throughout this year, but last night started getting an error, but it ended up working after retry and no changes to the usage of quill
.
This morning started getting the error again even after retries:
* notarization failed: unable to start submission: http status="400 Bad Request": body="<HTML><HEAD>\n<TITLE>Bad Request</TITLE>\n</HEAD><BODY>\n<H1>Bad Request</H1>\nYour browser sent a request that this server could not understand.<P>\nReference #7.4c822c17.1692969095.ec8f4c44\n</BODY>\n</HTML>\n"
Is there something that could be done in quill
to address this or is this an issue on the apple web service for notarization?
Windows PE binaries also support codesigning and I think the mechanisms are very similar. I'm not 100% sold that support for signing windows binaries should be done in this tool, but wanted to at least put this out there.
What would you like to be added:
Support the same ability as cosign -r <reqs>
to override the default designated requirements.
What happened: Attempted to sign (and attach the fullchain), which resulted in the following error:
failed to verify certificate chain: x509: unhandled critical extension
What you expected to happen: Either sign/notarization or attach-fullchain to succeed
How to reproduce it (as minimally and precisely as possible):
Keychain
, export by selecting top-level certificate and private keygo run github.com/anchore/quill/cmd/quill@latest p12 describe <p12>
with successgo run github.com/anchore/quill/cmd/quill@latest p12 attach-chain <p12>
with failureAnything else we need to know?: It's totally possible I'm doing something wrong here...
Environment:
version
command: v0.4.1
(go version go1.22.4 darwin/arm64
)cat /etc/os-release
or similar): uname -a
: Darwin pikachu.local 23.5.0 Darwin Kernel Version 23.5.0: Wed May 1 20:12:58 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T6000 arm64
What happened:
When going to update to the latest version of bubbletea I found the following error:
# github.com/erikgeiser/promptkit/textinput
../../go/pkg/mod/github.com/erikgeiser/[email protected]/textinput/model.go:102:8: input.BackgroundStyle undefined (type "github.com/charmbracelet/bubbles/textinput".Model has no field or method BackgroundStyle)
What you expected to happen:
promptkit
and bubbletea
should be bumped to compatible versions if we ever upgrade the UI library
Anything else we need to know?:
context: erikgeiser/promptkit#22
Today we require that users run quill attach-chain <p12>
to craft a new p12 file that has the full cert chain for use in signing. Without this signing verification will fail. Another way to do this is to bake the Apple certs directly into quill such that the full chain (if it does not already exist in the p12) could be looked up at runtime.
The codesign
utility does something like this by looking up the remaining certs in the chain in the system keychain. This won't be possible for all end users of quill since they may not be running on a mac (there is no reason to assume that apple root certs will be on a linux box, for instance).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.