netlify / binary-info Goto Github PK

After waste my two days to figuring out how to deploy simple Go function to Netlify, I finally ended up this repository from Netlify support: https://answers.netlify.com/t/getting-no-functions-were-found-in-functions-directory-error-with-go-build/53107

I actually love the idea by using goblin library to detect which runtime we are currently using for the specified binary.

Description

I have a simple Go function that I want to deploy on Netlify. But netlify-cli always says: No Functions were found in functions directory. But wait. I 100% sure that function is there and something is wrong with build command. So I just dig dive from the netlify-cli to check what's going on under the hood. And finally ended up here.

What I have done

I cloned the repository, bumped goblin from 0.2 to 0.5.1 (latest), replaced ./tests/data/hello-world-go with mine. Then ran cargo test. And tadaaa. It failed.

Binary Details

$ file api

api: Mach-O 64-bit executable x86_64

$ exiftool api

ExifTool Version Number         : 12.27
File Name                       : api
Directory                       : .
File Size                       : 64 MiB
File Modification Date/Time     : 2022:03:10 21:47:19+03:00
File Access Date/Time           : 2022:03:10 23:00:52+03:00
File Inode Change Date/Time     : 2022:03:10 21:47:19+03:00
File Permissions                : -rwxr-xr-x
File Type                       : Mach-O executable
File Type Extension             :
MIME Type                       : application/octet-stream
CPU Architecture                : 64 bit
CPU Byte Order                  : Little endian
CPU Type                        : x86 64-bit
CPU Subtype                     : i386 (all) 64-bit
Object File Type                : Demand paged executable
Object Flags                    : (none)

$ uname -a

Darwin 21.2.0 Darwin Kernel Version 21.2.0: Sun Nov 28 20:28:54 PST 2021; root:xnu-8019.61.5~1/RELEASE_X86_64 x86_64

$ go env

GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/furkan.turkal/Library/Caches/go-build"
GOENV="/Users/furkan.turkal/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/furkan.turkal/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/furkan.turkal/go"
GOPRIVATE=""
GOPROXY="direct"
GOROOT="/usr/local/Cellar/go/1.17.6/libexec"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.17.6/libexec/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.17.6"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/nq/vxjjn3311fg4q263qsxrghpcpzgp66/T/go-build3722978256=/tmp/go-build -gno-record-gcc-switches -fno-common"

P.S: I did not override any Go flags either in my build script or netlify.toml file.

Problem

This is why $ netlify build command does not see my function:

$ readelf -a api

readelf: Error: Not an ELF file - it has the wrong magic bytes at the start

$ readelf -x .note.go.buildid api

readelf: Error: Not an ELF file - it has the wrong magic bytes at the start

Because we enabled following features in Cargo.toml: features = ["alloc", "elf32", "elf64", "endian_fd"]

MY GO BINARY IS NOT AN ELF FILE??? Wow. My file type is: Mach-O executable

I enabled the following features, but couldn't make tests pass: "elf32", "elf64", "mach32", "mach64", "pe32", "pe64", "archive", "endian_fd". Probably I'm close but here's what I have done:

    let mach = match Mach::parse(data) {
        Ok(mach) => {
            match mach {
                Mach::Fat(f) => {
                }
                Mach::Binary(b) => {
                    let sym = match b.symbols {
                        None => {}
                        Some(s) => {
                            // TODO
                        }
                    };

                }
            }
        },
        _ => return Ok(None),
    };

CONCERNS

I am able to see my buildid my issuing following command:

$ go tool buildid api

Gn1rf6fgQXQfCuwWMc95/7fGmW1EHdtMgmbrEjMYu/Gc_UQp7DnnVjdzcbHCNj/rIEVguKbmGsWY9t4fadj

!!! Passing -trimpath -ldflags=-buildid= flag to compiler will break runtime detection. Some developer might pass this flag to get better reproducible builds.

CONSIDER PROVIDING BETTER UX FOR DEVELOPERS

Everything began with the following issue:

No Functions were found in functions directory

Please clarify this message if we don't support given binary' file type:

We found <foo> binary in your functions directory. But file type is <bar>. We currently support following types: <baz, qux, etc>

Consider rebuild your binary in different format.

Bla bla bla...

Ref: netlify/go-functions-example#2

Additional Notes

Please clarify what kind of file types Netlify support, what is the right way to build Go binaries on the documentation: https://docs.netlify.com/functions/build-with-go/

I filed this issue because I'm someone who enjoys investigating strange things and who likes to give feedback. I'm not sure who to ping here, sorry in advance if I'm pinging wrong maintainers:

elf-cam:
@calavera

zip-it-and-ship-it:detect_runtime
@eduardoboucas

build:core_steps
@ehmicky

Feel free to create cross issues to link this one to provide better UX for developers and customers.

Thanks for reading. I'm really exhausted, and I think I deserve a good sleep now.

We should create integration tests for elf-cam. What we want to test here is if the generated package can be used as promised.

First step run

wasm-pack build --target nodejs

This generates ./pkg folder with the complete package which we can import in javascript/typescript.

import {} from './pkg/'

If we write the tests in typescript we can also test the generated types easily.

This is the example I used for manual testing:

import { readFileSync } from "fs";
import { detect, Platform } from "./pkg/binary-info.js";

const buffer = readFileSync("./tests/data/nsd_3.0.2_Darwin_arm64");

const x = detect(buffer);
console.log(x);
console.log(x.platform, x.arch, x.runtime);
console.log(x.platform === Platform.Darwin);

This is an issue generated by (for-internal-use-only)github-tools

Description

This repository has exceeded the development grace period, and the repo owner must decide if branch protections should be enforced.
If this repository houses code used in production in any capacity, branch protections must be enforced. Read (for-internal-use-only)here for more details

What do I need to do?

• You can control whether or not branch protections are enforced by exclusively creating either of the labels:
  'enforce-branch-protections: true' or 'enforce-branch-protections: false'
• If there is no branch protection control label, this issue will be recreated.
• Once you have created either label, you can close this issue.
• If you have issues or questions, please reach out to #internal-security-n-compliance on slack.

The hardcoded string rust_eh_personality has been hidden in Rust production builds for awhile now. Everything else about Rust serverless functions works (including functions that retain this string in debug builds).