Comments (8)
uhop
commented on August 16, 2024
Could you share your Dockerfile so I can repro the problem? All I need is the OS + Node part.
from node-re2.
Uzlopak
commented on August 16, 2024
Thank you for your fast feedback.
The error occurs in the few lines I provided in the above Dockerfile. Don't get me wrong... I get a binary but I don't know if RE2 is potentially buggy when used, as my c++ skills are limited. Also I have to potentially explain my colleagues if this is a problem or not.
In later build steps I copy the node_modules/re2 folder into the production image.
from node-re2.
uhop
commented on August 16, 2024
The error in the original post indicates that a downloaded image does not work. Next, it proceeds to build from sources. That's the expected behavior. I would look into why the downloaded image didn't work, but it shouldn't matter.
Regarding copying the folder into the production image — that is unsupported. re2 is a C++ project based on NAN and built by node-gyp. It is compiled to a platform-specific image. It means that the result depends on the details of OS, installed system libraries, a CPU architecture, processor supported features, Node version, and so on. That's why there is a fail-safe mechanism, which verifies the binary and rebuilds it if there is any problem with it.
For example, I used to build my project (not re2-related) on one type of AWS EC2 VM and deploy on bigger AWS EC2 VM only to get random crashes related to "unsupported instructions". So generally I would advise against it.
from node-re2.
uhop
commented on August 16, 2024
I googled for the error and it turned out to be a common problem for Alpine:
- grpc/grpc#6126
- https://stackoverflow.com/a/55568352/26394 (other answers outline the problem too).
- https://dustri.org/b/error-loading-shared-library-ld-linux-x86-64so2-on-alpine-linux.html
It looks like there is a missing link in /lib and Alpine doesn't come with the glibc loader by default requiring installing libc6-compat explicitly. I don't know if both are required as in the last link, or only one of them is sufficient (first two links). Please try and let me know so I can update Wiki with this information.
Thank you in advance!
from node-re2.
Uzlopak
commented on August 16, 2024
I already had tried those potential solutions, but none of them worked. Maybe it is no issue, as I get a binary file but just get that error.
FROM node:12.18.1-alpine as re2-builder
WORKDIR /opt
RUN apk add python make g++ libc6-compat \
&& npm install [email protected]
Building docker image...
Sending build context to Docker daemon 669.1MB
Step 1/25 : FROM node:12.18.1-alpine as re2-builder
---> 06a4a7b5263d
Step 2/25 : WORKDIR /opt
---> Using cache
---> 824ba1be20d2
Step 3/25 : RUN apk add python make g++ libc6-compat && npm install [email protected]
---> Running in d82870d4a550
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
(1/22) Installing binutils (2.33.1-r0)
(2/22) Installing gmp (6.1.2-r1)
(3/22) Installing isl (0.18-r0)
(4/22) Installing libgomp (9.2.0-r4)
(5/22) Installing libatomic (9.2.0-r4)
(6/22) Installing mpfr4 (4.0.2-r1)
(7/22) Installing mpc1 (1.1.0-r1)
(8/22) Installing gcc (9.2.0-r4)
(9/22) Installing musl-dev (1.1.24-r2)
(10/22) Installing libc-dev (0.7.2-r0)
(11/22) Installing g++ (9.2.0-r4)
(12/22) Installing libc6-compat (1.1.24-r2)
(13/22) Installing make (4.2.1-r2)
(14/22) Installing libbz2 (1.0.8-r1)
(15/22) Installing expat (2.2.9-r1)
(16/22) Installing libffi (3.2.1-r6)
(17/22) Installing gdbm (1.13-r1)
(18/22) Installing ncurses-terminfo-base (6.1_p20200118-r4)
(19/22) Installing ncurses-libs (6.1_p20200118-r4)
(20/22) Installing readline (8.0.1-r0)
(21/22) Installing sqlite-libs (3.30.1-r2)
(22/22) Installing python2 (2.7.18-r0)
Executing busybox-1.31.1-r9.trigger
OK: 212 MiB in 38 packages
npm WARN deprecated [email protected]: request has been deprecated, see https://github.com/request/request/issues/3142
> [email protected] install /opt/node_modules/re2
> node scripts/install-from-cache.js --artifact build/Release/re2.node
Trying https://github.com/uhop/node-re2/releases/download/1.15.0/linux-x64-72.br ...
Writing to build/Release/re2.node ...
> [email protected] verify-build /opt/node_modules/re2
> node scripts/verify-build.js
internal/modules/cjs/loader.js:1188
return process.dlopen(module, path.toNamespacedPath(filename));
^
Error: Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /opt/node_modules/re2/build/Release/re2.node)
at Object.Module._extensions..node (internal/modules/cjs/loader.js:1188:18)
at Module.load (internal/modules/cjs/loader.js:986:32)
at Function.Module._load (internal/modules/cjs/loader.js:879:14)
at Module.require (internal/modules/cjs/loader.js:1026:19)
at require (internal/modules/cjs/helpers.js:72:18)
at Object.<anonymous> (/opt/node_modules/re2/re2.js:3:13)
at Module._compile (internal/modules/cjs/loader.js:1138:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1158:10)
at Module.load (internal/modules/cjs/loader.js:986:32)
at Function.Module._load (internal/modules/cjs/loader.js:879:14)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] verify-build: `node scripts/verify-build.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] verify-build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm WARN Local package.json exists, but node_modules missing, did you mean to install?
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2020-06-22T05_51_14_903Z-debug.log
Building locally ...
> [email protected] rebuild /opt/node_modules/re2
> node-gyp rebuild
make: Entering directory '/opt/node_modules/re2/build'
CXX(target) Release/obj.target/re2/lib/addon.o
CXX(target) Release/obj.target/re2/lib/new.o
CXX(target) Release/obj.target/re2/lib/exec.o
CXX(target) Release/obj.target/re2/lib/test.o
CXX(target) Release/obj.target/re2/lib/match.o
CXX(target) Release/obj.target/re2/lib/replace.o
CXX(target) Release/obj.target/re2/lib/search.o
CXX(target) Release/obj.target/re2/lib/split.o
CXX(target) Release/obj.target/re2/lib/to_string.o
CXX(target) Release/obj.target/re2/lib/accessors.o
CXX(target) Release/obj.target/re2/lib/util.o
CXX(target) Release/obj.target/re2/vendor/re2/bitstate.o
CXX(target) Release/obj.target/re2/vendor/re2/compile.o
CXX(target) Release/obj.target/re2/vendor/re2/dfa.o
CXX(target) Release/obj.target/re2/vendor/re2/filtered_re2.o
CXX(target) Release/obj.target/re2/vendor/re2/mimics_pcre.o
CXX(target) Release/obj.target/re2/vendor/re2/nfa.o
CXX(target) Release/obj.target/re2/vendor/re2/onepass.o
CXX(target) Release/obj.target/re2/vendor/re2/parse.o
CXX(target) Release/obj.target/re2/vendor/re2/perl_groups.o
CXX(target) Release/obj.target/re2/vendor/re2/prefilter.o
CXX(target) Release/obj.target/re2/vendor/re2/prefilter_tree.o
CXX(target) Release/obj.target/re2/vendor/re2/prog.o
CXX(target) Release/obj.target/re2/vendor/re2/re2.o
CXX(target) Release/obj.target/re2/vendor/re2/regexp.o
CXX(target) Release/obj.target/re2/vendor/re2/set.o
CXX(target) Release/obj.target/re2/vendor/re2/simplify.o
CXX(target) Release/obj.target/re2/vendor/re2/stringpiece.o
CXX(target) Release/obj.target/re2/vendor/re2/tostring.o
CXX(target) Release/obj.target/re2/vendor/re2/unicode_casefold.o
CXX(target) Release/obj.target/re2/vendor/re2/unicode_groups.o
CXX(target) Release/obj.target/re2/vendor/util/pcre.o
CXX(target) Release/obj.target/re2/vendor/util/rune.o
CXX(target) Release/obj.target/re2/vendor/util/strutil.o
SOLINK_MODULE(target) Release/obj.target/re2.node
COPY Release/re2.node
make: Leaving directory '/opt/node_modules/re2/build'
npm WARN saveError ENOENT: no such file or directory, open '/opt/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open '/opt/package.json'
npm WARN opt No description
npm WARN opt No repository field.
npm WARN opt No README data
npm WARN opt No license field.
+ [email protected]
added 101 packages from 74 contributors and audited 101 packages in 83.967s
1 package is looking for funding
run `npm fund` for details
found 0 vulnerabilities
Removing intermediate container d82870d4a550
FROM node:12.18.1-alpine as re2-builder
WORKDIR /opt
RUN apk add python make g++ libc6-compat \
&& ln -s /lib/libc.musl-x86_64.so.1 /lib/ld-linux-x86-64.so.2 \
&& npm install [email protected]
Building docker image...
Sending build context to Docker daemon 669.1MB
Step 1/25 : FROM node:12.18.1-alpine as re2-builder
---> 06a4a7b5263d
Step 2/25 : WORKDIR /opt
---> Using cache
---> 824ba1be20d2
Step 3/25 : RUN apk add python make g++ libc6-compat && ln -s /lib/libc.musl-x86_64.so.1 /lib/ld-linux-x86-64.so.2 && npm install [email protected]
---> Running in 1ebed36ae9ab
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
(1/22) Installing binutils (2.33.1-r0)
(2/22) Installing gmp (6.1.2-r1)
(3/22) Installing isl (0.18-r0)
(4/22) Installing libgomp (9.2.0-r4)
(5/22) Installing libatomic (9.2.0-r4)
(6/22) Installing mpfr4 (4.0.2-r1)
(7/22) Installing mpc1 (1.1.0-r1)
(8/22) Installing gcc (9.2.0-r4)
(9/22) Installing musl-dev (1.1.24-r2)
(10/22) Installing libc-dev (0.7.2-r0)
(11/22) Installing g++ (9.2.0-r4)
(12/22) Installing libc6-compat (1.1.24-r2)
(13/22) Installing make (4.2.1-r2)
(14/22) Installing libbz2 (1.0.8-r1)
(15/22) Installing expat (2.2.9-r1)
(16/22) Installing libffi (3.2.1-r6)
(17/22) Installing gdbm (1.13-r1)
(18/22) Installing ncurses-terminfo-base (6.1_p20200118-r4)
(19/22) Installing ncurses-libs (6.1_p20200118-r4)
(20/22) Installing readline (8.0.1-r0)
(21/22) Installing sqlite-libs (3.30.1-r2)
(22/22) Installing python2 (2.7.18-r0)
Executing busybox-1.31.1-r9.trigger
OK: 212 MiB in 38 packages
npm WARN deprecated [email protected]: request has been deprecated, see https://github.com/request/request/issues/3142
> [email protected] install /opt/node_modules/re2
> node scripts/install-from-cache.js --artifact build/Release/re2.node
Trying https://github.com/uhop/node-re2/releases/download/1.15.0/linux-x64-72.br ...
Writing to build/Release/re2.node ...
> [email protected] verify-build /opt/node_modules/re2
> node scripts/verify-build.js
terminate called after throwing an instance of 'std::system_error'
what(): No error information
Done.
npm WARN saveError ENOENT: no such file or directory, open '/opt/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open '/opt/package.json'
npm WARN opt No description
npm WARN opt No repository field.
npm WARN opt No README data
npm WARN opt No license field.
+ [email protected]
added 101 packages from 74 contributors and audited 101 packages in 8.323s
1 package is looking for funding
run `npm fund` for details
found 0 vulnerabilities
FROM node:12.18.1-alpine as re2-builder
WORKDIR /opt
RUN apk add python make g++ gcompat \
&& ln -s /lib/libc.musl-x86_64.so.1 /lib/ld-linux-x86-64.so.2 \
&& npm install [email protected]
Building docker image...
Sending build context to Docker daemon 669.1MB
Step 1/25 : FROM node:12.18.1-alpine as re2-builder
---> 06a4a7b5263d
Step 2/25 : WORKDIR /opt
---> Using cache
---> 824ba1be20d2
Step 3/25 : RUN apk add python make g++ gcompat && ln -s /lib/libc.musl-x86_64.so.1 /lib/ld-linux-x86-64.so.2 && npm install [email protected]
---> Running in ca6d2a41f183
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
(1/23) Installing binutils (2.33.1-r0)
(2/23) Installing gmp (6.1.2-r1)
(3/23) Installing isl (0.18-r0)
(4/23) Installing libgomp (9.2.0-r4)
(5/23) Installing libatomic (9.2.0-r4)
(6/23) Installing mpfr4 (4.0.2-r1)
(7/23) Installing mpc1 (1.1.0-r1)
(8/23) Installing gcc (9.2.0-r4)
(9/23) Installing musl-dev (1.1.24-r2)
(10/23) Installing libc-dev (0.7.2-r0)
(11/23) Installing g++ (9.2.0-r4)
(12/23) Installing libucontext (0.1.3-r1)
(13/23) Installing gcompat (0.4.0-r0)
(14/23) Installing make (4.2.1-r2)
(15/23) Installing libbz2 (1.0.8-r1)
(16/23) Installing expat (2.2.9-r1)
(17/23) Installing libffi (3.2.1-r6)
(18/23) Installing gdbm (1.13-r1)
(19/23) Installing ncurses-terminfo-base (6.1_p20200118-r4)
(20/23) Installing ncurses-libs (6.1_p20200118-r4)
(21/23) Installing readline (8.0.1-r0)
(22/23) Installing sqlite-libs (3.30.1-r2)
(23/23) Installing python2 (2.7.18-r0)
Executing busybox-1.31.1-r9.trigger
OK: 212 MiB in 39 packages
ln: /lib/ld-linux-x86-64.so.2: File exists
The command '/bin/sh -c apk add python make g++ gcompat && ln -s /lib/libc.musl-x86_64.so.1 /lib/ld-linux-x86-64.so.2 && npm install [email protected]' returned a non-zero code: 1
FROM node:12.18.1-alpine as re2-builder
WORKDIR /opt
RUN apk add python make g++ gcompat \
&& npm install [email protected]
Building docker image...
Sending build context to Docker daemon 669.1MB
Step 1/25 : FROM node:12.18.1-alpine as re2-builder
---> 06a4a7b5263d
Step 2/25 : WORKDIR /opt
---> Using cache
---> 824ba1be20d2
Step 3/25 : RUN apk add python make g++ gcompat && npm install [email protected]
---> Running in 94d1f3eee966
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
(1/23) Installing binutils (2.33.1-r0)
(2/23) Installing gmp (6.1.2-r1)
(3/23) Installing isl (0.18-r0)
(4/23) Installing libgomp (9.2.0-r4)
(5/23) Installing libatomic (9.2.0-r4)
(6/23) Installing mpfr4 (4.0.2-r1)
(7/23) Installing mpc1 (1.1.0-r1)
(8/23) Installing gcc (9.2.0-r4)
(9/23) Installing musl-dev (1.1.24-r2)
(10/23) Installing libc-dev (0.7.2-r0)
(11/23) Installing g++ (9.2.0-r4)
(12/23) Installing libucontext (0.1.3-r1)
(13/23) Installing gcompat (0.4.0-r0)
(14/23) Installing make (4.2.1-r2)
(15/23) Installing libbz2 (1.0.8-r1)
(16/23) Installing expat (2.2.9-r1)
(17/23) Installing libffi (3.2.1-r6)
(18/23) Installing gdbm (1.13-r1)
(19/23) Installing ncurses-terminfo-base (6.1_p20200118-r4)
(20/23) Installing ncurses-libs (6.1_p20200118-r4)
(21/23) Installing readline (8.0.1-r0)
(22/23) Installing sqlite-libs (3.30.1-r2)
(23/23) Installing python2 (2.7.18-r0)
Executing busybox-1.31.1-r9.trigger
OK: 212 MiB in 39 packages
npm WARN deprecated [email protected]: request has been deprecated, see https://github.com/request/request/issues/3142
> [email protected] install /opt/node_modules/re2
> node scripts/install-from-cache.js --artifact build/Release/re2.node
Trying https://github.com/uhop/node-re2/releases/download/1.15.0/linux-x64-72.br ...
Writing to build/Release/re2.node ...
> [email protected] verify-build /opt/node_modules/re2
> node scripts/verify-build.js
terminate called after throwing an instance of 'std::system_error'
what(): No error information
Done.
npm WARN saveError ENOENT: no such file or directory, open '/opt/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open '/opt/package.json'
npm WARN opt No description
npm WARN opt No repository field.
npm WARN opt No README data
npm WARN opt No license field.
+ [email protected]
added 101 packages from 74 contributors and audited 101 packages in 8.691s
1 package is looking for funding
run `npm fund` for details
from node-re2.
uhop
commented on August 16, 2024
Thank you for samples. While I assume it works for you, I will look at what is going on Alpine. In the past we had problems with versions of glibc. I suspect that we have something like that again.
I can suppress errors so users are none the wiser, yet the output can provide a valuable information in cases like this one.
from node-re2.
uhop
commented on August 16, 2024
That was very useful. The problem is that Alpine uses musl rather than glibc. There is a compat library, but it doesn't emulate glibc at the level we need. In the upcoming version I am planning to produce musl versions to save the compilation time.
Meanwhile, in https://github.com/uhop/node-re2/tree/1.15.1 I started to suppress "scary" errors and made the verification more robust.
from node-re2.
uhop
commented on August 16, 2024
I added precompiled versions for musl-based distros (like Alpine) in 1.15.2. Could you check if it works for you?
BTW, when I was debugging it I've noticed some random segmentation faults when running in the cloud. According to Google it is some kind of a weird stack-related Node/Alpine bug, which was fixed in Node 13.x. Previous versions work fine on Alpine 3.9, but not on newer versions.
For example:
from node-re2.
Related Issues (20)
- getting ERR_DLOPEN_FAILED after running re2 on alpine docker HOT 15
- Replace provide different results for node-re2 / re2j / re2js / golang HOT 1
- 1.20.1 verification has failed HOT 3
- Bug in var RE2 = require("re2"); HOT 4
- Node unable to import re2 HOT 4
- Flags passed as a separate argument will be lost if pattern is a regexp literal HOT 1
- Error compiling on Node.js HOT 6
- arm64 Linux prebuilts HOT 20
- matchAll and replaceAll scale nonlinearly HOT 10
- Fails to import re2.node with when installing in node:20.9 arm alpine HOT 7
- Much slower install time on node v20 HOT 3
- New nonbacktracking algorithm for lookbehinds and lookarounds HOT 5
- exec() on a Buffer returns incorrect index
- Can't install package with bun HOT 4
- Some Occurance operators seem to error HOT 2
- Ship prebuilt binaries using optionalDependencies instead of install script HOT 3
- Memory usage regression in 1.21 HOT 6
- Make internal data garbage-collectable HOT 9
- Regression in 1.21.2 HOT 1
- Release arm64 linux-musl prebuilts HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from node-re2.