Giter Club home page Giter Club logo

synp's Introduction

Build Status Coverage Status JavaScript Style Guide

synp

Convert yarn.lock to package-lock.json and vice versa.

install

npm install -g synp

command line usage

yarn.lock => package-lock.json

yarn # be sure the node_modules folder dir and is updated
synp --source-file /path/to/yarn.lock
# will create /path/to/package-lock.json

package-lock.json => yarn.lock

npm install # be sure the node_modules dir exists and is updated
synp --source-file /path/to/package-lock.json
# will create /path/to/yarn.lock

Note: if all you need is to convert in this direction (package-lock.json => yarn.lock), as of 1.7.0, Yarn is able to import its dependency tree from npm’s package-lock.json natively, without external tools. Use the yarn import command.

programmatic usage

const { npmToYarn, yarnToNpm } = require('synp')

const libPath = '/path/to/my/lib'
const stringifiedYarnLock = npmToYarn(libPath)
const stringifiedPackageLock = yarnToNpm(libPath)

how does it work?

Since package-lock.json and yarn.lock use different methods in order to deterministically lock down dependency versions, oftentimes they do not contain all the information needed to be purely converted.

For this reason, synp uses the existing node_modules directory of the package to determine the package state and assist in the conversion.

For this reason, it is vital to make sure the node_modules directory of the package is current and was installed by the respective tool (eg. by yarn if converting to package-lock.json and by npm if converting to yarn.lock).

caveats

Bundled dependencies: For various reasons, this tool does not 'play well' with bundled dependencies. This should not be a problem because installing the packages later with the converted file will (by definition) update the proper packages in the file. If this is not the case for you, please open an issue/PR with your use case and I'd be happy to take a look.

Package checksums: Both yarn.lock and package-lock.json include package checksums for dependencies. Since npm is slowly moving to sha-512 checksums which yarn does not (yet) support, converting to package-lock.json will result in weaker checksums (that will still work!) and converting to yarn.lock can sometimes result in a corrupted result file. Thankfully, this issue is 100% solvable. In npm one can update the checksums simply by deleting the integrity field of all or relevant packages. In yarn this can be solved with the --update-checksums* flag when installing from the created file.

Format limitations: Some things that can be expressed in one format simply cannot be expressed in the other. These are (to the best of my knowledge) extreme edge cases and should not worry 99% of this tool's intended users. One example is package-lock.json's ability to translate the same semver string to different versions. (eg. one package requesting version ^1.0.1 of a dependency and receiving 1.0.5 and a different package requesting version ^1.0.1 of the same dependency and receiving 1.0.71. When translating to yarn.lock through synp both will receive the same version).

Optional packages: Like npm (npm/npm#17722), synp also has issues with optional dependencies across different platforms. This is because it uses node_modules as its state, and does not guess about packages that are not installed on the converting platform. Sadly, like npm the only way to avoid this issue is to perform the conversion on the platform that meets most optional dependencies and update the rest manually. If this is a major issue for you, adding some sort of automatic tooling for this can be discussed.

* At the time of this writing, the --update-checksums flag in yarn has been merged but not released yet. Please see: yarnpkg/yarn#4860

troubleshooting

  1. checksum mismatch when installing from converted file? In yarn use --update-checksums, in npm delete the integrity field from the offending package (have no fear! This will be updated upon installation).
  2. synp failing or not converting properly - remove the node_modules from the package to be converted, install it again (with yarn if converting to package-lock.json or npm if converting to yarn.lock) and run synp one more time.
  3. something else? - please open an issue/PR.

License

MIT

synp's People

Contributors

antongolub avatar bluelovers avatar dependabot[bot] avatar deteam avatar harryparkdotio avatar hsribei avatar imsnif avatar jneuendorf avatar kevinji avatar skeggse avatar styx avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

synp's Issues

Unknown token

Issue description:

synp -w --source-file ./yarn.lock

Are you reporting a conversion error? Please include a link to yarn.lock / package-lock.json:

# This file is generated by running "yarn install" inside your project.
# Manual changes might be lost - proceed with caution!

__metadata:
  version: 6
  cacheKey: 8
...
more thousands lines

Terminal Broken after initial conversion.

Issue description:

After initially trying out the command to convert yarn.lock to package-lock.json I got those weird infinite loop of symbols on my terminal. That kept on for hours before I force stopped it. It keeps happening when I try and run other npm commands like npm run start, even after uninstalling the synp package. Please see the screenshot attached.
![Screenshot 2022-01-10 160118](https://user-images.githubusercontent.com/61785879/148778722-4c805266-54e4-4613-a3de-18c42e6dadd9.png)

Cannot read property 'replace' of undefined

I'm getting this error at npmToYarnResolved because the integrity of the package ua-parser-js comes undefined.

The package is a dependency of a top-level dependency in my project.

Here's the package-json.lock snippet:

"amplitude-js": {
      "version": "3.7.0",
      "resolved": "https://registry.npmjs.org/amplitude-js/-/amplitude-js-3.7.0.tgz",
      "integrity": "sha512-8wvzyM0c6RV5MAsWxkB/C1CMthSIGE+SZE1a5yc5MBzHb7hUkb/d6e7YaadkddicjicMpbFkGziA1e6TBvqC1A==",
      "requires": {
        "@segment/top-domain": "https://registry.npmjs.org/@segment/top-domain/-/top-domain-3.0.0.tgz",
        "blueimp-md5": "https://registry.npmjs.org/blueimp-md5/-/blueimp-md5-2.10.0.tgz",
        "json3": "https://registry.npmjs.org/json3/-/json3-3.3.2.tgz",
        "lodash": "https://registry.npmjs.org/lodash/-/lodash-4.17.4.tgz",
        "ua-parser-js": "github:amplitude/ua-parser-js#ed538f16f5c6ecd8357da989b617d4f156dcf35d"
      },
      "dependencies": {
        "ua-parser-js": {
          "version": "github:amplitude/ua-parser-js#ed538f16f5c6ecd8357da989b617d4f156dcf35d"
        }
      }
    },

add support for pnpm

Issue description:

@pnpm is another package manager which solves many issues of yarn and npm. pnpm uses a shrinkwrap.yaml lockfile similar to the shrinkwrap file of npm but uses another approach when installing dependencies.

`dev` marker may not be set correctly for dependencies of scoped packages

This is probably a tree sorting bug. Dir nesting count does not always match to nesting depth of the packages.

const sortedNodeModules = Object.keys(nodeModulesTree).sort((a, b) => a.split(sep).length < b.split(sep).length ? -1 : 1)

// foo/node_modules/baz/node_modules/qux — gives 3
// @foo/bar/node_modules/@bar/baz/node_modules/@baz/qux — gives 6, but actual nesting depth is still 3

I think it should be replaced with something like

const count = (path) => (path.match(new RegExp(`${sep}node_modules${sep}`, 'g')) || []).length
const sortedNodeModules = Object.keys(nodeModulesTree).sort((a, b) => count(a) < count(b) ? -1 : 1)

package-lock v3 not working with npmToYarn

Issue description:

I recently updated my npm to v9 and discovered that some functionality built on synp no longer works.

My code calls npmToYarn() which ends up dying in buildYarnTree() when it passes undefined to flattenPackageLock.
After doing some digging, it appears that npm v9 does not populate the dependencies section as it was absent from my package-lock file. Hence the undefined value.

By modifying the line in question to pass packageLock.packages instead of packageLock.dependencies in my node_modules, I was able to get the npmToYarn() call to work. Perhaps changing it to packageLock.dependencies ?? packageLock.packages would enable package-lock v3 support without breaking backwards compatibility?

Workspace mode - yarn.lock conversion fails with invalid package.json in tests

Issue description:

In workspace mode, it seems like synp tries to read all package.json files in the node_modules, even if they are not the main package.json file for the package. We're trying to convert yarn.lock to package-lock.json and one of our sub-dependencies is resolve package, which includes a bunch of invalid package.json files in their test suite. This causes the conversion to fail with Unexpected end of JSON input error.

Are you reporting a conversion error? Please include a link to yarn.lock / package-lock.json:

Cannot link to the whole yarn.lock, but the relevant parts are

ts-node-dev@^1.1.8:
  version "1.1.8"
  resolved "https://registry.yarnpkg.com/ts-node-dev/-/ts-node-dev-1.1.8.tgz#95520d8ab9d45fffa854d6668e2f8f9286241066"
  integrity sha512-Q/m3vEwzYwLZKmV6/0VlFxcZzVV/xcgOt+Tx/VjaaRHyiBcFlV0541yrT09QjzzCxlDZ34OzKjrFAynlmtflEg==
  dependencies:
    chokidar "^3.5.1"
    dynamic-dedupe "^0.3.0"
    minimist "^1.2.5"
    mkdirp "^1.0.4"
    resolve "^1.0.0"
    rimraf "^2.6.1"
    source-map-support "^0.5.12"
    tree-kill "^1.2.2"
    ts-node "^9.0.0"
    tsconfig "^7.0.0"

resolve@^1.0.0, resolve@^1.1.6, resolve@^1.1.7, resolve@^1.10.0, resolve@^1.11.1, resolve@^1.12.0, resolve@^1.14.2, resolve@^1.15.1, resolve@^1.20.0, resolve@^1.22.0, resolve@^1.3.2, resolve@^1.8.1:
  version "1.22.1"
  resolved "https://registry.yarnpkg.com/resolve/-/resolve-1.22.1.tgz#27cb2ebb53f91abb49470a928bba7558066ac177"
  integrity sha512-nBpuuYuY5jFsli/JIs1oldw6fOQCBioohqWZg/2hiaOybXOft4lonv85uDOKXdf8rhyK159cxU5cDcK/NKk8zw==
  dependencies:
    is-core-module "^2.9.0"
    path-parse "^1.0.7"
    supports-preserve-symlinks-flag "^1.0.0"

Are you reporting a conversion error? Please also include a link to your package.json:

Warning displayed by 'checkWorkspace' has typo: `--with-workspaces` should be `--with-workspace`

Using > synp --source-file yarn.lock in a monorepo displays this warning:

Workspace (npm lockfile v2) support is experimental. Pass `--with-workspaces` flag to enable and cross your fingers. Good luck!

As a first time user, I took it at face value and added that flag, which resulted in this:

error: unknown option '--with-workspaces'

Quite a minor issue, but misleading to first-time users.

Aside from the fix for the warning text, another suggestion would be to automatically display the help along with the 'unknown option' error, as it does here for a different error:

> synp --with-workspace --source-file yarn.lock
destination file package-lock.json already exists, will not overwrite
Usage: synp [options]

Options:
  -V, --version                    output the version number
  -s, --source-file [source-file]  The path to the yarn.lock or package-lock.json to be converted
  -f, --force                      Force overwrite destination file
  --with-workspace                 Enable experimental npm lockfile v2 processing
  -h, --help                       display help for command

Can this be used without needing multiple package managers installed?

I tend to stick with the default npm CLI tool for my Node based work, so I don't have other package managers installed (like Yarn, pnpm, etc).

When I found this project, it sounded like something I could use to convert a yarn.lock file to package-lock.json so that npm install had a valid lockfile to work from... But it looks like synp is trying to build NPM's lockfile based on the currently installed modules.

Perhaps I'm simply misunderstanding the purpose of this tool, but I assumed from the description that it would allow me to convert a Yarn lockfile to NPM so that I don't need to use Yarn at all.. Is this assumption incorrect then?


Using this demo project as a use case..

  • Install with yarn and the project builds correctly.
  • Install with npm and the library build fails.
  • Convert yarn.lock with synp after installing with yarn, remove node_modules, and install with npm and the project builds correctly.

Not sure if this would be helpful at all here, but I did create a gist with the demo project's package.json and yarn.lock, the package-lock.json converted by synp and generated by npm, along with a `diff1 output of the two.

Handling of git dependencies seems broken

One of our legacy products depends upon another via a direct git dependency:

"dependencies": {
  "physics-engine": "https://github.com/americademy/physics-engine/#master",
  ...

In the old yarn.lock, this is encoded as:

"physics-engine@https://github.com/americademy/physics-engine/#master":
  version "0.0.2"
  resolved "https://github.com/americademy/physics-engine/#655e2bd4fd42e8845c94af5327ceabb85bdb41b6"
  dependencies:
    gulp-sourcemaps "^2.6.4"

synp reencodes this into the produced package-lock.json as:

"physics-engine": {
  "version": "0.0.2",
  "resolved": "https://github.com/americademy/physics-engine/",
  "integrity": "sha1-ZV4r1P1C6IRclK9TJ86ruFvbQbY=",
  "requires": {
    "gulp-sourcemaps": "2.6.4"
  }
},

Unfortunately, this doesn't quite seem to be npm's actual encoding of git dependencies? When I subsequently try and test the resulting file with npm ci, I get:

$ npm ci                                                             
npm WARN prepare removing existing node_modules/ before installation
npm ERR! code E404
npm ERR! 404 Not Found: [email protected]

I can work around this by removing the git dependency with yarn remove physics-engine before running synp, then re-adding it with npm install --save 'americademy/physics-engine', but it's less-than-ideal, and I suspect this is a bug in synp. (If this project had any further transitive dependencies, then I'd lose their locked version-resolutions by doing this workaround! Luckily, in my case, it doesn't.)

create package-lock.json successfully ,but package-lock.json is overwritten after "npm install"

Issue description:

yarn.lock => package-lock.json
when I create a new project with vue-cli@~4.5.0,it create a yarn.lock.
And then follow the steps

synp --source-file yarn.lock
// delete node_modules
npm install

package-lock.json will be rewritten and files in node_modules are not I want.Project can not start!

even if I use 'npm ci' ,package-lock.json is not rewritten but files in node_modules are also not I want.

dev dependencies are losing "dev" flag

Issue description:

Hello,
With last version 1.9.3 when converting yarn.lock, dev dependencies are losing their flag "dev", please compare lock files :

Are you reporting a conversion error? Please include a link to yarn.lock / package-lock.json:

package-lock.json converted with synp
package-lock.json from npm install
yarn.lock

Are you reporting a conversion error? Please also include a link to your package.json:

{ "name": "angular-version", "version": "0.0.0", "license": "MIT", "private": true, "dependencies": { "rxjs": "~6.6.2" }, "devDependencies": { "protractor": "~7.0.0" } }

Cannot convert undefined or null to object

I'm afraid I can't share the package-lock.json that I'm trying to convert to yarn.lock, because it's the company's material under protection. It's a big list of dependencies anyway, probably something difficult to handle.

Anyway:
$ synp --source-file ./package-lock.json
Cannot convert undefined or null to object

Usage: synp [options]

Options:

-V, --version                    output the version number
-s, --source-file [source-file]  The path to the yarn.lock or package-lock.json to be converted
-h, --help                       output usage information

package-lock.json does not contain deps meta of resolved entries

As for now:

{"chalk": {
          "version": "2.4.2",
          "resolved": "https://registry.npmjs.org/chalk/-/chalk-2.4.2.tgz",
          "integrity": "sha512-Mti+f9lpJNcwF4tWV8/OrTTtF1gZi+f8FqlyAdouralcFWFQWF2+NgCHShjkCb+IFBLq9buZwE1xckQU4peSuQ==",
          "requires": {
            "ansi-styles": "3.2.1",
            "escape-string-regexp": "1.0.5",
            "supports-color": "5.3.0"
          }
 }

As it should be:

{"chalk": {
          "version": "2.4.2",
          "resolved": "https://registry.npmjs.org/chalk/-/chalk-2.4.2.tgz",
          "integrity": "sha512-Mti+f9lpJNcwF4tWV8/OrTTtF1gZi+f8FqlyAdouralcFWFQWF2+NgCHShjkCb+IFBLq9buZwE1xckQU4peSuQ==",
          "requires": {
            "ansi-styles": "^3.2.1",
            "escape-string-regexp": "^1.0.5",
            "supports-color": "^5.3.0"
          },
          "dependencies": {
            "supports-color": {
              "version": "5.5.0",
              "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-5.5.0.tgz",
              "integrity": "sha512-QjVjwdXIt408MIiAqCX4oUKsgU2EqAGzs2Ppkm4aQYbjm+ZEWEcW4SfFNTr4uMNZma0ey4f5lgLrkB0aX0QMow==",
              "requires": {
                "has-flag": "^3.0.0"
              }
            }
          }
}

Why do I need a node_modules folder?

I came here, because I want to generate a package-lock.json from a yarn.lock, because I want to use NPM instead of YARN. Obviously, I have not installed the dependencies yet and I should install them with a lock file. I tried to use synp, but it's only for someone who has already the deps installed, but why?

Shouldn't everything the package-lock.json needs be in the yarn.lock, already?

Update README with information about `yarn import`

as of 1.7.0 yarn is able to import its dependency tree from npm’s package-lock.json natively, without external tools—yarn blog

It would be useful to have this on the README in case all the user wants is one-way package-lock.json => yarn.lock conversion.

Could not find parent dir!

Issue description:

Bet you didn't expect to get an issue report 10 minutes after releasing a new version, but the new release seems to break something in my workflow. I get the following error: Could not find parent dir!, while everything worked fine one hour ago.

Are you reporting a conversion error? Please include a link to yarn.lock / package-lock.json:

lockfiles.zip

Are you reporting a conversion error? Please also include a link to your package.json:

See attached *.zip

EDIT: FYI I ran npx synp --source-file yarn.lock

ChainAlert: npm package release (1.9.9) has no matching tag in this repo

Dear synp maintainers,
Thank you for your contribution to the open-source community.

This issue was automatically created to inform you a new version (1.9.9) of synp was published without a matching tag in this repo.

Our service monitors the open-source ecosystem and informs popular packages' owners in case of potentially harmful activity.
If you find this behavior legitimate, kindly close and ignore this issue. Read more

badge

Trouble converting with file:// urls

Issue description:

When running synp --source-file yarn.lock after a fresh yarn install I get this error:

The "url" argument must be of type string. Received type undefined

I couldn't find the message's text anywhere in this repo so it must come from some dependency.
Therefore, I have no idea where to look or what the error means exactly. 😞

Are you reporting a conversion error? Please include a link to yarn.lock / package-lock.json:

yarn.lock.zip

Are you reporting a conversion error? Please also include a link to your package.json:

Here part of the file (I can't publicly post it because it's a project at work):

{
    "name": "projectX

",
    "version": "0.0.1",
    "private": true,
    "scripts": {
        "test": "jest"
    },
    "dependencies": {
        "babel-polyfill": "^6.26.0",
        "bulma": "^0.7.1",
        "change-case": "^3.0.2",
        "classnames": "^2.2.6",
        "d3": "^5.7.0",
        "emmapi": "file:../emmapi",
        "fastclick": "^1.0.6",
        "fetch-everywhere": "^1.0.5",
        "fetch-headers": "^2.0.0",
        "immutable": "^3.8.2",
        "jquery": "^3.3.1",
        "js_utils": "file:../js_utils",
        "jsc-android": "^236355.0.0",
        "lodash.groupby": "^4.6.0",
        "lodash.memoize": "^4.1.2",
        "lodash.uniqueid": "^4.0.1",
        "moment": "^2.22.2",
        "prop-types": "^15.6.2",
        "react": "16.3.1",
        "react-day-picker": "^7.2.4",
        "react-decoration": "^2.0.0",
        "react-dom": "^16.3.2",
        "react-is": "^16.4.1",
        "react-native": "0.55.4",
        "react-native-communications": "^2.2.1",
        "react-native-device-info": "0.24.0",
        "react-native-fetch-blob": "^0.10.8",
        "react-native-keyboard-aware-scroll-view": "^0.7.4",
        "react-native-navigation": "^1.1.478",
        "react-native-pdf": "^3.0.16",
        "react-native-push-notification": "^3.0.2",
        "react-native-splash-screen": "^3.1.1",
        "react-native-vector-icons": "^4.6.0",
        "react-native-wkwebview-reborn": "^2.0.0",
        "react-redux": "^5.0.7",
        "react-spring": "^6.1.9",
        "react-with-gesture": "^2.0.4",
        "redux": "^4.0.0",
        "redux-logger": "^3.0.6",
        "redux-promise": "^0.6.0",
        "redux-thunk": "^2.2.0",
        "smoothscroll-polyfill": "^0.4.3",
        "url-polyfill": "^1.0.13"
    },
    "devDependencies": {
        "babel-cli": "^6.26.0",
        "babel-eslint": "^8.2.3",
        "babel-jest": "22.4.4",
        "babel-loader": "^7.1.4",
        "babel-plugin-module-resolver": "^3.1.1",
        "babel-plugin-preval": "^3.0.1",
        "babel-plugin-transform-decorators-legacy": "^1.3.4",
        "babel-preset-env": "^1.7.0",
        "babel-preset-react": "^6.24.1",
        "babel-preset-react-native": "4.0.0",
        "babel-preset-stage-0": "^6.24.1",
        "cross-env": "^5.2.0",
        "css-loader": "^0.28.9",
        "eslint": "^4.19.1",
        "eslint-plugin-import": "^2.12.0",
        "eslint-plugin-react": "^7.8.2",
        "extract-loader": "^2.0.1",
        "extract-text-webpack-plugin": "^4.0.0-beta.0",
        "fast-sass-loader": "^1.4.5",
        "file-loader": "^1.1.11",
        "globby": "^8.0.1",
        "html-inline": "^1.2.0",
        "jest": "22.4.4",
        "jsdom": "^11.11.0",
        "ncp": "^2.0.0",
        "node-sass": "^4.9.0",
        "postcss-loader": "^2.1.4",
        "react-test-renderer": "16.3.1",
        "resolve": "^1.7.1",
        "resolve-url-loader": "^2.3.0",
        "sass-loader": "^7.0.3",
        "svg-sprite-loader": "^4.1.3",
        "url-loader": "^1.1.2",
        "webpack": "4.5.0",
        "webpack-cli": "^2.0.14",
        "webpack-shell-plugin": "git+https://github.com/cdeutsch/webpack-shell-plugin.git#bee537d"
    },
    "jest": {
        "preset": "react-native"
    }
}

Finding issues

Issue description:

npm i: match of undefined

Issues like these are sometimes caused by package-lock.json missing some version fields.
You can quickly discover those via:

function check(o, k) {
  typeof o.version === "undefined" && console.log(`${k} has no version`);
  o.dependencies && Object.keys(o.dependencies).forEach(k => check(o.dependencies[k], k));
}
check(require("./package-lock"))

which you can also run inline like this:

node --eval 'function check(o, k) { typeof o.version === "undefined" && console.log(k + " has no version"); o.dependencies && Object.keys(o.dependencies).forEach(k => check(o.dependencies[k], k)); }; check(require("./package-lock"));'

Are you reporting a conversion error? Please include a link to yarn.lock / package-lock.json:

Are you reporting a conversion error? Please also include a link to your package.json:

The automated release is failing 🚨

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you can benefit from your bug fixes and new features again.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can fix this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit to the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here are some links that can help you:

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.


Invalid npm token.

The npm token configured in the NPM_TOKEN environment variable must be a valid token allowing to publish to the registry https://registry.npmjs.org/.

If you are using Two Factor Authentication for your account, set its level to "Authorization only" in your account settings. semantic-release cannot publish with the default "
Authorization and writes" level.

Please make sure to set the NPM_TOKEN environment variable in your CI with the exact value of the npm token.


Good luck with your project ✨

Your semantic-release bot 📦🚀

Does not include optionalDependencies when translating yarn.lock to package-lock.json

Issue description:

I'm using yarn with Next.js v14, which declares some optional dependencies. When I run yarn synp --source-file yarn.lock those dependencies do not get carried over into package-lock.json. When I start the next dev server, I see this:

 ⚠ Found lockfile missing swc dependencies, patching...
 ⚠ Lockfile was successfully patched, please run "npm install" to ensure @next/swc dependencies are downloaded

So, clearly it expected those optional dependencies to be reflected in the package-lock.json, but I can't get synp to generate a file that includes them.

Relevant snippet of yarn.lock:

"@swc/[email protected]":
  version "1.3.105"
  resolved "https://registry.yarnpkg.com/@swc/core-darwin-arm64/-/core-darwin-arm64-1.3.105.tgz#2960f8a87aed01b3850d5c37f05b51d9d3747141"
  integrity sha512-buWeweLVDXXmcnfIemH4PGnpjwsDTUGitnPchdftb0u1FU8zSSP/lw/pUCBDG/XvWAp7c/aFxgN4CyG0j7eayA==

"@swc/[email protected]":
  version "1.3.105"
  resolved "https://registry.yarnpkg.com/@swc/core-darwin-x64/-/core-darwin-x64-1.3.105.tgz#aaa502f902dec1e4735a0a37a4fcc3a2c0369882"
  integrity sha512-hFmXPApqjA/8sy/9NpljHVaKi1OvL9QkJ2MbbTCCbJERuHMpMUeMBUWipHRfepGHFhU+9B9zkEup/qJaJR4XIg==

"@swc/[email protected]":
  version "1.3.105"
  resolved "https://registry.yarnpkg.com/@swc/core-linux-arm-gnueabihf/-/core-linux-arm-gnueabihf-1.3.105.tgz#5a8629c75db4fe298ca3fdb18ecbc7888cd34759"
  integrity sha512-mwXyMC41oMKkKrPpL8uJpOxw7fyfQoVtIw3Y5p0Blabk+espNYqix0E8VymHdRKuLmM//z5wVmMsuHdGBHvZeg==

"@swc/[email protected]":
  version "1.3.105"
  resolved "https://registry.yarnpkg.com/@swc/core-linux-arm64-gnu/-/core-linux-arm64-gnu-1.3.105.tgz#9cae768a92944774511a569dffca45f1beab7cdd"
  integrity sha512-H7yEIVydnUtqBSUxwmO6vpIQn7j+Rr0DF6ZOORPyd/SFzQJK9cJRtmJQ3ZMzlJ1Bb+1gr3MvjgLEnmyCYEm2Hg==

"@swc/[email protected]":
  version "1.3.105"
  resolved "https://registry.yarnpkg.com/@swc/core-linux-arm64-musl/-/core-linux-arm64-musl-1.3.105.tgz#9244aabe4a7884d3d81df707d3e3c3478953a454"
  integrity sha512-Jg7RTFT3pGFdGt5elPV6oDkinRy7q9cXpenjXnJnM2uvx3jOwnsAhexPyCDHom8SHL0j+9kaLLC66T3Gz1E4UA==

"@swc/[email protected]":
  version "1.3.105"
  resolved "https://registry.yarnpkg.com/@swc/core-linux-x64-gnu/-/core-linux-x64-gnu-1.3.105.tgz#7fde7b8a2ab7f5152e2dc37abf95f9add2c46937"
  integrity sha512-DJghplpyusAmp1X5pW/y93MmS/u83Sx5GrpJxI6KLPa82+NItTgMcl8KBQmW5GYAJpVKZyaIvBanS5TdR8aN2w==

"@swc/[email protected]":
  version "1.3.105"
  resolved "https://registry.yarnpkg.com/@swc/core-linux-x64-musl/-/core-linux-x64-musl-1.3.105.tgz#a8d96fc532427f353696283c70ae13e7fcc18358"
  integrity sha512-wD5jL2dZH/5nPNssBo6jhOvkI0lmWnVR4vnOXWjuXgjq1S0AJpO5jdre/6pYLmf26hft3M42bteDnjR4AAZ38w==

"@swc/[email protected]":
  version "1.3.105"
  resolved "https://registry.yarnpkg.com/@swc/core-win32-arm64-msvc/-/core-win32-arm64-msvc-1.3.105.tgz#35edc1d898d7e9994fd6986b4c0085b3f1d59ec5"
  integrity sha512-UqJtwILUHRw2+3UTPnRkZrzM/bGdQtbR4UFdp79mZQYfryeOUVNg7aJj/bWUTkKtLiZ3o+FBNrM/x2X1mJX5bA==

"@swc/[email protected]":
  version "1.3.105"
  resolved "https://registry.yarnpkg.com/@swc/core-win32-ia32-msvc/-/core-win32-ia32-msvc-1.3.105.tgz#5e538fd63b648d8c5bf9c22371c5e273c208f961"
  integrity sha512-Z95C6vZgBEJ1snidYyjVKnVWiy/ZpPiIFIXGWkDr4ZyBgL3eZX12M6LzZ+NApHKffrbO4enbFyFomueBQgS2oA==

"@swc/[email protected]":
  version "1.3.105"
  resolved "https://registry.yarnpkg.com/@swc/core-win32-x64-msvc/-/core-win32-x64-msvc-1.3.105.tgz#efd50adf2fdc1b9e5ad4df0e165d48d71ea5d7f3"
  integrity sha512-3J8fkyDPFsS3mszuYUY4Wfk7/B2oio9qXUwF3DzOs2MK+XgdyMLIptIxL7gdfitXJBH8k39uVjrIw1JGJDjyFA==

"@swc/core@^1.3.82":
  version "1.3.105"
  resolved "https://registry.yarnpkg.com/@swc/core/-/core-1.3.105.tgz#2b78db067b2dc90f3f398a08bb649a3d2ad74e8e"
  integrity sha512-me2VZyr3OjqRpFrYQJJYy7x/zbFSl9nt+MAGnIcBtjDsN00iTVqEaKxBjPBFQV9BDAgPz2SRWes/DhhVm5SmMw==
  dependencies:
    "@swc/counter" "^0.1.1"
    "@swc/types" "^0.1.5"
  optionalDependencies:
    "@swc/core-darwin-arm64" "1.3.105"
    "@swc/core-darwin-x64" "1.3.105"
    "@swc/core-linux-arm-gnueabihf" "1.3.105"
    "@swc/core-linux-arm64-gnu" "1.3.105"
    "@swc/core-linux-arm64-musl" "1.3.105"
    "@swc/core-linux-x64-gnu" "1.3.105"
    "@swc/core-linux-x64-musl" "1.3.105"
    "@swc/core-win32-arm64-msvc" "1.3.105"
    "@swc/core-win32-ia32-msvc" "1.3.105"
    "@swc/core-win32-x64-msvc" "1.3.105"

Relevant snippet of package.json:

  "dependencies": {
    "next": "^14.1.0",
    …
  },

yarn.lock to package-lock.json conversion ignores workspaces

Issue description:

Workspaces may have specific versions of dependencies which would not get hoisted to the root level, but synp accounts only for root node_modules Looks like there's an issue with nmtree, which leads to a loss of information in resulting package-lock.json

test repo

produced package-lock.json does not have other lodash versions:

{"name":"test-yarn-audit-fix","version":"1.0.0","lockfileVersion":1,"requires":true,"dependencies":{"lodash":{"version":"3.10.1","resolved":"https://registry.yarnpkg.com/lodash/-/lodash-3.10.1.tgz","integrity":"sha1-W/Rejkm6QYnhfUgnid/RW9FAt7Y="}}}

UPD: fixed repo link

No matching version found for dependencies@undefined

Issue description:

After converting yarn.lock to package-lock.json I run npm install.
output is

npm i
npm ERR! code ETARGET
npm ERR! notarget No matching version found for dependencies@undefined
npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.

Disclaimer
I reinstalled node_modules fresh using yarn install, then I used following command to convert yarn.lock synp --source-file yarn.lock and it ran correctly and created package-lock.json

Attaching files

yarn.lock https://pastebin.com/FZWC7mUS
package-lock.json https://ufile.io/3rsmr
package.json https://pastebin.com/9fMwYfGt

Problem with converting from Yarn to NPM: dependencies@undefined

Steps to reproduce with attached files:

yarn
synp --source-file yarn.lock
rm -rf node_modules
npm i

Getting error:

No matching version found for dependencies@undefined

Using Yarn 0.27.5 and npm 5.6.0

Are you reporting a conversion error? Please include a link to yarn.lock / package-lock.json:

yarn.lock.txt

Are you reporting a conversion error? Please also include a link to your package.json:

package.json.txt

Not working with Yarn 3.x

Issue description:

Project is using Yarn 3.x, and it's 'pnp' feature, so, it has a yarn.lock file, but, since it's using pnp, there is no node_modules folder, and synp is complaining about that.

Cannot convert undefined or null to object error

Issue description:

Cannot convert undefined or null to object error.

Are you reporting a conversion error? Please include a link to yarn.lock / package-lock.json:

synp --source-file ./package-lock.json
synp --source-file package-lock.json

Are you reporting a conversion error? Please also include a link to your package.json:

{
  "name": "hoge",
  "version": "0.1.0",
  "private": true,
  "engines": {
    "node": "8.11.3",
    "npm": "5.6.0"
  },
  "dependencies": {
    "@types/chart.js": "^2.7.7",
    "@types/chartjs": "^0.0.31",
    "@types/detect-browser": "^2.0.1",
    "@types/history": "^4.6.2",
    "@types/lodash": "^4.14.116",
    "@types/material-ui": "^0.20.5",
    "@types/prop-types": "^15.5.2",
    "@types/react-redux": "^5.0.20",
    "@types/react-router-dom": "^4.2.3",
    "@types/react-router-redux": "^5.0.11",
    "@types/redux-form": "^7.0.13",
    "@types/redux-logger": "^3.0.5",
    "@types/redux-thunk": "^2.1.0",
    "amazon-cognito-identity-js": "^1.19.0",
    "amazon-cognito-js": "^1.1.0",
    "autoprefixer": "7.1.6",
    "aws-sdk": "^2.186.0",
    "axios": "^0.17.1",
    "babel-core": "6.25.0",
    "babel-eslint": "7.2.3",
    "babel-jest": "20.0.3",
    "babel-loader": "7.1.1",
    "babel-polyfill": "^6.26.0",
    "babel-preset-env": "^1.6.0",
    "babel-preset-react": "^6.24.1",
    "babel-preset-react-app": "^3.0.2",
    "babel-runtime": "6.23.0",
    "babelify": "^7.3.0",
    "case-sensitive-paths-webpack-plugin": "2.1.1",
    "chalk": "1.1.3",
    "chart.js": "^2.7.1",
    "classnames": "^2.2.5",
    "css-loader": "0.28.7",
    "d3": "^3.5.17",
    "decimal.js": "^10.0.1",
    "detect-browser": "^3.0.0",
    "dotenv": "4.0.0",
    "eslint": "4.4.1",
    "eslint-config-es2015-react": "^0.2.1",
    "eslint-config-react-app": "^2.0.0",
    "eslint-loader": "1.9.0",
    "eslint-plugin-flowtype": "2.35.0",
    "eslint-plugin-import": "2.7.0",
    "eslint-plugin-jsx-a11y": "5.1.1",
    "eslint-plugin-react": "7.1.0",
    "extract-text-webpack-plugin": "3.0.2",
    "file-loader": "0.11.2",
    "fork-ts-checker-webpack-plugin": "^0.2.8",
    "fs-extra": "3.0.1",
    "history": "^4.7.2",
    "html-webpack-plugin": "2.29.0",
    "jest": "20.0.4",
    "linq": "^3.0.9",
    "lodash": "^4.17.10",
    "material-ui": "^0.19.0",
    "object-assign": "4.1.1",
    "postcss-flexbugs-fixes": "3.2.0",
    "postcss-loader": "2.0.8",
    "promise": "8.0.1",
    "raf": "3.4.0",
    "rd3": "^0.8.0",
    "react": "^16.2.0",
    "react-bootstrap": "^0.31.2",
    "react-dev-utils": "4.2.1",
    "react-dom": "^16.2.0",
    "react-error-overlay": "^1.0.10",
    "react-redux": "^5.0.6",
    "react-redux-form": "^1.14.0",
    "react-router": "^4.0.0-2",
    "react-router-bootstrap": "^0.24.2",
    "react-router-dom": "^4.2.2",
    "react-router-redux": "^4.0.8",
    "react-scripts-ts": "^2.17.0",
    "redux": "^3.7.2",
    "redux-form": "^7.0.3",
    "redux-form-material-ui": "^4.3.3",
    "redux-logger": "^3.0.6",
    "redux-router": "^2.1.2",
    "redux-thunk": "^2.1.2",
    "rx": "^4.1.0",
    "source-map-loader": "^0.2.1",
    "style-loader": "0.19.0",
    "sw-precache-webpack-plugin": "0.11.4",
    "ts-jest": "^20.0.7",
    "ts-loader": "^2.3.7",
    "tsconfig-paths-webpack-plugin": "^2.0.0",
    "tslint": "^5.7.0",
    "tslint-react": "^3.2.0",
    "underscore": "^1.8.3",
    "url-loader": "0.6.2",
    "validator": "^8.1.0",
    "webpack": "3.8.1",
    "webpack-dev-server": "2.9.4",
    "webpack-manifest-plugin": "1.3.2",
    "whatwg-fetch": "2.0.3"
  },
  "scripts": {
    "start": "react-scripts-ts start",
    "build": "react-scripts-ts build",
    "test": "react-scripts-ts test --env=jsdom",
    "eject": "react-scripts-ts eject"
  },
  "devDependencies": {
    "@types/jest": "^22.1.0",
    "@types/node": "^9.3.0",
    "@types/react": "^16.0.34",
    "@types/react-dom": "^16.0.3",
    "ts-jest": "^20.0.7",
    "typescript": "^2.6.2"
  }
}

Fix npm v7 / package-lock v2 converter

npm WARN old lockfile 
npm WARN old lockfile The package-lock.json file was created with an old version of npm,
npm WARN old lockfile so supplemental metadata must be fetched from the registry.
npm WARN old lockfile 
npm WARN old lockfile This is a one-time fix-up, please be patient...
npm WARN old lockfile 
npm WARN audit 400 Bad Request - POST https://registry.npmjs.org/-/npm/v1/security/audits/quick - Bad Request
{
  statusCode: 400,
  error: 'Bad Request',
  message: 'Invalid package tree, run  npm install  to rebuild your package-lock.json'
}

sourceFile.split is not a function

I use npm and workspaces. When I try to run the command synp --source-file --with-workspace package-lock.json, I get the error sourceFile.split is not a function

Support codeload.github.com packages

Issue description:

I'm seeing the following when trying to run synp against one of our yarn.lock files:

$ synp
Invalid hex string

  Usage: synp [options]


  Options:

    -V, --version                    output the version number
    -s, --source-file [source-file]  The path to the yarn.lock or package-lock.json to be converted
    -h, --help                       output usage information

After digging in a bit, I noticed that it's failing on an entry that looks like this:

throng@mixmaxhq/throng#eb_support:
  version "4.0.0"
  resolved "https://codeload.github.com/mixmaxhq/throng/tar.gz/8a015a378c2c0db0c760b2147b2468a1c1e86edf"
  dependencies:
    lodash.defaults "^4.0.1"

It looks like this would need explicit support.

EDIT: it looks like something similar happens going from npm to yarn, with this npm lockfile entry:

"throng": {
  "version": "github:mixmaxhq/throng#8a015a378c2c0db0c760b2147b2468a1c1e86edf",
  "requires": {
    "lodash.defaults": "4.2.0"
  }
}

which produces this error:

TypeError: Cannot read property 'replace' of undefined
    at npmToYarnResolved (/.../node_modules/synp/lib/entry.js:17:28)
    at yarnEntry (/.../node_modules/synp/lib/entry.js:48:31)
    at Object.keys.reduce (/.../node_modules/synp/lib/tree.js:23:23)
    at Array.reduce (native)
    at buildYarnTree (/.../node_modules/synp/lib/tree.js:18:47)
    at npmToYarn (/.../node_modules/synp/index.js:35:22)
    at run (/.../node_modules/synp/cli/run.js:25:20)
    at Object.<anonymous> (/.../node_modules/synp/cli/synp.js:13:1)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)

Package-lock.json missing some deps compared to npm's version

Issue description:

After converting yarn.lock, generated package-lock.json is missing some dep declaration and some "dev" flags (using 1.9.4)
Capture d’écran 2020-10-07 à 17 26 01

Here all json/lock files : https://gofile.io/uploadFiles
Or directly attached (beware of .txt extension)
package-lock-npm.json.txt
package-lock-synp.json.txt
package.json.txt
yarn.lock.txt

I made a gitlab project for testing synp AND https://github.com/jeremylong/DependencyCheck.
https://gitlab.com/boloss/test-dependency-check
You can fork it if you want

Every module in package-lock.json generated by synp has empty "requires" field which result in running failure because of module loss

Issue description:

Background: I got an existed project which use yarn.lock and I want to convert yarn.lock to package.lock.

Steps:

  1. I run yarn install , generate node_modules.
  2. run synp --source-file yarn.lock just like the doc says. Generate package-lock.json successfully.
  3. run npm install. Got some "Cannot find module ..." error in the middle, and "remove 1842 packages... " info at the end. And, lots of dependencies in package-lock.json has been removed after running npm install!
  4. run npm run serve. Failed in error "Error: Cannot find module... "

Result: So, I converted yarn.lock to package-lock.json successfully and conveniently, but this package-lock.json cannot install module properly and of course I can't run this project using this package-lock.json.

What I found: I removed node_modules and package-lock.json and retry Step1 and 2. Find out that every module in new generated package-lock.json has empty "requires" field. I think that will result in lots of first level indirect dependency package in package-lock lose connection to other package and seems like they are dependencies of nothing. The package-lock.json whose content been cut a lot after Step3 can prove it .

Are you reporting a conversion error? Please include a link to yarn.lock / package-lock.json:

https://github.com/zhouyuyuyuu/temp/blob/main/yarn.lock

Are you reporting a conversion error? Please also include a link to your package.json:

https://github.com/zhouyuyuyuu/temp/blob/main/package.json

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.