Giter Club home page Giter Club logo

opensphere-build-resolver's Introduction

opensphere-build-resolver

Resolves sources and other input for the Google Closure Compiler, scss for sass/node-sass, template views for angular, and other items through a project's dependency tree.

Problem

You want to use npm to manage your project and its dependencies, but you need to resolve source, css, scss, templates, or other resources from those dependencies and pass them as arguments on the command line to tools like the google-closure-compiler, sass/node-sass, and others.

Solution

This.

Usage

Install it as a dependency of your package: npm install opensphere-build-resolver --save-dev

Update your package.json to make it an opensphere build project.

"build": {
  "type": "lib", // for libraries, or "app" for applications, "plugin" for plugins, or "config" for config
  "pluggable": false, // whether or not you want the resolver to look for plugins like <packageName>-*

  // the rest here are extra options for a specific resolver, in this case the Google Closure Compiler
  "gcc": {
    "js": ['path/to/something/**.js'], // some source that isn't configured in a directory that the gcc resolver will naturally find
    "extra_annotation_name": ["api"]
  } 
},
"scripts": {
  "resolver": "os-resolve someOutputDirectory"
}

Now run npm run resolver and note the output in the given directory. You will want to ensure that your project does not have dependencies that should really be devDependencies (test frameworks, build systems, etc.).

Lib or App Plugins

If your project has

"build": {
  "pluggable": true
}

in its package.json, then the resolver will look for packagename-plugin-yourPlugin modules in:

../
package/node_modules
rootPackage/node_modules

The plugin package should have a peer dependency defined for your package:

"peerDependencies" : {
  "packagename": "^1.2.3"
}

Config Packages

"build": {
  "type": "config"
  "config": "someConfigDirectory" or ["thing1.json", "thing2.json"],
  "priority": 50 // configs are merged from least to greatest
}

Any config value not ending in .json is assumed to be a directory. Configs within directories are merged in lexicographical order.

Dependency Management

While we prefer that all actual source dependencies (such as google-closure-library, openlayers, etc.) or sass dependencies be installed and managed as npm dependencies, the resolver does support bower dependencies as long as they are stored in bower_components in the root of your project.

Writing Plugins

Want to resolve more stuff? Write an opensphere-build-resolver plugin!

Your plugin should be named opensphere-build-resolver-<something> and have a peer dependency on opensphere-build-resolver.

Your plugin should have a opensphere-build-resolver-<something>/index.js file with exports like so:

module.exports = {
  // called for every package in the dependency tree
  resolver: function(currPackageJson, currPackageDirectory, depth): Promise (resolve value does not matter) // optional

  // called once after all packages have resolved
  postResolver: function(basePackageJson, outputDir): Promise (resolve value does not matter) // optional

  // called once after those ^
  writer: function(basePackageJson, outputDir) : Promise (resolve value does not matter) // optional
};

About

OpenSphere was developed at the National Geospatial-Intelligence Agency (NGA) in collaboration with BIT Systems. The government has "unlimited rights" and is releasing this software to increase the impact of government investments by providing developers with the opportunity to take things in new directions. The software use, modification, and distribution rights are stipulated within the Apache license.

Pull Requests

If you'd like to contribute to this project, please make a pull request. We'll review the pull request and discuss the changes. All pull request contributions to this project will be released under the Apache license.

Software source code previously released under an open source license and then modified by NGA staff is considered a "joint work" (see 17 USC § 101); it is partially copyrighted, partially public domain, and as a whole is protected by the copyrights of the non-government authors and must be released according to the terms of the original open source license.

License

Copyright 2017 BIT Systems

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

opensphere-build-resolver's People

Contributors

bradh avatar btuttle avatar dependabot[bot] avatar gregroyal avatar jsalankey avatar schmidtk avatar semantic-release-bot avatar wallw-teal avatar wisestmumbler avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

opensphere-build-resolver's Issues

Results of simple-grep may be incomplete

simple-grep does not aggregate the output of the grep call from Node's child_process.exec(). On some systems, this can result in incomplete results. This is currently manifesting as an incomplete require_all.js file on some machines/projects and can cause broken library builds. Replace simple-grep with something more robust.

Inconsistent library builds

The GCC plugin has been observed to produce inconsistent builds for library projects from one machine to the next. The order of goog.require statements in the generated .build/require-all.js file may change, which can impact the build result. This resulted in errors on one machine that were not found on another. The errors were valid (and fixed), but build results should be consistent.

To fix this, I would recommend sorting the goog.require's, as required by OpenSphere's linter rules.

Add support for resolving more vendor sources of SCSS

Currently the resolver has a special case in the SCSS plugin for including Compass in the build. We are going to need to be able to pull the Bootstrap 4 SCSS into our Sass compiles, so add a way to support that functionality.

Support goog.loadModule generation

The resolver needs to support processing files using goog.module() by wrapping the entire file contents in:

goog.loadModule(function(exports) {
    "use strict";

    … module source …

    return exports;
});
//# sourceURL=<original file>

(per goog.module wiki) and subsequently placing the new output in something like .build/module-src/same/path/and/filename.js.

Resolver using wrong order for settings files

When checking the output of a yarn run compile:resolve step, the order in which settings files are included appears to be incorrect or random.

Checking the output in .build/settings-debug.json sometimes it will be...

"overrides": [
"../library/config/settings.json",
"../opensphere/config/settings.json",

and if I run it a second time or more times eventually I get

"overrides": [
"../opensphere/config/settings.json",
"../library/config/settings.json",

The order needs to be consistent and opensphere should always come first, otherwise we are unable to override settings appropriately.

Clean up GCC source locations

The resolver uses a number of properties in package.json to locate source files for the build. Currently it uses:

  • build.gcc.gcc-src: Adds an array of source paths to the build.
  • build.gcc.gcc-src-overrides: This replaces the above paths for a specific package. For example, a plugin could replace the source paths for its application.
  • directories.src and directories.lib: The default paths to check if neither of the above are specified.
  • build.gcc.js: Additional paths to add.

The first three items are managed by the GCC src plugin, while the last one is added by the options plugin. To keep the configuration more in line with what it's being used for, I would suggest removing the first three items in favor of build.gcc.js.

Both gcc-src and directories can be easily replaced, but the overrides will require additional work. That property should be replaced with a mechanism to override any property for another package. It doesn't appear source overrides are currently used by any known projects so we could remove that support for now.

Errors while resolving should stop the resolver

Right now we attempt to continue as much as possible. However, this ends up being unclear when builds complete but something ends up missing in the resulting artifact.

Best test of continuing with errors is:

  • start in an external plugin with tests
  • rm -rf .build/ ../opensphere/.build
  • yarn test

Resolve onboarding directory

The resolver does not handle the onboarding value directory, which means that plugins cannot integrate into the onboarding subsystem cleanly.

It looks like some copy'n'paste of the views code could handle this.

Incorrect priority when using opensphere as library dependency

Currently, Opensphere is at the same level as a middle library that depends on Opensphere

Example: main.html is in opensphere and main.html is in middle-library, middle libraries main.html should win. Currently openspheres main.html gets deployed to the dist directory overriding middle-libraries main.html.

opensphere  middle-library
        \      /
         application

When the override hierarchy should be

os
 \
  middle-library
    \
   application

The workaround is to not have the files in the same folder structure or change the file to a different name to prevent conflicts

Windows test compile resolution errors

> [email protected] compile:resolve C:\...\opensphere-yarn-workspace\workspace\opensphere
> os-resolve --outputDir .build --defineRoots $(cat .build/version) "--exclude" "resources" "scss"
C:\...\opensphere-yarn-workspace\node_modules\opensphere-build-resolver\plugins.js:22
    return new RegExp(pattern);
           ^
SyntaxError: Invalid regular expression: /\(plugins\|opensphere-build-resolver-)resources\/: Unmatched ')'
    at new RegExp (<anonymous>)
    at pluginToRegex (C:\...\opensphere-yarn-workspace\node_modules\opensphere-build-resolver\plugins.js:22:12)
    at Array.map (<anonymous>)
    at Object.load (C:\...\opensphere-yarn-workspace\node_modules\opensphere-build-resolver\plugins.js:26:34)
    at Object.<anonymous> (C:\...\opensphere-yarn-workspace\node_modules\opensphere-build-resolver\resolve.js:32:27)
    at Module._compile (internal/modules/cjs/loader.js:956:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:973:10)
    at Module.load (internal/modules/cjs/loader.js:812:32)
    at Function.Module._load (internal/modules/cjs/loader.js:724:14)
    at Function.Module.runMain (internal/modules/cjs/loader.js:1025:10)
    at internal/main/run_main_module.js:17:11

From running yarn test in opensphere.

Node: 12.13.0, npm: 6.12.0, yarn: 1.19.1

Fix lint errors in tests

Tests are not currently linted, but should follow the same rules as source files. Add tests to the lint script and fix all errors.

Clean up output

The resolver tree was changed to process breadth-first rather than depth-first. That change kinda screws up the output, which had a better appearance under the depth-first approach.

The depth-first approach is wrong under the hood, so don't do that. But do clean up the output so that it looks nicer or is easier to trace.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.