Giter Club home page Giter Club logo

mvs's Introduction

MVS

mvs is a flexible tool and library based on Go mods.

Use and create module systems with Minimum Version Selection semantics and manage dependencies go mod style. Mix any set of language, code bases, git repositories, package managers, and subdirectories. Manage polyglot and monorepo codebase dependencies with 100% reproducible builds from a single tool.

Features

  • Based on go mods MVS system, aiming for 100% reproducible builds.
  • Recursive dependencies, version resolution, and code instrospection.
  • Custom module systems with custom file names and vendor directories.
  • Control configuration for naming, vendoring, and other behaviors.
  • Polyglot support for multiple module systems and multiple languages within one tool.
  • Works with any git system and supports the main features from go mods.
  • Convert other vendor and module systems to MVS or just manage their files with MVS.

Language support:

  • golang - delegates to the go tool for the commands mirrored here
  • cuelang - builtin in default using the custom module feature
  • hofmods - extends Cue with advanced code generation capabilities
  • custom - Create your own locally or globally with .mvsconfig files

Upcoming languages: Python and JavaScript so they can have an MVS system and the benefits, and mvs can start supporting and fetching from package registries. These language implementations will have flexibility to manage with mvs and the common toolchain to varying degrees. Pull requests for improved language support are welcome.

The main difference from go mods is that mvs, generally, is not introspecting your code to determine dependencies. It relies on user management of the <lang>.mods file. Since golang is exec'd out to, introspection is supported, and as more languages improve, we look to similarly improve this situation in mvs. A midstep to full custom implementation will be a introspection custom module with some basic support using file globs and regex lists.

Note, we also default to the plural <lang>.mods/sums files and <lang.mod>/ vendor directory. This is 1) because cuelang reads from cue.mod directory, and 2) because it is less likely to collide with existing directories. You can also configure more behavior per language and module than go mods. The go mods is shelled out to as a convience, and often languages impose restrictions.

Install

Usage

# Print known languages in the current directory
mvs info

# Initialize this folder as a module
mvs init <lang> <module-path>

# Add your requirements
vim <lang>.mods  # go.mod like file

# Pull in dependencies, no args discovers by *.mods and runs all
mvs vendor [langs...]

# See all of the commands
mvs help

Module File

The module file holds the requirements for project. It has the same format as a go.mod file.

# These are like golang import paths
#   i.e. github.com/hofstadter-io/mvs
module <module-path> 

# Information about the module type / version
#  some systems that take this into account
# go = 1.14
<lang> = <version>

# Required dependencies section
require (
  # <module-path> <module-semver>
  github.com/hof-lang/cuemod--cli-golang v0.0.0      # This is latest on HEAD
  github.com/hof-lang/cuemod--cli-golang v0.1.5      # This is a tag v0.1.5 (can omit 'v' in tag, but not here)
)

# replace <module-path> => <module-path|local-path> [version]
replace github.com/hof-lang/cuemod--cli-golang => github.com/hofstadter-io/hofmod-cli-golang v0.2.0
replace github.com/hof-lang/cuemod--cli-golang => ../../cuelibs/cuemod--cli-golang

Custom Module Systems

.mvsconfig.cue allows you to define custom module systems. With some simple configuration, you can create and control and vendored module system based on go mods. You can also define global configurations.

See the custom-modder docs to learn more about writing you own module systems.

This is the current Cue modder configuration:

cue: {
  Name: "cue"
  Version: "0.1.1"
  ModFile: "cue.mods"
  SumFile: "cue.sums"
  ModsDir: "cue.mod/pkg"
  MappingFile: "cue.mod/modules.txt"
  InitTemplates: {
    "cue.mod/module.cue": """
      module "{{ .Module }}"
      """
    }

  VendorIncludeGlobs: []
  VendorExcludeGlobs: [
    "/.git/**",
    "**/cue.mod/pkg/**",
  ]
}

Development

Want to help out?

Here are some commands if you want to develop mvs.

Make sure you have go 1.14 and cue 0.1.1 are installed. Development setup requires the hof tool as well.

# Fetch mvs code
git clone https://github.com/hofstadter-io/mvs
cd mvs

# Fetch golang deps and build
go mod vendor
go build

# Run local mvs
./mvs help

# Fetch cue/hof deps
./mvs vendor cue

# Generate code
hof gen        # Generate gocode for the cmd implementation
git status # should be no difference

You may also like to look at the hofmod-cli project. We use this to generate the CLI code and files for CI. The mvs uses our hof tool for code generation of itself and is also imported into the hof tool as both cue and go modules, embedding the mvs cli as the subcommand hof mod.

You can find us in the cuelang slack for now.

Motivation

  • MVS has better semantics for vendoring and gets us closer to 100% reproducible builds.
  • JS and Python can have MVS while still using the remainder of the tool chains.
  • Prototype for cuelang module and vendor management.
  • We need a module system for our hof-lang project.

Links about go mods

Using go modules

Go and Versioning

More about version selection

mvs's People

Contributors

philipdexter avatar verdverm avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

mvs's Issues

Support '@' in the org name for requires/replaces

we can still prefix with a domain name, I think that's part of the syntax

This is mainly for javascript


Separate, but came to mind, ...

for both JS and Python, or any other language, where you can import w/o the domain, we should prefix with the package manager domain that would normally be used, during the convert process.

after extracting "import" statements during introspection, we will need to walk back any possible subdirectory packages to find the root of the module and do dependency stuff from there.

One Modder

Let's get down to one modder by adding fields for the execs that match up to mvs commands (i.e. what the go modder is right now)

Then we can remove the exec modder, change the custom modder into the default, drop the interface, and have simpler and more consistent processing.

Modder Rework

  • merge dirs lib/mod and lib/modder
  • convert everything to Billy FS
  • make a bunch of utility functions
  • rework loading logic, handle sum / mapping files
  • modules should handle own processing where possible
  • recursive deps processing

Merging modders

Can we merge modders, so that if I want to only override a portion of a default modder, I don't need to specify the entire config.

while still in yaml format

  • try to unmarshal yaml into an already existing modder, add omitempty to the struct's field tags
  • if that doesn't work, do by hand, but we should still add the omitempty tags
    but then again, will the omitempty result in an empty slice? how do / don't replace a default slice with values, with an empty slice, or not replace it and take the defaults

Once we migrate the .mvsconfig to cue, this probably all gets done there, maybe we do that sooner rather than later

v0.0.0 not working

README.md states:

github.com/hof-lang/cuemod--cli-golang v0.0.0      # This is latest on HEAD

This does not seem to be the case:

  1. Init CUE module:
$ mvs init cue example.com/test
  1. Modify cue.mods
    As there is no mvs get command, I assume I need to edit cue.mods by hand.
module example.com/test

+ require (
+   github.com/hof-lang/cuemod--cli-golang v0.0.0
+ )

cue 0.1.1
  1. Run mvs vendor
$ mvs vendor
Merge   github.com/hof-lang/[email protected]    => github.com/hof-lang/[email protected]
open /home/tbraack/.mvs/mod/cue/github.com/hof-lang/[email protected]: no such file or directory
While calculating billy dir hash for "cue.mod/pkg"
open /home/tbraack/.mvs/mod/cue/github.com/hof-lang/[email protected]: no such file or directory

Which repository is taken does not matter, can also reproduce with others. Using a proper tagged version works fine

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.