Giter Club home page Giter Club logo

crosspm's Introduction

CrossPM

Deploy codacy pypi license

Documentation

Actual version always here: http://devopshq.github.io/crosspm

Introduction

CrossPM (Cross Package Manager) is a universal extensible package manager. It lets you download and as a next step - manage packages of different types from different repositories.

Out-of-the-box modules:

  • Adapters

    • Artifactory
    • Artifactory-AQL (supported since artifactory 3.5.0):
    • files (simple repository on your local filesystem)
  • Package file formats

    • zip
    • tar.gz
    • nupkg (treats like simple zip archive for now)

Modules planned to implement:

  • Adapters

    • git
    • smb
    • sftp/ftp
  • Package file formats

    • nupkg (nupkg dependencies support)
    • 7z

We also need your feedback to let us know which repositories and package formats do you need, so we could plan its implementation.

The biggest feature of CrossPM is flexibility. It is fully customizable, i.e. repository structure, package formats, packages version templates, etc.

To handle all the power it have, you need to write configuration file (crosspm.yaml) and manifest file with the list of packages you need to download.

Configuration file format is YAML, as you could see from its filename, so you free to use yaml hints and tricks, as long, as main configuration parameters remains on their levels :)

crosspm's People

Contributors

alkovpro avatar allburov avatar antonzolotukhin avatar fishhead108 avatar fuzzmz avatar iaroslavscript avatar icq8680 avatar nikolasj avatar pre-commit-ci[bot] avatar simik13 avatar smithx avatar sseuzss avatar tim55667757 avatar tomskdiver avatar vasokot avatar vpag avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

crosspm's Issues

Unable to require particular pkg version

Found in ff4ad02.

dependencies.txt entry without wildcards in version like this

package  1.2.3  master

always causes

Error: pattern [1.2.3] does not match any version of package [package] for branch [master]

even though this version exists.

ImportError: No module named 'crosspm'

Found in ff4ad02.

> py cpm.py

Traceback (most recent call last):
  File "cpm.py", line 31, in <module>
    import crosspm
ImportError: No module named 'crosspm'

Hint: cpm.py module is located just under crosspm directory. Please move it one level up, or fix imports.

Add logic to resend http requests in a case of connection problems

Example of code:

    def do_request(self, method, url, **kwargs):

        responce = None
        n        = self.REQUEST_RETRY_TIME_SEC // self.REQUEST_RETRY_PAUSE_SEC

        with requests.Session() as session:

            for i in range( n ):

                try:

                    responce = session.request( method, url, **kwargs )
                    break

                except ( requests.exceptions.ChunkedEncodingError, ) as e:

                    self.log_msg_w( '\nWARNING! ChunkedEncodingError exception occuired!\n\t{} {}\n\tHeaders: {}\n\tStatus code: {}\n\tReason:{}'.format(
                        type( e ),
                        e,
                        e.response.headers,
                        e.response.status_code,
                        e.response.reason,
                    ))

                    responce = e.response

                    break

                except ( requests.exceptions.RequestException, requests.exceptions.ConnectionError, requests.exceptions.TooManyRedirects, requests.exceptions.Timeout, ) as e:

                    self.log_msg_w( '\nException occuired. Retry after {} sec\n\t{}\n\t{}'.format(
                        self.REQUEST_RETRY_PAUSE_SEC,
                        type( e ),
                        e,
                    ))

                    time.sleep( self.REQUEST_RETRY_PAUSE_SEC )

                except Exception as e:

                    self.log_msg_w( '\nUnknown Exception occuired. Retry after {} sec\n\t{}\n\t{}'.format(
                        self.REQUEST_RETRY_PAUSE_SEC,
                        type( e ),
                        e,
                    ))

                    time.sleep( self.REQUEST_RETRY_PAUSE_SEC )

            else:

                raise MaxRequestRetry()


        return responce

Exeption when call without args

~$ python -m"crosspm.cpm"
Traceback (most recent call last):
  File "/usr/lib/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/media/main/devpython/3.4_crosspm/lib/python3.4/site-packages/crosspm/cpm.py", line 222, in <module>
    main()
  File "/media/main/devpython/3.4_crosspm/lib/python3.4/site-packages/crosspm/cpm.py", line 217, in main
    args.check_args( args )
AttributeError: 'Namespace' object has no attribute 'check_args'

However:

~$ python -m"crosspm.cpm" --help
usage: PROG [-h] {download,promote,pack} ...

positional arguments:
  {download,promote,pack}
                        sub-command help
    download            a download help
    promote             a promote help
    pack                a pack help

optional arguments:
  -h, --help            show this help message and exit

Документация похоже устарела

Заметил что на данный момент имеются различия в документации и README.
Например нет такого примера запуска:

crosspm lock [DEPS] [DEPSLOCK] [options]

Разработоть стандарт файловой структуры CrossPM-пакета

Это должно нам дать следующие возможности:

  1. Производить валидацию пакетов - верификация пакета перед заливкой его в артифакторий поможет нам не пропускать в паблик поломанные пакеты;
  2. Начать разработку решения проблемы с транзитивными зависимостями;

Minor refactoring required

  • use logging instead of sys.stderr
  • use logging.exception instead of print error without stack
  • rename cpm.py to main.py
  • find out is it possible to replace "parser.call_cmd" with "parser.action"
  • replace any CMAKEPM with CROSSPM
  • remove sys.exit from api

PyInstaller for win/nix binary packages

build-on-server.cmd

REM -- Check/install prerequisites
ECHO ##teamcity[blockOpened name='Installing tools/libs']
ECHO ##teamcity[message text='--------  PyInstaller  --------']
%PYTHON3_DIR%\Scripts\pip3.exe install PyInstaller
IF %ERRORLEVEL% NEQ 0 EXIT /B 1
ECHO ##teamcity[blockClosed name='Installing tools/libs']

ECHO ##teamcity[blockOpened name='Compiling CrossPM']
PUSHD "cpm"
SET _CMD="%PYTHON3_DIR%\Scripts\pyinstaller.exe" ^
    --onefile ^
    --clean ^
    --noconfirm ^
    --distpath=".\dist" ^
    --workpath=".\build" ^
    --paths="." ^
    ".\cpm.py"
ECHO %_CMD%
%_CMD%
IF %ERRORLEVEL% NEQ 0 EXIT /B 1

ECHO Moving stuff
MKDIR "%OutDirectory%\cpm"
MOVE ".\dist\*.exe" "%OutDirectory%\cpm\"
IF %ERRORLEVEL% NEQ 0 EXIT /B 1
POPD
ECHO ##teamcity[blockClosed name='Compiling CrossPM']

build-on-server.sh

## -- Check/install prerequisites
echo "##teamcity[blockOpened name='Installing tools/libs']"
echo "##teamcity[message text='--------  PyInstaller  --------']"
pip3 install PyInstaller || exit 1
echo "##teamcity[blockClosed name='Installing tools/libs']"

echo "##teamcity[blockOpened name='Compiling CrossPM']"
pushd "cpm"
_CMD="pyinstaller
    --onefile
    --clean
    --noconfirm
    --distpath=./dist
    --workpath=./build
    --paths=.
    ./cpm.py"
echo $_CMD | sed -e 's/\n//g'
$_CMD
(( $? )) && exit 1

echo "Moving stuff"
mkdir ${OutDirectory}/cpm
mv -vf ./dist/* ${OutDirectory}/cpm
(( $? )) && exit 1
popd
echo "##teamcity[blockClosed name='Compiling CrossPM']"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.