Giter Club home page Giter Club logo

lgl's Introduction

lgl - python packaging manager launcher

libgirl launcher is a packaging manager to create python project

lgl demo

Installation

pip install lgl

Features

  • Create project directory.
  • Find lacking packge and install after analyze source code.
  • manage import forms for the package.
  • Launch application.

Usage

$ lgl
usage: lgl [-h] {run,install,fmt,init} ...

positional arguments:
  {run,install,fmt,init}
    run                 see `run -h`
    install             see `install -h`
    fmt                 see `fmt -h`
    init                see `init -h`

optional arguments:
  -h, --help            show this help message and exit

$ lgl init project-name # create project directory
$ lgl install           # install depending libraries.
$ lgl fmt               # manage `import` forms
$ lgl run [arguments]   # launch application

Author

Team Libgirl([email protected])

License

Licensed under the Apache License 2.0 License.

lgl's People

Contributors

snmsts avatar cataska avatar shakascchen avatar

Stargazers

本本 avatar avatar avatar avatar avatar

Watchers

avatar James Cloos avatar 本本 avatar

Forkers

kc17

lgl's Issues

lgl install should support pip install in some way

Use case

[]
Collecting package metadata: done
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

  - sagemaker

Current channels:

  - https://repo.anaconda.com/pkgs/main/osx-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/free/osx-64
  - https://repo.anaconda.com/pkgs/free/noarch
  - https://repo.anaconda.com/pkgs/r/osx-64
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

But SageMaker is available via pip install

possibility of auto inserting from ... import ...

Suggestion

Collect and divide different cases into different difficulty levels.

Examples

Level 0 (trivial = done): package is already installed and the insertion can be handled by importmagic

Level 1 (easy): function name is unique

In this sample case, deploy_config_from_estimator has a unique name.

from sagemaker.workflow.airflow import deploy_config_from_estimator

deploy_config = deploy_config_from_estimator(estimator=TF_estimator, 
        instance_type = 'ml.m5.large')

Level 4 (difficult): function name is a terminology, though not unique

scaler = MinMaxScaler(feature_range=(0, 1))
wltw_stock_prices = scaler.fit_transform(wltw_stock_prices)

train_size = int(len(wltw_stock_prices) * 0.67)
test_size = len(wltw_stock_prices) - train_size
train, test = wltw_stock_prices[0:train_size,:], wltw_stock_prices[train_size:len(wltw_stock_prices),:]

look_back = 1
testX, testY = create_dataset(test, look_back)
    
logger.info("read csv 7")
# reshape into X=t and Y=t+1
testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-16-eda95deb75c4> in <module>
     11 logger.info("read csv 4")
     12 wltw_stock_prices = wltw_stock_prices.reshape(-1, 1)
---> 13 scaler = MinMaxScaler(feature_range=(0, 1))
     14 wltw_stock_prices = scaler.fit_transform(wltw_stock_prices)
     15 

NameError: name 'MinMaxScaler' is not defined

PermissionError on lgl run

When execute lgl run on lgl_target_project, it happens to a PermissionError:

PS C:\Users\kevin-tsai\Desktop\lgl_target_project> lgl run lgl_live_demo/lgl_live_demo.py Traceback (most recent call last): File "c:\users\kevin-tsai\miniconda3\lib\runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "c:\users\kevin-tsai\miniconda3\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\kevin-tsai\Miniconda3\Scripts\lgl.exe\__main__.py", line 9, in <module> File "c:\users\kevin-tsai\miniconda3\lib\site-packages\lgl\lgl.py", line 60, in main args.handler(args) File "c:\users\kevin-tsai\miniconda3\lib\site-packages\lgl\lgl.py", line 12, in command_run lgl.install_module(["./"]) File "c:\users\kevin-tsai\miniconda3\lib\site-packages\lgl\__init__.py", line 44, in install_module result=result+list_unresolved(file) File "c:\users\kevin-tsai\miniconda3\lib\site-packages\lgl\__init__.py", line 25, in list_unresolved return get_unresolve.list_unresolved(path) File "c:\users\kevin-tsai\miniconda3\lib\site-packages\lgl\get_unresolve.py", line 9, in list_unresolved with open(path) as f: PermissionError: [Errno 13] Permission denied: './'

OS : Windows

lgl fmt ./ failed for nested packages

tabemasu.models is a package under the folder structure

.
├── setup.py
└── tabemasu
    ├── __init__.py
    └── models
        └── __init__.py
$ lgl fmt ./
Traceback (most recent call last):
  File "/usr/local/miniconda3/envs/tabemasu/bin/lgl", line 10, in <module>
    sys.exit(main())
  File "/usr/local/miniconda3/envs/tabemasu/lib/python3.7/site-packages/lgl/lgl.py", line 72, in main
    args.handler(args)
  File "/usr/local/miniconda3/envs/tabemasu/lib/python3.7/site-packages/lgl/lgl.py", line 35, in command_fmt
    lgl.add_import(args.file_names)
  File "/usr/local/miniconda3/envs/tabemasu/lib/python3.7/site-packages/lgl/__init__.py", line 47, in add_import
    return get_unresolve.add_import(result,index)
  File "/usr/local/miniconda3/envs/tabemasu/lib/python3.7/site-packages/lgl/get_unresolve.py", line 22, in add_import
    with open(path) as f:
FileNotFoundError: [Errno 2] No such file or directory: './tabemasu.models/'

incorporate pipenv?

to include virtual environment management and dependency lock. To incorporate pipenv could be the quickest way.

lgl install should skip symbols that are builtin modules or those can be solved by lgl fmt

related to #12

Environment

Prepare the environment by
cloning the lgl_target_project and execute

lgl_target_project/$ lgl install ./

Issue case

File is data.py

$ lgl install data.py 
[]
WARNING: The conda.compat module is deprecated and will be removed in a future release.
Collecting package metadata: done
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

  - os
  - json
  - minmaxscaler

Current channels:

  - https://repo.anaconda.com/pkgs/main/osx-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/free/osx-64
  - https://repo.anaconda.com/pkgs/free/noarch
  - https://repo.anaconda.com/pkgs/r/osx-64
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

$ lgl fmt data.py 
$ lgl install data.py 
# Now it's ok

CondaVerificationError on lgl install

When execute lgl install on lgl_target_project:

PS C:\Users\Nancy-tsai\Desktop\lgl_target_project> lgl install .\

after Preparing transaction was done, the verifying transaction failed and happens to a long string of CondaVerificationError messages:

Preparing transaction: done
Verifying transaction: failed

CondaVerificationError: The package for tornado located at C:\Users\kevin-tsai\Miniconda3\pkgs\tornado-6.0.2-py37he774522_0
appears to be corrupted. The path 'Lib/site-packages/tornado-6.0.2.dist-info/INSTALLER'
specified in the package manifest cannot be found.

CondaVerificationError: The package for tornado located at C:\Users\kevin-tsai\Miniconda3\pkgs\tornado-6.0.2-py37he774522_0
appears to be corrupted. The path 'Lib/site-packages/tornado-6.0.2.dist-info/LICENSE'
specified in the package manifest cannot be found.

OS : Windows

lgl fmt put "import, from...import" statement together and remove duplicate import

Use case input

import logging
import numpy as np
from sklearn.preprocessing import MinMaxScaler
from train import create_dataset

logger = logging.getLogger(__name__)

logger.info("read csv 2")
wltw = prices_dataset[prices_dataset['symbol']=='WLTW']

logger.info("read csv 3")
wltw_stock_prices = wltw.close.values.astype('float32')

logger.info("read csv 4")
wltw_stock_prices = wltw_stock_prices.reshape(-1, 1)
scaler = MinMaxScaler(feature_range=(0, 1))
wltw_stock_prices = scaler.fit_transform(wltw_stock_prices)

logger.info("read csv 5")
train_size = int(len(wltw_stock_prices) * 0.67)
test_size = len(wltw_stock_prices) - train_size
train, test = wltw_stock_prices[0:train_size,:], wltw_stock_prices[train_size:len(wltw_stock_prices),:]
    
logger.info("read csv 6")
look_back = 1
testX, testY = create_dataset(test, look_back)
    
logger.info("read csv 7")
# reshape into X=t and Y=t+1
testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))

from numpy import newaxis
import numpy as np
import matplotlib.pyplot as plt

def plot_results_multiple(predicted_data, true_data):
    plt.plot(true_data)
    plt.plot(predicted_data, 'g')
    plt.show()
    
#predict lenght consecutive values from a real one
def predict_sequences_multiple(predictor, firstValue,length):
    prediction_seqs = []
    curr_frame = firstValue

Use case output

import logging
import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import MinMaxScaler
from train import create_dataset
from numpy import newaxis


logger = logging.getLogger(__name__)

logger.info("read csv 2")
wltw = prices_dataset[prices_dataset['symbol']=='WLTW']

logger.info("read csv 3")
wltw_stock_prices = wltw.close.values.astype('float32')

logger.info("read csv 4")
wltw_stock_prices = wltw_stock_prices.reshape(-1, 1)
scaler = MinMaxScaler(feature_range=(0, 1))
wltw_stock_prices = scaler.fit_transform(wltw_stock_prices)

logger.info("read csv 5")
train_size = int(len(wltw_stock_prices) * 0.67)
test_size = len(wltw_stock_prices) - train_size
train, test = wltw_stock_prices[0:train_size,:], wltw_stock_prices[train_size:len(wltw_stock_prices),:]
    
logger.info("read csv 6")
look_back = 1
testX, testY = create_dataset(test, look_back)
    
logger.info("read csv 7")
# reshape into X=t and Y=t+1
testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))


def plot_results_multiple(predicted_data, true_data):
    plt.plot(true_data)
    plt.plot(predicted_data, 'g')
    plt.show()
    
#predict lenght consecutive values from a real one
def predict_sequences_multiple(predictor, firstValue,length):
    prediction_seqs = []
    curr_frame = firstValue

lgl install should install missing dependency in dependency

In this case, sagemaker is already installed. However, its dependency tensorflow isn't.
lgl install should also install tensorflow.

$ lgl install stock.py 
[]
$ python stock.py 
Traceback (most recent call last):
  File "stock.py", line 4, in <module>
    from sagemaker.tensorflow import TensorFlow
  File "/usr/local/miniconda3/lib/python3.6/site-packages/sagemaker/tensorflow/__init__.py", line 22, in <module>
    from sagemaker.tensorflow.estimator import TensorFlow  # noqa: E402, F401
  File "/usr/local/miniconda3/lib/python3.6/site-packages/sagemaker/tensorflow/estimator.py", line 27, in <module>
    from sagemaker.tensorflow.model import TensorFlowModel
  File "/usr/local/miniconda3/lib/python3.6/site-packages/sagemaker/tensorflow/model.py", line 20, in <module>
    from sagemaker.tensorflow.predictor import tf_json_serializer, tf_json_deserializer
  File "/usr/local/miniconda3/lib/python3.6/site-packages/sagemaker/tensorflow/predictor.py", line 20, in <module>
    from tensorflow.core.framework import tensor_pb2  # pylint: disable=no-name-in-module
ModuleNotFoundError: No module named 'tensorflow'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.