arecarn / dploy Goto Github PK

Provides the functionality of GNU Stow in a cross platform CLI tool and python module

License: MIT License

Python 100.00%

symbolic-links python cli cross-platform stow dotfiles dotfiles-manager dotfiles-installer osx linux windows

dploy's Introduction

Dploy

Dploy is a tool for creating symbolic links similarly to GNU Stow. It is provided as a CLI tool and Python 3.6.2+ module and supports Windows, Linux, and OSX.

Dploy's command stow creates symbolic links to the contents of source directories or packages in a specified destination directory. Repeating the stow command with the same arguments will confirm that the contents of the package have been symbolically linked.

Dploy's command unstow removes symbolic links that resulted from stow commands. Repeating the unstow command with the same arguments will confirm that the links to stowed packages have been removed.

Installation

Latest Release: pip install dploy
Development Version: pip install git+https://github.com/arecarn/dploy.git

Basic CLI Usage

dploy stow <source-directory>... <destination-directory>
dploy unstow <source-directory>... <destination-directory>
dploy --help

Rationale

Dploy started out as simple Python script to create symbolic links to my dotfiles for Windows, Mac, and Linux. Over time I keep improving and tweaking my script to suit my needs, but I was running into a problem. Keeping all the files I wanted to link in a config file was becoming a real pain in the neck.

I started looking for another solution to solve my problem, and found many alternatives but none of them seemed to be a good fit. The solution that seemed the most promising was using GNU Stow. It seemed like the most simple elegant solution to the problem. The only issue was that it didn't support Windows.

Then I thought to myself, why can't I just create my own version of Stow that work on Windows, Linux and OSX. So after that my I started morphing simple python script into what would become Dploy and learned a lot more about python in the process.

How does it compare with GNU Stow?

Below are just a few few major points of comparison between GNU stow and Dploy.

Like GNU Stow Dploy runs in two passes. First by collecting the actions required to complete the command and verifying that the command can completed without any issues. If no issues are detected then the second pass executes these actions are execute to complete the command. Otherwise Dploy will exit and indicate why the command can not be completed. This way a stow or unstow operation is atomic and never partially done.
Like Stow, Dploy supports tree folding and tree unfolding.
Unlike Stow, Dploy requires an explicit source(s) and a destination directory.
Unlike Stow, Dploy does not have any concept of ownership, but will only operate on symbolic links and the creation or removal of directories for these symbolic links.

dploy's People

Contributors

Stargazers

Watchers

Forkers

sleach thieryl code-monkeys ro6 harkabeeparolus whompyjaw joelvaneenwyk mdelite menghuu

dploy's Issues

unstow behaves is differently than stow -D

stow --target target source (target dir is empty)then stow -D --target target source
SAME as
dploy stow source target then dploy unstow source target

BUT
stow --target target source then make a dir in dest (target dir is not empty, has a folder which name same as a folder in source) then stow -D --target target source
NOT SAME as
dploy stow source target then make a dir in dest (target dir is not empty, has a folder which name same as a folder in source) then dploy unstow source target

Actually, this doesn't have a big impact, but you should know that they are still different

/tmp/stow
❯ tree -a
.
├── source
│   ├── aaa
│   │   ├── bbb
│   │   ├── dot-aaa
│   │   └── dot-ccc
│   │       ├── bbb
│   │       └── dot-aaa
│   └── dot-bbb
└── target

5 directories, 5 files
/tmp/stow
❯ stow --target target source
/tmp/stow
❯ tree
.
├── source
│   ├── aaa
│   │   ├── bbb
│   │   ├── dot-aaa
│   │   └── dot-ccc
│   │       ├── bbb
│   │       └── dot-aaa
│   └── dot-bbb
└── target
    ├── aaa -> ../source/aaa
    └── dot-bbb -> ../source/dot-bbb

6 directories, 6 files
/tmp/stow
❯ stow -D --target target source
/tmp/stow
❯ tree
.
├── source
│   ├── aaa
│   │   ├── bbb
│   │   ├── dot-aaa
│   │   └── dot-ccc
│   │       ├── bbb
│   │       └── dot-aaa
│   └── dot-bbb
└── target           <------------------------- same as dploy

5 directories, 5 files
/tmp/stow
❯ mkdir target/aaa
/tmp/stow
❯ stow --target target source^C
/tmp/stow
❯ tree
.
├── source
│   ├── aaa
│   │   ├── bbb
│   │   ├── dot-aaa
│   │   └── dot-ccc
│   │       ├── bbb
│   │       └── dot-aaa
│   └── dot-bbb
└── target
    └── aaa

6 directories, 5 files
/tmp/stow
❯ stow --target target source
/tmp/stow
❯ tree
.
├── source
│   ├── aaa
│   │   ├── bbb
│   │   ├── dot-aaa
│   │   └── dot-ccc
│   │       ├── bbb
│   │       └── dot-aaa
│   └── dot-bbb
└── target
    ├── aaa
    │   ├── bbb -> ../../source/aaa/bbb
    │   ├── dot-aaa -> ../../source/aaa/dot-aaa
    │   └── dot-ccc -> ../../source/aaa/dot-ccc
    └── dot-bbb -> ../source/dot-bbb

7 directories, 8 files
/tmp/stow
❯ stow -D --target target source
/tmp/stow
❯ tree
.
├── source
│   ├── aaa
│   │   ├── bbb
│   │   ├── dot-aaa
│   │   └── dot-ccc
│   │       ├── bbb
│   │       └── dot-aaa
│   └── dot-bbb
└── target
    └── aaa              <------------------------- NOT same as dploy

6 directories, 5 files

with dploy

/tmp/stow
❯ tree
.
├── source
│   ├── aaa
│   │   ├── bbb
│   │   ├── dot-aaa
│   │   └── dot-ccc
│   │       ├── bbb
│   │       └── dot-aaa
│   └── dot-bbb
└── target

5 directories, 5 files
/tmp/stow
❯ dploy stow source target
dploy stow: link target/aaa => ../source/aaa
dploy stow: link target/dot-bbb => ../source/dot-bbb
/tmp/stow
❯ tree
.
├── source
│   ├── aaa
│   │   ├── bbb
│   │   ├── dot-aaa
│   │   └── dot-ccc
│   │       ├── bbb
│   │       └── dot-aaa
│   └── dot-bbb
└── target
    ├── aaa -> ../source/aaa
    └── dot-bbb -> ../source/dot-bbb

6 directories, 6 files
/tmp/stow
❯ dploy unstow source target
dploy unstow: unlink target/aaa => ../source/aaa
dploy unstow: unlink target/dot-bbb => ../source/dot-bbb
/tmp/stow
❯ tree
.
├── source
│   ├── aaa
│   │   ├── bbb
│   │   ├── dot-aaa
│   │   └── dot-ccc
│   │       ├── bbb
│   │       └── dot-aaa
│   └── dot-bbb
└── target              <------------------------- same as stow

5 directories, 5 files
/tmp/stow
❯ mkdir target/aaa
/tmp/stow
❯ tree
.
├── source
│   ├── aaa
│   │   ├── bbb
│   │   ├── dot-aaa
│   │   └── dot-ccc
│   │       ├── bbb
│   │       └── dot-aaa
│   └── dot-bbb
└── target
    └── aaa

6 directories, 5 files
/tmp/stow
❯ dploy stow source target
dploy stow: link target/aaa/bbb => ../../source/aaa/bbb
dploy stow: link target/aaa/dot-aaa => ../../source/aaa/dot-aaa
dploy stow: link target/aaa/dot-ccc => ../../source/aaa/dot-ccc
dploy stow: link target/dot-bbb => ../source/dot-bbb
/tmp/stow
❯ tree
.
├── source
│   ├── aaa
│   │   ├── bbb
│   │   ├── dot-aaa
│   │   └── dot-ccc
│   │       ├── bbb
│   │       └── dot-aaa
│   └── dot-bbb
└── target
    ├── aaa
    │   ├── bbb -> ../../source/aaa/bbb
    │   ├── dot-aaa -> ../../source/aaa/dot-aaa
    │   └── dot-ccc -> ../../source/aaa/dot-ccc
    └── dot-bbb -> ../source/dot-bbb

7 directories, 8 files
/tmp/stow
❯ dploy unstow source target
dploy unstow: unlink target/aaa/bbb => ../../source/aaa/bbb
dploy unstow: unlink target/aaa/dot-aaa => ../../source/aaa/dot-aaa
dploy unstow: unlink target/aaa/dot-ccc => ../../source/aaa/dot-ccc
dploy unstow: unlink target/dot-bbb => ../source/dot-bbb
dploy unstow: remove directory target/aaa
/tmp/stow
❯ tree
.
├── source
│   ├── aaa
│   │   ├── bbb
│   │   ├── dot-aaa
│   │   └── dot-ccc
│   │       ├── bbb
│   │       └── dot-aaa
│   └── dot-bbb
└── target                       <------------------------- NOT same as stow

5 directories, 5 files

/tmp/stow
❯ dploy --version
dploy 0.1.2
/tmp/stow
❯ stow --version
stow (GNU Stow) version 2.4.0

Pytest leaves some garbage in tmpdir

After running pytest, there are some files and folders left in the tmpdir used by pytest. I have verified this on Linux and macOS, but I have not tried it on Windows.

I suspect this is a result of the function remove_execute_permission() in tests/utils.py, and maybe also remove_read and remove_write.

On Python <= 3.7, this is mostly invisible, because pytest doesn't complain. But on Python >= 3.8, after all the pytest tests have passed, this leads to warnings such as:

/users/traal/python/dploy/.venv/lib/python3.9/site-packages/_pytest/pathlib.py:87: PytestWarning: (rm_rf) unknown function <built-in function lstat> when removing /tmp/pytest-of-traal/garbage-fa89668f-46fb-4f9e-991b-55f88c40df17/test_unstow_folding_with_multi3/source_b/aaa:
<class 'PermissionError'>: [Errno 13] Permission denied: 'aaa'
  warnings.warn(

Regardless of Python version, here's an example of the underlying problem:

❯ ( cd /tmp/pytest-of-traal/pytest-current && find >/dev/null )
find: ‘./test_unstow_folding_with_multi3/source_b/aaa’: Permission denied
find: ‘./test_stow_with_write_only_sour1/source_a/aaa’: Permission denied
find: ‘./test_stow_with_source_dir_with0/source_a/aaa/ccc’: Permission denied
find: ‘./test_stow_unfolding_with_write0/source_a/aaa’: Permission denied
find: ‘./test_unstow_with_write_only_so0/source_a’: Permission denied
find: ‘./test_stow_with_write_only_sour0/source_a’: Permission denied
find: ‘./test_stow_with_source_with_no_0/source_a/aaa’: Permission denied
find: ‘./test_unstow_with_write_only_de0/source_a/aaa’: Permission denied
find: ‘./test_stow_unfolding_with_first0/source_a/aaa’: Permission denied

I am not sure how to fix this, because I don't know enough about pytest yet. I think that the removed permissions should be restored somehow, after the tests are run, so that the files can be removed cleanly.

dploy creates broken symlinks/junctions when linking to junction dir

After linking dotfiles to junction dir they cannot be opened.

Silently fails when stowing if source and destination are in different drives.

P:\dotfiles>dploy stow vim p:\test
dploy stow: link p:\test\.vim => ..\dotfiles\vim\.vim
dploy stow: link p:\test\.vimrc => ..\dotfiles\vim\.vimrc

P:\dotfiles>dploy stow vim c:\Users\user

P:\dotfiles>

As you can see stowing from P:\dotfiles\vim to P:\test works fine. But changing the destination to C:\Users\user does nothing.

Update:
I set up a directory junction C:\User\user\dotfiles => P:\dotfiles and stowing from there works fine, but it is a hassle.

Blanks lines in .dploystowignore cause silent failures of stow

I expect that blank likes would not effect the behavior of the .dploystowignore at all

Missing type hints

Is there any interest in supporting type hints? From a package user's point of view, the high-level entry points could be annotated:

With a stub dploy/__init__.pyi:

import pathlib
from typing import Optional, Union, List
Path = Union[str, pathlib.Path]

def stow(
    sources: List[Path],
    dest: Path,
    is_silent: bool = True,
    is_dry_run: bool = False,
    ignore_patterns: Optional[List[str]] = None
) -> None: ...
def unstow(
    sources: List[Path],
    dest: Path,
    is_silent: bool = True,
    is_dry_run: bool = False,
    ignore_patterns: Optional[List[str]] = None
) -> None: ...
def link(
    source: Path,
    dest: Path,
    is_silent: bool = True,
    is_dry_run: bool = False,
    ignore_patterns: Optional[List[str]] = None
) -> None: ...

And an empty py.typed.

The builder would also need to deploy the '*.pyi' and 'py.typed' into the output. For setuptools:

[tool.setuptools.package-data]
"*" = ["*.pyi", "py.typed"]

But, I don't know about poetry.

mypy would use the stub files in place of the real code.

More aggressive would be to annotate __init__.py, but I see there is a "requires python 3.3", which I think would break python 3.3 (import typing was added in 3.5).

Or, the package itself could use type annotations internally (ie: mypy --strict), but that's probably another discussion and larger change.

Effectively, two new files would be added which would not interfere with any runtime use of the module, but would allow mypy to typecheck calls to dploy.(stow, unstow, link).

Release new version to pypi.org

Last version on pip is 0.1.2, 32 new commits arrived to this repo after this version, it will be nice if you can release new version and upload it to pypi.org.

dploy stow is unable to create symbolic links on a Windows network drive

add `--no-folding`-like option similar to stow

More info on this option can be found here: https://linux.die.net/man/8/stow https://news.ycombinator.com/item?id=7927930

Don't give up on linking just because some files already exist

I often end up running dploy into a directory where some subset of the files I'm linking to already exist. I'm glad I get warnings about that, and that it doesn't clobber the files, but I would prefer if it still went ahead and linked any files where there wasn't a conflict. If this behavior doesn't work for others' use cases as the default behavior, it could be hidden behind a flag.