Giter Club home page Giter Club logo

yii-dev-tool's Introduction

Yii3 development environment

This repo contains tools to set up a development environment for Yii3 packages. It allows working on separate packages and test the result in other packages at the same time. Could be used for third party packages as well.

Requirements

Manual install

If you prefer manual install, you need:

Vagrant

If you prefer Vagrant, you only need to install Vagrant and VirtualBox, because our Vagrant environment already contains Ubuntu, PHP and Composer. See our Vagrant documentation for details.

Docker

If you prefer Docker, you only need to install Docker.

See our Docker documentation for details.

Manual install

git clone https://github.com/yiisoft/yii-dev-tool
cd yii-dev-tool
composer install
./yii-dev install

The above command will clone all Yii3 packages and run composer install in them. You may select packages by providing a second argument:

./yii-dev install di,rbac,yii-cycle,view

Note: In case you are using PhpStorm, run ide command once to configure the project directories for installed packages to avoid infinite indexing cycle.

Upgrade

To upgrade yii-dev-tool to the latest version, run the following commands:

cd yii-dev-tool
git pull
composer update

Configuration

To customize the configuration of yii-dev-tool, create your local configuration packages.local.php using file packages.local.php.example as example. In this file you will find examples of all available configuration options.

Some commands such as github/protect-branch, github/settings, release/make require a GitHub auth token. Therefore, you need to create a personal access token on GitHub. Then save it in config/github.token file.

Usage example

Objective

Suppose we want to work on three interdependent packages:

  • yiisoft/demo
  • yiisoft/view
  • yiisoft/i18n

Package demo depends on package view, and package view depends on package i18n.

Suppose we want to add new features to package i18n, and then use them in package view. After that, we will need to run the tests in package demo and make sure that everything works correctly.

Step 1: create forks

Go to the page of each repository and click the "Fork" button:

Suppose my nickname on Github is "samdark". Then I will get three forks:

  • samdark/demo
  • samdark/view
  • samdark/i18n

For your nickname you will get other fork names.

Step 2: install yii-dev-tool

Now install yii-dev-tool:

git clone https://github.com/yiisoft/yii-dev-tool
cd yii-dev-tool
composer install

Step 3: configure yii-dev-tool to use forks

In order for yii-dev-tool to use our forks, they must be configured.

It is possible to specify the owner for all packages in the owner-packages.php file

return 'samdark';

Or create your configuration:

cd yii-dev-tool
cp packages.local.php.example packages.local.php

Specify the forks in config packages.local.php:

$packages = [
    'demo' => 'samdark/demo',
    'view' => 'samdark/view',
    'i18n' => 'samdark/i18n',
];

See example.

Step 4: install packages

In order to work with Github via SSH, you have to add your public SSH key to Github account. Add key if you have not done it before.

Now install the packages:

cd yii-dev-tool
./yii-dev install demo,view,i18n

This command clones the fork repositories from GitHub to the local directory yii-dev-tool/dev/, sets upstream for them and executes composer install in each package. Then symlinks will be created:

  • yii-dev-tool/dev/demo/vendor/yiisoft/view -> yii-dev-tool/dev/view
  • yii-dev-tool/dev/view/vendor/yiisoft/i18n -> yii-dev-tool/dev/i18n

Due to these symlinks, changes in packages will immediately affect the operation of packages that depend on them. It is very convenient for development.

Step 5: create a git branch for work

Create a new feature-x branch in the repositories:

cd yii-dev-tool
./yii-dev git/checkout feature-x demo,view,i18n

Step 6: writing the code

Now make the necessary changes to the code of package i18n in folder yii-dev-tool/dev/i18n. Next, make changes to the code of package view in folder yii-dev-tool/dev/view. And, finally, change package demo in folder yii-dev-tool/dev/demo.

Step 7: run the tests

Make sure the tests pass. For instance, package view tests can be run with the following command:

cd yii-dev-tool/dev/view
./vendor/bin/phpunit

Step 8: commit and push the changes to the fork repositories

Commit the changes:

cd yii-dev-tool
./yii-dev git/commit "Add feature X" demo,view,i18n

Push the new code to remote repositories:

cd yii-dev-tool
./yii-dev git/push demo,view,i18n

Step 9: create pull requests

Go to the pages of the original repositories and create a PR in each:

Note: Don't create PR from master directly, instead create a new branch.

Final notes

That's all. We developed new functionality and submitted it for review ๐Ÿ™‚ Of course, the steps will be slightly different for different tasks and configurations.

Remember that yii-dev-tool contains many other commands:

Command Aliases Description
changelog/add Adds a changelog entry into CHANGELOG.md into under development section.
exec e Executes the specified console command in each package.
composer/update cu Update composer dependencies in packages.
git/checkout checkout Creates, if not exists, and checkout a git branch.
git/clone clone Package repositories cloning.
git/commit commit Add and commit changes into each package repository.
git/pull pull Pull changes from package repositories.
git/push push Push changes into package repositories.
git/status status, s Show git status of packages.
git/pr/create pr Create a pull request at GitHub.
github/forks forks Create forks of repositories
github/sync sync Sync forks from upstream repositories
github/settings - Change settings of a GitHub repository.
list l List enabled packages.
install i Clone packages repositories and install composer dependencies.
update u Pull changes from packages repositories and update composer dependencies.
lint - Check packages according to PSR-12 standard.
release/make - Make a package release.
replicate/files - Copy files specified in config/replicate/files.php into each package.
replicate/composer-config - Merge config/replicate/composer.json into composer.json of each package.
replicate/copy-file - Copy file into each package.
test t Test packages.
enable - Enable packages.
disable - Disable packages.
switch - Enable specified packages and disable others.

If you encounter any problems, create an issue โ€“ and we'll try to help you.

yii-dev-tool's People

Contributors

arhell avatar arogachev avatar billheaton avatar cebe avatar dagpro avatar dependabot-preview[bot] avatar dependabot[bot] avatar devanych avatar fantom409 avatar julian-b90 avatar loveorigami avatar machour avatar mapeveri avatar nex-otaku avatar pistej avatar roxblnfk avatar rugabarbo avatar rustamwin avatar samdark avatar sankaest avatar schmunk42 avatar stregubov avatar stylecibot avatar terabytesoftw avatar viktorprogger avatar vjik avatar xepozz avatar yiiliveext avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

yii-dev-tool's Issues

Proxy git arguments

Proxying git command arguments as is i.e. ./yii-dev git/commit --message="xxxx" --amend etc. would make working with git batch commands more convenient.

Alternative idea is to have a command that just does a batch execution of any commands in each repository.

What's about MVC collectors/reflection/name validators?

Sorry if this is an inappropriate question for current package, but where contained MVC reflection/helpers? What classes/methods have that's functionality?

How get list of

  • module/application controllers/commands
  • controller actions
  • route controllers and actions (controllers/actions matches route)

How check/validate

  • class is controller/command
  • method/class is action

How convert

  • class name to controller/command id
  • method/class name to action id

Package circular dependency detector

A command that resolves package dependencies and checks if there are circles. If there are, highlights problematic packages.

It should take into account require-dev of the current package as well.

Add fork command

Would be nice to have command that will fork origin repository and mark in packages.php to use forked repository.

Related to #95

Ability to test any packages

I think "test" command may be very handy.
For example, I've modified one related package and want to test another packages.

Problem

Modified package can't be tested as an integration of some packages.

User story

As a developer I want to call in console ./yii-dev test [packages delimited with comma] and all selected packages should be tested.

Possible solution

Implement composer test command in any yiisoft/* packages.
./yii-dev test [packages delimited with comma] will check if command is available then it will be called.

Use local composer

Instead of using a composer binary that is assumed to be in the path I propose downloading our own binary and running it with a different configuration directory.

This will allow us, for example, to enable: https://github.com/hirak/prestissimo for faster installation.

Remove phan support

We have migrated from phan to psalm partially.
I think we need to remove any phan mentions from packages.

Do not echo information that is clear from context

For example:

$ ./yii-dev status
โ–ถ Git status of package docs

 M guide/en/README.md

โ–ถ Git status of package di

โœ” nothing to commit, working tree clean

โ–ถ Git status of package injector

โœ” nothing to commit, working tree clean

Since we are aware that we executed status command there is no need to remind it:

$ ./yii-dev status
โ–ถ docs

 M guide/en/README.md

โ–ถ di

โœ” nothing to commit, working tree clean

โ–ถ injector

โœ” nothing to commit, working tree clean

`yii-dev install` should collect error information

Composer errors are hard to spot, the install command should:

  • Catch exit codes
  • Summarize errors at the end of execution

At the very least it should provide a note like "some packages have dependency issues".

Allow to use upstream branch to checkout when fork is used

Would be nice to have an ability to use upstream/master when you using a fork.
master from fork can be outdated and you will work with outdated code base.

I think it can be configured in packages.php like the following:

return [
    // ....
    'yii-web' => [
        'upstream' => 'yiisoft/yii-web',
        'origin' => 'xepozz/yii-web',
    ],
    // ....
];

or any handy ways.

`yii-dev update` for fork :fatal: No such remote 'origin upstream'

OS: Windows 10
packages.local.php:

$packages = [
    'auth' => 'https',
    'yii-console' => 'https://github.com/roxblnfk/yii-console.git',
    'yii-cycle' => 'https://github.com/roxblnfk/yii-cycle.git',
    'yii-web' => 'https://github.com/roxblnfk/yii-web.git',
    'yii-demo' => 'https://github.com/roxblnfk/yii-demo.git',
];

Can't execute yii-dev update for any forked package

yii-dev update auth works good

yii-dev update yii-console works not good :
image

upstream and origin configured automatically:
image

Make `yii-dev update` run `git/pull` before `composer update`

This change will make able to update all packages with one call yii-dev update. Now I need to run two commands one after another: yii-dev git/pull and yii-dev update.
If anyone want to have composer/update command we can make separated command.

Involve people in the development of Yii3, show yii-dev-tool usage examples

We do not have detailed explanations on how to use yii-dev-tool. Especially not enough examples of use. This can reduce the involvement of people in the development.

I propose to add:

  • Workflow examples in yii-dev-tool README
  • Cross references in yiisoft/docs
  • Forum post

Related docs:

Related forum posts:

Make it usable for other projects

Now this tool is used only for Yii 3, but it would be convenient to use it for developing other projects, for example, a project based on modules.

What needs to be changed/added:

  • Extract core files to yiisoft/dev-tool and add dependency it to yii-dev-tool

  • Make configuration of repositories possible, decouple from yii

  • Add a command that creates a package from a template

  • Add a command to run unit tests in packages

Ability to echo less messages

Ability to echo less messages is helpful for group commands such as ./yii-dev status. Instead of outputting

โ–ถ Git status of package yii-base-web

โœ” nothing to commit, working tree clean

โ–ถ Git status of package yii-project-template

โœ” nothing to commit, working tree clean

โ–ถ Git status of package yii-demo

 M src/Controller/SiteController.php

when the option is on we can display only items requiring attention:

โ–ถ Git status of package yii-demo

 M src/Controller/SiteController.php

Ready-to-use Vagrant configuration

All this time I have been using my Vargant configuration for yii-dev-tool.

This is a ready-to-use Vagrant playground that can be used to develop Yii3 packages. This is an alternative to Docker. It contains:

  • Ubuntu 18.04
  • PHP 7.2
  • Composer with prestissimo plugin
  • xDebug
  • unzip, htop, mc (Midnight Commander)
  • some other handy presets

In the future, we can add web configurations for packages like yii-demo to this Vagrant configuration.

I suggest adding it directly to yii-dev-tool repository so that it is available to run right from the yii-dev-tool package.

Version docker-compose

I have this problem with docker-compose when I run docker-compose run --rm php bash:

ERROR: Version in "./docker-compose.yml" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a supported version (e.g "2.2" or "3.3") and place your service definitions under the services key, or omit the version key and place your service definitions at the root of the file to use version 1.
For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/

The solution for this was change the version in docker-compose like this:

version: '3.4'
services:
  php:
    image: yiisoftware/yii-php:7.4-apache
    working_dir: /repo
    volumes:
      - .:/repo
      # For composer usage in container; NOTE! There may be performance issues, see also https://github.com/docker/for-mac/issues/77
      - ~/.composer-docker/cache:/root/.composer/cache:delegated

Change version 2.4 to 3.4.

Mi version of docker-compose is:

docker-compose version 1.17.1

Can I send a PR to update?

Thanks!

Support for dependencies on different versions of the same package

Now the tool is usable for work, because many packages are in dev status.

But let's imagine the following situation after the packages are released:

  1. Package A requires version ^2.0 of package X.
  2. Package B requires version ^1.7 of package X.
  3. Package C requires dev-master of package X.

But the tool only installs one version of the X package and creates symlinks for one version. Manual version control with so many repositories can be very tedious...

I currently have no solutions for this problem. But we need to think about this beforehand.

Group some commands and add aliases

Current state

Now the commands look like this:

20 12 07-14 19 45

Proposal

  1. Group ungrouped commands.
  2. Create aliases for frequently used commands.

Plan

  1. Group commands (rename):
  • exec -> shell/exec
  • install -> package/install
  • update -> package/update
  • lint -> package/lint
  • test -> package/test
  1. Create aliases:
Command Alias How call will look
composer/fix-dependencies โ€“ โ€“
git/checkout-branch co ./yii-dev co master yii-demo,db,arrays
git/commit c ./yii-dev c "Add ability to use feature X" yii-demo,db,arrays
git/pr/create pr ./yii-dev pr "My first yii3 PR" yii-demo,db,arrays
git/pull pull ./yii-dev pull yii-demo,db,arrays
git/push push ./yii-dev push yii-demo,db,arrays
git/status ss ./yii-dev ss yii-demo,db,arrays
github/settings โ€“ โ€“
release/make โ€“ โ€“
package/install โ€“ โ€“
package/lint lint ./yii-dev lint yii-demo,db,arrays
release/make โ€“ โ€“
package/test test ./yii-dev test yii-demo,db,arrays
package/update upd ./yii-dev upd yii-demo,db,arrays
replicate/composer-config โ€“ โ€“
replicate/copy-file โ€“ โ€“
replicate/files โ€“ โ€“
travis/ensure-cronjob โ€“ โ€“
travis/update-slack-config โ€“ โ€“
shell/exec โ€“ โ€“

Update command shouldn't install package

For example, I have part of all packages (only 10). I want to update all these packages and I don't want to install other packages.
When I do ./yii-dev update it starts installing all missed packages.
I think we shouldn't install missed packages in update mode

Dependencies checker / fixer

Could be a handy tool that checks dependencies are correct:

  1. Get package composer.json. Read require and require-dev packages.
  2. For each package get its root namespace from its composer.json, search for its usage within src and tests.
  3. If used in src or config/common.php / config/web.php - should be in require.
  4. If used in tests / config/tests.php only - should be in require-dev.
  5. If not used - should be removed.

Package configuration structure

When I was working on PR #21 I realized that logic of declaring packages in the package configuration is strange...

Suppose the main package configuration looks like this:

<?php
$packages = [
    'yiisoft/di' => 'di',
    'yiisoft/injector' => 'injector',
    'yiisoft/factory' => 'factory',
    'yiisoft/access' => 'access',
];
if (file_exists($localFile = __DIR__ . '/packages.local.php')) {
    $packages = array_merge($packages, require $localFile);
}
return $packages;

Suppose my own packages.local.php looks like this:

<?php

$packages = [
    'rugabarbo/di' => 'di',
    'rugabarbo/docs' => 'docs',
    'rugabarbo/factory' => 'factory',
    'rugabarbo/yii-captcha' => 'yii-captcha',
];

return $packages;

The array keys are the names of GitHub repositories, and the values are the names of /yii-dev-tool/dev/ subdirectories. It would be logical for my package factory to replace yii package factory after merge, but after merge we get the following array:

[
    'yiisoft/di' => 'di',
    'yiisoft/injector' => 'injector',
    'yiisoft/factory' => 'factory',
    'yiisoft/access' => 'access',
    'rugabarbo/di' => 'di',
    'rugabarbo/docs' => 'docs',
    'rugabarbo/factory' => 'factory',
    'rugabarbo/yii-captcha' => 'yii-captcha',
];

Here we see that folders di and factory are each connected to two different repositories, but the logic of the tool is such that the code from only one repository will actually be installed in the folder. And the most interesting: which one will be installed depends on the sequence of user actions :)

So something is wrong here. Function array_merge() doesn't seem to be suitable for merging these configurations or the configuration structure itself should be different.

Using --http fails...

If you use --httpd the result is that the help text is displayed. The problem is that adding the option moves where the command is located in $argv.

To get it working, I separated the command line processing into a new class: src/command/Args.php and using getopt() to parse with both long and short arguments specified.

Added --verbose for future use.

Will wait on pull request until I get a go signal. You can preview at BillHeaton/yii-dev-tool. Look for yii-dev-tool-fix-28.

Checkout only installed packages

When you want to reset all packages to master branch you get an errors

Package requirements repository is not cloned.
To fix, run the command:

  ./yii-dev install requirements

You can also disable the package in packages.local.php
See packages.local.php.example for configuration examples.

and the same for other not installed packages.

I expect when I do git/checkout master that will be checkout only installed packages.

No Way Easy Way To Setup Initial Forks

Apparently there is no easy way to set up the initial forks on the packages listed packages.php

./yii-dev setup

would be most helpful. Any suggestions for interim would be gratefully accepted.

Setup Travis crons

As briefly discussed on slack, it's becoming harder and harder to understand build failures in Yii 3 repos, as packages are inter-dependent and a tiny change in A can brake B without us noticing for days.

As a counter measure, could we setup Travis cronjobs as suggested by @rob006 on all repositories to run like every 2 days? (still unsure about the right periodicity)

Unexpected behaviour while adding new config

I tried to setup new package to app-api and I had a problem:
Composer don't read actual file composer.json from vendor folder, it cached them in vendor/composer/installed.json instead.
Because of this behaviour new package won't be added to config folder.

Composer version 2.0.8 2020-12-03 17:20:38
yii-dev-tool: 466ca89ee96dfe24e037f4d0a870d00e3d8eaf28 (commit hash)

Make build for phpbenchmarks.com

Would be nice to make builds for phpbenchmarks.com after that it will support PHP 7.4

  • Framework
    • Hello world
    • REST API
  • Template ( ? )
  • Json Serializer ( ? )

Doesn't work well with Composer binaries

When used with yii-console that registers its binary via the following

"bin": [
    "bin/yii"
],

there is no way to reliably determine path to project's autoload.php. Therefore, the tool doesn't work with command line applications.

Seems either we find a way to solve the problem or find out another workflow that suits it better.

Check SSH key for Github

If a developer did not setup SSH key for Github account, installation fails.

[email protected]: Permission denied (publickey).

Error message is quite confusing.

I do not have access to my own repo with fork? But why? It is mine, why do I have no permissions?

Requirement to set up SSH key before installation is not obvious.

I suggest that tool should check access to repo before taking any action.

With SSH key:

Checking access to "user/fork-name"... OK
Cloning "user/fork-name"...

Without SSH key:

Checking access to "user/fork-name"... DENIED
Seems like you have not installed SSH key to you Github account.
Key is required to work with repository via SSH.
See here for instructions: https://docs.github.com/en/github/authenticating-to-github/adding-a-new-ssh-key-to-your-github-account

There is no need to check access every single time. Just check when repository is cloned first time, it should be enough.

Automatically configure git upstream for repository forks

Command ./yii-dev install [<packages>] can automatically set upstream for repository forks. It's can be achieved by internal execution of git command git remote add upstream https://github.com/yiisoft/<repository>.git.

For those developers who use forks, this would eliminate most of the routine when setting up the environment.

Migrate to Symfony console components

Commands logic is getting bigger and harder. At the same time, we are building more and more console "bicycles". We can replace our console components with Symfony ready-made components to simplify development.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.