joachim-n / dorgflow Goto Github PK

Personally i like git format-patch patches, as maintainer i can just git am them and authort is in.
Also, although it is not well-known, you can put multiple patches in a file with something like
git format-patch origin/master..master --stdout >foo-patch
This way i can have a patch with multiple atomar commits.

Maybe that's cool?

Deprecated: Return type of Symfony\Component\Console\Helper\HelperSet::getIterator() should either be compatible with IteratorAggregate::getIterator(): Traversable, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /Users/joachim/Sites/_sandbox/dorgflow/vendor/symfony/console/Helper/HelperSet.php on line 111

applyPatch() checks STDERR for any errors from git when doing 'git apply' to apply a patch.

This will say whether the patch failed to apply.

However, it can also contain messages that don't matter, such as:

warning: schema.module has type 100644, expected 100755\n

The check for errors needs to be more specific.

Due to issues running drush (possibly solved by now?) my CLI PHP version is 5.6.x

dorgflow requires PHPUnit 6 which requires php ^7.0

phpunit/phpunit 6.0.7 requires php ^7.0 -> your PHP version (5.6.26) does not satisfy that requirement.

I had these branches:

• 1234-actually-made-with-dorgflow
• 1234-made-by-hand-later-because-I-forgot (active)

Dorgflow complained that I was not on the feature branch.

I'm working on an Atrium module (oa_toolbar) where my main branch is "7.x-2.x". When I was in an issue feature branch created by dorgflow, I used "dorgflow cleanup" and it complained "Could not find master branch".

See for example https://www.drupal.org/api-d7/node/2002514.json

Not sure if this is because the file was uploaded with the initial issue creation, or because it's an issue that predates the current system with the file field on the node.

The problem is somewhere in GitLog::getFeatureBranchLog().

Thank you for tool!
But I think it will be useful give full example for particular project. Commands from A to Z.

For example:
cd /var/www/patches
git clone --branch 8.7.x https://git.drupal.org/project/drupal.git
cd drupal
dorgflow https://www.drupal.org/project/drupal/issues/2652236

I've spend full day to understand, that I must receive project via "git clone" and not via "composer require ..." (because "composer require" command get repo with different branches).

Today I am trying out drogflow the first time. I used it a few times for new or smaller issues and it looked really nice.

Wehn I started to use it with https://www.drupal.org/project/drupal/issues/2831944 my branch is completely broken. I assume it's because of not applying patches.

$ dorgflow https://www.drupal.org/project/drupal/issues/2831944 Hello, this is Dorgflow! Detected master branch 8.6.x. Fetching node 2831944 from drupal.org. Created feature branch 2831944-Implement-media-source-plugin-for-remote-video-via-oEmbed. Patch 2831944_4.patch did not apply. Patch 2831944-10.patch did not apply. Patch 2831944-26.patch did not apply. Patch 2831944-30.patch did not apply. Patch 2831944-31.patch did not apply. Patch 2831944-32.patch did not apply. Patch 2831944-34.patch did not apply. Patch 2831944-36.patch did not apply. Patch 2831944-37.patch did not apply. Patch 2831944-40.patch did not apply. Patch 2831944-42.patch did not apply. Patch 2831944-51.patch did not apply. Patch 2831944-58.patch did not apply. Patch 2831944-60.patch did not apply. Patch 2831944-63.patch did not apply. Patch 2831944-64.patch did not apply. Patch 2831944-65.patch did not apply. Patch 2831944-67.patch did not apply. Patch 2831944-68.patch did not apply. Patch 2831944-71.patch did not apply. Patch 2831944-73.patch did not apply. Applied and committed patch 2831944-87.patch. Applied and committed patch 2831944-91.patch. Applied and committed patch 2831944-92.patch. Applied and committed patch 2831944-95.patch. Applied and committed patch 2831944-98.patch. Applied and committed patch 2831944-103.patch. Applied and committed patch 2831944-113.patch. Applied and committed patch 2831944-115.patch. Applied and committed patch 2831944-116.patch. Applied and committed patch 2831944-119.patch. Applied and committed patch 2831944-120.patch. Applied and committed patch 2831944-123.patch.

Pros:

• provides nice formatting of output
• provides autocomplete of commands
• isolates output; easier for testing (though note that PHPUnit can handle and test for print statements in tests).

Con:

• more overhead
• need to look at how it handles a set of different commands
• ...?

I'm getting this error message when trying to run dorgflow {ISSUE_URL}:

Hello, this is Dorgflow!
Detected master branch 8.x-1.x.
Fetching node 3132912 from drupal.org.
Created feature branch 3132912-Override-getIdealConditionsLimit-and-make-it-configurable.
There are no patches to apply.
PHP Fatal error:  Uncaught TypeError: Return value of "Dorgflow\Command\LocalSetup::execute()" must be of the type int, "NULL" returned. in /home/emerson/drupal/contrib/dorgflow/vendor/symfony/console/Command/Command.php:258
Stack trace:
#0 /home/emerson/drupal/contrib/dorgflow/vendor/symfony/console/Application.php(912): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#1 /home/emerson/drupal/contrib/dorgflow/vendor/symfony/console/Application.php(264): Symfony\Component\Console\Application->doRunCommand(Object(Dorgflow\Command\LocalSetup), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#2 /home/emerson/drupal/contrib/dorgflow/vendor/symfony/console/Application.php(140): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#3 /home/emerson/drupal/contrib/dorgflow/dor in /home/emerson/drupal/contrib/dorgflow/vendor/symfony/console/Command/Command.php on line 258

php --version

PHP 7.2.29-1+ubuntu18.04.1+deb.sury.org+1 (cli) (built: Mar 20 2020 13:54:39) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.2.29-1+ubuntu18.04.1+deb.sury.org+1, Copyright (c) 1999-2018, by Zend Technologies
    with Xdebug v2.9.3, Copyright (c) 2002-2020, by Derick Rethans

Eg: https://www.drupal.org/project/config_devel/issues/2923795#new

https://www.drupal.org/project/drush_iq

Someone else has made a follow-on project for drush_iq as well, but I don't remember where it is now.

Suppose that during initial setup of a feature branch, patch 1 did not apply, and patch 2 did.

Then 'dorgflow update' attempts to apply patch 1 again, and fails, and produces a message to that effect.

It should instead skip any patches older than what's already on the branch.

This is the scenario:

• make a patch with $ dorgflow
• realize you've missed things off, so don't upload it
• make more commits
• make a patch with $ dorgflow

The patch file written in the first step is overwritten, which is fine. But the interdiff is taken from that commit, rather than the correct prior patch.

Before "git diff-files" you should execute "git update-index --refresh"
See: https://stackoverflow.com/questions/36367190/git-diff-files-output-changes-after-git-status

Otherwise, git diff-files show me modifications, that I've never done. And I have "Git repository is not clean. Aborting." exception.

[03:05 PM]-[vagrant@local]-[/var/www/drupal_8/web/modules/contrib/field_display_label]-[...]
$ git diff-files
:100644 100644 d3f52e261af30ab4a4b58077e752605caba46e93 0000000000000000000000000000000000000000 M README.txt
:100644 100644 0dd03a509fd01506e5daecf0519683217f0cd1c0 0000000000000000000000000000000000000000 M config/schema/field_display_label.schema.yml
:100644 100644 fda071b0a3befeb8ba58bad996bae1f3641adce6 0000000000000000000000000000000000000000 M field_display_label.info.yml
:100644 100644 5a6dddfcd91dc02e9fde92b71a75fe33be4889e2 0000000000000000000000000000000000000000 M field_display_label.module

[03:06 PM]-[vagrant@local]-[/var/www/drupal_8/web/modules/contrib/field_display_label]-[...]
$ git update-index --refresh

[03:06 PM]-[vagrant@local]-[/var/www/drupal_8/web/modules/contrib/field_display_label]-[$]
$ git diff-files
$

I had this branch:

Patch for Drupal.org. Comment (expected): 2; file: 2909573-2.commerce_license.Allow-order-items-to-already-have-a-license-set-on-them-during-order-sync.patch. Automatic commit by dorgflow.
tweak
Patch for Drupal.org. Comment (expected): 2; file: 2909573-2.commerce_license.Allow-order-items-to-already-have-a-license-set-on-them-during-order-sync.patch. Automatic commit by dorgflow.
Fix

After the first patch, I spotted a problem and made one more commit -- 'tweak'. The rerolled patch had the same comment number, which is correct.

I uploaded that patch. Then I made another fix.

Patch for Drupal.org. Comment (expected): 2; file: 2909573-2.commerce_license.Allow-order-items-to-already-have-a-license-set-on-them-during-order-sync.patch. Automatic commit by dorgflow.
Fix fix
Patch for Drupal.org. Comment (expected): 2; file: 2909573-2.commerce_license.Allow-order-items-to-already-have-a-license-set-on-them-during-order-sync.patch. Automatic commit by dorgflow.
tweak
Patch for Drupal.org. Comment (expected): 2; file: 2909573-2.commerce_license.Allow-order-items-to-already-have-a-license-set-on-them-during-order-sync.patch. Automatic commit by dorgflow.
Fix

That's not correct -- the middle patch was uploaded by this point, so d.org should have been checked.

Particularly with Drupal core, it's hard to remember what your master branch is when taking a diff to look at progress so far on a feature.

What would be useful is 'dorgflow diff' which is basically a wrapper around 'git diff', passes any options along, and knows the master branch to diff against.

Purpose:

• Better install experience

Details:

• I hit issues during the install process due to a composer symfony version issue, I have put up a PR to fix those #52. Overall though we need a process that doesn't involve composer for end users as composer results in "hard to install" experiences.

Options:

• Building a phar file like drupal-check: https://github.com/mglaman/drupal-check/wiki/Build-From-Source

I tend to work on a lot of different issues at the same time. So I have lots of branches.

When I type: "dorgflow 2868879" I get the message:
"The feature branch 2868879-Wrong-user-link-in-oamessages already exists. Use the update command, rebasing first if necessary. Aborting."

Yes, I know...can you please just switch me to that branch so I don't have to type a really long "git checkout" command?

I can only use "dorgflow update" when I'm within the feature branch. If I'm within a different feature branch, or back in master then it doesn't work. So the error message is misleading.

So, as a feature request, I'd love to see "dorgflow issue#" switch me to the feature branch if it already exists. Would save a lot of typing for me.

If there is an existing feature branch for an issue, but it's not reachable, because the master branch has had new commits since the branching point, then doing 'dorgflow ISSUE' will destroy that branch and re-create it at the tip of master.

This is because createNewBranch() uses plumbing commands which don't check for the prior existence of a branch. Creating a new branch when it already exists effectively just moves its tip to the current SHA.

Fix:

1. fix createNewBranch() to check for the existence of a branch and throw an exception if there is one already, to prevent any future loss of branches.
2. feature branch detection should use ALL branches, not just reachable ones.

Rather than do a git rebase, which could cause conflicts and require manual handling, we should instead reparent commits -- see http://blog.plover.com/prog/git-reorder.html for details.

The comment indexes of the patches attached to a node are not available in the d.org REST API. So we currently don't have access to them (short of scraping the node page from the web and doing HTML parsing on the HTML table...).

This is the relevant d.org issue: https://www.drupal.org/node/2815359

When I make "dorgflow https://www.drupal.org/project/drupal/issues/2652236" I have exception:
"Detected master branch 8.6.x, but it is not the current branch."

After I have make "git symbolic-ref HEAD refs/heads/8.6.x" all is OK.
See: https://confluence.atlassian.com/fishkb/setting-a-valid-head-on-your-git-repository-260772024.html

See https://stackoverflow.com/questions/6336440/how-can-i-make-git-am-git-apply-work-fuzzy-like-the-patch-command/44699442#44699442 for details.

The script currently uses 'git apply', which throws an error if the patch doesn't apply perfectly.

However, it's useful to be able to apply patches with fuzz. If we add the -3 option, then at least some fuzzy patches will work (I don't know enough about the internals of git's 3-way merge to know if it's exactly identical to patch fuzz...)

(The reason we use git apply and not patch is that we need to commit any files that the patch adds, and using patch and git add wouldn't know to add only those files and might add other untracked files.)

Dorgpatch (this project's predecessor) is able to detect a secondary branch if it has a name of the form '12345-tests'. It's probably too complex to create this branch automatically from the list d.org patches.

But an alternative approach would be to do a git diff on only certain folders. We know where tests are going to be located, so this approach might work:

• list all files which have changed
• identify the ones which have tests. For contrib, I think this is /tests/ and /src/Tests. For core I'm not sure sure.
• output a patch of just the changes to those files.

I used to use dorgflow a lot for my D7 modules. Now in D8 we are using composer for everything. It is unclear from the README instructions how to properly install dorgflow when working on D8 projects. The section about using lakedrops/dorgflow in the Packagist isn't clear.

What I'd like to do is simply install dorgflow globally via composer global require vendor/dorgflow but that doesn't seem to work. Perhaps the patch for putting dorgflow into the vendor/bin folder will help. But we also need the README updated with better instructions for installing.

To start with, you need to have an up to date git clone of the project you want to work on, be it Drupal core or a contrib project.

Sometimes i don't have a project clone or want a new... ;-)

I am working on this issue that adds a README.txt to the project. Even though there were 9 patches and commits to the issue before I started to use dorgflow on the project, the README.txt is still untracked after dorgflow is given the issue.

See https://thephp.cc/news/2016/02/questioning-phpunit-best-practices for details.

(braindump follows)

My vague plan was originally to split Situation into a class that does the analysis (Analyser) and the Situation that just gets things from DataSource classes.

But there's a bit of a circular dependency in the way we get data:

• anything to do with the feature branch, such as the git branch log, depends on the analysis of the issue number
• analysis of the issue number may depend on the git current branch

This means that arranging this into 3 services, Situation, Analyser, Git Executor, with the DataSource classes handled by Situation, won't work.

Second thought is to have all of these as a set of services, with a hierarchy of dependencies. To limit the number of classes, I was thinking to merge things together and maybe do away with the Fetchers, as actually, very little goes on in the parsing part of the DataSources.

But having a single combined GitInfo service won't work, as it both depends on and is depended on by the Analyser:

git current branch -> analyser -> git feature branch

So the git side of things needs to be split up.

Which produces a list of services and their dependencies something like this:

• git feature branch log: analyser
• d.org API: analyser
• analyser: d.org, git info
• other git info
• user input
• current folder
• git executor

Nice work! (like a lot of stuff from you ;-)

cgr joachim-n/dorgflow --stability dev
...would not do...

Can you announce this to packagist.org?

Currently running dorgflow help returns:

I would expect that this would list the commands that can be run (diff, status, update, apply, cleanup, purge) with dorgflow with a help text.

My preference for location is ".." (the directory where the module dir is in) to not clutter up the repo itself. Also in old non-composer projects it is handy to have all patches in the module dir.

My preference for name is MODULE-PATCH-COMMENT-TEXT.patch. By having the module name as prefix, in old non-composer projects i can easily and error-pronely spot if a module is patched.

Motivation
I use drupal-composer/drupal-project as a starter for all my Drupal sites. It installs a .git folder for Drupal Core within the core directory.

Problem
When creating a Core patch with dorgflow, the patch won't apply when uploading it to the Drupal.org issue queue. This is because the core is not in the patch. It needs to be because git clone --branch 8.6.x https://git.drupal.org/project/drupal.git will place the .git folder outside of core.

Solution
Fix the patch file:

• Search for ([-+]{3} [ab])/(assets|config|includes|lib|misc|modules|profiles|scripts|tests|themes)/
• Replace by $1/core/$2/

Would be handy to have a dorgflow commit command which calls git commit -am "[message]" where the message is the suggested message from d.o so thatthe issue number and the author is being mentioned for credit purposes.

Of course, sometimes you need other commit messages but I guess that 80+ % of all commits I'm using that default and such a command would save a lot of typing - and typos.

Error messages that mention branches and the like need to give the actual name of the branch.

Maybe better create patch not in project folder, but in parent folder?

A fresh install of dorgflow installs symfony/console 5.0.0 which then throws a fatal error:

PHP Fatal error: Uncaught TypeError: Return value of "Dorgflow\Command\LocalSetup::execute()" must be of the type int, NULL returned. in .../vendor/symfony/console/Command/Command.php:258

I have worked around this for now by downgrading in composer.json:

"symfony/console": "^3.2"

See symfony/symfony#33747

If your local log is clean (ish!) then it's a good start point for your issue comment, eg:

- fixed typo in method call
- changed machine name of permission
- added comment

Would be handy if the CreatePatch command would output this so you can copy and paste it to the comment.

We currently use an empty commit to mark the point when a user makes a local patch. (This is so that future patches can make an interdiff against it, and also so the user can easily see where work was.)

Git rebase removes empty commits, which means that if the user rebases their feature branch against the master branch, these a lost.

Git rebase has a --keep-empty option, which allows these empty commits to be kept, however, using this option is buggy: any commit that fails to apply (which is always a possibility with a rebase) cause the rebase to completely break: see http://git.661346.n2.nabble.com/PATCH-0-2-rebase-broken-behavior-with-keep-empty-td7643204.html

So we need a different system.

The only thing I can think of is to make TWO commits:

• one that adds a dummy file, with a placeholder commit message
• one that removes the file, and has the meaningful commit message with all the data.

This is a bit ugly, but prevents the commit from being empty.

Questions:

• does git accept a commit of an empty file? Or does this dummy file also need to contain something?
• what's a safe name to call the file? 'dorgflow-dummy.txt' maybe?

(Another idea would be to use tags instead of commits, but those don't survive a rebase either.)

Error:

$ dorgflow
PHP Warning: require_once(/var/www/drupal_8/vendor/joachim-n/dorgflow/vendor/autoload.php): failed to open stream: No such file or directory in /var/www/drupal_8/vendor/joachim-n/dorgflow/dorgflow on line 6

I think we should replace:
require_once __DIR__ . '/vendor/autoload.php';
with:
require_once __DIR__ . '/../../autoload.php';