eliziumnet / loopz Goto Github PK

For any command using Write-HostFeItemDecorator, the current display of just a single line for each item processed can result in too wide an output causing excessive wrapping ruining the resultant output. An example of this is Rename-ForeachFsItem (Rename-Many).

We need to define appropriate options for Write-HostFeItemDecorator that will enable its output to be properly formatted over multiple lines for each iterated item.

Add another parameter to the command called Context, which adds contextual info. This will allow other new higher-order commands to be built on top of Rename-Many and inject the higher-order info into it.

• Title (goes into the header)
• Item Message ('Rename Item')
• Summary Message ('Rename Summary')

Derive Invoke-ForeachFile from TerminalBuddy

Trigger needs to be clarified.
Summary needs to be resolved (this migt be a UI only thing )

Currently, patterns provided by the user (for Pattern, Anchor & With parameters) provided to the RegEx constructor without any options. That means for examle that case sensitivity is on by default, without a means to change this by the user.

Pattern parameters should be able to support global flags at the end of the pattern preceded with a forward slash: eg 'abc/i'. The .Net flags are documented at https://docs.microsoft.com/en-us/dotnet/standard/base-types/regular-expression-options and should be expressed as a string of comma separated values (use codes as described at regex101).

Factory function new-RegularExpression, needs to be modified to support this functionality.

standardise emoji usage, by assigning specific emojis to well defined actions, so this can be made to be uniform accross multiple commands

Need a global Write-HostFeFsItem decorator script-block.
Define a global Summary script-block.

This is a big change, but the right one to do, unfortunately for little payback. Use get-command/invoke on block rather than using the & op on the named function string. This will fix the problem using test functions in Pester tests.

See FuncitonInfo Class

branch: use-command-ref

Minor cosmetic refinements.

Eg some properties are currently displayed without quotes.

Need to check for trigger and break requests

Currently, the function writes a single key-value pair denoted by 'ITEM-LABEL' & 'ITEM-VALUE' in PassThru. This is too limiting for the client. The client needs to be able to defined any number of custom properties.

The custom properties should be defined in PassThru as 'PROPERTIES', which represents a collection of key/value pairs which are then just written out. We can leave in ITEM-LABEL/ITEM-VALUE if the client only needs to display a single property, but a pre-condition of the function is that either 'PROPERTIES' can exist or 'ITEM-LABEL' & 'ITEM-VALUE', but not both.

CopyFiles switch should be able to work independently from CreateDirs. currently, CopyFiles has not effect unless CreateDirs is also set. There is a useful usecase that requires this decoupling. If you want to mirror and directory tree, but in the destination tree, you only want to copy over some files that match a particluar filter (includes/excludes), then this won't work unless CreateDirs. If you also set CreateDirs, then this could result in empty directories in the destination, as a result of there being files that fail to pass the filter, but the directroy would be created anyway.

The much better way around this is to remove the need to also set the CreateDirs flag.

branch: feature/copyfiles-without-createdirs

I forgot to add this functionality to Invoke-ForeachFsItem and Invoke-TraverseDirectory, Should implement the same behaviour as is already implemented in Invoke-MirrorDirectoryTree using parameters:

• DirectoryIncludes
• DirectoryExcludes
• FileIncludes
• FileExcludes

Running Xatch periodically throws errors like this:

_________________________________________________________________________________
   [🎶] Conversion Summary (Gatecrasher- Global Sound System (Longitude)) | ("Count" -> "17", "Skipped" -> "0", "Triggered" -> "True", "From" -> "flac", "To" -> "mp3")
_________________________________________________________________________________
   [📁] Audio Directory // ["No" => "850", "Album" => "Gatecrasher- Global Sound System (Longitude)"]
MethodInvocationException: C:\Users\Plastikfan\Documents\PowerShell\Modules\Elizium.Loopz\Elizium.Loopz.psm1:1306
Line |
1306 |      $adapted.Invoke(
     |      ~~~~~~~~~~~~~~~~
     | Exception calling "Invoke" with "5" argument(s): "Exception calling "Invoke" with "1" argument(s): "Exception calling "Invoke" with "4" argument(s): "Exception calling "Invoke" with "1" argument(s): "Exception calling
     | ".ctor" with "2" argument(s): "Count cannot be less than zero. (Parameter 'count')"""""

However, processing that directory by itself, does not cause the same error to be thrown. Only happens in the full batch run.

Ocurring on this line in Invoke-TraverseDirectoryTree:

    $adapted.Invoke(
      $_underscore,
      $_passThru['LOOPZ.TRAVERSE.CONDITION'],
      $_passThru,
      $PassThru['LOOPZ.TRAVERSE.INVOKEE'],
      $_trigger
    );

bulk file system item renamer.

Alias: rnfsi

This is a problem because errors are being absorbed and hidden which makes problem resolution difficult. Its better to crash and burn fast then to hide the errors.

Plastikfan@JANUS  ~\dev\github\PoSh\Loopz   feature/header-block ≢ +1 ~1 -0 !                                                                                            [09:26]
λ cd .\Elizium.Loopz\
Plastikfan@JANUS  ~\dev\github\PoSh\Loopz\Elizium.Loopz   feature/header-block ≢ +2 ~1 -0 !                                                                              [09:29]
λ inv-bu
Build . C:\Users\Plastikfan\dev\github\PoSh\Loopz\Elizium.Loopz\Elizium.Loopz.build.ps1
Task /./Clean
Done /./Clean 00:00:00.0283399
Task /./Build/Compile
Missing output 'C:\Users\Plastikfan\dev\github\PoSh\Loopz\Elizium.Loopz\Output\Elizium.Loopz\Elizium.Loopz.psm1'.
Done /./Build/Compile 00:00:00.4513523
Task /./Build/CreateManifest/CopyPSD
VERBOSE: Performing the operation "Copy File" on target "Item: C:\Users\Plastikfan\dev\github\PoSh\Loopz\Elizium.Loopz\Elizium.Loopz.psd1 Destination: C:\Users\Plastikfan\dev\github\PoSh\Loopz\Elizium.Loopz\Output\Elizium.Loopz\Elizium.Loopz.psd1".
Done /./Build/CreateManifest/CopyPSD 00:00:00.0242980
Task /./Build/CreateManifest/UpdatePublicFunctionsToExport
Done /./Build/CreateManifest/UpdatePublicFunctionsToExport 00:00:00.0204947
Done /./Build/CreateManifest 00:00:00.0479371
Task /./Build/Ana/Analyse
ERROR: Object reference not set to an instance of an object.
At C:\Users\Plastikfan\dev\github\PoSh\Loopz\Elizium.Loopz\Elizium.Loopz.build.ps1:175 char:3
+   Invoke-ScriptAnalyzer -Path .\Output\ -Recurse
+   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
At C:\Users\Plastikfan\dev\github\PoSh\Loopz\Elizium.Loopz\Elizium.Loopz.build.ps1:174 char:1
+ task Analyse {
+ ~~~~~~~~~~~~~~
At C:\Users\Plastikfan\dev\github\PoSh\Loopz\Elizium.Loopz\Elizium.Loopz.build.ps1:7 char:1
+ task Ana Analyse
+ ~~~~~~~~~~~~~~~~
At C:\Users\Plastikfan\dev\github\PoSh\Loopz\Elizium.Loopz\Elizium.Loopz.build.ps1:5 char:1
+ task Build Compile, CreateManifest, Ana
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
At C:\Users\Plastikfan\dev\github\PoSh\Loopz\Elizium.Loopz\Elizium.Loopz.build.ps1:2 char:1
+ task . Clean, Build, Tests, Stats
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Build FAILED. 9 tasks, 1 errors, 0 warnings 00:00:01.1740783
Invoke-ScriptAnalyzer: C:\Users\Plastikfan\dev\github\PoSh\Loopz\Elizium.Loopz\Elizium.Loopz.build.ps1:175
Line |
 175 |    Invoke-ScriptAnalyzer -Path .\Output\ -Recurse
     |    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Object reference not set to an instance of an object.

⨯ Plastikfan@JANUS  ~\dev\github\PoSh\Loopz\Elizium.Loopz   feature/header-block ≢ +2 ~1 -0 !                                                                            [09:29]
λ inv-bu

Line 175 is the invoke:

task Analyse {
  Invoke-ScriptAnalyzer -Path .\Output\ -Recurse
}

The Directory/File name is set to the new name and does not preserve the old name when WhatIf not active; functionality is still correct though.

This is possibly causing this error:

[-] invoke-ConversionBatch.given: blah.should:  204ms (203ms|1ms)
 ParameterBindingValidationException: Cannot bind argument to parameter 'Path' because it is null.
 MethodInvocationException: Exception calling "Invoke" with "4" argument(s): "Cannot bind argument to parameter 'Path' because it is null."
 CmdletInvocationException: Exception calling "Invoke" with "4" argument(s): "Cannot bind argument to parameter 'Path' because it is null."
 MethodInvocationException: Exception calling "Invoke" with "4" argument(s): "Exception calling "Invoke" with "4" argument(s): "Cannot bind argument to parameter 'Path' because it is null.""
 CmdletInvocationException: Exception calling "Invoke" with "4" argument(s): "Exception calling "Invoke" with "4" argument(s): "Cannot bind argument to parameter 'Path' because it is null.""
 MethodInvocationException: Exception calling "Invoke" with "1" argument(s): "Exception calling "Invoke" with "4" argument(s): "Exception calling "Invoke" with "4" argument(s): "Cannot bind argument to parameter 'Path' because it is null."""
 CmdletInvocationException: Exception calling "Invoke" with "1" argument(s): "Exception calling "Invoke" with "4" argument(s): "Exception calling "Invoke" with "4" argument(s): "Cannot bind argument to parameter 'Path' because it is null."""
 MethodInvocationException: Exception calling "Invoke" with "5" argument(s): "Exception calling "Invoke" with "1" argument(s): "Exception calling "Invoke" with "4" argument(s): "Exception calling "Invoke" with "4" argument(s): "Cannot bind argument to parameter 'Path' because it is null.""""
 CmdletInvocationException: Exception calling "Invoke" with "5" argument(s): "Exception calling "Invoke" with "1" argument(s): "Exception calling "Invoke" with "4" argument(s): "Exception calling "Invoke" with "4" argument(s): "Cannot bind argument to parameter 'Path' because it is null.""""
 MethodInvocationException: Exception calling "Invoke" with "5" argument(s): "Exception calling "Invoke" with "5" argument(s): "Exception calling "Invoke" with "1" argument(s): "Exception calling "Invoke" with "4" argument(s): "Exception calling "Invoke" with "4" argument(s): "Cannot bind argument to parameter 'Path' because it is null."""""
 at <ScriptBlock>, C:\Users\Plastikfan\Documents\PowerShell\Modules\Elizium.Loopz\Elizium.Loopz.psm1:1308
 at Invoke-ForeachFsItem<Process>, C:\Users\Plastikfan\Documents\PowerShell\Modules\Elizium.Loopz\Elizium.Loopz.psm1:404
 at Invoke-TraverseDirectory, C:\Users\Plastikfan\Documents\PowerShell\Modules\Elizium.Loopz\Elizium.Loopz.psm1:1457
 at Invoke-MirrorDirectoryTree, C:\Users\Plastikfan\Documents\PowerShell\Modules\Elizium.Loopz\Elizium.Loopz.psm1:944
 at invoke-ConversionBatch, C:\Users\Plastikfan\dev\github\PoSh\Xatch\Elizium.Xatch\Internal\invoke-ConversionBatch.ps1:212
 at <ScriptBlock>, C:\Users\Plastikfan\dev\github\PoSh\Xatch\Elizium.Xatch\Tests\Internal\invoke-ConversionBatch.tests.ps1:59
Tests completed in 713ms

This is happening in Xatch, when WHAT-IF set to true.

Also there is some ambiguity of the WHAT-IF entry in PassThru. Make sure they all use WHAT-IF, not LOOPZ.MIRROR.WHAT-IF

First, the crumb should not only be displayed if the user has defined a message in the PassThru as 'LOOPZ.HEADER-BLOCK.MESSAGE'

When a message is defined, sometimes that is displayed incorrectly eg:

[🛡️] ---------------------------------------------------------------------------------------------- ------ [ Rename ] ---

In this example (Rename-ForeachFsItem), there is a gap in the dashes, why?

branch: boilerplate

Both these functions do not capture the invoke result, which contains the trigger and break requests and the Product. These should be handled correctly and consistently with ForeachFsItem.

For some reason, "Directory"/"File" item is missing from the output if the path contains square brackets. This is probably due to using Path somewhere instead of LiteralPath. Only manifests itself when WhatIf is not active.

Setting Literal currently throws up an error.

Remove the current Literal param and replace it with a positional parameter LiteralPattern, which should be used as the Literal version of Pattern, so we don't specify Literal as a separate flag. The literal version of With should be LiteralWith. There should also be a LiteralTarget.

• Literal and Pattern are mutually exclusive.
• With / LiteralWith/Target/LiteralTarget can be used with Literal & Pattern.
• Literal parameters support wildcards (except LiteralTarget).

So the first 2 positional parameters should be LiteralPattern and LiteralWith. This way the user can specify

Rename-ForeachFsItem "old" "new"

and this would be the intuitive way to use the command.

If the user wants to use the more advanced functionality using dynamic regular expressions, then -Pattern and -With come into play and must be named explicitly.

EDIT: we can't allow a wildcard to be used with target/with/pattern, because it is too blunt an instrument. Wildcards are suitbale for narrowing result sets like you would with Get-ChildItem, but not for selecting sub strings; this is why regex is required.

Derive Invoke-ForeachDirectory from XLDPlusPlus.Invoke-ForeachDirectory (iterate.ps1)

Trigger needs to be clarified.
Summary needs to be resolved (this migt be a UI only thing )

The initial version of Rename-ForeachFsItem caches all pipeline items before piping them to Invoke-ForeachFsItem. This technique is undesirable if the pipeline is large. There is a way around this issue and as the solution is involved, it warrants a separate issue outside of the initial implementation of the command. We can use this as a blue print for the implementation of other commands.

See the answer posted to my stackoverflow question.

Also see this blog post Proxy Functions: Spice Up Your PowerShell Core Cmdlets

Combines the functionality of invoke-foreachFile and invoke-foreachDirectory, both of which are now redundant. Define an extra 2 switches, File and Directory, which work in the same way as they do on Get-ChildItem. If neither of these flags are present, then
invoke-ForeachFsItem will invoke the specified function/scriptblock for appropriate files and directories.

Also need to define a global Summary variable that can be used by the client.

Should be done in a generic manner so that other commands can re-use the functionality. Implement via shell object.

To be consistent with the convention of naming functions that are designed to work with invoke-foreachFile/Directory/FsItem containing 'Fe', to indicate foreach.

The message as defined under 'LOOPZ.WH-FOREACH-DECORATOR.MESSAGE' is static. It needs to be made dynamic and be able to change depending on the result of invoking the invokee.

Define a new PassThru entry under key 'LOOPZ.WH-FOREACH-DECORATOR.GET-MESSAGE' that should be a scriptblock that calling side can define.

branch: dynamic-wh-fe-message

Currently WHAT-IF is scoped: LOOPZ.MIRROR.WHAT-IF. This does not make sense. WHAT-IF should be targeted at every command in the passThru chain. LOOPZ.MIRROR.WHAT-IF should simply be WHAT-IF.

This is a breaking change.

Currently, Write-HostFeItemDecorator is not using scoped names such as LOOPZ.FN.ITEM. It is a good idea to us scoped names to avoid the chance of name clashes and over-writing entries by accident.

WhatIf setting should not be applied to New-Item, because it causes other code to break as required directories have not been created, leading to $null being passes as Path parameter.

[-] invoke-ConversionBatch.given: blah.should:  204ms (203ms|1ms)
 ParameterBindingValidationException: Cannot bind argument to parameter 'Path' because it is null.
 MethodInvocationException: Exception calling "Invoke" with "4" argument(s): "Cannot bind argument to parameter 'Path' because it is null."
 CmdletInvocationException: Exception calling "Invoke" with "4" argument(s): "Cannot bind argument to parameter 'Path' because it is null."
 MethodInvocationException: Exception calling "Invoke" with "4" argument(s): "Exception calling "Invoke" with "4" argument(s): "Cannot bind argument to parameter 'Path' because it is null.""
 CmdletInvocationException: Exception calling "Invoke" with "4" argument(s): "Exception calling "Invoke" with "4" argument(s): "Cannot bind argument to parameter 'Path' because it is null.""
 MethodInvocationException: Exception calling "Invoke" with "1" argument(s): "Exception calling "Invoke" with "4" argument(s): "Exception calling "Invoke" with "4" argument(s): "Cannot bind argument to parameter 'Path' because it is null."""
 CmdletInvocationException: Exception calling "Invoke" with "1" argument(s): "Exception calling "Invoke" with "4" argument(s): "Exception calling "Invoke" with "4" argument(s): "Cannot bind argument to parameter 'Path' because it is null."""
 MethodInvocationException: Exception calling "Invoke" with "5" argument(s): "Exception calling "Invoke" with "1" argument(s): "Exception calling "Invoke" with "4" argument(s): "Exception calling "Invoke" with "4" argument(s): "Cannot bind argument to parameter 'Path' because it is null.""""
 CmdletInvocationException: Exception calling "Invoke" with "5" argument(s): "Exception calling "Invoke" with "1" argument(s): "Exception calling "Invoke" with "4" argument(s): "Exception calling "Invoke" with "4" argument(s): "Cannot bind argument to parameter 'Path' because it is null.""""
 MethodInvocationException: Exception calling "Invoke" with "5" argument(s): "Exception calling "Invoke" with "5" argument(s): "Exception calling "Invoke" with "1" argument(s): "Exception calling "Invoke" with "4" argument(s): "Exception calling "Invoke" with "4" argument(s): "Cannot bind argument to parameter 'Path' because it is null."""""
 at <ScriptBlock>, C:\Users\Plastikfan\Documents\PowerShell\Modules\Elizium.Loopz\Elizium.Loopz.psm1:1308
 at Invoke-ForeachFsItem<Process>, C:\Users\Plastikfan\Documents\PowerShell\Modules\Elizium.Loopz\Elizium.Loopz.psm1:404
 at Invoke-TraverseDirectory, C:\Users\Plastikfan\Documents\PowerShell\Modules\Elizium.Loopz\Elizium.Loopz.psm1:1457
 at Invoke-MirrorDirectoryTree, C:\Users\Plastikfan\Documents\PowerShell\Modules\Elizium.Loopz\Elizium.Loopz.psm1:944
 at invoke-ConversionBatch, C:\Users\Plastikfan\dev\github\PoSh\Xatch\Elizium.Xatch\Internal\invoke-ConversionBatch.ps1:212
 at <ScriptBlock>, C:\Users\Plastikfan\dev\github\PoSh\Xatch\Elizium.Xatch\Tests\Internal\invoke-ConversionBatch.tests.ps1:59
Tests completed in 713ms

For Invoke-MirrorDirectoryTree, in a WhatIf sceanrio, we should be able to create Directories; this saves other code from breaking. WhatIf should still apply to file creation.

In the same way there is a Summary block, the compound functions also need to have a Header block.

branch: header-block

Currently, ITEM-VALUE can only accept a static string, which makes this not as useful as desired. There is little to no value in having a static value that is displayed for each item in the pipleline, because it would be the same value for every item. The client should be able to provide a script block (stored in the passThru under key 'LOOPZ.WH-FOREACH-DECORATOR.GET-VALUE', when given a result and the underscore can determine what the item value should be.

The same criticsim can be made of 'LOOPZ.WH-FOREACH-DECORATOR.PROPERTIES'. However, this should be resolved by the invokee by populating the PSCustomObject returned with an addition property 'Properties' with an array of key/value pairs. Write-HostFeFsItem should pick up these values and add them to the themedPairs collection. Actually, instead of simply overriding 'LOOPZ.WH-FOREACH-DECORATOR.PROPERTIES', we can simply just add to the themedPairs collection.

Actually, on further consideration, should we just remove custom properties from the PassThru altogether? ITEM-VALUE could be replaced with a property on the PSCustomObject invokee result, or perhaps even better let's completely do away with ITEM-LABEL/ITEM-VALUE and just rely on PROPERTIES/Properties instead; this would make the framework simpler to use.

This allows client code to access the number of processed items in compound functions, outside of the the Summary object if so provided.
All compound functions should implement this.
branch: add-count-to-passthru

Some property values are too long to be displayed on the same line as other properties. Eg when processing directories, the fullpath is too often far too long a string, but its useful to see this displayed. It is not suitbale for it to be displayed in a key/value list. It is better for this to be displayed on their own line.

Add a new PassThru entry 'LOOPZ.SUMMARY-BLOCK.WIDE-PAIRS', which is an array of key/value pairs, where each pair is displayed on its own line.

This needs to be implemented for the sake of completeness and consistency. Invoke-ForeachFsItem already provides this using parameter sets and this should also apply to TraverseDirectory and MirrorDirectoryTree.

See https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/set-strictmode?view=powershell-7

Derive from TerminalBuddy

Dervive from XLDPlusPlus.Invoke-ConversionBatch (.psm1)

Applying a directory filter can be too brutal. Currently, child directories that match the filter will not be selected if it contains any ancestors that do not also match the filter. We need to modify this behaviour and give the client an option to change this.

Invoke-TraverseDirectory contains the following offending code:

    [System.IO.DirectoryInfo[]]$directoryInfos = Get-ChildItem -Path $Path `
      -Directory | Where-Object { $Condition.Invoke($_) }

    if ($directoryInfos) {
      $directoryInfos | Invoke-ForeachFsItem -Directory -Block $adapter `
        -PassThru $PassThru -Condition $Condition;
    }

and Invoke-MirrorDirectoryTree contains:

  [scriptblock]$filterDirectories = {
    [OutputType([boolean])]
    param(
      [System.IO.DirectoryInfo]$directoryInfo
    )
    Select-Directory -DirectoryInfo $directoryInfo `
      -Includes $DirectoryIncludes -Excludes $DirectoryExcludes;
  }

  Invoke-TraverseDirectory -Path $resolvedSourcePath `
    -SourceDirectoryBlock $doMirrorBlock -PassThru $PassThru -Summary $summary `
    -Condition $filterDirectories;

We could introduce a new flag on Invoke-TraverseDirectory, lets say PluckDescendents (meaning that we can pluck matching descendents, even when its parents may not match the filter), which could be use whilst processing the top level directory. If the flag is set, we don't use the recurse functionality. Instead, we issue a separate Get-ChildItem request with the Recurse parameter set, then apply the filter on this resultant set which would allow us to see those matching descendents. The key here is that we only ever want to issue a single Invoke-ForeachFsItem request when PluckDescendents is set.

The existing functionlaity should still stand because that is what the client may want.

Write-HostFeItemDecorator needs to be able to affirm the Product. Currently, the Product s written without any affrmation:

    # Get Product if it exists
    #
    [string]$productLabel = [string]::Empty;
    if ($invokeResult -and $invokeResult.psobject.properties.match('Product') -and $invokeResult.Product) {
      $productValue = $getResult.Invoke($invokeResult.Product);
      $productLabel = $PassThru.ContainsKey('LOOPZ.WH-FOREACH-DECORATOR.PRODUCT-LABEL') `
        ? $PassThru['LOOPZ.WH-FOREACH-DECORATOR.PRODUCT-LABEL'] : 'Product';

      if (-not([string]::IsNullOrWhiteSpace($productLabel))) {
        $themedPairs += , @($productLabel, $productValue);
      }
    }

In particular, its this line:

$themedPairs += , @($productLabel, $productValue);

that needs to be modified. A third parameter, its affirmation needs to ebe passed in here.

Cut indicator in missing when the user specifies a Pattern but does not specify either a Paste or LiteralWith. The outcome is correct, but the indicator ([✂️] Cut) is missing from the output.

But its only a cut operation for Upate-Match action so doesnt apply to Move-Match, because Move-Match implies that either an Anchor has been specified or Start/End has been specified, meaning that its a move operation.

Public functions not exported from the module.