Hi All,

I think this might not be an UmbPack issue, but instead an issue with the code on Our, but I thought it would make sense to raise the issue here so that you can see it.

You can see in the following screen shot that we have uploaded a new version of the "FontAwesome Icon Pack" and that the package shows that it supports all the minor v8 versions, however on the right hand side it shows that two of the minor versions are un-tested. I believe we have uploaded things correctly using UmbPack, so I think this is potentially a bug?

Cheers,

Chris

I have a build script running on AzurePipelines which is successfully pushing packages to our, but as part of that deploy I have an archive flag set to archive any previous versions of the package, however it doesn't look to be working as the old packages remain un-archived on our.

For example, with this command that was run on my build server

umbpack push D:\a\1\BuildArtifacts\packages\Vendr.PaymentProviders.Opayo.1.0.0.zip -k %OUR_UMBRACO_API_KEY% -w v860,v850,v840,v830,v820 -c false -a Vendr.PaymentProviders.Opayo.*.zip

The existing package Vendr.PaymentProviders.Opayo.0.1.0.zip should have been archived, and indeed the build script logged that this occurred

Archived 1 packages matching the archive pattern.
Extracting info from package.xml file
Name: Vendr.PaymentProviders.Opayo
Version: 1.0.0

Uploading Vendr.PaymentProviders.Opayo.1.0.0.zip to our.umbraco.com ...
The package 'Vendr.PaymentProviders.Opayo.1.0.0.zip' was successfully uploaded to our.umbraco.com

However when I go to Our and check the list of packages, I can still see that 0.1.0 is unarchived

Given that the build script is logging the fact that a package is being archived, I'm assuming the UmbPack tool is correctly locating the package and making the request, but it appears something on the Our end is not actually archiving the file.

write some basic documentation, and example usage for the command line tool. (for me to do!)

Creating a NuGet package using the SDK/csproj properties is now really easy by using dotnet pack. This uses the Pack MSBuild target underneath and we can already piggyback on this by adding the following target to your csproj:

<Target Name="UmbPack" AfterTargets="Pack">
    <Exec Command="umbpack pack package.xml -v $(Version) -o $(PackageOutputPath) -p Configuration=$(Configuration)" />
</Target>

This does however also require UmbPack to be installed as global dotnet tool. It's possible to write a native MSBuild task that can be included into your project by simply installing a single NuGet package, which removes this requirement, but can also automatically add the above target to your project.

We can also re-use a lot of the existing package properties to automatically populate the package.xml and introduce extra input properties to specify the path to the XML file, e.g.:

• IsUmbPackable - A boolean value that specifies whether the project can be packed. The default value is true.
• UmbPackFile - Relative or absolute path to the .xml file being use for packaging.
• UmbPackBasePath - Base path for the .xml file.
• UmbPackProperties - Semicolon separated list of key=value pairs.

We can start by first requiring the path to the package.xml to be set, but eventually might be able to auto-generate it.

Feature request

I've watched the demo of Warrens prototype for the CLI tool pushing packages to Our and I had the thought.

What if you want to push a beta release and not replace/set the current release?

My thought is add a flag, "--beta" to the CLI interface that allows a package file to be pushed up and flagged as a beta release.

This could fall back into the Github action's as well so that if your commit is tagged with "-beta" then it would auto put this into command line call.

Type of feature

CLI switch option

Our feature

Might need a modification to Our to allow a package file to be uploaded as a beta

When installing the tool you see the following:

Is it right that there are two different version numbers? Seems a bit confusing to me.

Is "Umbraco Package CLI" still the right name to show here?

Output file package.xml generated by command UmbPack init contains a tag <licence>, but it should be <license> whitch is supported by Umbraco package standard.
This misspell also make occour an error durning package installation in Umbraco 7, because <license> tag is required by this Umbraco version.

Trying to install this via the docs here:

https://our.umbraco.com/documentation/Extending/Packages/UmbPack/#installing-umbpack-locally

and get the following error:

Seems you have to include a version number like so:

dotnet tool install --global Umbraco.Tools.Packages --version 0.9.3

The above was taken from the Nuget listing itself: https://www.nuget.org/packages/Umbraco.Tools.Packages/

When you successfully push to Our you get a message like this:

The message "The package ... was successfully uploaded" is in there twice. And in the first instance it's missing the file name parameter.

About to raise a PR with a fix

Turns out my missing file issues #24 are caused by me running umbpack pack from the solution root but the package.xml is a folder deeper. The file references within package.xml are all relative to its location but the tool assumes that the working directory is where I ran the command from.

Not sure what makes more sense here. Logically it should make sense that the files in the package.xml should be relative to it I think so it shouldn't matter where I run it from?

It would be nice if the files section in the packages.xml (to be fed into UmbPack) accepted the same format as the output packages.xml. A potential use case for this might be if someone has an existing package they want to use the tool with, without rewriting the files section.

Currently the format of the tool requires the files section to be the following:

<folder path="src/xxx/App_Plugins/duug" orgPath="App_Plugins/duug" />
<file path="src/xxx/bin/release/xxx.dll" orgPath="bin/xxx.dll" />

The Umbraco package.xml uses the below format:

<file>
  <guid>package.manifest</guid>
  <orgPath>src/duug/App_Plugins/duug</orgPath>
  <orgName>package.manifest</orgName>
</file>

Based on the requirements, I think we need to flesh out the command line a little to give us more space to expand/separate the functions of the CLI.

I am proposing we follow a similar command structure to that of the nuget command (given we are essentially pushing and packing in a similar way)

UmbPack <command> <args> [options]

Push          pushes a package to our.umbraco.com

Pack          Creates a Umbraco package from a given folder and package.xml

Init          Creates a package.xml file from a series of questions

The push command is the code as is now, so pushing to our.umbraco.org, with the additional options required to do this.

UmbPack push mypackagefile.zip -key xxxxx-xxxxx-xxxxx

The Pack command will take a folder and munge everything in the folder into a Umbraco package.zip file - either finding the package.xml in the folder or the user says where it is, you can set the version when you do this too

UmbPack Pack <folder> -version 1.0.2 

the Init command works like npm init, in that it will ask you a series of questions and at the end of it generate the package.xml file.

UmbPack Init
Package Name (folder) > My Package
Version (1.0.0) >
Description >
Author >
....

In v7 you can generate packages directly in the backoffice that ends up in this structure:

• pkg.zip
  -- folder with guid name
  --- package.xml
  --- other files

but the tool needs them to be of the structure:

• pkg.zip
  -- package.xml
  -- other files

otherwise we get a null ref error:

Unhandled exception. System.AggregateException: One or more errors occurred. (Object reference not set to an instance of an object.)
 ---> System.NullReferenceException: Object reference not set to an instance of an object.
   at Umbraco.Packager.CI.Parse.PackageXml(String packagePath) in C:\Umbraco\UmbracoRepos\Umbraco-Packager-CLI\src\Parse.cs:line 20
   at Umbraco.Packager.CI.Verbs.PushCommand.RunAndReturn(PushOptions options, PackageHelper packageHelper) in C:\Umbraco\UmbracoRepos\Umbraco-Packager-CLI\src\Verbs\PushCommand.cs:line 77
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at Umbraco.Packager.CI.Program.<>c__DisplayClass1_0.<InternalMain>b__2(PushOptions opts) in C:\Umbraco\UmbracoRepos\Umbraco-Packager-CLI\src\Program.cs:line 60
   at CommandLine.ParserResultExtensions.WithParsed[T](ParserResult`1 result, Action`1 action)
   at Umbraco.Packager.CI.Program.InternalMain(String[] args, PackageHelper packageHelper) in C:\Umbraco\UmbracoRepos\Umbraco-Packager-CLI\src\Program.cs:line 58
   at Umbraco.Packager.CI.Program.Main(String[] args) in C:\Umbraco\UmbracoRepos\Umbraco-Packager-CLI\src\Program.cs:line 35
   at Umbraco.Packager.CI.Program.<Main>(String[] args)

One of the Push options is to specify whether the package file you push should be the new current file.

We call the internal property Publish for some reason, would be good to update the naming to avoid confusion, both the property itself and the helptext tied to it.

https://github.com/umbraco/UmbPack/blob/dev/src/Verbs/PushCommand.cs#L33

Been playing with the "push" command. It all works which is lovely.

However I seeing a problem with this going forward. How do I get the version number into my build? Currently I'm just using some scripts in my package.json but the "push" command requires a version number so I'm having to hardcode that into my scripts. This isn't ideal.

Wondered if I can just give it a glob or regex and it then would use that. This would allow me to use wild card filenames:

umbpack push -k [MY-API-KEY] ./dist/TestPackage_*.zip

That way I wouldn't have to keep updating my script. This would only "work" if there is only one match in the folder. If that was the case then I would be happy if it failed with a helpful message:

"Multiple files match your package path (./dist/TestPackage_*.zip), either make your path more specific or remove unwanted files (did you forget to clean up old packages?)"

If the tool needs to know the version number well that should be in the package.xml file anyway so it could get it from there.

Once we have created documentation (#5 ) we should update the readme of this repo with info on using and also running the tool locally.

It seems the specified name from package.xml isn't preserved when building the package.
E.g. Our.Umbraco.ColorPickr seems to generate Our_Umbraco_ColorPickr_1.0.0.zip and replacing . with _.

umbpack pack .\package.xml

while building NuGet package does keep the dot in the naming:

nuget pack .\package.nuspec

In v0.9.5 we can overide the name, but it would also require to add the version manually.

umbpack pack .\package.xml -n Our.Umbraco.ColorPickr_1.0.0.zip

Creating the package.xml through the init command it will create something like:

      <requirements type="strict">
        <major>8</major>
        <major>0</major>
        <major>0</major>
      </requirements>

The 3x major needs to be major, minor, patch - currently packages packed based on this will give an error when installed.

With the nuget package, you can have variables in your nuspec file like $SomeProp$ and then in the pack command you can pass properties in line -Properties SomeProp=SomeValue and then the token in the nuspec file will be replaced. This is really useful when things like the source directory are not known until build time.

It would be great if we could do the same with the UmbPack pack command

We are creating a new Umbraco Package where we are deploying Templates that become part of the Umbraco website.

The syntax from the existing createdPackages.config file would suffice, e.g.

<templates>1102,1063,1060,1075,1088,1080,1107,1085</templates>

Then for each Template ID the template is exported into the package.xml file into the <Templates> section, e.g.

<Templates>
    <Template>
      <Name>My Template</Name>
      <Alias>MyTemplate</Alias>
      <Master>MyMasterTemplate</Master>
      <Design><![CDATA[@inherits Umbraco.Web.Mvc.UmbracoViewPage<ContentModels.MyTemplate>
@using ContentModels = Umbraco.Web.PublishedModels;
@{
	Layout = null;
}]]></Design>
    </Template>
</Templates>

It would be nice to be able to automate the package build process using UMBPACK but without this feature we are using the "Create Package" function within the Backoffice.

.NET 5.0

Installed UmbPack 1.0.0
Run umbpack init

Got the below exception:

D:\Repos\Umbraco.MegaNavV8\Builds\UmbPack> umbpack init
Unhandled exception. System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.Resources.MissingManifestResourceException: Could not find the resource "Umbraco.Packager.CI.Properties.HelpTextResource.resources" among the resources "UmbPack.Properties.Defaults.resources", "UmbPack.Properties.HelpTextResource.resources", "UmbPack.Properties.Resources.resources" embedded in the assembly "UmbPack", nor among the resources in any satellite assemblies for the specified culture. Perhaps the resources were embedded with an incorrect name.
   at System.Resources.ManifestBasedResourceGroveler.HandleResourceStreamMissing(String fileName)
   at System.Resources.ManifestBasedResourceGroveler.GrovelForResourceSet(CultureInfo culture, Dictionary`2 localResourceSets, Boolean tryParents, Boolean createIfNotExists)
   at System.Resources.ResourceManager.InternalGetResourceSet(CultureInfo culture, Boolean createIfNotExists, Boolean tryParents)
   at System.Resources.ResourceManager.GetString(String name, CultureInfo culture)
   at Umbraco.Packager.CI.Properties.HelpTextResource.get_HelpPack() in D:\a\UmbPack\UmbPack\src\Properties\HelpTextResource.Designer.cs:line 100
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.RuntimePropertyInfo.GetValue(Object obj, BindingFlags invokeAttr, Binder binder, Object[] index, CultureInfo culture)
   at System.Reflection.RuntimePropertyInfo.GetValue(Object obj, Object[] index)
   at CommandLine.Infrastructure.LocalizableAttributeProperty.GetLocalizedValue()
   at CommandLine.Infrastructure.LocalizableAttributeProperty.get_Value()
   at CommandLine.VerbAttribute.get_HelpText()
   at CommandLine.Core.Verb.FromAttribute(VerbAttribute attribute)
   at CommandLine.Core.Verb.<>c.<SelectFromTypes>b__11_2(<>f__AnonymousType12`2 <>h__TransparentIdentifier0)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Enumerable.Any[TSource](IEnumerable`1 source, Func`2 predicate)
   at CommandLine.Core.InstanceChooser.MatchVerb(Func`3 tokenizer, IEnumerable`1 verbs, IEnumerable`1 arguments, StringComparer nameComparer, Boolean ignoreValueCase, CultureInfo parsingCulture, Boolean autoHelp, Boolean autoVersion, IEnumerable`1 nonFatalErrors)
   at CommandLine.Core.InstanceChooser.<>c__DisplayClass0_0.<Choose>b__0()
   at CommandLine.Core.InstanceChooser.Choose(Func`3 tokenizer, IEnumerable`1 types, IEnumerable`1 arguments, StringComparer nameComparer, Boolean ignoreValueCase, CultureInfo parsingCulture, Boolean autoHelp, Boolean autoVersion, IEnumerable`1 nonFatalErrors)
   at CommandLine.Parser.ParseArguments(IEnumerable`1 args, Type[] types)
   at CommandLine.ParserExtensions.ParseArguments[T1,T2,T3](Parser parser, IEnumerable`1 args)
   at Umbraco.Packager.CI.Program.InternalMain(String[] args, PackageHelper packageHelper) in D:\a\UmbPack\UmbPack\src\Program.cs:line 56
   at Umbraco.Packager.CI.Program.Main(String[] args) in D:\a\UmbPack\UmbPack\src\Program.cs:line 33
   at Umbraco.Packager.CI.Program.<Main>(String[] args)

It is currently supported that you can enable / disable the ability to set the package being uploaded as the current package which is controlled by the -c command line argument.

However, this argument is not following command line etiquette in that boolean options should be set or unset purely by their presence. Ie, currently, to control whether an upload should be made current or not you can do the following:

// Default is to make current
umbpack push -k [APIKEY] .\UmbPackTest_1.0.0.zip

// Explicitly make current
umbpack push -k [APIKEY] -c true .\UmbPackTest_1.0.0.zip

// Don't make the package current
umbpack push -k [APIKEY] -c false .\UmbPackTest_1.0.0.zip

Where as I think the proper use should be

// Make current
umbpack push -k [APIKEY] -c .\UmbPackTest_1.0.0.zip

// Don't make  current
umbpack push -k [APIKEY] .\UmbPackTest_1.0.0.zip

I understand the reason for this is so that the default behavior is to make the package the current one, and to change it now would be a breaking change, but I think it would be better to be consistent with command line etiquette and have to be a bit more explicit about making current, than using a custom syntax in order to support an inverted boolean option.

It seems the contributors and the child contributor element is not supported from this tool.

It was added some time ago in the Umbraco packager in backoffice using textstring array property editor. umbraco/Umbraco-CMS#4385

Currently when generating the package.xml file + zip files, all files are flattened into a root directory, except where file names would overlap then the file is prefixed with a guid.

The issue with this is it's hard to debug a package file and it's contents as you can't tell where a file goes without looking through the package.xml file so it would be great if we could maintain the relative folder structure in the zip file. Given the use of ZipFile.CreateFromDirectory which the docs say this maintains folder structures, this should allow us to achieve this.

The only thing I'm not sure of is whether the installer will handle folders ok, but given Umbraco generated zips include a folder, I don't think it should be an issue.

Firstly this is great my package now releases all through github actions https://github.com/Matthew-Wise/umbraco-forms-rte/

init
With some of the options it is not clear what purpose they serve. So it would be nice if the was a short explanation.

push
Would be great if the -w param was like nuget so we can add a range.

Thanks!
Matt

With rising interest in UmbPack we are of course interested in having more people contribute. Contribution guidelines that give a quick overview of the codebase and how you can test things would be nice to make it easier for newcomers to dive in!

Should cover things like:

• Building the tool locally and testing the init and pack commands from the source
• Testing the push command with a local version of Our
• Overview of the Our apis UmbPack pushes to and what is possible there

@mattbrailsford anything you had trouble figuring out when you made PR's in the past you'd have loved to be documented somewhere? 🙂
@nathanwoulfe, likewise if you hit a wall with something related to contributing to UmbPack please add it here so we can get it documented!

Hello Umbraco Package Team! 👋

I've been using UmbPack to push updates for my Contentment package for a while now, but I'm getting a niggly feeling that something isn't quite right. I can't put my finger on it, and feels to be so many moving parts and code that I can't figure out if there is an issue with UmbPack or Our Umbraco code.

What's the problem? OK, here goes...

On Our, if I list all "Backoffice Extensions" category. I can see my package Contentment is listed. 🎉
https://our.umbraco.com/packages/?category=Backoffice%20extensions

If I change the version filter to "Version 8". It isn't listed. 🤔
https://our.umbraco.com/packages/?category=Backoffice%20extensions&version=8

Side note, I also notice that Doc Type Grid Editor is missing! That weird! 🤔

I check the other major versions and none of them list my Contentment package. Something odd there. I'm checking all the settings I have available on my project page - I must be doing something wrong, right? What haven't I ticked? 🤔

Then I think, well I am using UmbPack to upload my updated package files. Maybe DTGE is using UmbPack too - unsure. Who else uses it, I think The Matt Brailsford does for Vendr... okay, let's take a look.

Vendr is listed under "Website Utilities" category. "Hey Vendr!" 😆 🤦‍♂️ It's listed, ace!
https://our.umbraco.com/packages/?category=Website%20utilities

Change the version filter to "Version 8", and....

...it's not there! Also "MegaNavV8" is missing too... well that's gotta be for v8 right?! Does it use UmbPack? Unsure. 🤔

I had a quick look over the Our Umbraco source-code... but honestly, I got totally lost.

So raising it here, hopefully not to gather dust, but to someone to stumble across at some point and think "you know what, that is a bit weird" and I've not gone completely mad. 😝

If I attempt to push a package where the file name already exists, umbpack throws an exception and my build breaks. In most cases that's probably the preferred behaviour, because it indicates that something isn't right in the build. However, I like to be difficult 😃

Example for my use case is the newly improved Nesting Contently build, where the build generates two packages - one being the NC datatype, and a second containing the optional property value converters. If I tag release 1.1.0 that has changes to the datatype but not the PVC package, the resulting build generates NestingContently.Umbraco_1.1.0.zip and NestingContently.Umbraco.ValueConverters_1.0.0.zip. Both are then pushed, but the latter fails because it is unchanged from the previous release, which throws an exception and breaks the build.

That's currently not a huge issues because the push task is the final step in the build, so it still completes but the build shows as failed. Ideally pushing a duplicate file name would return a warning, allowing the build to continue.

We have a few help commands that are incomplete, need to run through them all and decide what goes in.

Running umbracopackage --help definitely needs to inform on the 3 verbs we offer!
But the help text for each verb also seem a bit unfinished 🙂

Good overview on them here: #5

If I mess up with my file references in the package.xml file (as I just did) it all blows up with a file not found exception. That sort of works but its not a friendly error.

I'd rather it checked that it can find all the files it needs and list out any it can't in one list. That way I don't have to keep fixing and rechecking it for each file in turn (ok so I got a few wrong in a row, bite me).

Little bits like this make the developer experience much nicer/smoother

I pushed a package to Our. The command successfully extracted the version as 1.4 from the package.xml:

Is it correct that on Our my package is still showing as version 1.0?

This push was setting this file as the current version (the default behaviour of 'push'), so I was kinda expecting the current version to be updated to match the version in package.xml. Should that have happened?

I would love to see the packaging logic separated out into a separate assembly / package - e.g. Umbraco.Packaging.

This would offer a number of benefits:

• The ability for 3rd party libraries to create packages in code (e.g. my MSBuild task, Umbraco)
• One central place for managing all packaging logic

The CMS and UmbPack could then depend on the same Umbraco.Packaging package, while the CLI would retain it's package upload abilities (I don't think these would belong in the CMS).

Currently the only way to do this would be by shipping the EXE, which could be flagged by antivirus etc...

Happy to help with this if it's something the Package Team thinks could work.

Does the default value for the WorksWith option need to be manually updated when a new Umbraco minor version is released?

https://github.com/umbraco/UmbPack/blob/dev/src/Verbs/PushCommand.cs#L41

At the time of writing, v8.6.4 is the current version, and default value for WorksWith is "v850".

https://our.umbraco.com/documentation/Extending/Packages/UmbPack/ doesn't make any reference to the archiving flag, and I assume it should.

Here's a comment from Matt Brailsford explaining some issues he had getting umbPack working earlier today:

"... the path to the package should come first. Archive doesn’t have to be last but it needs to be followed by another flag so that the parser know when you are done and changing context."

So think we need to double check how things needs to be ordered, and then update the tool help text and the Our documentation accordingly.

A few things I noticed when testing the tool on Our while working on umbraco/OurUmbraco#511

1:

We set a baseUrl in a few different spots, would be nice to set it once in a config file somewhere to make it a bit easier to test locally.

• https://github.com/umbraco/Umbraco-Packager-CLI/blob/master/src/Program.cs#L113
• https://github.com/umbraco/Umbraco-Packager-CLI/blob/master/src/Verify.cs#L65

2:

We have a few things that Our needs to be set, and we set them to some defaults but may want to allow some of them to be input:

https://github.com/umbraco/Umbraco-Packager-CLI/blob/master/src/Program.cs#L128-L132

var content = new StreamContent(fileInfo.OpenRead());
content.Headers.ContentDisposition = new ContentDispositionHeaderValue("form-data")
{
    Name = "file",
    FileName = fileInfo.Name
};
form.Add(content);
form.Add(new StringContent("true"), "isCurrent");
form.Add(new StringContent("4.7.2"), "dotNetVersion");
form.Add(new StringContent("package"), "fileType");
form.Add(new StringContent("[{Version: 'v820'}]"), "umbracoVersions");

Making the isCurrent setting optional would fix #2

The dotNetVersion option doesn't matter much, as we are considering getting rid of that entirely on Our, but the versions would make a lot of sense to make something people can input.

On the Our site we expect it in this format:
[{Version: 'v820, v830, v840'}]

Note: If any of the versions here are "invalid" it will leave the versions completely blank.. Maybe something we should validate and send errors about in the Our Api? 🤔

I don't know if you already considered these for your changes @KevinJump ? 🙂

Twitter thread from Matt Brailsford below describing both scenario and how it could work:

Source: https://twitter.com/mattbrailsford/status/1263011754075979776

Niice! This should make deploying Umbraco packages so much easier.
When a package is published, is the current package file on our archived? And what about if the package isn’t the “current” pkg ? Eg Vendr has a main pkg + payment provider pkgs listed.

The core pkg is always “current”. If I published a payment provider pkg would the existing one be archived? If not, maybe an -a flag can be passed with a pattern to match and archive any matches?

Currently, when using the pack command it replaces .'s in a package name with an _. This is messing with my historic file naming strategy of using periods as separators. This also goes for the _ added between the file name and the version number.

Clearly it doesn't have a huge problem with periods in the file name as you still use them in the version number part of the file name.

Ultimately I think we should give more control over the generated filename so people can choose how they want it formatted.

Maybe we can just allow the passing in of a file name flag to completely override the generated name with an explicit value. I think this would be the simplest approach.