Comments (12)
From discussion today, it seems there are multiple parts here:
- ok to use
--all
withget
sub-command fordsc.exe
- reject any input JSON when
--all
is used, we can always support filtering in the future if there's a valid scenario - for resources, it makes sense to have resources return JSONLINES instead of a single JSON object containing an array so that the instances don't need to be cached in memory to emit the entire object
- dsc.exe would collect the results and emit a valid configuration doc. We have two options:
resources:
name: IIS
properties:
- website: A
foo: bar
- website: B
foo: a
- website: C
foo: C
resources:
type: IIS
properties:
website: A
foo: bar
type: IIS
properties:
website: B
foo: a
type: IIS
properties:
website: C
foo: C
We agreed to go with the 2nd option pending customer feedback.
from dsc.
Thinking about this further, I think it may be better to start with an export
operation rather than an --all
switch to get
as it may make it more clear to users that this is distinct from get
since the semantics are very different and export
would make more sense to use as declaring support in a resource manifest.
from dsc.
The specific requirement for dsc resource export
is limited in scope to known scenarios that support it. Two partners would be WinGet and M365Dsc. For example, you have written a JSON configuration that is intended to get the configuration details for 10 known applications (WinGet must already have an install/configure script).
@michaeltlombardi, good description of get all
. I am pinging a few stakeholders that used it in the past for inventory.
from dsc.
@michaeltlombardi just like get/set/test, the export
operation can be done at the config level or resource level. With dsc config export
, you would get a valid config that can immediately be applied elsewhere. With dsc resource export
, you would get JSONLINES.
from dsc.
Is the idea to have return type of the export be an array of hashtables from the Get or a string representation of the resulting current configuration in JSON/YAML?
from dsc.
Intent is that the output could be turned around and reused for get/set/test so needs to be a valid resource config. This means if it's an array of values, then the resource needs to have it's config as an array already.
Thinking about this further, it may also make sense to support the case where export
would return multiple instances and dsc.exe
would wrap each one as a config for the resource. It'll be up to the resource to decide which model works best.
from dsc.
Modeling this as a Restful API would make enumerating all instances of a given resource feel natural
from dsc.
Thinking about this again today and considering the following use cases:
- I want to retrieve the current state for all instances of a single resource (what IIS sites exist on this machine?) for inspection.
- I want to retrieve the current state for all instances of a list of resources (What IIS sites and application pools exist on this machine, which features are installed) for inspection.
- I want to get the current state for all instances in a group of resources so I can use the output as a starting point for further configuration.
Case 1
For this case, I think that to maintain backward compatibility with existing resources, it makes the most sense to assume that a resource can only return a single instance from a get
call.
If the resource's manifest has a key that opts into a return-all functionality, like _enumerable
or _supportsGetAll
, then the caller knows they can use the get method on the resource without any additional input to retrieve state for all instances of that resource. Alternatively, with the --all
parameter on the subcommand.
For example:
# Get a single instance of an IIS site
dsc resource get --name Site --module Microsoft.IIS --properties "name=foo"
# Get all IIS sites (no parameters)
dsc resource get --name Site --module Microsoft.IIS
# Get all IIS sites (indicating flag)
dsc resource get --name Site --module Microsoft.IIS --all
Case 2
For retrieving all instances for multiple resources, I think it would be reasonable for the caller to just loop calls to dsc
.
foreach ($Resource in @('Site', 'ApplicationPool', 'Feature')) {
dsc resource get --name $Resource --module Microsoft.IIS --all
}
Case 3
To enumerate the current state of an arbitrary set of resources on the target and get the return output as a configuration document, I think you're correct that the dsc config
command needs an export
subcommand. I don't think that the DSC Resource API surface semantics should be extended for an export
method. If the resource has marked itself as supporting the retrieval of all instances of itself on a target, dsc
knows whether it can export that resource. Similarly, that information can be advertised when authoring a configuration file directly.
I wonder if another approach, re-using the existing grouping model, is to have a built-in ExportGroup
. This group's validation could fail any entry in it that doesn't support full enumeration.
An example of a configuration for this purpose might be:
# ./exportable.config.yaml
resources:
- name: my resources
type: DSC/ExportGroup
properties:
resources:
- name: Websites
type: Microsoft.IIS/Site
- name: App Pools
type: Microsoft.IIS/ApplicationPools
- name: IIS Features
type: Microsoft.IIS/Feature
dependsOn:
- '[DSC/AssertionGroup]IsWindows'
- name: IsWindows
type: DSC/AssertionGroup
properties:
resources:
- name: os
type: Microsoft/OSInfo
properties:
family: Windows
- name: Example Variable
type: Microsoft.Windows/EnvironmentVariable
properties:
name: EXAMPLE
value: An arbitrary string
_ensure: present
And a user would call it with:
# Output to terminal
dsc config export --file ./exportable.config.yaml
# Capture in new config file
dsc config export --file ./exportable.config.yaml > webfarm.config.yaml
When you call dsc config export
, any resources not included in a DSC/ExportGroup
get ignored but appended to the output config. I can also see wanting to apply the assertions before export.
Aside
Another reason to support the "enumerate all instances of this resource" key in a resource manifest is to enable the caller to decide whether it wants to enumerate all of the instances of a resource at once or one at a time. Some resources might functionally have to enumerate the full list every time and filter it to only return the specified instance.
If a configuration includes 2-3 of those resources, the performance impact might not be noticeable. If the configuration includes 20 of those resources, knowing whether it can intelligently decide to pre-cache the instances would be useful. That also enables improved integration with higher-order tools that also have a model for enumerating the full list of instances for a resource.
It also makes DSC Resources more useful as investigative tools on a system.
Conclusion
I think adding a new config key for a resource to advertise whether it supports enumerating every instance of itself, an ExportGroup
resource, and an export
subcommand to dsc config
would be a relatively neat solution to this problem without increasing the complexity or cognitive load for users and resource authors too much. Resource authors just need to indicate if their resource supports the operation, config authors need to know about the subcommand and resource group.
For a user who wants to enumerate the list of instances without worrying about the return information being a usable configuration document, calling dsc resource get [--all]
without defining properties seems like a coherent UX that is similar to behavior in PowerShell, only filtering the list of resources if provided additional information or failing if the query requires mandatory information.
from dsc.
I think dsc resource|config export
is a very different operation from dsc resource get --all
- they're related, insofar as only resources that can support --all
can also be used for export
- but definitely different use cases.
I'm not sure I see how we can support export
without supporting --all
- but --all
doesn't require anything (I think?) but an entry in the resource manifest indicating what argument to pass to the resource to return all instances and wiring dsc resource get
to handle that flag / return an array of instances when the flag is used.
Edit for clarification:
By this I mean that there's two operations a caller might want to perform on a resource:
- I want to retrieve all instances of some resource
R
- to cache or to view, whatever. I want just the array of instances as return output. - I want to export the current state of some resource
R
- to use for baseline or reporting, etc. I want the DSC configuration document representing those resources. I might even want to export multiple resources at once.
This implies to me that the two operations are dsc resource get --all
and config | dsc resource export
. The export call likely uses the same code path as resource get --all
, but they're distinct operations.
from dsc.
We're looking at the same thing over in WinGet for:
And over at Dev Home for:
from dsc.
@michaeltlombardi just like get/set/test, the
export
operation can be done at the config level or resource level. Withdsc config export
, you would get a valid config that can immediately be applied elsewhere. Withdsc resource export
, you would get JSONLINES.
To clarify, I don't mean that there's no value in export
at the resource level, just that it's a distinct use case from get --all
, even though the underlying mechanism for retrieving the full set of instances for a resource is the same (call the command with the correct flags).
Another thought, there's no way for extant PowerShell DSC Resources to participate in this - would that require an overload for Get
or a new method?
from dsc.
Another thought, there's no way for extant PowerShell DSC Resources to participate in this - would that require an overload for
Get
or a new method?
That would require:
- updating PSDesiredStateConfiguration module to, for example, when 'Get' operation is invoked without any filters - treat it as 'get-all' and pass it down to the resource;
- correspondingly update the classic PS resource to accept the whatever mechanism is used in 1st point.
from dsc.
Related Issues (20)
- Unable to reuse `reference()` in set operation from get and test
- The `int()` function incorrectly handles non-integer numbers HOT 4
- Update `dsc --version` to include build info
- Resources in parallel unless dependson HOT 4
- Create JSON/YAML for configs that only use /*powershell adapter, from PowerShell script HOT 1
- Installing resources HOT 1
- localization support
- Include metadata in result objects HOT 2
- Some static-analysis support HOT 1
- Intent for this implementation - will it only be an Intermediate language? HOT 3
- Extend exitCodes to support Hexidecimal error codes HOT 1
- Add tests for canonical schemas
- `IncludeGroup` to include an external configuration HOT 5
- Enable `dsc config resolve` for included configs HOT 2
- Install PowerShell7 resource HOT 6
- Long running mode for DSC HOT 2
- Resource to manage Windows env vars HOT 1
- Define semantic exit codes for the PowerShell adapters HOT 2
- Automatically handle messages from PSDSC resources
- Use trace messaging to indicate composite resources are being ignored HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dsc.