oras-project / artifacts-spec Goto Github PK

Proposing we release v1.0.0-draft.2 for doc and security updates.

Capture how artifact authors can provide localized strings and logos for how their artifact can be visualized in registry specific tooling.
This surfaces the details captured here: https://github.com/opencontainers/artifacts/blob/f9710ad60b6de311843df88d8f8fa1e6d0d44b50/authoring-artifacts.md#optional-publishing-the-artifact-type

This is a dupe of OCI Artifacts/Add localized strings and icons for artifact types #45

We can either create the schema and spec here and submit as a proven PR to OCI Artifacts, or track under OCI Artifacts. Either way, we'll need this for ORAS Artifacts for authors to identify their types.

In filtering examples, you reference the artifact type application/vnd.org.cncf.notary.v2. I think you mean to use application/vnd.cncf.notary.v2 (no org.), since this makes it seem like it's a reverse-domain-name scheme referring to cncf.org which is unaffiliated with the CNCF (https://cncf.io).

This is moot if filtering is pinned for later (#68), or if specific example media types are removed (#66), but in case those don't come to pass, we should fix the typo.

Shouldn't https://github.com/oras-project/artifacts-spec/blob/1.0.0-draft.1/examples/net-monitor-oci-image.json have the artifactType field?

Currently the containerd project is used only for the referrers api and does not need to be present in the notary-project since nv2 doesn't depend on containerd.
We need to move notaryproject/containerd to oras-project/containerd.

@SteveLasker can you please transfer the repository over.

Since io.cncf.oras.artifact.created is a pre-defined annotation key and will be used for Referrers sorting, we should provide this value in specs-go/v1, just like we do for media types.

The mediaType field is present in the artifact manifest spec, presumably copied over from the OCI image manifest or image index specs. According to those specs, it's a field that is kept around for backwards compatibility / reserved for use. I'm not sure that reasoning applies here, so I wonder if we should either:

• remove it.
• better explain its purpose in the new artifact manifest.

ref: https://github.com/oras-project/artifacts-spec/blob/main/artifact-manifest.md#oras-artifact-manifest-properties

As registries capabilities evolve, it will take time for all registries to enable the capabilities. Having a list of capabilities isn't limited to Notary or Artifacts. This issue tracks the design of an API for discovering the capabilities of a specific registry implementation, or even the capabilities of a specific repo within the registry. The response should provide the product/project/instance of the registry and a version, enabling an artifact client to cache the capabilities for subsequent interactions with the registry.

Existing Patterns

Discovering capabilities is a well-known pattern, such as smart device phones that implement capabilities based on the cost/model of the hardware or version of the software. Providing a list of capabilities enables applications to be written and released as one application (or client), that may check the capability of the device for what the application may provide. A device may have location services, wifi, bluetooth, a light, headset, nfc, multiple windows. The application checks the capability of the device, possibly prompting the user for authorization to use that capability, and proceeds based on that capabilities enablement or authorization of the user.

This same pattern can be used for registries. It's proposed to land this capability check with the artifact.manifest to set the stage for future capability checks.

Examples

1. What manifests are supported by the registry? (docker.manifest, docker index, oci manifest, oci index, artifact.manifest, ...)
2. _catalog api
3. Max size of each manifest type
4. Max size of layers
5. Max number of layers/blobs, possibly per manifest
6. Max size of annotations
7. Does the referrers/ API support filtering based on artifactType and/or annotations
8. Does the registry/repo support searching
9. Does the registry support eventing?
10. Does the registry/repo support metadata

Design Discussion

A name/value api can be provided, enabling a list of capabilities the registry can respond with, allowing registry clients with the ability for how they may proceed.

There should be a set of standard registry properties, simply defined in the root, with a set of registry specific capabilities defined with an annotation style prefix.

/oras/v1/capabilities?capabilityName
For example:
/oras/v1/capabilities?supportedManifests would respond with:

{
  "registry": "com.azure.registry",
  "version": "6.4.202107062010",
  "lastUpdate": "2021-08-10T20:18-06:00",
  "manifestSupport": {
    "application/vnd.docker.distribution.manifest.v1+json": {
      "maxLayerSize": "200000",
      "maxManifestSize": "4000"
    },
    "application/vnd.docker.distribution.manifest.v2+json": {
      "maxLayerSize": "200000",
      "maxManifestSize": "4000"
    },
    "application/vnd.docker.distribution.manifest.list.v2+json": {
      "maxLayerSize": "200000",
      "maxManifestSize": "4000"
    },
    "application/vnd.oci.image.manifest.v1+json": {
      "maxLayerSize": "200000",
      "maxManifestSize": "4000"
    },
    "application/vnd.oci.image.index.v1+json": {
      "maxLayerSize": "200000",
      "maxManifestSize": "4000"
    },
    "application/vnd.cncf.oras.artifact.manifest.v1": {
      "maxLayerSize": "200000",
      "maxManifestSize": "4000"
    }
  },
  "repositories":
    {
      "pathMaxLength": "256",
      "depth": 99999
    },
  "_catalog": "true",
  "com.azure.registry.quarantine": "enabled",
  "com.azure.registry.cmk": "enabled",
  "com.azure.registry.geo-replication.regions": [
    "eastus",
    "westus"
  ]
}

The capabilities MAY be implemented at the repository level: /oras/v1/{repository}/capabilities?capabilityName

Now or Later

For the immediate timeframe, the ORAS libraries will likely just query if the referrers/ API exists to proxy the support of the artifacts.manifest. If we can land a solid design of this API in the next several weeks, we can ship the ORAS libraries to be more robust.

Tagging OCI Distribution Spec extension proposal for possible alignment.

As golang documentation states

A version identifies an immutable snapshot of a module, which may be either a release or a pre-release. Each version starts with the letter v, followed by a semantic version. See Semantic Versioning 2.0.0 for details on how versions are formatted, interpreted, and compared.

In short, golang requires tags starting with the letter v. Therefore, it is suggested to have v1.0.0-draft.1 instead of 1.0.0-draft.1. Otherwise, 1.0.0-draft.1 will be resolved to a pseudo-version v0.0.0-20210914235636-eecc5d95bcee.

The artifacts-spec defines the io.cncf.oras.artifact.created annotation for date/time sorting of artifacts, based on a created date.

io.cncf.oras.artifact.created date and time on which the artifact was created (string, date-time as defined by RFC 3339)

The annotation is currently is defined with RFC 3339

This issue re-opens the discussion if there are valid reasons to switch to another format, such as:

• ISO 8601

The created date will be used by the referrers API to sort (asc or desc) references. For instance, a list of scan results could be sorted to get the first (original), or last (current).
A list of annotations may be added, where the same value is set to a new value, over time.
A user may want the original or latest value of the annotation.

• Are there specific indexing services that make one format better over another?
• Are there other scenarios where another datetime format was chosen, and we want to avoid conversion?

An OCI image manifest is not a valid artifact manifest. But it's listed in Example ORAS Artifact Manifests (the first item).

@SteveLasker @shizhMSFT Should we reference this file instead?

The link How does ORAS Artifacts relate to OCI Artifacts? is broken.

I didn't find a possible matching paragraph to be linked to.

Was the pagination mechanism described in the current spec intentionally different than the one that already exists for tags?

I'd prefer to not introduce a new way of paginating, and I wasn't aware of tag pagination until now. Thoughts?

If people are in agreement, I'll make a PR.

The PR describing the OCI Artifact Manifest spec has been merged. The schema described in the PR seems to be the same as the ORAS Artifact spec. How will the two be reconciled? Will the ORAS Artifact spec be dropped in favor of the standard? Will the oras cli be implemented with the new OCI Artifact spec?

We've been using the logical namespace of org.cncf.oras.* but the image spec calls out a suggestion to use internet reverse namespace:

Keys MUST be unique within this map, and best practice is to namespace the keys.
Keys SHOULD be named using a reverse domain notation - e.g. com.example.myKey.

We could use land.oras to reflect https://oras.land/
Or, we could ask CNCF to create a oras.cncf.io domain.

The thing I like about the cncf.io sub-domain is it would align with a DiD based identity provider.

Motivation

Right now it appears that in many use cases we consider all the artifacts referencing an image to be indivisible from the image. What this means is in scenarios where the image needs to be moved from system A to system B, all of the artifacts associated with the image need to be moved with the image.

During that move operation, discovering all the artifacts associated with the image can be a costly operation that involves many recursive calls to the referrer API. Consider the following example:

Problem

If image net-monitor:v1 has daily scans, and each scan is signed, there would be the following image structure:

net-monitor:v1
   -> scan-result-1
       --> scan-result-1-signature
   -> scan-result-2
       --> scan-result-2-signature
...
   -> scan-result-365
       --> scan-result-365-signature

To move net-monitor:v1 and all its associated artifacts, the following network calls would be needed

GET registry/v2/.../net-monitor:v1

GET oras/.../net-monitor:v1/referrers
GET registry/v2/.../scan-result-1
GET oras/.../scan-result-1/referrers
GET registry/v2/.../scan-result-1-signature
GET oras/.../scan-result-1-signature/referrers      // To make sure it has no children

GET registry/v2/.../scan-result-2
GET oras/.../scan-result-2/referrers
GET registry/v2/.../scan-result-2-signature
GET oras/.../scan-result-2-signature/referrers      

...

GET registry/v2/.../scan-result-365
GET oras/.../scan-result-365/referrers
GET registry/v2/.../scan-result-365-signature
GET oras/.../scan-result-365-signature/referrers 

A total of 731 referrers calls is needed just to move this one image. This is extremely computationally expensive and may cause livelocks or be exploited for DDoS attacks.

Proposed solution

The referrers API would accept a recursive=true query parameter.
When this is true, it would return all the artifacts transitively referencing an image in a flat list.

GET oras/.../net-monitor:v1/referrers&recursive=true

Result:
{
  "references": [
    {
      "digest": {scan-result-1.digest},
      "mediaType": "application/vnd.cncf.oras.artifact.manifest.v1+json",
      "artifactType": "oras.scan.result",
      "size": 312
    },
    {
      "digest": {scan-result-1-signature.digest},
      "mediaType": "application/vnd.cncf.oras.artifact.manifest.v1+json",
      "artifactType": "oras.signature",
      "size": 312
    },
...
   {
      "digest": {scan-result-365.digest},
      "mediaType": "application/vnd.cncf.oras.artifact.manifest.v1+json",
      "artifactType": "oras.scan.result",
      "size": 312
    },
    {
      "digest": {scan-result-365-signature.digest},
      "mediaType": "application/vnd.cncf.oras.artifact.manifest.v1+json",
      "artifactType": "oras.signature",
      "size": 312
    },
  ]
}

This would allow all 731 calls to this API to be shortened to just 1 call.

Please LGTM if folks are in favor or archiving the repository REJECT with reasoning.

• @SteveLasker
• @sajayantony
• @justincormack
• @michaelb990

Update status as per outcome of this issue - #118

Proposing we release artifacts-spec-1.0.0-DRAFT1 as we have completed the milestone. Release process is defined here.

High level deliverables as as follows:

• Decide on main commit for release - eecc5d9
• Cut branch named 1.0.0-draft.1 based on agreed upon main commit
• Cut a tag named 1.0.0-draft.1
• Update README.md

/cc @SteveLasker @michaelb990 @justincormack @sajayantony

Can the maintainers please respond with a +1 comment once you are in favor of cutting this release.

Does the spec for the referrers API currently allow for cross-repository references? This has been a user requirement for a few organizations that want to put signatures in a separate repository from the images themselves.

This is mostly done for security and permissions: developers that can sign images may not need write access to the repository the images are stored in. Registries could solve this with fine-grained permissions, but many only implement permissions at the repository-level.

https://github.com/oras-project/artifacts-spec/blob/v1.0.0-draft.1/manifest-referrers-api.md#paging-results

If there are more items, the URL for the next collection is encoded in a RFC5988 Link header, as a "next" relation. Clients SHOULD treat this as an opaque value and not try to construct it themselves.

In current v1.0.0-draft.1 spec, the next link in pagination result is an absolute url that the client will treat as an opaque value. The challenge of the requirement is the absolute url will not work if the registry is running behind a gateway/proxy. The following flow illustrates the problem. In this case, client is only able to connect to myregistry.gateway.io.

client => myregistry.gateway.io(gateway) => myregistry.azurecr.io (registry)

If the next link returned by the registry is an absolute path, eg, http://myregistry.azurecr.io/oras/artifacts/v1/hello-world/manifests/sha256:3c3a4604a545cdc127456d94e421cd355bca5b528f4a9c1905b15da2eb4a4c6b/referrers?n=5&nextToken=abc, the client won't be able to follow the link to request the next page items.

We propose to loose the constraint to allow relative path in the paging result. For the above example, the next link will be /oras/artifacts/v1/hello-world/manifests/sha256:3c3a4604a545cdc127456d94e421cd355bca5b528f4a9c1905b15da2eb4a4c6b/referrers?n=5&nextToken=abc.
The client will parse the url and compose the full url with the appropriate host: https://myregistry.gateway.io/oras/artifacts/v1/hello-world/manifests/sha256:3c3a4604a545cdc127456d94e421cd355bca5b528f4a9c1905b15da2eb4a4c6b/referrers?n=5&nextToken=abc

(First off: Feel free to ask me to open separate issues for these, I wasn't sure whether spamming issues would be less disruptive than opening one laundry list issue.)

(Also: Apologies in advance if these issues have already been discussed and resolved, if there are links to discussions where these have already been hashed out I'm happy to go read those instead)

I've read through the referrers spec proposal (this specific version, for future readers), and have some feedback I thought I'd share, with the goal of clarifying/generalizing the spec, and toward making it easier to propose and accept into distribution-spec when the time comes.

Example media types

I'd suggest not explicitly referring to any specific real-world artifact type, e.g., Notary v2 signatures. When references are successful and prevalent we expect a lot of referencing artifact types to exist, and we don't want to make it seem like the spec is preferring or omitting one or another.

Instead, you can use some clearly-example-text artifact type (e.g., "application/vnd.com.example.signature") as a clearer placeholder value.

Similarly, I'd omit the aside "such as a Notary v2 client", with the same reasoning.

In a similar vein, the artifacts spec also lists example media types at https://github.com/opencontainers/artifacts/blob/master/artifact-authors.md#example-layer-types

I'm not sure the spec repo is a good place to list example media types. The list will by definition be incomplete, and having some included while others aren't, even ordering them non-alphabetically, can make it seem like the spec is preferring one type of artifact over another.

There have been examples in the OCI specs where an example media type can be confused to refer to a real expected value, confusing readers and implementors. We should take steps to avoid this confusion in new specs.

Paging

The proposal for pagination is at https://github.com/oras-project/artifacts-spec/blob/main/manifest-referrers-api.md#paging-results

It specifies consulting the Link rel="next" response header, which should be consider opaque (RFC5988 FTW).

distribution-spec's pagination is described at https://github.com/opencontainers/distribution-spec/blob/main/spec.md#content-discovery

distribution-spec specifies the parameters &n= and &last=<tagname> specifically, and except for specifying that implementations MUST NOT include a Link header when there are no more items, makes no mention of the Link rel="next" header or RFC5988 at all (though some registry implementations might use it; I don't know)

While I agree that leveraging RFC5988 is definitely better, I think consistency with the rest of distribution-spec is paramount. We don't want registry clients or implementations to have to implement two pagination flows, even if the new one is obviously better.

The proposal could instead suggest ("SHOULD") that registries fully leverage RFC5988, for both referrers and traditional tag listing. This SHOULD amendment could be proposed against distribution-spec today, separate from this proposal, probably contingent on a survey of registry implementations in the wild to see how they stack up.

If distribution-spec moves to suggest/SHOULD RFC5988 then perhaps the referrers API could require/MUST it, since it's a new API, but I wouldn't want to leave room for registries to have to support two pagination schemes, or for clients to have to negotiate or navigate them.

Filtering

distribution-spec doesn't currently have any concept of filtering list results.

While I definitely agree that filtering is useful, especially since we expect potentially many artifacts referring to an image, and since we'd prefer to have one interface of that clearly specified for compatibility, I'd suggest it might be easier to put a pin filtering for now to remove one more area of conversation and potential disagreement, and come back to address it after the bulk of the proposal has been ironed out and accepted into distribution-spec.

This should also lead to less churn and work for you if there are other changes to artifact types suggested, since you won't have to update the filtering specification to match those changes.

Slight typo: vnd.org.cncf....

In filtering examples, you reference the artifact type application/vnd.org.cncf.notary.v2. I think you mean to use application/vnd.cncf.notary.v2 (no org.), since this makes it seem like it's a reverse-domain-name scheme referring to cncf.org which is unaffiliated with the CNCF (https://cncf.io).

This is moot if filtering is pinned for later, or if specific example media types are removed, but in case those don't come to pass, we should fix the typo.

From the artifactmanifest.md doc I had some questions/clarification recommendations

The doc says phase one reference types.. but does not define reference types.. rather at the bottom of the doc I see that "A reference type is an artifact that doesn't have a lifecycle unto itself." Given that the subjectManifest is "optional" that does not jibe with this statement. I'd put the "is a" definition up top.. then later on here you can say something like when subjectManifest is present..

I think you are saying new base artifact focused media types like a new manifest.

if so maybe update the doc to add that specific clarity.

In the sentence an artifact manifest provides an optional collection of blobs and a reference to the manifest of another artifact. I think you mean something like "an artifact manifest provides an optional collection references to objects (called blobs) that have information/content associated with this manifest/object; and also an optional reference (called subjectManifest) that points to a manifest for which this artifact (manifest) is dependent."

In the following paragraph there are some issues.

Tagged referenceTypes

As signatures and SBoMs are not considered independent artifact types, they SHOULD NOT have a tag, simplifying the lifecycle management. As the subjectManifest is marked for deletion (refCount=0), the referenctType is also marked for deletion (refCount -1). However, these artifacts MAY have tags as future versions of the artifact manifest MAY support independent types.

typo: referenctType

I think you mean something like both the object referred to by the subjectManifest and all artifact manifest objects referring to the subject manifest object MUST have the same lifecycle (be marked for deletion along with the subject manifest object..

Looking over at the artifact descriptor definition

artifactType string

This OPTIONAL property defines the type or Artifact, differentiating artifacts that use the application/vnd.oras.manifest. When the descriptor is used for blobs, this property MUST be empty.

Suggest "This OPTIONAL property defines the (media) type of this artifact descriptor."

I don't understand the context switch to the following text "differentiating artifacts that use the application/vnd.oras.manifest. When the descriptor is used for blobs, this property MUST be empty."

In the example at the bottom:

{
  "mediaType": "application/vnd.cncf.oras.artifact.manifest.v1+json",
  "digest": "sha256:5b0bcabd1ed22e9fb1310cf6c2dec7cdef19f0ad69efa1f392e94a4333501270",
  "size": 7682,
  "artifactType": "org.cncf.notary.v2"
}

the text "cncf.notary.v2 signature" and ""org.cncf.notary.v2" don't seem to jibe.. or are confusing

Define error codes & behavior for the referrers API.

Most important questions in my mind:

• Should the /referrers API return results when the digest doesn't exist in the repository? (This seems like a possible scenario if an image is deleted but the reference artifacts aren't)
• Do we need any new error codes? Current list of codes are: here

Anything I'm missing?

Proposing we release artifacts-spec-1.0.0-rc.1
Account for issues and PRs tagged RC1

High-level deliverables as follows:

• Decide on main commit for release - 9ff738f
• Cut branch named v1.0.0-rc.1 based on agreed upon main commit
• Cut a tag named v1.0.0-rc.1
• Update README.md

/cc @SteveLasker, @michaelb990, @justincormack, @sajayantony

Can the maintainers please respond with a +1 comment once you are in favor of cutting this release.

(Apologies in advance if this has already been discussed and resolved, if there are links to discussions where these have already been hashed out I'm happy to go read those instead)

I've read through the referrers spec proposal (this specific version, for future readers), and have some feedback I thought I'd share, with the goal of clarifying/generalizing the spec, and toward making it easier to propose and accept into distribution-spec when the time comes.

I'd suggest not explicitly referring to any specific real-world artifact type, e.g., Notary v2 signatures. When references are successful and prevalent we expect a lot of referencing artifact types to exist, and we don't want to make it seem like the spec is preferring or omitting one or another.

Instead, you can use some clearly-example-text artifact type (e.g., "application/vnd.com.example.signature") as a clearer placeholder value.

Similarly, I'd omit the aside "such as a Notary v2 client", with the same reasoning.

In a similar vein, the artifacts spec also lists example media types at https://github.com/opencontainers/artifacts/blob/master/artifact-authors.md#example-layer-types

I'm not sure the spec repo is a good place to list example media types. The list will by definition be incomplete, and having some included while others aren't, even ordering them non-alphabetically, can make it seem like the spec is preferring one type of artifact over another.

There have been examples in the OCI specs where an example media type can be confused to refer to a real expected value, confusing readers and implementors. We should take steps to avoid this confusion in new specs.

I think it is unnecessary because:

1. Manifest spec defines what the manifest is. Lifecycle management is about object storage, and seems out-of-place in a manifest spec. Current OCI manifest specs don’t define lifecycle management.
2. Differences in lifecycle management between registries does not affect cross-registry portability of artifacts. Therefore there is no immediate need for everyone to agree on one particular behavior.
3. Registries currently implement object cleanup based on their own use-cases (public vs private, free vs paid). Adding a requirement in the spec will force all registries to adopt the same behavior for reference artifacts, which may be inconsistent with their current object cleanup behavior.

I do agree that the spec should not prevent registries managing object lifecycles efficiently. Preventing reference cycles that could occur due to the reverse-ownership relation introduced by the subjectManifest field is important in this regard.

Therefore I propose that we remove the “Lifecycle Management” section from artifact-manifest.md, and instead document that preventing reference cycles was a goal when designing this spec.

On https://github.com/oras-project/artifacts-spec/blob/main/artifact-manifest.md there is a link which points towards https://github.com/oras-project/artifacts-spec/blob/main/annotations.md#rules which is currently a dead link.

(Apologies in advance if this has already been discussed and resolved, if there are links to discussions where these have already been hashed out I'm happy to go read those instead)

I've read through the referrers spec proposal (this specific version, for future readers), and have some feedback I thought I'd share, with the goal of clarifying/generalizing the spec, and toward making it easier to propose and accept into distribution-spec when the time comes.

The proposal for pagination is at https://github.com/oras-project/artifacts-spec/blob/main/manifest-referrers-api.md#paging-results

It specifies consulting the Link rel="next" response header, which should be consider opaque (RFC5988 FTW).

distribution-spec's pagination is described at https://github.com/opencontainers/distribution-spec/blob/main/spec.md#content-discovery

distribution-spec specifies the parameters &n= and &last=<tagname> specifically, and except for specifying that implementations MUST NOT include a Link header when there are no more items, makes no mention of the Link rel="next" header or RFC5988 at all (though some registry implementations might use it; I don't know)

While I agree that leveraging RFC5988 is definitely better, I think consistency with the rest of distribution-spec is paramount. We don't want registry clients or implementations to have to implement two pagination flows, even if the new one is obviously better.

The proposal could instead suggest ("SHOULD") that registries fully leverage RFC5988, for both referrers and traditional tag listing. This SHOULD amendment could be proposed against distribution-spec today, separate from this proposal, probably contingent on a survey of registry implementations in the wild to see how they stack up.

If distribution-spec moves to suggest/SHOULD RFC5988 then perhaps the referrers API could require/MUST it, since it's a new API, but I wouldn't want to leave room for registries to have to support two pagination schemes, or for clients to have to negotiate or navigate them.

The proposal here is to have a fork of distribution under the ORAS project at https://github.com/oras-project/distribution . This will enable the ORAS working group to iterate over the artifact spec and provide a reference implementation PR for distribution which enables APIs like the referrers API that is a part of this proposal.

Currently these changes are being presented under https://github.com/notaryproject/distribution/tree/reference-types and given that the API is meant to support more than signatures it makes sense to iterate and release these under this working group and stage these for upstream donation to the OCI and Distribution.

Proposing we release artifacts-spec-1.0.0-rc.2

Based on feedback and discussions with @michaelb990, I’ve updated the wording here:

rc.2 provides projects consuming the ORAS artifacts-spec an updated tag to develop against.

In the meantime, we continue to actively participate in the OCI Reference Type Working Group. We hope to be able to migrate to an OCI-supported solution before releasing 1.0.0.

High-level deliverables as follows:

• Decide on main commit for release - eddd1d8
• Cut branch named v1.0.0-rc.2 based on agreed upon main commit
• Cut a tag named v1.0.0-rc.2
• Cut a release named v1.0.0-rc.2

Can the maintainers please respond with a +1 comment once you are in favor of cutting this release?

1. Build and Push $IMAGE can be done with oras copy

2. Push an SBoM can be done with oras attach

The artifact spec defines the response to the discover API here:

200 OK
Content-Length: <length>
Content-Type: application/json

{
    "extensions": [
        {
            "name": "cncf.oras.referrers",
            "url": "https://github.com/oras-project/artifacts-spec/blob/main/manifest-referrers-api.md",
            "description": "ORAS referrers listing API",
            "endpoints": [
              "_oras/artifacts/referrers"
            ]
        }
    ]
}

The name of the oras extension is defined to be cncf.oras.referres in the artifact-spec. However, this violates the naming extension defined in the distribution spec for extensions here.

The response to the discover API should be:

200 OK
Content-Length: <length>
Content-Type: application/json

{
    "extensions": [
        {
            "name": "_oras",
            "url": "https://github.com/oras-project/artifacts-spec/blob/main/manifest-referrers-api.md",
            "description": "ORAS referrers listing API",
            "endpoints": [
              "_oras/artifacts/referrers"
            ]
        }
    ]
}

OCI Artifacts uses the manifest.config.mediaType to capture the unique artifact types.
This puts an additional burden to use iana.org mediaTypes, and longer formatting: application/vnd.org.project.subproject.version+config-file-format
The OCI Artifact standard is captured in the Artifact Authors content.

For Artifact Manifest, we're decoupling from the manifest.config.mediaType, enabling optional config objects to be used/or shared across artifact types, while an artifact can declare it's helm, wasm, notary v2, spdx sbom, nydus, etc.

For the artifactType, we've started using: org.[org-name].project.version. However, we haven't clarified the format in the artifact-manifest spec.

Currently, the artifact descriptor does not define some properties that exist in the image-spec v1.0.1 descriptor, namely:

• urls
• annotations
• data (reserved field)

Any client authoring an ORAS Artifact Manifest should be able to use these properties in the subjectManifest or blobs objects.

The artifact descriptor spec should be updated to include these additional properties.

With the increasing proliferation of new artifacts linked to manifests (e.g. sbom, scan reports, attestations, signatures...), there is a need to efficiently index the content of these artifacts to provide certain search capabilities. Obviously due to the heterogeneity of these artifacts, these search capabilities should be specific to each artifact type.

As an example, some of our use stories are:

• Get a list of images affected by a specific CVE .
• Get a list of images affected by a compromised signature.
• Find the latest scan report linked to a specific image digest.
• Get a list of images affected by a specific dependency.
• ...

Likewise, we are not sure that the usage of annotations for each artifact manifest would be enough to satisfy the aforementioned user stories.

This issue tracks a proposal to change blobs to descriptors in the oras artifacts manifest

While this proposal makes a name change from blobs to descriptors, we are not changing the definition or scope of the property at this time. The name change is similar to changing subjectManifest to subject, enabling future changes.

{
  "artifactType": "application/vnd.cncf.notary.v2",
  "blobs": [
    {
      "mediaType": "application/tar",
      "digest": "sha256:9834876dcfb05cb167a5c24953eba58c4ac89b1adf57f28f2f9d09af107ee8f0",
      "size": 32654
    }
  ],
  "subject": {
    "mediaType": "application/vnd.oci.image.manifest.v1+json",
    "digest": "sha256:73c803930ea3ba1e54bc25c2bdc53edd0284c62ed651fe7b00369da519a3c333",
    "size": 16724
  }
}

becomes:

{
  "artifactType": "application/vnd.cncf.notary.v2",
  "descriptors": [
    {
      "mediaType": "application/tar",
      "digest": "sha256:9834876dcfb05cb167a5c24953eba58c4ac89b1adf57f28f2f9d09af107ee8f0",
      "size": 32654
    }
  ],
  "subject": {
    "mediaType": "application/vnd.oci.image.manifest.v1+json",
    "digest": "sha256:73c803930ea3ba1e54bc25c2bdc53edd0284c62ed651fe7b00369da519a3c333",
    "size": 16724
  }
}

Background

At KubeCon US 2021, @dmcgowan, @mikebrow and @SteveLasker were discussing how the artifacts spec might support index like scenarios. Including the ability for the oras artifact-spec to support redirects from one manifest request (tag/digest), to another based on some (TBD) element of data.

A future, possible flow could be:

1. On ingestion, a distribution instance (registry) would evaluate the mediaType in the descriptor.
2. If the mediaType is know to be a manifest, supported by the registry, it would parse the manifest and possibly track additional lifecycle management links.
3. If the mediaType is unknown, it would be treated as an opaque blob.

The above flow supports a fallback model where descriptors are treated as blobs, if unknown to the registry instance. However, it enables a registry to opt-into support of other manifest types.

Why Change Now

"Data lasts indefinitely"

In two words: "future proof".

As the oras artifacts spec is being adopted, it needs to account for future possibilities. The name change does not guarantee the artifact spec will support index scenarios, but it does free up a property name to support those scenarios as the maintainers have time to evaluate the impact.

I am wondering how ordering could be implemented to know which artifacts are more recent than other. Let's imagine a scenario where I push an image with one artifact representing the scan report generated that day. One day later, I generate a new scan report that also associate to the same image, but this time with vulnerabilities.

When i am filtering the artifacts linked to this image, how could we detect which is the latest artifact, or in my case, the most recent scan report ?.

I believe this scenario could be also applied to other type of artifacts.

We have the media type defined under https://github.com/notaryproject/artifacts and this should move into the artifact specs similar to https://github.com/opencontainers/image-spec/tree/main/specs-go

We can move the the following specs https://github.com/aviral26/artifacts/tree/prototype-3/specs-go/v2 so that oras-project/distribution#2

Moving the discussion from #88 over to here.

I'm curious what the future plans for this specification are, and how it is expected to interact with the work happening in the OCI Reference Types WG. A few questions:

Are both expected to be supported long term?

Is this an alternative to the work produced from the OCI, or an extension? What's the best way to think about how they're related?

Is there a difference in functionality? How should client tooling decide which to use?

I'm a bit worried this is going to lead to fragmentation of the OCI specifications and registry operators/clients will be forced to choose to support one or the other here. Even if registries support both, it could lead to a confusing story for end users and clients. Have there been any discussions or plans on how this will be surfaced to end users/clients?

We see more and more people confused about the project status and relations with OCI artifact spec, like #117
and
https://cloud-native.slack.com/archives/C02AJS1BUTX/p1665691786144209

@michaelb990 @oras-project/oras-artifacts-spec-maintainers Let's create a Q&A list to explain the project status and relationship with OCI spec in this repo. I will draft a doc first and request your review.

(Apologies in advance if this has already been discussed and resolved, if there are links to discussions where these have already been hashed out I'm happy to go read those instead)

I've read through the referrers spec proposal (this specific version, for future readers), and have some feedback I thought I'd share, with the goal of clarifying/generalizing the spec, and toward making it easier to propose and accept into distribution-spec when the time comes.

distribution-spec doesn't currently have any concept of filtering list results.

While I definitely agree that filtering is useful, especially since we expect potentially many artifacts referring to an image, and since we'd prefer to have one interface of that clearly specified for compatibility, I'd suggest it might be easier to put a pin filtering for now to remove one more area of conversation and potential disagreement, and come back to address it after the bulk of the proposal has been ironed out and accepted into distribution-spec.

This should also lead to less churn and work for you if there are other changes to artifact types suggested, since you won't have to update the filtering specification to match those changes.

The current proposed referrers API returns the full manifest of each artifact. This will cause scaling issues, especially as registries add support for the data field in the manifest which would allow a manifest to contain full blobs in the manifest json. I also believe that it complicates the registry implementations because it means that the API now returns 2 types of information -- metadata about the reference relationship AND the full manifest of each reference. This may cause issues for registry operators if they need/want to authorize those operations separately or if that data is stored in two separate locations.

I've put together a doc comparing the existing proposal with the changes I'm proposing to return descriptors instead. I will follow this issue with a PR changing the API to a list of descriptors.

https://hackmd.io/83xqSoc6S8-PJI67vVC_0g?view

This tracks the changing of the /referrers API from /oras/artifacts/v1/{repository}/manifests/{digest}/referrers to the OCI Distribution extension proposal #111

I believe it should be, but it currently reads differently than the other fields. The description of mediaType is:

This field contains the mediaType of this document, differentiating from image-manifest and image-index. The mediaType for this manifest type MUST be application/vnd.cncf.oras.artifact.manifest.v1+json, where the version WILL change to reflect newer versions.

vs e.g. artifactType

The REQUIRED artifactType is a unique value, as registered with iana.org. The artifactType values are equivalent to the values used in the manifest.config.mediaType in OCI Artifacts. Examples include sbom/example, application/vnd.cncf.notary.v2. For details on creating a unique artifactType, see OCI Artifact Authors Guidance

The OCI image manifest & image index specs both require manifest media type to be present and I believe the artifact manifest should also require this field. If that opinion is shared by others, I will submit a PR to add a REQUIRED note to the description of the mediaType field in the artifact manifest.

Thanks!

What do we want to label the fall release?

For a client CLI, the version is less important as you can state the user must upgrade to a newer CLI version.

However, registries will store these manifests for an indefinite amount of time.

• We assume there will be newer additions,
• We want a versioning schema that allows newer clients to differentiate between versions

Questions:

• Do we want to call this a 0.1 release?
• Do we call this 1.0 release, and any non-breaking changes would be 1.1, while larger, possibly breaking changes that registries should opt-into would be 2.0?

Where this matters:

The artifact manifest has a mediaType of application/vnd.cncf.oras.artifact.manifest.v1+json
What should the value of .vN+json be?

The ORAS Artifact Manifest Spec states that,

subject descriptor
An OPTIONAL reference to any existing manifest within the repository. When specified, the artifact is said to be dependent upon the referenced subject.

However, the Manifest Referrers API says that,

referrers: A list of artifact descriptors that reference the given manifest. The list MUST include these references even if the given manifest does not exist in the repository. The list MUST be empty if there are no artifacts referencing the given manifest.

Which means, the Artifacts returned by the Referrers API may have a non-existing subject. Isn't this behavior inconsistent with the Artifact spec?
If the behavior of Referrers API is by design, should we update the Artifact Spec to keep them consistent?

While OCI finalizes the working group process and decides to enable the Reference Types working group, the OCI and Reference Types working group agreed to run these efforts in parallel. The incubation of the artifacts-spec is being developed under: https://github.com/oras-project/artifacts-spec/

As products and services move forward supporting reference types, they will store user artifacts, which will be supported for an indefinite period of time.

These new artifacts, with reference type support, are persisted with the new artifact.manifest (spec).
The current manifest.mediaType is application/vnd.cncf.oras.artifact.manifest.v1+json to avoid any conflict with OCI branding.

If OCI adopts the reference type working group before a release of the artifacts-spec is made, we can change the mediaType for user content stored. If not, the tooling can simply use the cncf.oras string. Both are Linux Foundation orgs, so in the end, it's just a string end-users have no knowledge about. That said, if we can unify these, we can avoid the different branded mediaTypes, like application/vnd.docker.distribution.manifest.v1+json and application/vnd.oci.image.manifest.v1+json