append annotations instead of overwrite?

[rchincha]

oras push with annotations appears to assume a full update. Is there any way to perform an append to existing annotations?

[sajayantony]

+1 on this. I would like to propose an amend command that would

1. Add an annotation to an existing manifest
2. Put the new manifest
3. Delete the old manifest by digest so that we don’t have a dangling manifest but maybe have and option to skip delete(flag)

[qweeah]

Add an annotation to an existing manifest
Put the new manifest
Delete the old manifest by digest so that we don’t have a dangling manifest but maybe have and option to skip delete(flag)

@rchincha Will this functionally cover what you need?

[rchincha]

https://github.com/opencontainers/distribution-spec/blob/main/spec.md#pushing-manifests

Note the two options that are possible.
By digest, it is a read-copy-update operation but by tag, it is a single in-situ update.
Thoughts?

[shizhMSFT]

Amending manifests can have unintended consequences that can affect existing attached artifacts, associated tags, and associated manifest list / index.

• Scenario 1: Amending annotations of a signed image
  • Modifying a manifest will result in a manifest with a new digest. The signature of the previous manifest gets invalidated immediately.
• Scenario 2: Amending annotations of an image tagged with latest, v1.1.0, v1.1, v1.
  • Modifying a manifest will result in a manifest with a new digest, which is not tagged.
  • If the previous manifest gets deleted, all associated tags are deleted.
  • Since there is no common API to reverse search the tags of the manifest being amended, the client has to scan all tags, which is inefficient with a repository full of tags, in order to tag the new manifest with the same tags.
• Scenario 3: Amending annotations of an image manifest, which is in a manifest list.
  • Modifying a manifest will result in a manifest with a new digest. Therefore, the corresponding manifest list is required to be updated.

[shizhMSFT]

+1 on this. I would like to propose an amend command that would

1. Add an annotation to an existing manifest
2. Put the new manifest
3. Delete the old manifest by digest so that we don’t have a dangling manifest but maybe have and option to skip delete(flag)

Although we don't have a command like oras manifest amend --annotation key=value, this operation can be equivalently achieved using the oras CLI built from the main branch by

1. oras manifest fetch localhost:5000/test:latest --descriptor -o manifest.json
2. Edit manifest.json with any tools
3. oras manifest push localhost:5000/test:latest manifest.json
   • Optionally, push with multiple tags once [Multiple tags at the once #355] get resolved.
4. Delete the old manifest by oras manifest delete localhost:5000/test@sha256:xxxx using the digest obtained from step 1.

[rchincha]

A slight enhancement ...

oras manifest amend --annotation key=value
^ a single key/val

(OR) this

oras manifest amend --annotation key=value,key=value,key=value...

[rchincha]

Amending manifests can have unintended consequences that can affect existing attached artifacts, associated tags, and associated manifest list / index.

* **Scenario 1**: Amending annotations of a signed image
  
  * Modifying a manifest will result in a manifest with a new digest. The signature of the previous manifest gets invalidated immediately.

* **Scenario 2**: Amending annotations of an image tagged with `latest`, `v1.1.0`, `v1.1`, `v1`.
  
  * Modifying a manifest will result in a manifest with a new digest, which is not tagged.
  * If the previous manifest gets deleted, all associated tags are deleted.
  * Since there is no common API to reverse search the tags of the manifest being amended, the client has to scan all tags, which is inefficient with a repository full of tags, in order to tag the new manifest with the same tags.

* **Scenario 3**: Amending annotations of an image manifest, which is in a manifest list.
  
  * Modifying a manifest will result in a manifest with a new digest. Therefore, the corresponding manifest list is required to be updated.

Signatures or any other invariant will be invalidated since the image manifest itself is changing, which may be ok and would requiring re-signing etc.

When we compose, extend or distribute images based off of other images, this is really the normal.

[shizhMSFT]

A slight enhancement ...

oras manifest amend --annotation key=value ^ a single key/val

(OR) this

oras manifest amend --annotation key=value,key=value,key=value...

If we decide to do this, a suggested command is

oras manifest amend --annotation key=value --annotation key=value ...

which is consistent with oras push.

[shizhMSFT]

/cc @FeynmanZhou @yizha1 for inputs.

[shizhMSFT]

There are more questions:

• We can amend an image manifest. How about OCI Index / manifest list?
• What's the UX to delete an annotation entry? Note: empty string is a valid value.

[FeynmanZhou]

Although we don't have a command like oras manifest amend --annotation key=value, this operation can be equivalently achieved using the oras CLI built from the main branch by

1. oras manifest fetch localhost:5000/test:latest --descriptor -o manifest.json

2. Edit manifest.json with any tools

3. oras manifest push localhost:5000/test:latest manifest.json

   • Optionally, push with multiple tags once [Multiple tags at the once #355] get resolved.

4. Delete the old manifest by oras manifest delete localhost:5000/test@sha256:xxxx using the digest obtained from step 1.

I think it's significant to support amend annotations to simplify the process above.

The effects on the existing attached artifacts, associated tags, and associated manifest list / index are acceptable but should be explicitly informed to users in the documentation or somewhere.

If we decide to do this, a suggested command is

oras manifest amend --annotation key=value --annotation key=value ...

The sample above only considered the case of appending new annotations. Should Delete or Modify annotations also be considered in the amending case?