Skip to content

Add more tests#537

Merged
mihaimaruseac merged 3 commits intosigstore:mainfrom
stefanberger:more_tests
Oct 9, 2025
Merged

Add more tests#537
mihaimaruseac merged 3 commits intosigstore:mainfrom
stefanberger:more_tests

Conversation

@stefanberger
Copy link
Copy Markdown
Contributor

@stefanberger stefanberger commented Oct 7, 2025

Summary

Add test cases for v1.0.1 signatures.
Also add a test that first creates key, certificate, and sigstore type of signatures with the currently active library and tests verification against several older versions of the library that are installed in a venv. This serves the purpose to ensure backwards compatibility. Then also use the older version of the library and model_signing tool to verify against the key, certificate, and sigstore signatures we have in the test directory.

Checklist
  • All commits are signed-off, using DCO
  • All new code has docstrings and type annotations
  • All new code is covered by tests. Aim for at least 90% coverage. CI is configured to highlight lines not covered by tests.
  • Public facing changes are paired with documentation changes
  • Release note has been added to CHANGELOG.md if needed

Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
@stefanberger stefanberger requested review from a team as code owners October 7, 2025 17:43
@stefanberger stefanberger force-pushed the more_tests branch 2 times, most recently from 6cbe799 to 2a8af3f Compare October 7, 2025 18:15
@stefanberger stefanberger marked this pull request as draft October 8, 2025 13:49
@stefanberger
Copy link
Copy Markdown
Contributor Author

Putting this into draft mode while I experiment with the oidc-token from here for sigstore signing.

Create a signature with the currently active model signing library and test
it against old versions of the library installed into a venv. Use the older
versions of the library to test against the pre-created signatures located
in the test directory.

Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
@stefanberger stefanberger marked this pull request as ready for review October 8, 2025 15:24
@stefanberger
Copy link
Copy Markdown
Contributor Author

It can now sign with sigstore using that test-token as well. It has problems with --use_staging, though, so I cannot use this option.

@spencerschrock
Copy link
Copy Markdown
Contributor

It has problems with --use_staging, though, so I cannot use this option.

This is specifically for trying to verify new signatures on old versions? I believe rekor v2 is live in staging, which is why you're having this issue (verifying a rekorv2 entry with sigstore-python v3.6.5 which doesnt have support for them yet?) .

I understand verifying old signatures with new versions, but do we need tests for verifying new signatures using old versions?

@stefanberger
Copy link
Copy Markdown
Contributor Author

stefanberger commented Oct 8, 2025

It has problems with --use_staging, though, so I cannot use this option.

This is specifically for trying to verify new signatures on old versions? I believe rekor v2 is live in staging, which is why you're having this issue (verifying a rekorv2 entry with sigstore-python v3.6.5 which doesnt have support for them yet?) .

I got strange errors with insufficient time sources.

I understand verifying old signatures with new versions, but do we need tests for verifying new signatures using old versions?

  1. I think it's good to maintain backwards compatibility (for as long as we can) so that signatures created with a newer version of the library still verify with older versions of it, especially if the model-signing library was integrated into a much more complex environment where updating it could be a hassle and users may not know what to do if a signature doesn't verify. FYI, I am trying to have signature verification used by vLLM where installations of vLLM and a potential future plugin could use one version of the library while models get signed with a newer version: Add plugin registry and callbacks for AI model validation vllm-project/vllm#26309
  2. I also want to prevent PR Replace sigstore_proto_buf with sigstore_models #533 from introducing unnecessary backwards compatibility issues.
  3. For our own awareness I think it's good to know what version of the library can/cannot be used to verify a signature created by a specific version of the library. This can be deduced from the test cases.

Copy link
Copy Markdown
Member

@mihaimaruseac mihaimaruseac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nit in renaming some files, but otherwise I'd say to get this in for now, even though we might need to do a refactoring of the testing to also take into account that the spec has been moved to a separate repo. But for the next release they can be here.

Comment thread scripts/tests/test-sign-verify-allversions.sh
@mihaimaruseac mihaimaruseac enabled auto-merge (squash) October 9, 2025 16:52
Also adjust the testrunner to pick up test cases with suffix .sh.

Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
auto-merge was automatically disabled October 9, 2025 17:27

Head branch was pushed to by a user without write access

Copy link
Copy Markdown
Member

@mihaimaruseac mihaimaruseac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@mihaimaruseac mihaimaruseac merged commit a15963c into sigstore:main Oct 9, 2025
51 checks passed
@stefanberger stefanberger deleted the more_tests branch October 9, 2025 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants