Skip to content

feat: General purpose clipping filter#96

Merged
floriankrb merged 9 commits into
mainfrom
general-purpose-clipper
Jun 13, 2025
Merged

feat: General purpose clipping filter#96
floriankrb merged 9 commits into
mainfrom
general-purpose-clipper

Conversation

@lzampier
Copy link
Copy Markdown
Member

Description

This code defines a custom data filter plugin called Clipper for the Anemoi framework. The purpose of the Clipper filter is to clip (limit) the values of a specified field to a given range.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

Issue Number

#84

Code Compatibility

  • I have performed a self-review of my code

Code Performance and Testing

  • I have added tests that prove my fix is effective or that my feature works
  • I ran the complete Pytest test suite locally, and they pass

Dependencies

  • I have ensured that the code is still pip-installable after the changes and runs
  • I have tested that new dependencies themselves are pip-installable.

Documentation

  • My code follows the style guidelines of this project
  • I have updated the documentation and docstrings to reflect the changes
  • I have added comments to my code, particularly in hard-to-understand areas

Additional Notes

This is just a first draft that works both for 2d and 3d variables. It would be nice to apply the same clipper to multiple variables at once, if the required bounding values are the same. As it is, a new clipping instance must be added for each variable to the dataset creation recipe.

@lzampier lzampier changed the title General purpose clipping filter faet: General purpose clipping filter May 18, 2025
@lzampier lzampier changed the title faet: General purpose clipping filter feat: General purpose clipping filter May 18, 2025
Comment thread src/anemoi/transform/filters/clipper.py Outdated
@github-actions github-actions Bot added the tests label May 26, 2025
@floriankrb
Copy link
Copy Markdown
Member

As we have "rescale" (verb) in the config, I changed the name from "clipper" (noun) to "clip" (verb)

@lzampier lzampier marked this pull request as ready for review May 30, 2025 17:13
@lzampier
Copy link
Copy Markdown
Member Author

lzampier commented Jun 2, 2025

From @anaprietonem:

The tests look good, but it would be great if either @lzampier or @floriankrb could update the input/output interface to match the ones here https://github.com/ecmwf/anemoi-transform/pull/93/files#diff-a990ae3dbc875f09016f33e9258352e6e8b5cc3fd3f7a50d2a2eef97e198e856, so trying to use the one from anemoi.transform.sources import source_registry, which can either take a source file or a simple dictionary containing the earthkit field(s).

@floriankrb floriankrb mentioned this pull request Jun 13, 2025
9 tasks
@floriankrb floriankrb merged commit f70b1ab into main Jun 13, 2025
68 checks passed
@floriankrb floriankrb deleted the general-purpose-clipper branch June 13, 2025 09:07
@github-project-automation github-project-automation Bot moved this from Now In Progress to Done in Anemoi-dev Jun 13, 2025
anaprietonem pushed a commit that referenced this pull request Jul 14, 2025
🤖 Automated Release PR

This PR was created by `release-please` to prepare the next release.
Once merged:

1. A new version tag will be created
2. A GitHub release will be published
3. The changelog will be updated

Changes to be included in the next release:
---


##
[0.1.12](0.1.11...0.1.12)
(2025-07-14)


### Features

* Add orography to surface geopotential filter
([#115](#115))
([9b7fd8d](9b7fd8d))
* Add SingleFieldFilter
([#104](#104))
([914d1df](914d1df))
* Filter migration from anemoi-datasets + testing
([#93](#93))
([abfb33d](abfb33d))
* General purpose clipping filter
([#96](#96))
([f70b1ab](f70b1ab))
* Migrate sum filter
([#121](#121))
([3b5596e](3b5596e))
* Set resolution metadata in filter operation.
([#98](#98))
([7844fea](7844fea))


### Bug Fixes

* Docs sphinx dependency python>=3.11
([#116](#116))
([29f6290](29f6290))
* Fixed typo in error message
([#120](#120))
([242c836](242c836))

---
> [!IMPORTANT]
> Please do not change the PR title, manifest file, or any other
automatically generated content in this PR unless you understand the
implications. Changes here can break the release process.
> ⚠️ Merging this PR will:
> - Create a new release
> - Trigger deployment pipelines
> - Update package versions

 **Before merging:**
 - Ensure all tests pass
 - Review the changelog carefully
 - Get required approvals

[Release-please
documentation](https://github.com/googleapis/release-please)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants