Recipe Test Results Version 2.14.

[bettina-gier]

Recipe Test results for the new release 2.14.

Environment: ToolEnv_v214_rc01.txt

Current Status: Running recipes with Core Release 2.14.0rc1
Runs can be found on Levante /scratch/b/b309137/esmvaltool-2.14rc0/ and https://esmvaltool.dkrz.de/shared/esmvaltool/v2.14.0rc1/debug.html

Links to the VM and comparison to previous results will be added once all the recipes have been run.

[bettina-gier]

Slowly populating this list when going through the failed recipes:

Likely cause backwards incompatible changes
Failures likely related to the preprocessor group coordinates (ESMValGroup/ESMValCore#2787):

Recipe	Maintainer	Failure Reason	Fixed?
recipe_calculate_gwl_exceedance_stats	@rswamina	Diagnostic Failure: expected year coordinate	#4352
recipe_climate_patterns	@gregrmunday	Diagnostic Failure: "Expected to find exactly 1 'month_number' coordinate, but found none."	#4352
recipe_kcs	unmaintained	Diagnostic failure: DataArray object has no attribute 'year'	#4352 (but still fails), add to list of broken recipes in #4354
recipe_seaice_sensitivity	@NParsonsMO	"Expected to find exactly 1 'year' coordinate, but found none.	#4352

OBS data issues:

Missing data:

Recipe	Maintainer	Failure Reason	Fixed?
recipe_aod_aeronet_assess	@ehogan	Missing data for Dataset: od440aer, AERmon, OBS6, AERONET, mon, 20250726; Levante newest version of that dataset 20250823	Runs when changing version to 20250823 -> @ehogan check version on JASMIN?
recipe_check_obs	@valeriupredoi	Missing data for ESACCI-SNOW, day v2.0	Yes, Data has been added to levante datapool
recipe_enso_feedback	@flicj191	Missing data for Dataset: zos, Omon, OBS6, CMEMS, mon, SEALEVEL-GLO-PHY-L4	Yes, with provided data it runs
recipe_weathertyping_CMIP6		Several CMIP6 and ERA5 daily data missing, we knew that when we merged the PR	Not planned; Has been recently merged with shared full runs

native 6:

Recipe	Maintainer	Failure Reason	Fixed?
recipe_climwip_brunner2019_med	@lukasbrunner	rsus: ValueError: Unable to convert from 'Unit('1.15740740740741e-05 kg.s-4')' to 'Unit('W m-2')' (see #3694).	Retired by #4350

Several key errors if all native 6 configurations are active and not commented out (will rerun these one by one)
Affected:

• recipe_climwip_brunner20esd
• recipe_create_ranges_obs
• recipe_galytska23jgr
• recipe_hydro_forcing
• recipe_model_benchmarking_diurnal_cycle
• recipe_model_evaluation_basics
• recipe_model_evaluation_clouds_cycles
• recipe_ref_annual_cycle_region
• recipe_ref_timeseries_region
• recipe_ref_trend_regions

Misc Failures:

Recipe	Maintainer	Failure Reason	Fixed?
recipe_autoassess_landsurface_surfrad	@valeriupredoi	Diagnostic Failure: line 457 in diag_scripts/autoassess/autoassess_area_base.py in run_area(config): list index out of range	#4377
recipe_daily_era5	unmaintained	Diagnostic Failure: KeyError: 'CMIP6', if CMOR_TABLES["CMIP6"].get_variable(mip, info["short_name"]) line 28 cmorizers/era5.py; Possibly due to ESMValGroup/ESMValCore#2946	#4363
recipe_ecs_constraints	@schlunma	Diagnostic Failure: KeyError: 0	#4356
recipe_eyring13jgr_fig12, recipe_ipccwg1ar6ch3_fig_3_19	@LisaBock	Hung up with no error message for 4 hours at step: Computing and saving data for preprocessing task eyring13jgr_fig12/ua / fig_3_19/ua	Works with --max-parallel-tasks=1
recipe_ipccwg1ar5ch3_atmosphere	@LisaBock	Diagnostic Error: line 418 in ipcc_ar6/precip_anom.ncl: Scubscript out of range	No
recipe_martin18grl , recipe_spei	@katjaweigel	Error in library(R.utils) : there is no package called ‘R.utils’	Yes #4355
recipe_ipccwg1ar6ch3_atmosphere	@LisaBock	Diagnostic Error: Subscript out of range, error in subscript #0; line 428 ipcc_ar6/precip_anom.ncl	No
recipe_ipccwg1ar6ch3_fig_3_42_b	@LisaBock	Diagnostic Error: fatal:systemfunc: cannot create child process:[errno=12] (reran twice, error occurred twice but at different times)	No

More often occuring:

In provenance: ValueError: Attributes have not been read yet. Call the to_iris method first to read the attributes from the file.

-> Fixed by ESMValGroup/ESMValCore#2992.

Affected recipes and maintainer:

• recipe_capacity_factor: unmaintained
• recipe_esacci_lst: @morobking
• recipe_flato13ipcc_figures_938_941_cmip3: @katjaweigel
• recipe_flato13ipcc_figures_938_941_cmip6: @katjaweigel
• recipe_preprocessor_test: Mattia (probably should change that)

Preprocessor Posix error TypeError: 'PosixPath' object is not iterable
Affected recipes and maintainer:

• recipe_lauer22jclim_fig3-4_zonal (@axel-lauer )
• recipe_lauer22jclim_fig5_lifrac (@axel-lauer )
• recipe_ref_scatterplot (@LisaBock )
• recipe_schlund20esd (@schlunma)

[bettina-gier]

Random things I noticed while running:

• While we have a DKRZ example config file, the AUX directory is not included - should we have this directory also noted somewhere as it's in the public DATA folder? (refer to Adding figures of Bock and Lauer (2024) #3526 )

[schlunma]

Attempting to fix the recipes with missing group coordinates right now!

[katjaweigel]

Because it happens for two recipes where I'm maintainer, I tried to look at:
"In provenance: ValueError: Attributes have not been read yet. Call the to_iris method first to read the attributes from the file.
Affected recipes and maintainer:"
Unfortunately I get lost with the Classes in the core:

For recipe_flato13ipcc_figures_938_941_cmip6
the error surfaces at esmvalcore/io/local.py line 851 and points back to
esmvalcore/_provenance.py line 269

"to_iris" is found both in esmvalcore/io/local.py and esmvalcore/io/protocol.py, where the "DataElement" is defined which seemingly should be used in esmvalcore/_provenance.py.
I'm not sure, why it apparently needs "to_iris" (and which of them), but I rather assume, that it should not be used at all, because the idea is "delay reading the attributes ... until after the data has been loaded" (esmvalcore/_provenance.py)? ( @bouweandela )

[bouweandela]

I'll take a look at that tomorrow

[flicj191]

Thanks! for recipe_enso_feedback we have the data on Gadi and a pr for the cmoriser #3952 but I didn't add the downloader because copernicus marine has a toolbox that has to be installed to download through python.. I can copy some files over if you need?

[katjaweigel]

recipe_martin18grl , recipe_spei do not have all necessary R packages anymore with the new environment (probably due to #4306 ). The package r-lintr usually installs also r-utils, which is used in esmvaltool/diag_scripts/droughts/spei.R. It is not r-spei directly, which needs that package, but it is used to read something to prepare data for r-spei (using commandArgs which is part of r-utils) (@lukruh so it would probably be possible to use something else in droughts/spei.R,? It could be enough to replace this line:
setwd(dirname(commandArgs(asValues=TRUE)$file))
)
The other calls to commandArgs should also work with the base version of it, see:
https://cran.r-project.org/web/packages/R.utils/refman/R.utils.html#commandArgs
and
https://cran.r-project.org/doc/manuals/r-patched/packages/base/refman/base.html#commandArgs

I did so far not succeed to include r-utils in the dependencies itself.

[bettina-gier]

Thanks! for recipe_enso_feedback we have the data on Gadi and a pr for the cmoriser #3952 but I didn't add the downloader because copernicus marine has a toolbox that has to be installed to download through python.. I can copy some files over if you need?

If you could copy those OBS to levante or give us a link where we could download them for this testing it would be great! Ideally that cmorizer should be merged before we feature freeze the Tool. Is it ready for review?

[schlunma]

Looking at the posix error now

[bouweandela]

In provenance: ValueError: Attributes have not been read yet. Call the to_iris method first to read the attributes from the file.

Created ESMValGroup/ESMValCore#2991 and am trying to figure out why this happens.

[katjaweigel]

recipe_martin18grl , recipe_spei do not have all necessary R packages anymore with the new environment (probably due to #4306 ). The package r-lintr usually installs also r-utils, which is used in esmvaltool/diag_scripts/droughts/spei.R. It is not r-spei directly, which needs that package, but it is used to read something to prepare data for r-spei (using commandArgs which is part of r-utils) (@lukruh so it would probably be possible to use something else in droughts/spei.R,? It could be enough to replace this line: setwd(dirname(commandArgs(asValues=TRUE)$file)) ) The other calls to commandArgs should also work with the base version of it, see: https://cran.r-project.org/web/packages/R.utils/refman/R.utils.html#commandArgs and https://cran.r-project.org/doc/manuals/r-patched/packages/base/refman/base.html#commandArgs

I did so far not succeed to include r-utils in the dependencies itself.

Including r.utils into the environment works now (after I understood, that it is not r-utils, but r-r.utils) and solves the issue with recipe_martin18grl and recipe_spei. So I started PR 4355 to change the environment.