[Bug]: error for spaceKLIP v2.2.0: Exception: Error determining best reference for 'sirskernel' = Unknown reference type 'sirskernel' #268
Description
Check Existing Issues
- Yes, I have checked existing issues to ensure this problem hasn't already been reported.
Instrument or Category
NIRCam Stage 1/2 Pipeline
Description
I recently installed the latest spaceKLIP version 2.2.0. But when I run the stage 1 reduction spaceKLIP.coron1pipeline.run_obs(), I have the following errors. The older version (installed in a different virtual environment) works fine in the same computer.
---------------------------------------------------------------------------
CrdsLookupError Traceback (most recent call last)
File [~/spaceKLIP/spaceKLIP/coron1pipeline.py:729](https://vscode-remote+ssh-002dremote-002bstokes-002ephysics-002eucsb-002eedu.vscode-resource.vscode-cdn.net/home/jwzhang/Desktop/tutorial_spaceKLIP/~/spaceKLIP/spaceKLIP/coron1pipeline.py:729), in run_single_file(fitspath, output_dir, steps, verbose, **kwargs)
728 with all_logging_disabled(log_level):
--> 729 res = pipeline.run(fitspath)
730 except Exception as e:
File [~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/jwst/stpipe/core.py:147](https://vscode-remote+ssh-002dremote-002bstokes-002ephysics-002eucsb-002eedu.vscode-resource.vscode-cdn.net/home/jwzhang/Desktop/tutorial_spaceKLIP/~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/jwst/stpipe/core.py:147), in JwstStep.run(self, *args, **kwargs)
132 """
133 Run the step.
134
(...) 145 The step output
146 """
--> 147 result = super().run(*args, **kwargs)
148 if not self.parent:
File [~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/stpipe/step.py:505](https://vscode-remote+ssh-002dremote-002bstokes-002ephysics-002eucsb-002eedu.vscode-resource.vscode-cdn.net/home/jwzhang/Desktop/tutorial_spaceKLIP/~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/stpipe/step.py:505), in Step.run(self, *args)
504 if self.prefetch_references:
--> 505 self.prefetch(*args)
506 try:
File [~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/stpipe/step.py:630](https://vscode-remote+ssh-002dremote-002bstokes-002ephysics-002eucsb-002eedu.vscode-resource.vscode-cdn.net/home/jwzhang/Desktop/tutorial_spaceKLIP/~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/stpipe/step.py:630), in Step.prefetch(self, *args)
629 if len(args) and len(self.reference_file_types) and not self.skip:
--> 630 self._precache_references(args[0])
File [~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/stpipe/pipeline.py:266](https://vscode-remote+ssh-002dremote-002bstokes-002ephysics-002eucsb-002eedu.vscode-resource.vscode-cdn.net/home/jwzhang/Desktop/tutorial_spaceKLIP/~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/stpipe/pipeline.py:266), in Pipeline._precache_references(self, input_file)
263 with self.open_model(
264 input_file, asn_n_members=1, asn_exptypes=["science"]
265 ) as model:
--> 266 self._precache_references_opened(model)
267 except (ValueError, TypeError, OSError):
File [~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/stpipe/pipeline.py:292](https://vscode-remote+ssh-002dremote-002bstokes-002ephysics-002eucsb-002eedu.vscode-resource.vscode-cdn.net/home/jwzhang/Desktop/tutorial_spaceKLIP/~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/stpipe/pipeline.py:292), in Pipeline._precache_references_opened(self, model_or_container)
290 else:
291 # precache a single model object
--> 292 self._precache_references_impl(model_or_container)
File [~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/stpipe/pipeline.py:319](https://vscode-remote+ssh-002dremote-002bstokes-002ephysics-002eucsb-002eedu.vscode-resource.vscode-cdn.net/home/jwzhang/Desktop/tutorial_spaceKLIP/~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/stpipe/pipeline.py:319), in Pipeline._precache_references_impl(self, model)
314 self.log.info(
315 "Prefetching reference files for dataset: %r reftypes = %r",
316 model.meta.filename,
317 fetch_types,
318 )
--> 319 crds_refs = crds_client.get_multiple_reference_paths(
320 model.get_crds_parameters(), fetch_types, model.crds_observatory
321 )
323 ref_path_map = dict(list(crds_refs.items()) + list(ovr_refs.items()))
File [~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/stpipe/crds_client.py:53](https://vscode-remote+ssh-002dremote-002bstokes-002ephysics-002eucsb-002eedu.vscode-resource.vscode-cdn.net/home/jwzhang/Desktop/tutorial_spaceKLIP/~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/stpipe/crds_client.py:53), in get_multiple_reference_paths(parameters, reference_file_types, observatory)
52 log.set_log_time(True)
---> 53 return _get_refpaths(parameters, tuple(reference_file_types), observatory)
File [~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/stpipe/crds_client.py:65](https://vscode-remote+ssh-002dremote-002bstokes-002ephysics-002eucsb-002eedu.vscode-resource.vscode-cdn.net/home/jwzhang/Desktop/tutorial_spaceKLIP/~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/stpipe/crds_client.py:65), in _get_refpaths(data_dict, reference_file_types, observatory)
64 with crds_cache_locking.get_cache_lock():
---> 65 bestrefs = crds.getreferences(
66 data_dict,
67 reftypes=reference_file_types,
68 observatory=observatory,
69 )
70 return {
71 filetype: filepath if "N/A" not in filepath.upper() else "N/A"
72 for (filetype, filepath) in bestrefs.items()
73 }
File [~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/crds/core/heavy_client.py:127](https://vscode-remote+ssh-002dremote-002bstokes-002ephysics-002eucsb-002eedu.vscode-resource.vscode-cdn.net/home/jwzhang/Desktop/tutorial_spaceKLIP/~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/crds/core/heavy_client.py:127), in getreferences(parameters, reftypes, context, ignore_cache, observatory, fast)
125 # Attempt to cache the recommended references, which unlike dump_mappings
126 # should work without network access if files are already cached.
--> 127 best_refs_paths = api.cache_references(
128 final_context, bestrefs, ignore_cache=ignore_cache)
130 return best_refs_paths
File [~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/crds/client/api.py:996](https://vscode-remote+ssh-002dremote-002bstokes-002ephysics-002eucsb-002eedu.vscode-resource.vscode-cdn.net/home/jwzhang/Desktop/tutorial_spaceKLIP/~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/crds/client/api.py:996), in cache_references(pipeline_context, bestrefs, ignore_cache)
989 """Given a CRDS context `pipeline_context` and `bestrefs` dictionary, download missing
990 reference files and cache them on the local file system.
991
(...) 994 Returns: { reference_keyword : reference_local_filepath ... }
995 """
--> 996 wanted = _get_cache_filelist_and_report_errors(bestrefs)
998 if config.S3_RETURN_URI:
File [~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/crds/client/api.py:1040](https://vscode-remote+ssh-002dremote-002bstokes-002ephysics-002eucsb-002eedu.vscode-resource.vscode-cdn.net/home/jwzhang/Desktop/tutorial_spaceKLIP/~/miniconda3/envs/spaceklip_v2_2_0/lib/python3.11/site-packages/crds/client/api.py:1040), in _get_cache_filelist_and_report_errors(bestrefs)
1039 if last_error is not None:
-> 1040 raise last_error
1041 return wanted
CrdsLookupError: Error determining best reference for 'sirskernel' = Unknown reference type 'sirskernel'
During handling of the above exception, another exception occurred:
RuntimeError Traceback (most recent call last)
Cell In[7], line 1
----> 1 spaceKLIP.coron1pipeline.run_obs(database=database,
2 steps={'group_scale': {'skip': False},
3 'dq_init': {'skip': False},
4 'saturation': {'n_pix_grow_sat': 1,
5 'grow_diagonal': False},
6 'ipc': {'skip': True},
7 'superbias':{'skip': False},
8 'refpix': {'odd_even_columns': True,
9 'odd_even_rows': True,
10 'nlower': 4,
11 'nupper': 4,
12 'nleft': 4,
13 'nright': 4,
14 'nrow_off': 0,
15 'ncol_off': 0},
16 'linearity': {'skip': False},
17 'dark_current': {'skip': True},
18 'persistence': {'skip': True},
19 'jump': {'rejection_threshold': 4.,
20 'three_group_rejection_threshold': 4.,
21 'four_group_rejection_threshold': 4.,
22 'maximum_cores': 'all'},
23 'subtract_1overf': {'model_type': 'savgol',
24 'sat_frac': 0.5,
25 'combine_ints': True,
26 'vertical_corr': True,
27 'nproc': 4},
28 'ramp_fit': {'save_calibrated_ramp': False,
29 'maximum_cores': 'all'},
30 'gain_scale': {'skip': False}},
31 subdir='stage1')
File [~/spaceKLIP/spaceKLIP/coron1pipeline.py:957](https://vscode-remote+ssh-002dremote-002bstokes-002ephysics-002eucsb-002eedu.vscode-resource.vscode-cdn.net/home/jwzhang/Desktop/tutorial_spaceKLIP/~/spaceKLIP/spaceKLIP/coron1pipeline.py:957), in run_obs(database, steps, subdir, overwrite, quiet, verbose, **kwargs)
955 else:
956 if not quiet: log.info(' --> Coron1Pipeline: processing ' + tail)
--> 957 _ = run_single_file(fitspath, output_dir, steps=steps,
958 verbose=verbose, **kwargs)
960 if skip_revert:
961 # Need to make sure we don't skip later files if we just didn't want to mask_groups for this file
962 steps['mask_groups']['skip'] = False
File [~/spaceKLIP/spaceKLIP/coron1pipeline.py:731](https://vscode-remote+ssh-002dremote-002bstokes-002ephysics-002eucsb-002eedu.vscode-resource.vscode-cdn.net/home/jwzhang/Desktop/tutorial_spaceKLIP/~/spaceKLIP/spaceKLIP/coron1pipeline.py:731), in run_single_file(fitspath, output_dir, steps, verbose, **kwargs)
729 res = pipeline.run(fitspath)
730 except Exception as e:
--> 731 raise RuntimeError(
732 'Caught exception during pipeline processing.'
733 '\nException: {}'.format(e)
734 )
735 finally:
736 try:
RuntimeError: Caught exception during pipeline processing.
Exception: Error determining best reference for 'sirskernel' = Unknown reference type 'sirskernel'
Error traceback output
No response
What operating system are you using?
No response
What version of Python are you running?
No response
What Python packages do you have installed?
No response
Additional context or information
No response