Update dark db#98
Conversation
| self.uncal_files = files | ||
| logging.info(f"Ingesting {len(files)} Files: {files}") | ||
| if self.db_handler: | ||
| # TODO - this is no longer intended to be run in segments so pipeline_cmd is no longer needed |
There was a problem hiding this comment.
What do you mean by segments? Do you mean stages? like the prep stage?
| @@ -132,6 +139,12 @@ def prep_pipeline(self, file_list=None): | |||
|
|
|||
| self.prepped_files.append(prep_output_file_path) | |||
| self.qc.check_prep_pipeline() # TODO - speak with rick about what to do on QC Failures | |||
There was a problem hiding this comment.
Let's gracefully exit a prep pipeline instance if a romancal step fails. That means there is something wrong with the data or data model. We shouldn't expect errors in prep pipelines and we wouldn't want to continue either.
|
|
||
| # TODO - populate routine based DB entries for rfp_log_pro | ||
| # self.db_handler.db_entry.rfp_log_pro. | ||
| # verify with rick, no updates in DB for superdark? |
There was a problem hiding this comment.
No. It's not as intense of computation as it was before. Before it was lots of processing, intermediate product used to derive more derived products down stream. Back then I could see keeping more eyes on it. Now I dont see the need for that.
| REF_TYPE_DETECTORSTATUS = "DETECTORSTATUS" | ||
| REF_TYPE_DISTORTION = "DISTORTION" | ||
| REF_TYPE_ETC = "ETC" | ||
| REF_TYPE_FGS_MASK = "FGS_MASK" |
There was a problem hiding this comment.
Did you do the meta maker classes for this also or not yet? One of us should for Sierra I think.
2 participants
Create database (rfp_log_pro) update for the entirety of the dark pipeline run.