feat: add new record consumers and enhance data import commands with detailed help descriptions#3665
Open
tchoumi313 wants to merge 18 commits intomainfrom
Open
feat: add new record consumers and enhance data import commands with detailed help descriptions#3665tchoumi313 wants to merge 18 commits intomainfrom
tchoumi313 wants to merge 18 commits intomainfrom
Conversation
…detailed help descriptions
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds three RecordConsumer classes (Folder, ElementaryAction, Processing), routes FOLDER/ELEMENTARY_ACTION/PROCESSING through them in process_excel_file, implements domain/field/M2M resolution and conflict policies; and expands CLI Data Wizard help with a per-command Changes
Sequence DiagramsequenceDiagram
participant User as "User"
participant CLI as "CLI\n(clica)"
participant Dispatcher as "process_excel_file"
participant Consumer as "RecordConsumer\n(Folder/ElementaryAction/Processing)"
participant DB as "Database"
User->>CLI: run import command (upload Excel)
CLI->>Dispatcher: process_excel_file(model_type, file, options)
Dispatcher->>Dispatcher: select consumer based on model_type
Dispatcher->>Consumer: instantiate & pass file chunk
Consumer->>Consumer: resolve domain, map fields, normalize status
Consumer->>Consumer: resolve M2M (nature, assigned_to, filtering_labels)
Consumer->>Consumer: apply conflict policy (Stop/Skip/Update)
Consumer->>DB: create or update records
DB-->>Consumer: persist results
Consumer-->>Dispatcher: return import summary
Dispatcher-->>CLI: relay summary
CLI-->>User: display import result
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment Tip CodeRabbit can enforce grammar and style rules using `languagetool`.Configure the |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@backend/data_wizard/views.py`:
- Around line 1231-1239: The current code uses
Folder.objects.filter(name__iexact=domain_name).first() which silently picks an
arbitrary parent when multiple folders share the same name; replace this with a
safe lookup that rejects ambiguous matches: use
Folder.objects.filter(name__iexact=domain_name) to get the queryset, then if
qs.count() == 0 return the existing Error, if qs.count() > 1 return an Error
indicating multiple parent folders found (so callers can resolve ambiguity),
otherwise use qs.first().id (or qs.get() to let MultipleObjectsReturned surface)
to set parent_folder_id; update references to parent_folder and parent_folder_id
accordingly.
- Around line 1397-1404: The code resolves assigned_to emails to User IDs but
Processing.assigned_to is a M2M to Actor; replace the User lookup with an Actor
lookup and set data["assigned_to"] to Actor PKs. Specifically, change the
User.objects.filter(...) call to
Actor.objects.filter(user__email__in=emails).values_list("id", flat=True) (or
Actor.objects.filter(email__in=emails) if Actor has its own email field) so the
resulting IDs match the Processing.assigned_to relation.
- Around line 1351-1395: The update path is dropping aliased M2M fields and
won't clear stale relations; when preparing update data (see _build_update_data
and prepare_create/prepare_update logic) add support for source aliases: map
"processing_nature" -> "nature" and "labels" -> "filtering_labels" so aliased
columns are included in update payloads, and ensure that if those aliased fields
are present but blank you emit an explicit empty list to clear the M2M relation;
likewise treat "assigned_to" as a clearable blank M2M (same behavior as "owner")
so an empty/blank assigned_to input will remove existing assignees rather than
preserving them.
In `@cli/clica.py`:
- Around line 388-397: Update the help text strings (the "help" value in
cli/clica.py and the similar help block around the 524-531 region) to list the
actual backend column names the importer reads: use inherent_proba,
current_proba, residual_proba, current_impact (and similar exact field names
used by backend/data_wizard/views.py), and for TPRM imports mention
provider_entity_ref_id and solution_ref_id rather than provider_entity or
solution. Locate and edit the "help" text blocks in clica.py that describe
CSV/Excel columns and replace the incorrect *_probability names and TPRM column
names with the backend-expected identifiers so the CLI guidance matches views.py
expectations.
- Around line 324-337: The CLI command entries (e.g., the "import_folders"
command dict) disable conflict handling (supports_conflict: False and a help
note saying conflict management is unsupported) while the backend uses
RecordConsumer.process_records() which supports --on-conflict (skip/update).
Change the command entries that follow this pattern (including the shown
"import_folders" entry and the other similar import commands around the same
region) to enable conflict handling by setting supports_conflict: True, remove
or update the help text that says conflict management is unsupported, and ensure
the CLI wiring forwards the parsed --on-conflict option into the import flow so
RecordConsumer.process_records() receives the conflict mode (skip/update) from
the CLI.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: a81d8f00-3f73-4764-baf1-3af78dcb5549
📒 Files selected for processing (2)
backend/data_wizard/views.pycli/clica.py
monsieurswag
requested changes
Mar 12, 2026
Contributor
There was a problem hiding this comment.
As the biggest part of this PR is about replacing old code with new one, don't forget to remove all the dead code created by your new code.
All the following methods seem to be dead code now:
LoadFileView._process_elementary_actions is dead codeLoadFileView._process_foldersLoadFileView._process_processings
Look for dead code and delete all of it, beside that everything's fine.
Don't forget to address @coderabbitai comments too.
Plus i did some changes, check if it didn't break anything on your side.
… and folders in LoadFileView
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: add new record consumers and enhance data import commands with detailed help descriptions
Summary by CodeRabbit
New Features
Documentation