Skip to content

🎨 Optimize storage access layer: replace GROUP BY subquery with EXISTS in get_readable_project_ids#9031

Merged
matusdrobuliak66 merged 5 commits intoITISFoundation:masterfrom
matusdrobuliak66:optimize-project-list-storage
Apr 15, 2026
Merged

🎨 Optimize storage access layer: replace GROUP BY subquery with EXISTS in get_readable_project_ids#9031
matusdrobuliak66 merged 5 commits intoITISFoundation:masterfrom
matusdrobuliak66:optimize-project-list-storage

Conversation

@matusdrobuliak66
Copy link
Copy Markdown
Collaborator

@matusdrobuliak66 matusdrobuliak66 commented Apr 15, 2026

What do these changes do?

Optimizes _list_user_projects_access_rights_with_read_access in the storage service's access_layer.py by replacing the GROUP BY + jsonb_object_agg subquery JOIN with a simple EXISTS subquery — same pattern applied to the webserver's project listing in #9028.

Related issue/s

How to test

Dev-ops

@sonarqubecloud
Copy link
Copy Markdown

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Optimizes project readability queries by replacing GROUP BY-based access-rights subqueries with correlated EXISTS predicates, aligning storage-service access-layer behavior with the previously optimized webserver project listing approach.

Changes:

  • Refactors storage _list_user_projects_access_rights_with_read_access to use EXISTS for read-access filtering.
  • Updates webserver legacy projects repository to use EXISTS for access-rights checks and to split folder-scope logic into ROOT/SPECIFIC/ALL query shapes.
  • Adds/extends unit tests in both webserver and storage to cover folder-scope behavior and readable-project-id retrieval.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
services/web/server/tests/unit/with_dbs/02/test_projects_crud_handlers__list_with_query_params.py Adds folder-related listing/search/pagination test cases to validate folder-scope query behavior.
services/web/server/src/simcore_service_webserver/projects/_projects_repository_legacy.py Rewrites access-rights filtering to EXISTS and introduces folder-scope-specific query strategies.
services/storage/tests/unit/test_db_access_layer.py Adds tests for get_readable_project_ids across private/shared/no-access scenarios.
services/storage/src/simcore_service_storage/modules/db/access_layer.py Replaces join+GROUP BY subquery with EXISTS for readable project ID listing in storage access layer.

Comment thread services/storage/tests/unit/test_db_access_layer.py
@matusdrobuliak66 matusdrobuliak66 enabled auto-merge (squash) April 15, 2026 11:24
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 15, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 66.88%. Comparing base (c50e568) to head (48a5451).
⚠️ Report is 1 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (c50e568) and HEAD (48a5451). Click for more details.

HEAD has 31 uploads less than BASE
Flag BASE (c50e568) HEAD (48a5451)
unittests 32 1
Additional details and impacted files
@@             Coverage Diff             @@
##           master    #9031       +/-   ##
===========================================
- Coverage   87.34%   66.88%   -20.47%     
===========================================
  Files        2056      844     -1212     
  Lines       80962    39469    -41493     
  Branches     1451      182     -1269     
===========================================
- Hits        70714    26397    -44317     
- Misses       9837    13022     +3185     
+ Partials      411       50      -361     
Flag Coverage Δ
integrationtests 64.04% <ø> (+0.01%) ⬆️
unittests 86.82% <100.00%> (+0.60%) ⬆️
Components Coverage Δ
pkg_aws_library ∅ <ø> (∅)
pkg_celery_library ∅ <ø> (∅)
pkg_dask_task_models_library ∅ <ø> (∅)
pkg_models_library ∅ <ø> (∅)
pkg_notifications_library ∅ <ø> (∅)
pkg_postgres_database ∅ <ø> (∅)
pkg_service_integration ∅ <ø> (∅)
pkg_service_library ∅ <ø> (∅)
pkg_settings_library ∅ <ø> (∅)
pkg_simcore_sdk 69.55% <ø> (-16.40%) ⬇️
agent ∅ <ø> (∅)
api_server ∅ <ø> (∅)
autoscaling ∅ <ø> (∅)
catalog ∅ <ø> (∅)
clusters_keeper ∅ <ø> (∅)
dask_sidecar ∅ <ø> (∅)
datcore_adapter ∅ <ø> (∅)
director ∅ <ø> (∅)
director_v2 78.81% <ø> (-12.82%) ⬇️
dynamic_scheduler ∅ <ø> (∅)
dynamic_sidecar 73.91% <ø> (-13.59%) ⬇️
efs_guardian ∅ <ø> (∅)
invitations ∅ <ø> (∅)
payments ∅ <ø> (∅)
resource_usage_tracker ∅ <ø> (∅)
storage 86.82% <100.00%> (ø)
webclient ∅ <ø> (∅)
webserver 59.36% <ø> (-27.42%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c50e568...48a5451. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@matusdrobuliak66 matusdrobuliak66 merged commit a00f890 into ITISFoundation:master Apr 15, 2026
92 of 96 checks passed
matusdrobuliak66 added a commit that referenced this pull request Apr 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

a:storage issue related to storage service

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants