Add cloud-sync: optional R2 / S3 / Azure sync for memories and config#1330
Closed
ikarolaborda wants to merge 1 commit intooraios:mainfrom
Closed
Add cloud-sync: optional R2 / S3 / Azure sync for memories and config#1330ikarolaborda wants to merge 1 commit intooraios:mainfrom
ikarolaborda wants to merge 1 commit intooraios:mainfrom
Conversation
Ships an experimental feature to sync serena memories and configuration to
a cloud backend. Additive union semantics — never silently overwrites
divergent content.
- New CLI group: serena cloud-sync {configure, test, status, push, pull,
fix-perms}.
- Dashboard panel at /dashboard/cloud-sync.html with credential form,
dry-run plan preview, and push/pull actions.
- Providers: Cloudflare R2 and Amazon S3 via boto3 (endpoint_url override
also enables MinIO / Ceph RGW / SeaweedFS), Azure Blob via
azure-storage-blob (lazy-imported).
- CloudStorageProvider ABC as the only seam; CloudSyncService takes the
provider via constructor (DIP). A guard test asserts business-logic
modules import no SDKs.
- Conflict preservation: sha256 compare; conflicts leave both sides
intact (remote saved as <path>.cloud-<ts> sibling on pull;
--force-push / --force-pull opt-in).
- Credentials only in ~/.serena/cloud-sync.env (chmod 0600, atomic
write, mask sentinel preserves existing secrets on re-save). Dashboard
routes require a per-process local token and reject non-localhost
callers.
- Dry-run-first on both CLI and dashboard.
- EXPERIMENTAL; user-triggered only. 5 MiB per-object cap.
- Tests: 24 unit tests with an in-memory FakeCloudProvider; LocalStack
and Azurite integration skeletons (auto-skip when not configured).
- Full design doc at docs/cloud-sync-feature-plan.md.
This was referenced Apr 13, 2026
Contributor
|
Hi. Sry, 4150 loc of changes is the limit of PRs that we accept before prior discussion in an issue, so closing this. Jokes aside, if this is a feature you're interested in, feel free to open an issue and start a discussion there |
Contributor
|
FYI @ikarolaborda in the past we have worked on https://github.com/oraios/accsr/ for syncing data between cloud and local, it was used in many production projects. Have a look if you're interested, supports all providers |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #1328. Supersedes #1329 (closed — same branch renamed).
Adds an experimental, opt-in
cloud-syncfeature so Serena's memories and configuration can be mirrored to a user-provided cloud bucket across machines. Additive union — no silent overwrites.What it does
serena cloud-sync {configure, test, status, push, pull, fix-perms}./dashboard/cloud-sync.htmlwith credential form, dry-run plan preview, and push/pull actions.boto3(theendpoint_urloverride also enables MinIO / Ceph RGW / SeaweedFS), and Azure Blob viaazure-storage-blob(lazy-imported so R2/S3 users never load it).How it's shaped
CloudStorageProviderABC is the only seam;CloudSyncServicereceives the provider via constructor. A guard test (test_no_boto3_in_business_logic) fails if any SDK leaks intosync/diff/inventory/scope/hash_util/credentials/provider.<path>.cloud-<ts>; on push the remote is kept.--force-push/--force-pullare opt-in.~/.serena/cloud-sync.env(chmod 0600, atomic write; a****sentinel on re-save preserves existing secrets). Dashboard routes require a per-process local token and reject non-localhost callers.Scope (explicit MVP non-goals)
project.local.yml(opt-in only).Tests
FakeCloudProvider.LOCALSTACK_ENDPOINT/AZURITE_ACCOUNTenv is set.Docs
Design rationale, data model, failure/idempotency contract, and acceptance criteria in
docs/cloud-sync-feature-plan.md.