Skip to content

feature: doppler in-platform migration and unified external migrations UI#5875

Open
IgorHorta wants to merge 50 commits intomainfrom
igor/eng-4649-create-migration-tool-from-doppler-to-infisical
Open

feature: doppler in-platform migration and unified external migrations UI#5875
IgorHorta wants to merge 50 commits intomainfrom
igor/eng-4649-create-migration-tool-from-doppler-to-infisical

Conversation

@IgorHorta
Copy link
Copy Markdown
Contributor

Summary

  • Add Doppler app connection and in-platform migration (org settings + secret overview import).
  • Unify External Migrations settings: single table, provider picker, v3 UI, row actions menu.
  • Rename vault migration config storage to external_migration_configs with provider; filter Vault API responses to vault-only rows.
  • Docs: Doppler guide, overview and Vault setup updates.

Test plan

  • Run DB migration; verify Vault and Doppler configs in External Migrations.
  • Doppler: App Connection, add config, Import from settings and from project overview.
  • Vault: existing namespace config and secret overview import still work.

Made with Cursor

Add migration to rename vault_external_migration_config to
external_migration_configs, introduce nullable namespace and provider
field, and register the Zod schema plus Knex typings.

Made-with: Cursor
Update joins and queries to use external_migration_configs (renamed table)
while preserving connection hydration behavior.

Made-with: Cursor
Implement Doppler connection schema, service helpers, and v1 router;
register enum, maps, and factory wiring alongside existing app connections.

Made-with: Cursor
Add Doppler config CRUD, project/environment listing, and secret import;
scope vault migration endpoints to provider=vault so Doppler rows do not
break response validation. Register v3 external-migration routes.

Made-with: Cursor
Add Doppler to APP_CONNECTION_MAP, hooks enums and option types, form
component, and integration logo asset.

Made-with: Cursor
Add query keys, Doppler config mutations, import mutation, and Doppler
project/environment fetchers.

Made-with: Cursor
Replace per-provider sections with provider picker, merged v3 table and
row actions menu, Doppler config/import modals, Vault/Doppler docs link
in header, and remove legacy Vault-only section component.

Made-with: Cursor
Add Add from Doppler to the overview Add Secret menu when a Doppler
migration config exists, with modal for project/environment selection
and useImportDopplerSecrets targeting the current folder.

Made-with: Cursor
Document unified in-platform tooling, update Vault namespace setup for
Add configuration flow, add Doppler migration page and nav entry.

Made-with: Cursor
@linear
Copy link
Copy Markdown

linear bot commented Mar 31, 2026

@mintlify
Copy link
Copy Markdown

mintlify bot commented Mar 31, 2026

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
infisical 🟢 Ready View Preview Mar 31, 2026, 12:18 AM

@maidul98
Copy link
Copy Markdown
Collaborator

maidul98 commented Mar 31, 2026

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

…perations

Add explicit checks to ensure updateDopplerExternalMigration and
deleteDopplerExternalMigration only operate on Doppler configs, not other
provider types. This prevents accidental modification or deletion of wrong
config types.
… for Doppler imports

Validate that the Doppler connection exists and is accessible before updating
a migration config. When importing Doppler secrets, detect if the operation
requires approval and return status indicating approval-required or imported
state. This allows the frontend to surface different messaging based on
whether secrets are imported immediately or pending approval.
…erations

Add response schema to updateDopplerExternalMigrationV3 and
deleteDopplerExternalMigrationV3 endpoints showing the returned config object.
Update importDopplerSecretsV3 response to include status field alongside
imported count. Fix type casting for raw secret values to ensure TypeScript
type safety.
…count

Update useImportDopplerSecrets mutation to return the API response object
containing status and imported count, allowing the caller to conditionally
handle approval-required vs imported states.
…andle import status

Upgrade from v2 (Modal, FormControl, Input) to v3 (Dialog, Field, UnstableInput,
Tooltip) components with improved styling. Add useEffect to reset form when modal
opens. Update success notification to show different message based on import
status: either submission for approval or count of imported secrets. Add reset()
call to cleanup handlers.
Upgrade from v2 (Modal, ModalContent, FormControl) to v3 (Dialog,
DialogContent, Field, Tooltip) components. Standardize form structure with
consistent spacing via form className. Update footer layout with DialogFooter
component and proper button ordering (Cancel, Create/Update).
Replace FontAwesome icons with lucide-react (Plus). Migrate custom documentation
badge to DocumentationLinkBadge component from v3. Update semantic color tokens
from mineshaft theme to modern v3 palette (border, card, foreground, muted).
Refactor button to use v3 variant system with proper spacing.
The Doppler mark fills more of its asset bounds than Vault. Use a smaller px
width (36px) for Doppler compared to default 50px to achieve visual size
parity with Vault icon.
…ystem

Upgrade ExternalMigrationsTab modals and supporting components from v2 to v3:
- EnvKeyPlatformModal, GenericDropzone, InPlatformMigrationSection
- SelectImportFromPlatformModal, SelectInPlatformMigrationProviderModal
- VaultNamespaceConfigModal, VaultPlatformModal
- New MigrationConfigDeleteDialog component

Changes include: v2→v3 component imports, updated semantic colors (border,
card, foreground, muted), Dialog/Field/Tooltip replacements for v2 Modal/
FormControl, lucide-react icon usage, improved form spacing and button styling.
@IgorHorta IgorHorta force-pushed the igor/eng-4649-create-migration-tool-from-doppler-to-infisical branch from 8f53bc9 to 8d81e31 Compare March 31, 2026 01:10
@IgorHorta IgorHorta changed the title ENG-4649: Doppler in-platform migration and unified external migrations UI feature: Doppler in-platform migration and unified external migrations UI Mar 31, 2026
…ayer

SQL treats NULL != NULL so the unique constraint never fires for Doppler rows
(which have namespace = null). Add an explicit existence check for the same
orgId + provider + connectionId combination before creating, throwing
BadRequestError if a config already exists.
…indOne filter type

- Rename `vaultConfig` to `existingConfig` in getVaultConnectionForConfig for clarity
- Add `connectionId` to the `findOne` filter type to support Doppler uniqueness check
@IgorHorta IgorHorta marked this pull request as ready for review March 31, 2026 19:05
@IgorHorta IgorHorta requested a review from varonix0 April 1, 2026 01:02
@gitguardian
Copy link
Copy Markdown

gitguardian bot commented Apr 4, 2026

⚠️ GitGuardian has uncovered 2 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
29206045 Triggered Generic Password 02d396a backend/src/ee/services/pki-scep/pki-scep-service.ts View secret
28944329 Triggered PostHog Project API Key 94043e1 docs/docs.json View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secrets safely. Learn here the best practices.
  3. Revoke and rotate these secrets.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@varonix0 varonix0 requested a review from victorvhs017 April 8, 2026 02:57
Comment on lines +25 to +66
if (!(await knex.schema.hasTable(TableName.ExternalMigrationConfig))) {
await knex.schema.createTable(TableName.ExternalMigrationConfig, (t) => {
t.uuid("id", { primaryKey: true }).defaultTo(knex.fn.uuid());
t.uuid("orgId").notNullable();
t.foreign("orgId").references("id").inTable(TableName.Organization).onDelete("CASCADE");
t.string("provider").notNullable();
t.binary("encryptedConfig").notNullable();
t.uuid("connectionId");
t.foreign("connectionId").references("id").inTable(TableName.AppConnection);
t.timestamps(true, true, true);
});

await createOnUpdateTrigger(knex, TableName.ExternalMigrationConfig);

if (await knex.schema.hasTable(TableName.VaultExternalMigrationConfig)) {
const existingVaultConfigs = await knex(TableName.VaultExternalMigrationConfig).select(
selectAllTableCols(TableName.VaultExternalMigrationConfig)
);

await Promise.all(
existingVaultConfigs.map(async (vaultConfig) => {
const { encryptor } = await kmsService.createCipherPairWithDataKey({
orgId: vaultConfig.orgId,
type: KmsDataKey.Organization
});

const config = {
namespace: vaultConfig.namespace
};
const { cipherTextBlob: encryptedConfig } = encryptor({ plainText: Buffer.from(JSON.stringify(config)) });

await knex(TableName.ExternalMigrationConfig).insert({
orgId: vaultConfig.orgId,
provider: ExternalMigrationProviders.Vault,
encryptedConfig,
connectionId: vaultConfig.connectionId
});
})
);
}
}
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This migration is not idempotent. If it fails in one of the promises, the migration will be marked as failed. When we try again, we'll skip the data migration because the table already exists.

We could do this inside a transaction, and a failed promise would also rollback the table creation. But only if we don't have too many existingVaultConfigs to migrate, which I'm assuming is the case here for EU and US.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optional because they were already missing:

Can we add some audit logs for the add, update, and delete external migrations requests?

Comment on lines +94 to +99
while (hasMore && page <= maxPages) {
// eslint-disable-next-line no-await-in-loop
const res = await request.get<{ environments: TDopplerEnvironment[] }>(`${DOPPLER_API_URL}/v3/environments`, {
params: { project: projectSlug, page, per_page: perPage },
headers: { Authorization: `Bearer ${apiToken}` }
});
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://docs.doppler.com/reference/environments-list. Their API doesn't have page or per_page parameters here. This pagination may be dead code.

});
}

return { status: VaultImportStatus.Imported, imported: Object.keys(dopplerSecrets).length };
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should rename this VaultImportStatus to ExternalMigrationImportStatus or something like that

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In v3 components we use yellow for configurations inside the project; here, we should have these buttons blue, as we do in the suborg tab

Image Image

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, and the modal button too:

image

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore the placement of the comment.


I don't understand why we need this configuration here, where we are only selecting the app connection:

Image

Couldn't we just select the app connection here and cut that previous step completely?

Image

Comment on lines +8 to +15
import {
Button,
FilterableSelect,
FormControl,
Modal,
ModalClose,
ModalContent
} from "@app/components/v2";
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we update these to use v3?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are showing these loading indicators as soon as we open the modal, which is a bit misleading. We should show these when we are actually fetching.

Image

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've created this project:

Image

But nothing is showing up here:

Image

I followed the Doppler app connection documentation, and what we are missing there is that when you create a service account, the default role doesn't have any permissions.

Image

It only worked after I added some here:

Image

We should update the docs to make it clear because we recommend the creation of a new service account. Also, we should list the permissions we need to do the migration, then the user can use a least privilege service account.

Comment on lines +93 to +95
<Info>
**First time in this project?** Create a [project](/documentation/platform/project) and environments if you have not already. The overview import always targets the project you are viewing.
</Info>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't understand this part. It seems to be mixing project and environment concepts. I think we can remove this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants