Skip to content

Add downloading of changed element file and changed element property in get changeset links #333

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Draighneys wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Add downloading of changed element file and changed element property in get changeset links #333

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
348c8c5
first simple implementation
Draighneys Jan 21, 2026
b2f30e4
added public download method without tests
Draighneys Jan 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions clients/imodels-client-authoring/src/base/types/apiEntities/ChangesetInterfaces.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,5 @@ import { DownloadedFileProps } from "../CommonInterfaces";

/** Changeset metadata along with the downloaded file path. */
export type DownloadedChangeset = Changeset & DownloadedFileProps;

export type DownloadedChangedElements = DownloadedChangeset;
6 changes: 6 additions & 0 deletions clients/imodels-client-authoring/src/operations/changeset/ChangesetOperationParams.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import {

import {
DownloadProgressParam,
GenericAbortSignal,
RetryParams,
TargetDirectoryParam,
} from "../../base/types";
Expand Down Expand Up @@ -59,6 +60,11 @@ export interface CreateChangesetParams extends IModelScopedOperationParams {
changesetProperties: ChangesetPropertiesForCreate;
}

export type DownloadChangedElementsFileParams = GetSingleChangesetParams &
TargetDirectoryParam & {
abortSignal?: GenericAbortSignal;
};

/** Parameters for single Changeset download operation. */
export type DownloadSingleChangesetParams = GetSingleChangesetParams &
TargetDirectoryParam &
Expand Down
65 changes: 65 additions & 0 deletions clients/imodels-client-authoring/src/operations/changeset/ChangesetOperations.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import {

import {
DownloadProgressParam,
DownloadedChangedElements,
DownloadedChangeset,
GenericAbortSignal,
RetryParams,
Expand All @@ -31,6 +32,7 @@ import {
CreateChangesetParams,
DownloadChangesetListParams,
DownloadSingleChangesetParams,
DownloadChangedElementsFileParams,
} from "./ChangesetOperationParams";
import { LimitedParallelQueue } from "./LimitedParallelQueue";

Expand Down Expand Up @@ -127,6 +129,24 @@ export class ChangesetOperations<
return this.downloadChangeset({ ...params, changeset });
}

/**
* Downloads a changed elements file from a changeset identified by either index or id. If the file does not exist
* in the changeset, an error is thrown. This operation does not retry the download on failure and does not check if the
* file already exists in the target directory.
* @param {DownloadSingleChangesetParams} params parameters for this operation. See {@link DownloadSingleChangesetParams}.
* @returns downloaded changed element. See {@link DownloadedChangedElements}.
*/
public async downloadChangedElements(
params: DownloadChangedElementsFileParams
): Promise<DownloadedChangedElements> {
await this._options.localFileSystem.createDirectory(
params.targetDirectoryPath
);

const changeset: Changeset = await this.querySingleInternal(params);
return this.downloadChangedElementsFile({ ...params, changeset });
}

/**
* Downloads Changeset list. Internally the method uses {@link ChangesetOperations.getRepresentationList} to query the
* Changeset collection so this operation supports most of the the same url parameters to specify what Changesets to
Expand Down Expand Up @@ -214,6 +234,47 @@ export class ChangesetOperations<
};
}

private async downloadChangedElementsFile(
Copy link
Copy Markdown
Author

@Draighneys Draighneys Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we will need fileSize for the changed element file if we want to have a retry policy.

params: IModelScopedOperationParams &
DownloadChangedElementsFileParams & { changeset: Changeset }
): Promise<DownloadedChangedElements> {
const changedElementsWithPath: DownloadedChangedElements = {
...params.changeset,
filePath: path.join(
params.targetDirectoryPath,
this.createChangedElementsFileName(params.changeset.id)
),
};

let loggedError: Error | undefined;
try {
Copy link
Copy Markdown
Author

@Draighneys Draighneys Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to delete the file if it exists

Copy link
Copy Markdown
Contributor

@azonys azonys Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we won't have retry then I would say yes, because existing file may be incomplete.

const downloadLink = params.changeset._links.download;
assertLink(downloadLink);
await downloadFile({
storage: this._options.cloudStorage,
localPath: params.targetDirectoryPath,
abortSignal: params.abortSignal,
url: downloadLink.href,
storageType: downloadLink.storageType,
});
return changedElementsWithPath;
} catch (error) {
this.throwIfAbortError(error, params.changeset, params.abortSignal);
if (error instanceof Error) loggedError = error;
throw new IModelsErrorImpl({
code: IModelsErrorCode.ChangedElementsDownloadFailed,
message: `Failed to download changedElements File. Changeset id: ${
params.changeset.id
}, changeset index: ${params.changeset.index}, error: ${JSON.stringify(
loggedError
)}.`,
originalError: loggedError,
statusCode: undefined,
details: undefined,
});
}
}

private async downloadChangeset(
params: IModelScopedOperationParams &
TargetDirectoryParam & { changeset: Changeset } & DownloadProgressParam &
Expand Down Expand Up @@ -344,6 +405,10 @@ export class ChangesetOperations<
return `${changesetId}.cs`;
}

private createChangedElementsFileName(changesetId: string): string {
return `${changesetId}.ndjson.gz`;
}

private async provideDownloadCallbacks(
params: DownloadChangesetListParams
): Promise<[DownloadCallback, DownloadFailedCallback] | undefined> {
Expand Down
1 change: 1 addition & 0 deletions clients/imodels-client-management/src/base/types/IModelsErrorInterfaces.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ export enum IModelsErrorCode {
BriefcaseNotFound = "BriefcaseNotFound",
CannotAcquire = "CannotAcquire",
ChangesetDownloadFailed = "ChangesetDownloadFailed",
ChangedElementsDownloadFailed = "ChangedElementsDownloadFailed",
ChangesetExists = "ChangesetExists",
ChangesetExtendedDataNotFound = "ChangesetExtendedDataNotFound",
ChangesetGroupNotFound = "ChangesetGroupNotFound",
Expand Down
5 changes: 5 additions & 0 deletions clients/imodels-client-management/src/base/types/apiEntities/ChangesetInterfaces.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,11 @@ export interface ChangesetLinks extends MinimalChangesetLinks {
currentOrPrecedingCheckpoint: Link | null;
/** Link from where to download the Changeset file. Link points to a remote storage. */
download: StorageLink | null;
/** Link from where to download the changed elements file. Link points to a remote storage.
*
* Note: This property is experimental.
*/
changedElements: StorageLink | null;
/**
* Link where to upload the Changeset file. Link points to a remote storage. IMPORTANT: this link
* is never present in any of the Changeset instances returned from methods in this client. This property
Expand Down
5 changes: 5 additions & 0 deletions tests/imodels-clients-tests/src/integration/authoring/ChangesetOperations.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,7 @@ describe("[Authoring] ChangesetOperations", () => {
expectedLinks: {
namedVersion: false,
checkpoint: false,
elements: false,
},
isGetResponse: false,
});
Expand Down Expand Up @@ -229,6 +230,7 @@ describe("[Authoring] ChangesetOperations", () => {
expectedLinks: {
namedVersion: false,
checkpoint: false,
elements: false,
},
isGetResponse: false,
});
Expand Down Expand Up @@ -285,6 +287,7 @@ describe("[Authoring] ChangesetOperations", () => {
expectedLinks: {
namedVersion: changesetHasNamedVersion,
checkpoint: true,
elements: true,
},
expectedTestChangesetFile: testChangesetFile,
});
Expand Down Expand Up @@ -349,6 +352,7 @@ describe("[Authoring] ChangesetOperations", () => {
expectedLinks: {
namedVersion: changesetHasNamedVersion,
checkpoint: true,
elements: true,
},
expectedTestChangesetFile: testChangesetFile,
});
Expand Down Expand Up @@ -423,6 +427,7 @@ describe("[Authoring] ChangesetOperations", () => {
expectedLinks: {
namedVersion: changesetHasNamedVersion,
checkpoint: true,
elements: true,
},
expectedTestChangesetFile: testChangesetFile,
});
Expand Down
3 changes: 3 additions & 0 deletions tests/imodels-clients-tests/src/integration/management/ChangesetOperations.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -267,6 +267,7 @@ describe("[Management] ChangesetOperations", () => {
expectedLinks: {
namedVersion: true,
checkpoint: true,
elements: true,
},
isGetResponse: true,
});
Expand Down Expand Up @@ -308,6 +309,7 @@ describe("[Management] ChangesetOperations", () => {
expectedLinks: {
namedVersion: true,
checkpoint: true,
elements: true,
},
isGetResponse: true,
});
Expand Down Expand Up @@ -349,6 +351,7 @@ describe("[Management] ChangesetOperations", () => {
expectedLinks: {
namedVersion: changesetHasNamedVersion,
checkpoint: true,
elements: true,
},
isGetResponse: true,
});
Expand Down
7 changes: 7 additions & 0 deletions utils/imodels-client-test-utils/src/assertions/NodeOnlyAssertions.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@ export async function assertChangeset(params: {
expectedLinks: {
namedVersion: boolean;
checkpoint: boolean;
elements: boolean;
};
isGetResponse: boolean;
}): Promise<void> {
Expand Down Expand Up @@ -151,6 +152,11 @@ export async function assertChangeset(params: {
shouldLinkExist: params.expectedLinks.checkpoint,
});

assertOptionalLink({
actualLink: params.actualChangeset._links.changedElements,
shouldLinkExist: params.expectedLinks.elements,
});

if (params.isGetResponse) {
expect(params.actualChangeset._links.download).to.exist;
expect(params.actualChangeset._links.download!.href).to.not.be.empty;
Expand All @@ -173,6 +179,7 @@ export async function assertDownloadedChangeset(params: {
expectedLinks: {
namedVersion: boolean;
checkpoint: boolean;
elements: boolean;
};
}): Promise<void> {
await assertChangeset({
Expand Down
Loading