feat(optimizer): add data model — schema, entities, DTOs, converters by mkuchenbecker · Pull Request #527 · linkedin/openhouse

mkuchenbecker · 2026-04-03T18:17:52Z

Summary

PR 0 of 7 in the optimizer stack. Design doc.

Introduces the optimizer service module with the foundational data model layer. No repositories, controllers, or service layer yet -- those follow in subsequent PRs.

Changes

Schema (optimizer-schema.sql): DDL for table_operations, table_stats, table_stats_history, and table_operations_history. Compatible with MySQL (prod) and H2 in MySQL mode (tests).

JPA entities: TableOperationsRow, TableOperationsHistoryRow, TableStatsRow, TableStatsHistoryRow with JSON column support via vladmihalcea hibernate-types.

Models/DTOs/enums: OperationType, OperationStatus, OperationHistoryStatus, TableStats (nested snapshot + delta with addedSizeBytes), JobResult, all request/response DTOs.

Converter: JPA AttributeConverter for JobResult JSON column.

Mapper: MapStruct OptimizerMapper for entity to DTO conversion.

Build wiring: settings.gradle include, build.gradle dockerPrereqs dependency.

Testing Done

Manually Tested on local docker setup. Please include commands ran, and their output.
Added new tests for the changes made.
Updated existing tests to reflect the changes made.
No tests added or updated. Please explain why. If unsure, please feel free to ask for help.
Some other form of testing like staging or soak time in production. Please explain.

This PR contains only the data model (entities, DTOs, converters) with no repositories or service logic to test. Repository tests follow in PR 1. Verified:

./gradlew :services:optimizer:compileJava passes
./gradlew compileJava (full project) passes with no regressions
Spotless formatting passes

Additional Information

Breaking Changes
Deprecations
Large PR broken into smaller PRs, and PR plan linked in the description.

Optimizer Stack

PR	Branch	Content
#527 (this)	`optimizer-0`	Schema, entities, models/DTOs, converters, mapper, build wiring
#tbd	`optimizer-1`	Repos + repo tests + context test
#tbd	`optimizer-2`	REST service + shared optimizer module
#tbd	`optimizer-3`	Stats pipeline: housetables + tables service stats client
#tbd	`optimizer-4`	Analyzer app
#tbd	`optimizer-5`	Scheduler app
#tbd	`optimizer-6`	Spark BatchedOFD app
#tbd	`optimizer-7`	Infra, docker-compose, smoke test, demo

Introduces the optimizer service module with: - MySQL/H2 schema for table_operations, table_stats, table_stats_history, and table_operations_history - JPA entities with JSON column support (vladmihalcea hibernate-types) - All model/DTO/enum types: OperationType, OperationStatus, TableStats, CompleteOperationRequest, JobResult, OperationMetrics, etc. - JPA AttributeConverters for JobResult and OperationMetrics JSON columns - MapStruct mapper (OptimizerMapper) for entity→DTO conversion - Spring Boot application shell and build wiring (settings.gradle, build.gradle dockerPrereqs) No repositories, controllers, or service layer yet — those follow in subsequent PRs. Co-Authored-By: Claude Opus 4.6 <[email protected]>

...ces/optimizer/src/main/java/com/linkedin/openhouse/optimizer/api/model/OperationMetrics.java

services/optimizer/src/main/java/com/linkedin/openhouse/optimizer/api/model/OperationType.java

...s/optimizer/src/main/java/com/linkedin/openhouse/optimizer/api/model/TableOperationsDto.java

...izer/src/main/java/com/linkedin/openhouse/optimizer/api/model/TableOperationsHistoryDto.java

services/optimizer/src/main/java/com/linkedin/openhouse/optimizer/api/model/TableStats.java

services/optimizer/src/main/java/com/linkedin/openhouse/optimizer/api/model/TableStatsDto.java

...optimizer/src/main/java/com/linkedin/openhouse/optimizer/api/model/TableStatsHistoryDto.java

...r/src/main/java/com/linkedin/openhouse/optimizer/api/model/UpsertTableOperationsRequest.java

...imizer/src/main/java/com/linkedin/openhouse/optimizer/api/model/UpsertTableStatsRequest.java

...timizer/src/main/java/com/linkedin/openhouse/optimizer/entity/TableOperationsHistoryRow.java

- Remove OperationMetrics class and converter; stats are read directly from table_stats instead of duplicating into operations - Remove orphanFilesDeleted/orphanBytesDeleted from history entity, DTO, and schema; operation-specific data belongs in the result JSON - Add addedSizeBytes to CommitDelta for tracking write volume - Fix OperationType javadoc to describe current state, not roadmap - Fix TableOperationsHistoryRow javadoc: written on operation complete, not by Spark app directly - Add field comments to all DTOs and request objects Co-Authored-By: Claude Opus 4.6 <[email protected]>