OPENJPA-2940 - Jakarta Persistence 3.2 #142
Open
rzo1 wants to merge 191 commits intoapache:masterfrom
Open
Conversation
* Updated tentative version to 4.2.0-SNAPSHOT * Updated java version to 17
* Updating project to exclude almost all java.security deprecated calls * Passes default profile tests, but fails with postgres * Fixed some non-deterministic tests that fail with postgresql
* Tested and passed XML support using postgresql-17 as target db
* Replacing string number constructors * Removing dangling SecurityContext references
* removed TestSecurityContext because it is terminally deprecated since 17 and already removed in current JDK versions * updated h2-2 test profile jdbc url to remove strict definition * updated openjpa-slice and openjpa-xmlstore pom system variables definitions * updated GH actions workflows to use test-h2-2 profiles
* Project passes tests on derby, h2-2, postgres:latest, mysql:lts, mariadb:lts
* Updated dependency version * Added API new methods to API implementation classes with methods that throw UnsupportedOperationException, except for four methods in EntityManagerImpl that required proper implementations to pass tests * Project is still passing tests on derby and postgresql, at least
* Added XML JPA 3.2 schema and definitions * Added configuration support by 3.2 version * Added SchemaManager impl and corresponding methods in BrokerFactory interface * Added concrete working (not for h2-2) implementation of SchemaManager methods for JDBCBrokerFactory * Added concrete working impl for EMF#getName()
* Reverting unnecessary changes * Fixing broken map synchronization
* Changing signature of BrokerFactory API on schema dealing validate method * Adding test to check if validate operation throws exception when it fails * Changing GH CI workflow to allow usage of both self-hosted and GH hosted runners * Tested on derby, h2-2, postgresql:latest, mysql:lts, mariadb:lts
* Implementing emf creation passing PersistenceConfiguration
* Removing unused import in BrokerImpl * Implemented new PersistenceUnitUtil load methods
* Moved PUU loading tests to test unit already present * Updated test unit to junit 4.x format
…UUID and VARCHAR columns Override setUnknown() and setTyped() in PostgresDictionary to bind UUID values as string with Types.OTHER. The PG JDBC driver sends this as PostgreSQL's "unknown" pseudo-type (OID 705), which PostgreSQL can implicitly cast to both native UUID and VARCHAR column types. This is needed because PostgreSQL cannot implicitly cast between its native UUID type and VARCHAR in WHERE comparisons: - setObject(idx, uuid, Types.OTHER) fails on VARCHAR columns - setString(idx, uuid.toString()) fails on native UUID columns - setObject(idx, uuid.toString(), Types.OTHER) works for both Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…iantSchemaGeneration When SpecCompliantSchemaGeneration is true (TCK mode), use contains() so the dropped-table entry persists across multiple buildSchema EMFs within the same schema generation flow. When false (normal mode), use remove() so the entry is consumed and subsequent test classes can re-create the table.
…uotes from numeric identifiers The toDBName() override unconditionally stripped double-quote delimiters from all identifiers to fix case-sensitivity issues. However, identifiers that start with a digit (e.g., column name "1") require quoting in PostgreSQL DDL. Now only strips quotes when the unquoted form starts with a letter. Fixes 5 test classes: TestResultClsAnnotation, TestResultClsXml, TestSimpleXmlEntity, TestXmlOverrideEntity, TestMixedMappingLocation.
…ripping quotes The previous fix only checked the first character of quoted identifiers. Identifiers containing spaces (e.g., "nsc DelSet") were incorrectly unquoted, producing invalid SQL on PostgreSQL. Now validates all characters match PostgreSQL's unquoted identifier rules.
…CompliantSchemaGeneration The unconditional SCHEMA_CASE_LOWER caused convertSchemaCase() to lowercase delimited identifiers during metadata lookups. PostgreSQL preserves case for quoted identifiers, so mixed-case tables like "nsc DelSet" were not found when querying metadata with "nsc delset". This prevented proper DROP TABLE during schema synchronization. Now only forces LOWER in TCK mode (where it prevents "ITEM" vs item duplicate-table issues). In normal mode, uses PRESERVE so quoted identifiers with mixed case work correctly.
Test asserted delimitedCase=LOWER which is now only active in TCK mode. Normal PostgreSQL mode uses PRESERVE, making these assertions invalid. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Contributor
|
Working here to find and fix some regressions on pg. Do we have a target db clearance to release? I'm working on pg, h2(2.x), mariadb and mysql. All but h2-2 still failing. |
Author
I think PG17 should be fine with the next commit (at least on my PG17 runs, I only have 2 proxy failures similar to master) - but feel free to just commit direclty on this branch. I didn't look into mariadb and mysql though. |
Author
I think that we should come to a state in which we do not have regressions for existing users (but that is from the side line :) ) |
Contributor
|
I still get some errors with pg-17 (schemavalidation test is my current bisect focus). mariadb and mysql have much more errors or failures. |
Author
Should be gone with b6fde0b - mariadb and mysql might miss some dictionary updates for the new spec parts. Two things: 1.) Maybe we can share the surefire reports somewhere? Running the > 45min tests all the time is a bit cumbersome or do we have ASF Jenkins CI for it and can configure a run from this PR with multi databases? 2.) I am going to stop working on this branch for different rdbms for now so we avoid doing things multiple times. If you have any numbers or surefire reports for mariadb / mysql / pg17 (the few errors which are left), than feel free to drop them somewhere, so I can take another round :) |
…le support 4.30 shades ASM 9.9.1 which supports Java 25 (classfile major version 69). Resolves 'Unsupported class file major version 69' enhancer errors when running on JDK 25.
# Conflicts: # pom.xml
TestSchemaGenAnnotations and TestSchemaGenerationScripts hardcoded ANSI ADD CONSTRAINT ... FOREIGN KEY / DROP CONSTRAINT syntax, but MariaDB and MySQL legitimately emit the equivalent ADD FOREIGN KEY <name> / DROP FOREIGN KEY <name> form (constraintNameMode = CONS_NAME_MID, intentional in the MySQL/MariaDB dictionaries). Accept either form in the assertions.
The jpql.treatjoinon.TCustomer entity used @table(name="TCUSTOMER") which collides with xmlmapping.entities.Customer (composite EmbeddedId with NOT NULL countryCode). On shared-DB test infrastructures (MariaDB, PostgreSQL), this caused schema drop/recreate cycles to fail with 'ALTER TABLE TCUSTOMER DROP COLUMN id' since id is a PK in the xmlmapping schema. Rename to TTREAT_CUSTOMER to isolate the tables.
MariaDB Connector/J 3.5.4 reports per-row failure indexes correctly (unlike older Oracle/older drivers that flag the first row of the batch). A prior commit mistakenly grouped MariaDB with Oracle/Postgres in the 'first-row reported' assertion branch. Use the generic per-row assertion for MariaDB.
Instant, LocalDateTime, Date, and Calendar version columns left column precision unset, producing DATETIME(0) on MariaDB/MySQL by default. Two updates within the same whole second stored identical versions, defeating optimistic locking. Request 6 fractional digits (microseconds) in ColumnVersionStrategy.map() for temporal javaTypes when the user has not already specified precision. DATETIME(6) / TIMESTAMP(6) matches what Derby/PostgreSQL already use.
columns" This reverts commit 76e8293.
…B/MySQL MariaDB >= 10.2 and MySQL >= 5.7 support fractional-second precision on DATETIME/TIME columns. Bump dateFractionDigits to 6 inside the existing version-gated block so temporal columns use DATETIME(6)/TIME(6) by default. This fixes optimistic-locking on @Version Instant/LocalDateTime columns where two updates within the same whole second otherwise stored identical versions. Matches PostgreSQL and Derby defaults.
…r.proxy SingleFieldManager.proxy assumed the datastore value for a @TeMPOraL field always matches the field's declared type. Handler strategies (and some JDBC driver paths) can return a Calendar for a java.util.Date field or a Date for a Calendar field, which caused a ClassCastException at runtime (e.g. TCK StoredProcedureQuery Client1 where a GregorianCalendar proxy was assigned to an Employee.hireDate Date field). Coerce between Calendar and Date on store so the proxy receives a value of the correct type regardless of which the handler produced. Also fix a copy/paste bug in ProxyManagerImpl.generateAndLoadProxyCalendar which passed ProxyDate.class as the ancestor class used to resolve the proxy class loader's package/protection domain.
…Connection when tx is active Per JPA 3.2, the Connection handed to ConnectionConsumer/ConnectionFunction is on loan to the user code; its lifecycle belongs to the EntityManager/Broker when a transaction is active. Previously, the finally blocks in both EntityManagerImpl.runWithConnection and callWithConnection unconditionally closed the returned wrapper (a ClientConnection around the broker's RefCountConnection). On most drivers (Derby, H2, PostgreSQL) the close path is benign because RefCountConnection._retain is true during an active tx so free() is not invoked. MariaDB Connector/J 3.5.4, however, reacts more aggressively when its connection wrapper is closed inside an active transaction and discards pending work on the underlying connection, causing manual INSERTs issued from within runWithConnection to be lost. This manifests as TestCascadeManyToOneAndEM2.testRunWithConnection / testCallWithConnection failing on MariaDB with "Order should be found ...". Fix: only close the user-facing connection when no transaction is active, i.e. when the user borrowed a free-standing connection. During an active transaction the EM/broker retains ownership of the connection.
MySQL and MariaDB don't support ANSI 'NULLS FIRST' / 'NULLS LAST' on ORDER BY. Route the ordering-suffix emission through DBDictionary.appendNullsPrecedence so dialects can customize. Default implementation emits 'NULLS FIRST' / 'NULLS LAST' unchanged. MySQLDictionary and MariaDBDictionary override to emulate via the '<expr> IS NULL <sort>' auxiliary sort key. The emulation is skipped when MySQL's default NULL ordering (NULLs first for ASC, NULLs last for DESC) already matches the requested precedence.
…DK 24+ Ant's non-forked <java> task installs a SecurityManager to catch System.exit(), but System.setSecurityManager() throws UnsupportedOperationException on JDK 24+. The generator would fail silently (no failonerror), producing no build-time proxy classes and causing TestDetachNoProxy legacy-compat assertions to fail because runtime proxies are never tagged detachable=true. Add fork=true and failonerror=true so the generator runs in a separate JVM and surfaces any errors.
…r config - docker-compose-test-pg17.yml (PostgreSQL 17) - docker-compose-test-mariadb.yml (MariaDB 11.4) - docker-compose-test-mysql.yml (MySQL 8.4) The test-mysql-docker profile still used the obsolete mysql:mysql-connector-java GAV and com.mysql.jdbc.Driver class name. Update to com.mysql:mysql-connector-j and com.mysql.cj.jdbc.Driver so modern MySQL Connector/J (9.x) resolves and connects.
MySQL stores BOOLEAN columns as TINYINT(1). By default, MySQL Connector/J reports them as Types.BIT, which OpenJPA's MappingInfo.mergeColumn coerces as "numeric" and silently upgrades to VARCHAR — defeating SchemaManager.validate() detection of a genuine VARCHAR->BOOLEAN schema drift (TestJDBCSchemaManager.testValidate). Setting transformedBitIsBoolean=true makes Connector/J report TINYINT(1) as Types.BOOLEAN, which isNumericType() excludes. This matches MariaDB Connector/J's default behavior where the same test already passes.
Two PostgreSQL-specific fixes needed to make TestJDBCSchemaManager pass: 1. PostgreSQL's JDBC driver reports native boolean columns as Types.BIT. OpenJPA's generic column-type comparison treats BIT as numeric and silently coerces any VARCHAR mapping to 'upgrade' such columns to VARCHAR, which defeats SchemaManager.validate() detection of a genuine VARCHAR -> BOOLEAN schema drift. Override newColumn() to translate reflected columns whose native type name is 'bool' or 'boolean' back to Types.BOOLEAN so the type comparison treats them as their true logical type. 2. IDENTITY-column sequences are dropped implicitly with their owning table, but OpenJPA's schema reflection may still list the sequence as extant. Override getDropSequenceSQL to emit 'DROP SEQUENCE IF EXISTS' so refresh / truncate actions are idempotent and don't fail with 'sequence "..." does not exist' on PostgreSQL 17.
Truncate is a best-effort operation - the repo SchemaGroup may declare tables the dialect never physically created (e.g. OPENJPA_SEQUENCE_TABLE on dialects that prefer native sequences, or tables owned by generators that were never exercised, or tables qualified by a schema that's not on the connection's current search_path). Set _ignoreErrs around the DELETE FROM statements so individual missing tables are logged as warnings instead of aborting the whole truncate. Fixes TestJDBCSchemaManager.testTruncate on PostgreSQL 17.
…testing MariaDB uses 3306. Putting MySQL on 3307 externally (still 3306 inside the container) lets both run simultaneously on the same host, enabling parallel cross-DB test runs.
Author
|
Did look into MariaDB / MySQL and the local build on my OSX looks good. Maybe some one can some additional runs for the tests? 61e93dd might be worth a direct port to |
Contributor
|
@rzo1, thanks for the effort. I'll run on mariadb, mysql and pg-18 tomorrow. As soon I have some results, I'll post here. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi all, @solomax, @cristof, all
This PR builds on @cristof's JPA 3.2 work, completed with the assistance of generative AI (Claude).
It fully passes the JPA 3.2 TCK (at least on PostgreSQL).
Tests run on Java 21, though there are some local failures - similar to what I see on master, likely an OSX or Java 21 issue rather than something introduced here:
The PR is fairly large, so feel free to cherry-pick whatever is useful. I expect there are concerns and things to address before this is merge-ready.
A number of new tests have been added that mirror TCK behavior. Happy to drop those if they're unwanted.
That said, it was a fun three-week experiment :)