Skip to content

feat: improved ueberdb speed and esm compatibility #952

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
SamTV12345 wants to merge 3 commits into from
Closed

feat: improved ueberdb speed and esm compatibility #952

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f1ec026
feat: improved ueberdb speed and esm compatibility
SamTV12345 May 10, 2026
8aafae4
feat: improved ueberdb speed and esm compatibility
SamTV12345 May 10, 2026
c25d6c6
feat: improved ueberdb speed and esm compatibility
SamTV12345 May 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion databases/cassandra_db.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
* limitations under the License.
*/

import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase';
import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase.ts';
import {Client, types} from 'cassandra-driver';
import type {ArrayOrObject, ValueCallback} from 'cassandra-driver';
type ResultSet = types.ResultSet;
Expand Down
4 changes: 2 additions & 2 deletions databases/couch_db.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@
* limitations under the License.
*/

import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase';
import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase.ts';
import nano from 'nano';
import type {BulkObject} from './cassandra_db';
import type {BulkObject} from './cassandra_db.ts';

export default class Couch_db extends AbstractDatabase {
public db: nano.DocumentScope<string> | null;
Expand Down
2 changes: 1 addition & 1 deletion databases/dirty_db.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
*
*/

import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase';
import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase.ts';
import Dirty from 'dirty-ts';

type DirtyDBCallback = (p?:any, keys?: string[])=>{};
Expand Down
2 changes: 1 addition & 1 deletion databases/dirty_git_db.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase';
import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase.ts';
import {dirname} from 'node:path'
import {simpleGit} from 'simple-git'
/**
Expand Down
4 changes: 2 additions & 2 deletions databases/elasticsearch_db.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,14 @@
* limitations under the License.
*/

import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase';
import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase.ts';
import assert from 'assert';
import {equal} from 'assert';
import {Buffer} from 'buffer';
import {createHash} from 'crypto';
import {Client} from '@elastic/elasticsearch';
import type {estypes} from '@elastic/elasticsearch';
import type {BulkObject} from './cassandra_db';
import type {BulkObject} from './cassandra_db.ts';

const schema = '2';

Expand Down
2 changes: 1 addition & 1 deletion databases/memory_db.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase';
import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase.ts';


export default class MemoryDB extends AbstractDatabase {
Expand Down
2 changes: 1 addition & 1 deletion databases/mock_db.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import type {Settings} from '../lib/AbstractDatabase';
import type {Settings} from '../lib/AbstractDatabase.ts';

import events from 'events';

Expand Down
4 changes: 2 additions & 2 deletions databases/mongodb_db.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@
* limitations under the License.
*/

import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase';
import type {BulkObject} from './cassandra_db';
import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase.ts';
import type {BulkObject} from './cassandra_db.ts';
import {MongoClient} from 'mongodb';
import type {Collection, Db} from 'mongodb';

Expand Down
4 changes: 2 additions & 2 deletions databases/mssql_db.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@
*
*/

import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase';
import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase.ts';
import async from 'async';
import mssql from 'mssql';
import type {ConnectionPool} from 'mssql';
import type {BulkObject} from './cassandra_db';
import type {BulkObject} from './cassandra_db.ts';

type RowResult = {
key: string;
Expand Down
4 changes: 2 additions & 2 deletions databases/mysql_db.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@
* limitations under the License.
*/

import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase';
import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase.ts';
import util from 'util';
import type {BulkObject} from './cassandra_db';
import type {BulkObject} from './cassandra_db.ts';
import {createPool} from 'mysql2';
import type {ConnectionConfig, Pool, QueryError} from 'mysql2';

Expand Down
4 changes: 2 additions & 2 deletions databases/postgres_db.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@
* limitations under the License.
*/

import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase';
import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase.ts';
import async from 'async';
import * as pg from 'pg';
import type {BulkObject} from './cassandra_db';
import type {BulkObject} from './cassandra_db.ts';

export default class extends AbstractDatabase {
public db: pg.Pool;
Expand Down
4 changes: 2 additions & 2 deletions databases/postgrespool_db.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import type {Settings} from '../lib/AbstractDatabase';
import type {Settings} from '../lib/AbstractDatabase.ts';

import Postgres_db from './postgres_db'
import Postgres_db from './postgres_db.ts'

export default class PostgresDB extends Postgres_db {
constructor(settings:Settings) {
Expand Down
4 changes: 2 additions & 2 deletions databases/redis_db.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@
* limitations under the License.
*/

import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase';
import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase.ts';
import {createClient} from 'redis';
import type {RedisClientOptions} from 'redis';
import type {BulkObject} from './cassandra_db';
import type {BulkObject} from './cassandra_db.ts';

export default class RedisDB extends AbstractDatabase {
public _client: any
Expand Down
4 changes: 2 additions & 2 deletions databases/rethink_db.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@
* limitations under the License.
*/

import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase';
import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase.ts';
import r from 'rethinkdb';
import async from 'async';
import type {BulkObject} from './cassandra_db';
import type {BulkObject} from './cassandra_db.ts';

export default class Rethink_db extends AbstractDatabase {
public host: string;
Expand Down
2 changes: 1 addition & 1 deletion databases/rusty_db.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import AbstractDatabase from "../lib/AbstractDatabase";
import AbstractDatabase from "../lib/AbstractDatabase.ts";
import {KeyValueDB} from 'rusty-store-kv'

export default class Rusty_db extends AbstractDatabase {
Expand Down
4 changes: 2 additions & 2 deletions databases/sqlite_db.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import type {BulkObject} from './cassandra_db';
import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase';
import type {BulkObject} from './cassandra_db.ts';
import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase.ts';
import {SQLite} from "rusty-store-kv";

/**
Expand Down
4 changes: 2 additions & 2 deletions databases/surrealdb_db.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@
* limitations under the License.
*/

import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase';
import AbstractDatabase, {type Settings} from '../lib/AbstractDatabase.ts';
import {Surreal} from 'surrealdb';
import type {BulkObject} from './cassandra_db';
import type {BulkObject} from './cassandra_db.ts';

const DATABASE = 'ueberdb';
const WILDCARD = '*';
Expand Down
46 changes: 23 additions & 23 deletions index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,13 @@
* limitations under the License.
*/

import {Database as DatabaseCache, type Metrics} from './lib/CacheAndBufferLayer';
import {normalizeLogger} from './lib/logging';
import type {Settings} from './lib/AbstractDatabase';
import {Database as DatabaseCache, type Metrics} from './lib/CacheAndBufferLayer.ts';
import {normalizeLogger} from './lib/logging.ts';
import type {Settings} from './lib/AbstractDatabase.ts';

export type {Settings} from './lib/AbstractDatabase';
export type {Metrics, CacheSettings} from './lib/CacheAndBufferLayer';
export type {Logger} from './lib/logging';
export type {Settings} from './lib/AbstractDatabase.ts';
export type {Metrics, CacheSettings} from './lib/CacheAndBufferLayer.ts';
export type {Logger} from './lib/logging.ts';

// Database drivers are loaded lazily in initDB() so that only the selected
// backend's dependencies need to be installed.
Expand Down Expand Up @@ -89,44 +89,44 @@ export class Database {
private async initDB(): Promise<any> {
switch (this.type) {
case 'mysql':
return new (await import('./databases/mysql_db')).default(this.dbSettings as Settings);
return new (await import('./databases/mysql_db.ts')).default(this.dbSettings as Settings);
case 'postgres':
return new (await import('./databases/postgres_db')).default(this.dbSettings as Settings);
return new (await import('./databases/postgres_db.ts')).default(this.dbSettings as Settings);
case 'sqlite':
return new (await import('./databases/sqlite_db')).default(this.dbSettings as Settings);
return new (await import('./databases/sqlite_db.ts')).default(this.dbSettings as Settings);
case 'rustydb':
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return new (await import('./databases/rusty_db')).default(this.dbSettings as any);
return new (await import('./databases/rusty_db.ts')).default(this.dbSettings as any);
case 'mongodb':
return new (await import('./databases/mongodb_db')).default(this.dbSettings as Settings);
return new (await import('./databases/mongodb_db.ts')).default(this.dbSettings as Settings);
case 'redis':
return new (await import('./databases/redis_db')).default(this.dbSettings as Settings);
return new (await import('./databases/redis_db.ts')).default(this.dbSettings as Settings);
case 'cassandra':
return new (await import('./databases/cassandra_db')).default(this.dbSettings as Settings);
return new (await import('./databases/cassandra_db.ts')).default(this.dbSettings as Settings);
case 'dirty':
return new (await import('./databases/dirty_db')).default(this.dbSettings as Settings);
return new (await import('./databases/dirty_db.ts')).default(this.dbSettings as Settings);
case 'dirtygit':
return new (await import('./databases/dirty_git_db')).default(this.dbSettings as Settings);
return new (await import('./databases/dirty_git_db.ts')).default(this.dbSettings as Settings);
case 'elasticsearch':
return new (await import('./databases/elasticsearch_db')).default(
return new (await import('./databases/elasticsearch_db.ts')).default(
this.dbSettings as Settings,
);
case 'memory':
return new (await import('./databases/memory_db')).default(this.dbSettings as Settings);
return new (await import('./databases/memory_db.ts')).default(this.dbSettings as Settings);
case 'mock':
return new (await import('./databases/mock_db')).default(this.dbSettings as Settings);
return new (await import('./databases/mock_db.ts')).default(this.dbSettings as Settings);
case 'mssql':
return new (await import('./databases/mssql_db')).default(this.dbSettings as Settings);
return new (await import('./databases/mssql_db.ts')).default(this.dbSettings as Settings);
case 'postgrespool':
return new (await import('./databases/postgrespool_db')).default(
return new (await import('./databases/postgrespool_db.ts')).default(
this.dbSettings as Settings,
);
case 'rethink':
return new (await import('./databases/rethink_db')).default(this.dbSettings as Settings);
return new (await import('./databases/rethink_db.ts')).default(this.dbSettings as Settings);
case 'couch':
return new (await import('./databases/couch_db')).default(this.dbSettings as Settings);
return new (await import('./databases/couch_db.ts')).default(this.dbSettings as Settings);
case 'surrealdb':
return new (await import('./databases/surrealdb_db')).default(this.dbSettings as Settings);
return new (await import('./databases/surrealdb_db.ts')).default(this.dbSettings as Settings);
default:
throw new Error(`Invalid database type: ${this.type as string}`);
}
Expand Down
2 changes: 1 addition & 1 deletion lib/AbstractDatabase.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import {normalizeLogger, type Logger} from './logging';
import {normalizeLogger, type Logger} from './logging.ts';

const nullLogger = normalizeLogger(null);

Expand Down
2 changes: 1 addition & 1 deletion lib/CacheAndBufferLayer.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
*/

import {promisify} from 'node:util';
import type {Logger} from './logging';
import type {Logger} from './logging.ts';

type BulkOp = {
type: 'set' | 'remove';
Expand Down
3 changes: 1 addition & 2 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,6 @@
"surrealdb": "^2.0.3",
"testcontainers": "^11.14.0",
"typescript": "^6.0.3",
"vitest": "^4.1.5",
"wtfnode": "^0.10.1"
},
"repository": {
Expand All @@ -140,7 +139,7 @@
"build": "pnpm run build:js && pnpm run build:types",
"build:js": "pnpm exec rolldown -c rolldown.config.mjs",
"build:types": "pnpm exec tsc --emitDeclarationOnly",
"test": "vitest --test-timeout=120000",
"test": "node --experimental-strip-types --test --test-timeout=120000 test/**/*.spec.ts",
"ts-check": "tsc --noEmit",
Comment on lines 139 to 143
Copy link
Copy Markdown

@qodo-free-for-open-source-projects qodo-free-for-open-source-projects Bot May 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Ci test matrix broken 🐞 Bug ☼ Reliability

The new test script does not handle the existing workflow’s pnpm run test <db> invocation, so
the extra argument will be forwarded to node --test as a test file/pattern and can fail or run
unintended tests. This will break the npmpublish.yml test job matrix and can also cause every
matrix shard to run the entire test suite instead of a single DB target.
Agent Prompt 
## Issue description
The `test` script no longer supports `pnpm run test <db>` used by the GitHub Actions matrix, so the workflow will forward `<db>` to `node --test` as an extra positional argument (file/pattern), breaking CI and eliminating per-DB sharding.

## Issue Context
The workflow intentionally runs one DB suite per job (`matrix.db`). With Vitest this could be achieved by passing an argument; with Node’s test runner you need to either:
- pass an explicit test file/glob per db, or
- use `--test-name-pattern` to select tests by suite name.

## Fix Focus Areas
- package.json[134-145]
- .github/workflows/npmpublish.yml[11-53]

## Suggested fix approach
- Update the `test` script to accept an optional argument and translate it into a file glob (recommended). For example:
  - `pnpm run test couch` => run `test/couch/**/*.spec.ts`
  - `pnpm run test mysql` => run `test/mysql/**/*.spec.ts`
  - `pnpm run test mongo` => run `test/mongodb/**/*.spec.ts`
- Or update the workflow to call `node --test` with the correct file glob directly instead of passing a db argument.
- Ensure `pnpm run test` (no args) still runs the full suite locally if desired.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

"ts-check:watch": "tsc --noEmit --watch"
},
Expand Down
Loading
Loading