Skip to content

Improve query performance #32

@fcasson

Description

@fcasson
  • Queries on arbitrary metadata fields from CLI should pass a proper DB query directly to the server
  • Current implementation communicates all metadata (minus file metadata) and is not scalable to large databases
  • SQLAlchemy has known performance issues for large databases, see e.g. https://docs.sqlalchemy.org/en/14/faq/performance.html#result-fetching-slowness-core and SimDB may need an API to communicate directly with SQL.
  • Example: For a large database of 20,000 entries, simdb remote list -l 0 takes about 4 seconds, however requesting metadata takes longer, e.g. simdb remote list --limit 0 -m userid -m codeid -m machineid -m shotid -m dateid -m seqid takes 20s.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions