Skip to content

select/logsql: unify memory guard for large interactive log queries #1249

@func25

Description

@func25

Is your feature request related to a problem? Please describe

VictoriaLogs has memory guards in several query-processing paths such as sort, stats, top, uniq, facets, running_stats, and stream_context.

But the optimized /select/logsql/query?limit=N path for returning the last N logs does not appear to use the same kind of explicit memory guard. So large limit values from interactive clients such as vmui or Grafana datasource can trigger a backend path that keeps too many rows in memory and may lead to very high memory usage or even OOM/backend crashes.

Related to: VictoriaMetrics/victorialogs-datasource#613, #500

Describe the solution you'd like

VictoriaLogs should have a unified memory-guard strategy across all query execution paths, including the optimized limit query-arg path.

If a query would require too much memory, it should fail with a clear error instead of risking backend instability.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions