-
Notifications
You must be signed in to change notification settings - Fork 119
select/logsql: unify memory guard for large interactive log queries #1249
Description
Is your feature request related to a problem? Please describe
VictoriaLogs has memory guards in several query-processing paths such as sort, stats, top, uniq, facets, running_stats, and stream_context.
But the optimized /select/logsql/query?limit=N path for returning the last N logs does not appear to use the same kind of explicit memory guard. So large limit values from interactive clients such as vmui or Grafana datasource can trigger a backend path that keeps too many rows in memory and may lead to very high memory usage or even OOM/backend crashes.
Related to: VictoriaMetrics/victorialogs-datasource#613, #500
Describe the solution you'd like
VictoriaLogs should have a unified memory-guard strategy across all query execution paths, including the optimized limit query-arg path.
If a query would require too much memory, it should fail with a clear error instead of risking backend instability.