Skip to content

index.md

Latest commit

 

History

History
167 lines (104 loc) · 4.41 KB

index.md

File metadata and controls

167 lines (104 loc) · 4.41 KB

LogstashAgent

A control-plane agent for LogstashUI that fully manages the Logstash instance it runs alongside.

⚠️ Beta Release - This project is under active development. Features may change.

Overview

LogstashAgent is the host-side runtime for LogstashUI-managed instances.

It enrolls with LogstashUI, persists local agent state, checks in for policy and configuration changes, and applies those changes directly to the local Logstash installation.

Enrollment + Reconciliation Loop

Enroll with LogstashUI and continuously reconcile desired state to the local Logstash instance.

  • Enrollment mode: python src/logstashagent/main.py --enroll=<TOKEN> --logstash-ui-url=<URL>
  • Controller mode: python src/logstashagent/main.py --run
  • Agent state includes enrollment identity, policy assignment, and revision tracking

Pipeline Management API

Create, update, delete, validate, and inspect Logstash pipelines.

  • Endpoints: /_logstash/pipeline, /_logstash/pipeline/{pipeline_id}, /_logstash/pipeline/{pipeline_id}/logs, /_logstash/pipelines/status
  • Config persistence backed by pipelines.yml, conf.d, and metadata files

Host Configuration Management

Apply managed configuration to local Logstash runtime files and secure settings.

  • Controller updates logstash.yml, jvm.options, log4j2.properties, and keystore entries
  • Supports reconciliation and service restart flows for managed updates

Local State + Credential Protection

Persist agent identity and encrypted sensitive fields under package-local data storage.

  • State file: src/logstashagent/data/state.json
  • Encryption key: src/logstashagent/data/.secret_key
  • Log file: src/logstashagent/data/logs/logstashagent.log

Requirements

Software

For Managed Agent mode:

For Enrolled Controller mode (--run):

  • Python 3.12+
  • Access to managed Logstash settings/log paths
  • Network reachability to your LogstashUI instance

For Local Development:

Quick Start - Agent Mode

Tip

Use --run only after successful enrollment, because controller mode requires persisted enrollment state.

1. Install

cd LogstashAgent
uv sync

2. Configure

Copy and adjust the example config:

cp src/logstashagent/config/logstashagent.example.yml src/logstashagent/config/logstashagent.yml

3. Run agent process

python src/logstashagent/main.py

By default this starts the agent service (including management API) on 0.0.0.0:9600 unless overridden in config.

Enroll and Run Controller

1. Enroll the agent

python src/logstashagent/main.py --enroll=<BASE64_TOKEN> --logstash-ui-url=http://localhost:8080

2. Start controller mode

python src/logstashagent/main.py --run

3. Verify state files

  • src/logstashagent/data/state.json
  • src/logstashagent/data/.secret_key

Documentation

  • Configuration - Configuration options and settings for LogstashAgent
  • General - Build and deployment guides

Updating

Pull latest source and resync dependencies:

git pull
uv sync

Then restart the running agent process.

Limitations

  • Controller behavior depends on available host service managers (systemctl or service) for restart operations
  • Host filesystem permissions must allow managed writes to Logstash settings and metadata paths

Roadmap

  • Hardened host-mode lifecycle and service integration
  • Expanded policy diff/apply visibility and diagnostics
  • Additional keystore and secret-management workflows
  • Broader automated test coverage around simulation and controller reconciliation paths

Reporting Issues

Found a bug or have a feature request? Open an issue.

Contributing

Contributions are welcome.

Please open an issue to discuss large changes before submitting a pull request.

License

Copyright 2024–2026 Elasticsearch and contributors.

Licensed under the Apache License, Version 2.0. See LICENSE for details.