Skip to content

Commit 3ac393f

Browse files
committed
Add notices for ES output options not available when using Beat receivers
1 parent 6eca2c1 commit 3ac393f

2 files changed

Lines changed: 29 additions & 80 deletions

File tree

reference/fleet/elasticsearch-output.md

Lines changed: 23 additions & 80 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,19 @@ Token-based authentication is required in an [{{serverless-full}}](/deploy-manag
4242
::::
4343
4444
45+
## Beat receivers compatibility [elasticsearch-output-beat-receivers]
46+
47+
```{applies_to}
48+
stack: ga 9.3+
49+
```
50+
51+
Starting with version 9.3, {{agent}} uses [Beat receivers](/reference/fleet/elastic-agent-as-otel-collector.md#beat-receivers) to run Beat inputs within an embedded OpenTelemetry Collector. Some {{es}} output parameters are not supported when using Beat receivers:
52+
53+
* `escape_html`: Not supported.
54+
* `allow_older_versions`: Not applicable. Beat receivers always allow connections to older {{es}} versions.
55+
56+
For more details on Beat receivers, refer to [{{agent}} as an OTel Collector](/reference/fleet/elastic-agent-as-otel-collector.md).
57+
4558
## {{es}} output configuration settings [_es_output_configuration_settings]
4659

4760
The `elasticsearch` output type supports the following settings, grouped by category. Many of these settings have sensible defaults that allow you to run {{agent}} with minimal configuration.
@@ -237,6 +250,12 @@ The service principal name for the {{es}} instance is constructed from these opt
237250

238251
**Default:** `true`
239252

253+
::::{note}
254+
:applies_to: stack: ga 9.3.0+
255+
256+
This setting is not applicable when using [Beat receivers](elastic-agent://reference/fleet/elastic-agent-as-otel-collector.md#beat-receivers) with {{agent}}. Beat receivers always allow connections to older {{es}} versions.
257+
::::
258+
240259
`status_reporting.enabled` $$$output-elasticsearch-fleet-settings-status_reporting.enabled-setting$$$
241260
: (boolean) Whether status reporting is enabled for this output. When disabled, the output does not change its health status if there is a connectivity problem.
242261

@@ -251,88 +270,12 @@ Settings used to parse, filter, and transform data.
251270

252271
**Default:** `false`
253272

254-
`pipeline` $$$output-elasticsearch-pipeline-setting$$$
255-
: (string) A format string value that specifies the [ingest pipeline](/manage-data/ingest/transform-enrich/ingest-pipelines.md) to write events to.
256-
257-
```yaml
258-
outputs:
259-
default:
260-
type: elasticsearch
261-
hosts: ["http://localhost:9200"]
262-
pipeline: my_pipeline_id
263-
```
264-
265-
You can set the ingest pipeline dynamically by using a format string to access any event field. For example, this configuration uses a custom field, `fields.log_type`, to set the pipeline for each event:
266-
267-
```yaml
268-
outputs:
269-
default:
270-
type: elasticsearch
271-
hosts: ["http://localhost:9200"]
272-
pipeline: "%{[fields.log_type]}_pipeline"
273-
```
274-
275-
With this configuration, all events with `log_type: normal` are sent to a pipeline named `normal_pipeline`, and all events with `log_type: critical` are sent to a pipeline named `critical_pipeline`.
276-
277-
::::{tip}
278-
To learn how to add custom fields to events, see the `fields` option.
273+
::::{note}
274+
:applies_to: stack: ga 9.3.0+
275+
276+
This setting is not supported when using [Beat receivers](/reference/fleet/elastic-agent-as-otel-collector.md#beat-receivers) with {{agent}}.
279277
::::
280278

281-
See the `pipelines` setting for other ways to set the ingest pipeline dynamically.
282-
283-
`pipelines` $$$output-elasticsearch-pipelines-setting$$$
284-
: An array of pipeline selector rules. Each rule specifies the [ingest pipeline](/manage-data/ingest/transform-enrich/ingest-pipelines.md) to use for events that match the rule. During publishing, {{agent}} uses the first matching rule in the array. Rules can contain conditionals, format string-based fields, and name mappings. If the `pipelines` setting is missing or no rule matches, the `pipeline` setting is used.
285-
286-
Rule settings:
287-
288-
**`pipeline`**
289-
: The pipeline format string to use. If this string contains field references, such as `%{[fields.name]}`, the fields must exist, or the rule fails.
290-
291-
**`mappings`**
292-
: A dictionary that takes the value returned by `pipeline` and maps it to a new name.
293-
294-
**`default`**
295-
: The default string value to use if `mappings` does not find a match.
296-
297-
**`when`**
298-
: A condition that must succeed in order to execute the current rule.
299-
300-
All the conditions supported by processors are also supported here.
301-
302-
The following example sends events to a specific pipeline based on whether the `message` field contains the specified string:
303-
304-
```yaml
305-
outputs:
306-
default:
307-
type: elasticsearch hosts: ["http://localhost:9200"]
308-
pipelines:
309-
- pipeline: "warning_pipeline"
310-
when.contains:
311-
message: "WARN"
312-
- pipeline: "error_pipeline"
313-
when.contains:
314-
message: "ERR"
315-
```
316-
317-
The following example sets the pipeline by taking the name returned by the `pipeline` format string and mapping it to a new name that’s used for the pipeline:
318-
319-
```yaml
320-
outputs:
321-
default:
322-
type: elasticsearch
323-
hosts: ["http://localhost:9200"]
324-
pipelines:
325-
- pipeline: "%{[fields.log_type]}"
326-
mappings:
327-
critical: "sev1_pipeline"
328-
normal: "sev2_pipeline"
329-
default: "sev3_pipeline"
330-
```
331-
332-
With this configuration, all events with `log_type: critical` are sent to `sev1_pipeline`, all events with `log_type: normal` are sent to a `sev2_pipeline`, and all other events are sent to `sev3_pipeline`.
333-
334-
335-
336279
## HTTP settings [output-elasticsearch-http-settings]
337280

338281
Settings that modify the HTTP requests sent to {{es}}.

reference/fleet/es-output-settings.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,12 @@ Specify these settings to send data over a secure connection to {{es}}. In the {
8585

8686
**Default:** `true`
8787

88+
::::{note}
89+
:applies_to: stack: ga 9.3.0+
90+
91+
This setting is not applicable when using [Beat receivers](elastic-agent://reference/fleet/elastic-agent-as-otel-collector.md#beat-receivers) with {{agent}}. Beat receivers always allow connections to older {{es}} versions.
92+
::::
93+
8894
`backoff.init` $$$output-elasticsearch-fleet-settings-backoff.init-setting$$$
8995
: (string) The number of seconds to wait before trying to reconnect to {{es}} after a network error. After waiting `backoff.init` seconds, {{agent}} tries to reconnect. If the attempt fails, the backoff timer is increased exponentially up to `backoff.max`. After a successful connection, the backoff timer is reset.
9096

0 commit comments

Comments
 (0)