Docs: Deprecate Elasticsearch output options not available for Beat receivers

[vishaangelova]

Proposed commit message

This PR deprecates the escape_html, allow_older_versions, and non_indexable_policy.dead_letter_index ES output options starting from 9.5+ as these are not applicable/supported when using Beat receivers.

Contributes to #47537
Related PR: elastic/docs-content#6217

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
• I have added an entry in ./changelog/fragments using the changelog tool.

Preview

• docs/reference/auditbeat/elasticsearch-output.md
• docs/reference/filebeat/elasticsearch-output.md
• docs/reference/heartbeat/elasticsearch-output.md
• docs/reference/metricbeat/elasticsearch-output.md
• docs/reference/packetbeat/elasticsearch-output.md
• docs/reference/winlogbeat/elasticsearch-output.md

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)
• /test : Run the Buildkite pipeline.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: c8eae2c8-63f1-4aa4-858c-88cd3107c177

📥 Commits

Reviewing files that changed from the base of the PR and between 2a9c218 and 0d3007e.

📒 Files selected for processing (5)

• docs/reference/filebeat/elasticsearch-output.md
• docs/reference/heartbeat/elasticsearch-output.md
• docs/reference/metricbeat/elasticsearch-output.md
• docs/reference/packetbeat/elasticsearch-output.md
• docs/reference/winlogbeat/elasticsearch-output.md

✅ Files skipped from review due to trivial changes (1)

• docs/reference/metricbeat/elasticsearch-output.md

🚧 Files skipped from review as they are similar to previous changes (3)

• docs/reference/packetbeat/elasticsearch-output.md
• docs/reference/heartbeat/elasticsearch-output.md
• docs/reference/winlogbeat/elasticsearch-output.md

---

📝 Walkthrough

Hidden review stack artifact

Walkthrough

This PR updates Elasticsearch output reference docs for Auditbeat, Filebeat, Heartbeat, Metricbeat, Packetbeat, and Winlogbeat. It adds {applies_to} deprecation metadata marking escape_html, pipeline, pipelines, and allow_older_versions as stack: deprecated 9.5.0+. It replaces prior beta warnings for non_indexable_policy.dead_letter_index with {applies_to} metadata (stack: deprecated 9.5.0+, beta 9.0-9.4) and adds guidance to use the Elasticsearch data stream failure store and notes about Beat receiver incompatibility.

🚥 Pre-merge checks | ✅ 2

✅ Passed checks (2 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 47537-deprecate-output-parameters

• 🛠️ Update Documentation: Commit on current branch
• 🛠️ Update Documentation: Create PR

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

• Generate code and open pull requests
• Plan features and break down work
• Investigate incidents and troubleshoot customer tickets together
• Automate recurring tasks and respond to alerts with triggers
• Summarize progress and report instantly

Built for teams:

• Shared memory across your entire org—no repeating context
• Per-thread sandboxes to safely plan and execute work
• Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/reference/auditbeat/elasticsearch-output.md`:
- Line 483: The docs-content://...#beat-receivers anchor used in the note at the
top of the Elasticsearch output docs fails redirect-anchor resolution; replace
the problematic target
(docs-content://reference/fleet/elastic-agent-as-otel-collector.md#beat-receivers)
with a stable, resolvable link/anchor (for example the target page without the
docs-content:// scheme plus the correct section anchor or the canonical relative
path and anchor that the site build recognizes) and update the identical note in
the other modified Beat docs files in this PR to use the same resolvable
link/anchor so all instances resolve cleanly during the docs build.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: 8ffe358b-63de-48e0-8c4f-bfcaa72b8d78

📥 Commits

Reviewing files that changed from the base of the PR and between 80fb8bc and 97e2f33.

📒 Files selected for processing (6)

• docs/reference/auditbeat/elasticsearch-output.md
• docs/reference/filebeat/elasticsearch-output.md
• docs/reference/heartbeat/elasticsearch-output.md
• docs/reference/metricbeat/elasticsearch-output.md
• docs/reference/packetbeat/elasticsearch-output.md
• docs/reference/winlogbeat/elasticsearch-output.md

[github-actions]

🔍 Preview links for changed docs

• docs/reference/auditbeat/elasticsearch-output.md
• docs/reference/filebeat/elasticsearch-output.md
• docs/reference/heartbeat/elasticsearch-output.md
• docs/reference/metricbeat/elasticsearch-output.md
• docs/reference/packetbeat/elasticsearch-output.md
• docs/reference/winlogbeat/elasticsearch-output.md

[github-actions]

Vale Linting Results

Summary: 18 suggestions found

💡 Suggestions (18)

File	Line	Rule	Message
docs/reference/auditbeat/elasticsearch-output.md	426	Elastic.Versions	Use 'or later' instead of 'or newer' when referring to versions.
docs/reference/auditbeat/elasticsearch-output.md	428	Elastic.WordChoice	Consider using 'deactivate, deselect, hide, turn off' instead of 'disable', unless the term is in the UI.
docs/reference/auditbeat/elasticsearch-output.md	484	Elastic.Wordiness	Consider using 'also' instead of 'In addition'.
docs/reference/filebeat/elasticsearch-output.md	422	Elastic.Versions	Use 'or later' instead of 'or newer' when referring to versions.
docs/reference/filebeat/elasticsearch-output.md	424	Elastic.WordChoice	Consider using 'deactivate, deselect, hide, turn off' instead of 'disable', unless the term is in the UI.
docs/reference/filebeat/elasticsearch-output.md	480	Elastic.Wordiness	Consider using 'also' instead of 'In addition'.
docs/reference/heartbeat/elasticsearch-output.md	424	Elastic.Versions	Use 'or later' instead of 'or newer' when referring to versions.
docs/reference/heartbeat/elasticsearch-output.md	426	Elastic.WordChoice	Consider using 'deactivate, deselect, hide, turn off' instead of 'disable', unless the term is in the UI.
docs/reference/heartbeat/elasticsearch-output.md	483	Elastic.Wordiness	Consider using 'also' instead of 'In addition'.
docs/reference/metricbeat/elasticsearch-output.md	426	Elastic.Versions	Use 'or later' instead of 'or newer' when referring to versions.
docs/reference/metricbeat/elasticsearch-output.md	428	Elastic.WordChoice	Consider using 'deactivate, deselect, hide, turn off' instead of 'disable', unless the term is in the UI.
docs/reference/metricbeat/elasticsearch-output.md	485	Elastic.Wordiness	Consider using 'also' instead of 'In addition'.
docs/reference/packetbeat/elasticsearch-output.md	426	Elastic.Versions	Use 'or later' instead of 'or newer' when referring to versions.
docs/reference/packetbeat/elasticsearch-output.md	428	Elastic.WordChoice	Consider using 'deactivate, deselect, hide, turn off' instead of 'disable', unless the term is in the UI.
docs/reference/packetbeat/elasticsearch-output.md	485	Elastic.Wordiness	Consider using 'also' instead of 'In addition'.
docs/reference/winlogbeat/elasticsearch-output.md	422	Elastic.Versions	Use 'or later' instead of 'or newer' when referring to versions.
docs/reference/winlogbeat/elasticsearch-output.md	424	Elastic.WordChoice	Consider using 'deactivate, deselect, hide, turn off' instead of 'disable', unless the term is in the UI.
docs/reference/winlogbeat/elasticsearch-output.md	481	Elastic.Wordiness	Consider using 'also' instead of 'In addition'.

---

The Vale linter checks documentation changes against the Elastic Docs style guide.

To use Vale locally or report issues, refer to Elastic style guide for Vale.

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

docs/reference/heartbeat/elasticsearch-output.md (1)

293-376: ⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Missing deprecation notices for pipeline and pipelines options.

PR objectives state that both pipeline and pipelines should be deprecated, but neither section has the required {applies_to} deprecation block added. These options need deprecation metadata consistent with the other deprecated options.

Expected additions:

• After line 293 header, add deprecation block for pipeline
• After line 328 header, add deprecation block for pipelines

📝 Proposed fix

For pipeline section (after line 293):

 ### `pipeline` [pipeline-option-es]

+```{applies_to}
+stack: deprecated 9.5.0+
+```
+
 A format string value that specifies the ingest pipeline to write events to.

For pipelines section (after line 328):

 ### `pipelines` [pipelines-option-es]

+```{applies_to}
+stack: deprecated 9.5.0+
+```
+
 An array of pipeline selector rules. Each rule specifies the ingest pipeline to use for events that match the rule.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/reference/heartbeat/elasticsearch-output.md` around lines 293 - 376, Add
the missing deprecation metadata blocks for the pipeline and pipelines options:
insert an {applies_to} deprecation block after the "pipeline" header (the
"pipeline" option section) indicating "stack: deprecated 9.5.0+" and a short
quoted description, and likewise insert an {applies_to} deprecation block after
the "pipelines" header (the "pipelines" option section) with the same "stack:
deprecated 9.5.0+" metadata and a short quoted description; ensure the blocks
follow the same formatting style used for other deprecated options in the docs
so the pipeline and pipelines sections include the required deprecation notices.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@docs/reference/heartbeat/elasticsearch-output.md`:
- Around line 293-376: Add the missing deprecation metadata blocks for the
pipeline and pipelines options: insert an {applies_to} deprecation block after
the "pipeline" header (the "pipeline" option section) indicating "stack:
deprecated 9.5.0+" and a short quoted description, and likewise insert an
{applies_to} deprecation block after the "pipelines" header (the "pipelines"
option section) with the same "stack: deprecated 9.5.0+" metadata and a short
quoted description; ensure the blocks follow the same formatting style used for
other deprecated options in the docs so the pipeline and pipelines sections
include the required deprecation notices.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: fceb2a0c-c8ac-442f-a22a-c184d0eb12e4

📥 Commits

Reviewing files that changed from the base of the PR and between 97e2f33 and 6072ef3.

📒 Files selected for processing (6)

• docs/reference/auditbeat/elasticsearch-output.md
• docs/reference/filebeat/elasticsearch-output.md
• docs/reference/heartbeat/elasticsearch-output.md
• docs/reference/metricbeat/elasticsearch-output.md
• docs/reference/packetbeat/elasticsearch-output.md
• docs/reference/winlogbeat/elasticsearch-output.md

🚧 Files skipped from review as they are similar to previous changes (5)

• docs/reference/filebeat/elasticsearch-output.md
• docs/reference/winlogbeat/elasticsearch-output.md
• docs/reference/metricbeat/elasticsearch-output.md
• docs/reference/packetbeat/elasticsearch-output.md
• docs/reference/auditbeat/elasticsearch-output.md

[cmacknz]

LGTM, pinging @nimarezainia for a PM review since this is marking settings as deprecated.

[nimarezainia]

@cmacknz I think marking this as pending deprecation is ok. but I am wondering what we will do with customers who were using these settings (I have no way of determining that).

also these docs are changing the beats docs (which unfortunately have always been pointed to by the Elastic Agent output section). These fields are not being deprecated for beats. Only affects beatsreceivers.

[cmacknz]

also these docs are changing the beats docs (which unfortunately have always been pointed to by the Elastic Agent output section). These fields are not being deprecated for beats. Only affects beatsreceivers.

Dead letter index should be deprecated anyway, there's a way to handle this natively in ES now. Deprecating this is overdue, it never came out of beta.

I would also agree, we are only deprecating escape_html and allow_older_versions in Elastic Agent since it will use beats receivers going forward. We probably should only deprecate them in the Elastic Agent documentation to start.