Fix effective_config merge bug for OpAMP collectors#6988
Open
ycombinator wants to merge 3 commits intoelastic:mainfrom
Open
Fix effective_config merge bug for OpAMP collectors#6988ycombinator wants to merge 3 commits intoelastic:mainfrom
ycombinator wants to merge 3 commits intoelastic:mainfrom
Conversation
Contributor
|
This pull request does not have a backport label. Could you fix it @ycombinator? 🙏
|
The Elasticsearch partial update API recursively merges objects, so when a collector removes a pipeline from its config, the old pipeline key persists in the stored effective_config. This test creates a document with two pipelines, updates it with one, and asserts the removed pipeline is gone. Relates elastic#6877 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ycombinator
force-pushed
the
fix/6877-effective-config-merge
branch
from
d89970c to
008a0ab
Compare
ycombinator
added
bug
ycombinator
force-pushed
the
fix/6877-effective-config-merge
branch
from
c1c6a75 to
9c26062
Compare
Use a Painless script to update the agent document when
effective_config is present, replacing the {"doc": ...} partial
update that recursively merges nested objects. This ensures that
keys removed from the collector's config (e.g. deleted pipelines)
are no longer preserved in Elasticsearch.
Closes elastic#6877
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ycombinator
force-pushed
the
fix/6877-effective-config-merge
branch
from
9c26062 to
11e707d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the problem this PR solves?
When an OpAMP collector removes a pipeline from its configuration, the removed pipeline still appears in the
effective_configfield of the agent document in Elasticsearch. This happens because fleet-server uses Elasticsearch's partial document update API ({"doc": {...}}), which recursively merges objects rather than replacing them. So keys present in the oldeffective_configbut absent in the new one are preserved.For example, if the stored config has pipelines
{traces, metrics}and the collector reports only{traces}, the merge would keepmetricswhen it should be removed.How does this PR solve the problem?
When the checkin includes
effective_config, the update now uses a new Painless script (checkinWithEffectiveConfig.painless) that follows the same pattern asdeleteAuditFieldsOnCheckin.painless, directly assigning fields onctx._sourceto fully replaceeffective_configinstead of merging it. This is a new branch inflush()alongside the existing script and doc-merge paths.How to test this PR locally
Design Checklist
Checklist
./changelog/fragmentsusing the changelog toolRelated issues