fix(hlf-ordnode): mount emptyDir at /etc/hyperledger/fabric for Fabric 3.1.2+ upgrade#317
Open
fix(hlf-ordnode): mount emptyDir at /etc/hyperledger/fabric for Fabric 3.1.2+ upgrade#317
Conversation
…c 3.1.2+ compatibility Fabric 3.1.2 (hyperledger/fabric#5236, commit 77db242a) restructured the contents of the image's declared `VOLUME /etc/hyperledger/fabric`. On many container runtimes (notably containerd) this triggers a "runtime copy of volume" step at pod start that fails against the new 3.1.2 layout, causing orderer pods to crashloop after upgrading the orderer image past 3.1.1. Mounting an empty `emptyDir` over `/etc/hyperledger/fabric` overrides the image's declared VOLUME, so the runtime never performs the broken copy. The orderer continues to read its real configuration from `/var/hyperledger/fabric/config` (unchanged), so behavior is identical for existing 1.4.x / 2.x / 3.0.x / 3.1.0 / 3.1.1 deployments. Bump chart version 1.4.0 -> 1.4.1 and appVersion 1.4.3 -> 3.1.2. This fix is required to upgrade orderers to Fabric 3.1.2 and onwards.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Enables upgrading the orderer image to Hyperledger Fabric 3.1.2 and onwards by mounting an
emptyDirover/etc/hyperledger/fabricin thehlf-ordnodechart, overriding the image's declaredVOLUMEand bypassing a container-runtime regression.What happened
Fabric 3.1.2 (hyperledger/fabric#5236, commit
77db242a) restructured the contents of the image's declaredVOLUME /etc/hyperledger/fabric. On several container runtimes — most notably containerd, which is the default on the majority of managed Kubernetes distributions — this triggers a "runtime copy of volume" step at pod start that fails against the new 3.1.2 layout. The result: orderer pods crashloop the moment you bump the image past3.1.1.Symptom seen in the field:
CrashLoopBackOffimmediately after the image bump.3.1.1resolves it, confirming the issue is the 3.1.2 image layout interacting with the runtime's anonymous-volume copy behavior.The fix
Mount an empty
emptyDirover/etc/hyperledger/fabricin the orderer container. This overrides the image's declaredVOLUME, so the runtime never performs the broken copy step. The orderer continues to read its real configuration from/var/hyperledger/fabric/config(unchanged), so behavior is identical for existing 1.4.x / 2.x / 3.0.x / 3.1.0 / 3.1.1 deployments — this is purely a forward-compat unblocker.Changes
charts/hlf-ordnode/templates/deployment.yaml— addfabric-cfgemptyDirvolume and mount it at/etc/hyperledger/fabricin the orderer container.charts/hlf-ordnode/Chart.yaml— bump chartversion1.4.0->1.4.1, bumpappVersion1.4.3->3.1.2.Why this is safe for existing deployments
/etc/hyperledger/fabric, which the chart never wrote to before — there is no overlap with the existing/var/hyperledger/fabric/configConfigMap mount that the orderer actually reads.emptyDiris ephemeral and reset on each pod start, which is exactly what we want for a path the orderer doesn't consume.Test plan
helm template) and confirm thefabric-cfgvolume + mount appear on theorderercontainer.image.tag=3.1.2and confirm the orderer pod starts cleanly (no crashloop,/healthzreturns 200).3.1.1upgrades to3.1.2in place via the operator without data loss.image.tag=3.1.1(and 2.5.x) with this chart and confirm no behavior change.kubectl exec deploy/<orderer> -- ls /etc/hyperledger/fabricreturns empty (the emptyDir).