Releases: osodevops/kafka-backup
Releases · osodevops/kafka-backup
v0.15.3
What's Changed
Fixes
- fix(backup): persist progress metadata during backup (#103)
Full Changelog: v0.15.2...v0.15.3
Install kafka-backup-cli 0.15.3
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/osodevops/kafka-backup/releases/download/v0.15.3/kafka-backup-cli-installer.sh | shInstall prebuilt binaries via powershell script
powershell -ExecutionPolicy Bypass -c "irm https://github.com/osodevops/kafka-backup/releases/download/v0.15.3/kafka-backup-cli-installer.ps1 | iex"Install prebuilt binaries via Homebrew
brew install osodevops/tap/kafka-backupDownload kafka-backup-cli 0.15.3
| File | Platform | Checksum |
|---|---|---|
| kafka-backup-cli-aarch64-apple-darwin.tar.xz | Apple Silicon macOS | checksum |
| kafka-backup-cli-x86_64-apple-darwin.tar.xz | Intel macOS | checksum |
| kafka-backup-cli-x86_64-pc-windows-msvc.zip | x64 Windows | checksum |
| kafka-backup-cli-x86_64-unknown-linux-gnu.tar.xz | x64 Linux | checksum |
v0.15.2
What's Changed
Fixes
- fix(cli): resolve storage URLs for inspection commands (#102)
Full Changelog: v0.15.1...v0.15.2
Install kafka-backup-cli 0.15.2
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/osodevops/kafka-backup/releases/download/v0.15.2/kafka-backup-cli-installer.sh | shInstall prebuilt binaries via powershell script
powershell -ExecutionPolicy Bypass -c "irm https://github.com/osodevops/kafka-backup/releases/download/v0.15.2/kafka-backup-cli-installer.ps1 | iex"Install prebuilt binaries via Homebrew
brew install osodevops/tap/kafka-backupDownload kafka-backup-cli 0.15.2
| File | Platform | Checksum |
|---|---|---|
| kafka-backup-cli-aarch64-apple-darwin.tar.xz | Apple Silicon macOS | checksum |
| kafka-backup-cli-x86_64-apple-darwin.tar.xz | Intel macOS | checksum |
| kafka-backup-cli-x86_64-pc-windows-msvc.zip | x64 Windows | checksum |
| kafka-backup-cli-x86_64-unknown-linux-gnu.tar.xz | x64 Linux | checksum |
v0.15.1
What's Changed
Fixes
- fix(sasl): bind auth plugins to connected broker endpoint (#97)
Full Changelog: v0.15.0...v0.15.1
Install kafka-backup-cli 0.15.1
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/osodevops/kafka-backup/releases/download/v0.15.1/kafka-backup-cli-installer.sh | shInstall prebuilt binaries via powershell script
powershell -ExecutionPolicy Bypass -c "irm https://github.com/osodevops/kafka-backup/releases/download/v0.15.1/kafka-backup-cli-installer.ps1 | iex"Install prebuilt binaries via Homebrew
brew install osodevops/tap/kafka-backupDownload kafka-backup-cli 0.15.1
| File | Platform | Checksum |
|---|---|---|
| kafka-backup-cli-aarch64-apple-darwin.tar.xz | Apple Silicon macOS | checksum |
| kafka-backup-cli-x86_64-apple-darwin.tar.xz | Intel macOS | checksum |
| kafka-backup-cli-x86_64-pc-windows-msvc.zip | x64 Windows | checksum |
| kafka-backup-cli-x86_64-unknown-linux-gnu.tar.xz | x64 Linux | checksum |
0.15.0 - 2026-04-21
What's Changed
Features
- feat(sasl): pluggable SASL mechanism extension point with GSSAPI V1 (#96)
Fixes
- feat(sasl): pluggable SASL mechanism extension point with GSSAPI V1 (#96)
Full Changelog: v0.14.0...v0.15.0
Release Notes
Added
- SASL/GSSAPI (Kerberos) authentication via the
gssapiCargo feature.
Default builds remain Kerberos-free; opt in with
cargo build --features gssapi -p kafka-backup-cli. State machine
and credential hints adapted from @kthimjo's PR #95 — thank you.- New
SaslMechanism::Gssapienum variant. - New optional
SecurityConfigfields:sasl_kerberos_service_name,
sasl_keytab_path,sasl_krb5_config_path. GssapiPluginimplements RFC 4752 Phase 1 multi-roundgss_init_sec_context,
Phase 1→2 turnaround, Phase 2layer = 0x01(no security layer, no size)
wrap/unwrap, and KIP-368 re-authentication via fresh-context rebuild.GssapiPluginFactory— constructed from the operator-provided keytab +
krb5.conf + service name, validated eagerly at config time. The factory
binds the SPN hostname at.build()time (see Factory extension point
below), so each per-brokerKafkaClientauthenticates against the correct
per-broker SPN (kafka/brokerN.fqdn@REALM) on multi-broker clusters.- Process-wide
KRB5_ENV_LOCK: tokio::sync::Mutex<()>serialises
KRB5_CLIENT_KTNAME/KRB5_CONFIG/KRB5CCNAMEmutation during
credential acquisition — eliminates the multi-client env-var race
inherent tolibgssapi 0.9. - When a keytab is configured,
GssapiPluginisolates its credential
cache viaKRB5CCNAME=MEMORY:<ptr>. This prevents stale tickets in
the OS default ccache (common on macOSAPI:<uuid>caches) from
being preferred over a fresh TGT from the keytab — a failure mode
that surfaces as a cryptic broker-side
Authentication failed due to invalid credentials.
- New
- Factory extension point —
SecurityConfig.sasl_mechanism_plugin_factory: Option<SaslMechanismPluginFactoryHandle>replaces the prior
sasl_mechanism_plugin: Option<SaslMechanismPluginHandle>(both introduced
on this branch; neither has shipped).KafkaClient::authenticatecalls
factory.build(broker_host, broker_port)once per connection, receiving the
endpoint frombootstrap_servers[0]— whichPartitionLeaderRouterhas
already rewritten to the advertised per-brokerhost:portbefore spawning
pooled clients. This fixes one correctness bug and removes a latent one:- Multi-broker GSSAPI SPN (fixed). Non-bootstrap brokers now
authenticate against their own SPN
(kafka/brokerN.fqdn@REALM) rather than the bootstrap host's — the
standard librdkafka / JVM-client behaviour. - Per-connection GSSAPI state (removed as a latent risk). Each
pooledKafkaClientnow owns its ownGssapiPluginand its own
ClientCtx. A shared plugin across the pool would have been a
concurrency hazard even if it has not produced a visible failure in
the current test matrix.
SharedPluginFactory— convenience wrapper for stateless mechanisms
(PLAIN, OAUTHBEARER with a shared token provider); returns the same Arc
from everybuildcall.- New
SaslPluginError::FactoryFailed { mechanism, source }variant for
clean error surfaces at build time.
- Multi-broker GSSAPI SPN (fixed). Non-bootstrap brokers now
SaslMechanismPlugin::supports_reauth()capability flag — default
true(PLAIN, SCRAM, OAUTHBEARER continue to schedule KIP-368 live
re-auth);GssapiPluginoverrides tofalse. Apache Kafka does not
support live re-authentication for GSSAPI — Kerberos GSS-API contexts
are bound to the wire connection, and the broker rejects in-place
SaslAuthenticateafter the initial handshake. Matches librdkafka:
treat the broker-advertisedsession_lifetime_msas a
drain-and-reconnect timer rather than firing a reauth the broker will
reject. With the plugin opting out,KafkaClient::authenticateskips
spawn_reauth_taskentirely; the session expires naturally and the
next RPC reconnects through the normal auth path.- CLI plumbing: new flags
--sasl-mechanism,--sasl-keytab,
--sasl-krb5-config,--sasl-kerberos-service-nameonoffset-reset,
offset-reset-bulk, andoffset-rollbackcommands. YAML configs auto-wire
aGssapiPluginFactorywhensasl_mechanism: GSSAPIis set. A runtime
error surfaces if the CLI was built without--features gssapi. - Deduplicated CLI security-args parsing via
commands/security_args.rs
(#[derive(clap::Args)] SecurityCliArgs) — removes three copies of the prior
parse_security_confighelper. - Docker test fixture at
tests/sasl-gssapi-test-infra/— self-hosted MIT KDC
(Dockerfile.kdc), Apache Kafka 7.7.0 configured for
SASL_PLAINTEXT://kafka.test.local:9098withGSSAPIenabled, realm
TEST.LOCAL, keytab auto-generation with healthcheck gate. - Three
#[ignore]E2E tests: keytab happy-path, missing-keytab clear error,
KIP-368 reauth fires within broker's 60 s window
(crates/kafka-backup-core/tests/integration_suite/sasl_gssapi_tests.rs). - Full backup → restore roundtrip E2E test over GSSAPI
(sasl_gssapi_backup_restore_roundtrip): produces records, drives
BackupEngine+RestoreEnginewith topic remap, consumes from the
restored topic and asserts record count + payload. Runs at the default
connections_per_broker: 4now that each pooled connection owns its own
GssapiPluginvia the factory. - Factory-dispatch regression test
(sasl_plugin_mock_tests::factory_receives_per_broker_endpoint): a
CapturingFactoryassertsbuild(host, port)is called exactly once per
KafkaClientwith the endpoint frombootstrap_servers[0]. No Docker —
uses the in-processMockKafkaBrokerfixture. - Pool-isolation regression test
(sasl_plugin_mock_tests::pool_produces_distinct_plugin_per_kafkaclient):
N=3 separateMockKafkaBrokerinstances, NKafkaClients sharing one
SaslMechanismPluginFactory; asserts the factory is invoked once per
client with the correct endpoint and returns a pointer-distinct plugin
Arc each time. Turns item 2 above ("removed as a latent risk") into a
tested guarantee. - Scheduler-opt-out regression test
(sasl_plugin_mock_tests::reauth_scheduler_not_spawned_when_plugin_opts_out):
a plugin returningsupports_reauth() = falseconnects against a mock
that advertisessession_lifetime_ms: 60_000; virtual time is advanced
past the 80 % reauth deadline; the test assertsreauth_payloadis
never called and the mock sees exactly oneSaslAuthenticateframe. - Example YAML configs for operators:
config/gssapi-backup.yamland
config/gssapi-restore.yaml, driving the release binary end-to-end
against the fixture. - Release-binary CLI smoke script at
tests/sasl-gssapi-test-infra/run-cli-smoke.sh— builds
--release --features gssapiand exerciseskafka-backup backup
andkafka-backup restoreagainst the fixture, asserting exit codes,
manifest existence, and restored record count.
Build requirements
gssapifeature links against MIT krb5 at build time. Install:- macOS:
brew install krb5+ export
PKG_CONFIG_PATH="$(brew --prefix krb5)/lib/pkgconfig:…"(Apple's bundled
Heimdal does not expose the symbolslibgssapi 0.9requires). - Debian/Ubuntu:
apt-get install libkrb5-dev. - Fedora/RHEL:
dnf install krb5-devel.
- macOS:
Notes on GSSAPI re-authentication
- Apache Kafka does not support live KIP-368 re-authentication for the
GSSAPI mechanism — Kerberos GSS-API contexts are bound to the wire
connection and the broker rejects in-placeSaslAuthenticateafter
the initial handshake.GssapiPlugin::supports_reauth()returns
false, so the client no longer schedules a reauth task for GSSAPI
connections; the broker-advertisedsession_lifetime_msis treated
as a drain-and-reconnect window, matching librdkafka and the JVM
client behaviour. The connection lives out its session and the next
RPC transparently reconnects through the normal auth path.
Limitations
- The mock-broker test proves the factory contract (
buildis called with the
correct endpoint perKafkaClient). A multi-broker Docker GSSAPI fixture
that exercises distinct per-broker SPNs end-to-end is a planned follow-up. - Release binaries and the default Docker image do not include GSSAPI. Build
your own image with--build-arg FEATURES=gssapionce the downstream image
ships that arg.
Install kafka-backup-cli 0.15.0
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/osodevops/kafka-backup/releases/download/v0.15.0/kafka-backup-cli-installer.sh | shInstall prebuilt binaries via powershell script
powershell -ExecutionPolicy Bypass -c "irm https://github.com/osodevops/kafka-backup/releases/download/v0.15.0/kafka-backup-cli-installer.ps1 | iex"Install prebuilt binaries via Homebrew
brew install osodevops/tap/kafka-backupDownload kafka-backup-cli 0.15.0
| File | Platform | Checksum |
|---|---|---|
| kafka-backup-cli-aarch64-apple-darwin.tar.xz | Apple Silicon macOS | checksum |
| kafka-backup-cli-x86_64-apple-darwin.tar.xz | Intel macOS | checksum |
| [kafka-backup-cli-x86_64-pc-windows-msvc.zip](https://github.com/osodevops/kafka-backup/releases/download/v0.15.0/kafka-backu... |
0.14.0 - 2026-04-21
What's Changed
Features
- feat(kafka): SaslMechanismPlugin extension trait (0.14.0) (#94)
Dependencies
- feat(kafka): SaslMechanismPlugin extension trait (0.14.0) (#94)
Full Changelog: v0.13.5...v0.14.0
Release Notes
Added
- Pluggable SASL mechanism extension point (
SaslMechanismPlugintrait)
— lets downstream crates implement OAUTHBEARER, MSK IAM, or custom
SASL mechanisms without forkingkafka-backup-core.- Handshake + single- or multi-round
SaslAuthenticatedispatch. - KIP-368 re-authentication scheduler: spawns a task post-handshake
when the broker advertisessession_lifetime_ms > 0; fires
reauth_payloadat 80 % of the advertised lifetime with a 30 s
minimum floor and ±5 s jitter. - Default
interpret_server_errorhandles both RFC 7628 JSON and
Apache Kafka 3.5+ free-formerror_messagebytes. - New field
SecurityConfig.sasl_mechanism_plugin: Option<Arc<dyn SaslMechanismPlugin>>
(marked#[serde(skip)]— programmatic wiring only, no YAML surface).
- Handshake + single- or multi-round
- 14 unit tests + 4 integration tests exercising single-round,
multi-round, server-error, and scheduler paths against an
in-process Kafka-wire mock (no Docker required). #[ignore]E2E test against Confluent cp-kafka 7.7.0 configured for
SASL_PLAINTEXT + OAUTHBEARER with the bundled unsecured-JWS validator.
Fixture:tests/sasl-oauth-test-infra/.- Example:
examples/custom_sasl_plugin.rs— minimal static-token
OAUTHBEARER plugin (reference implementation).
Changed
- SASL dispatch in
KafkaClientunified: the four duplicated
sasl_{plain,scram}_auth{,_raw}methods collapse into a single
dispatch function called by both initial-connect and reconnect.
Behaviour for existingPLAIN/SCRAM-SHA-256/SCRAM-SHA-512
configurations is unchanged.
Install kafka-backup-cli 0.14.0
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/osodevops/kafka-backup/releases/download/v0.14.0/kafka-backup-cli-installer.sh | shInstall prebuilt binaries via powershell script
powershell -ExecutionPolicy Bypass -c "irm https://github.com/osodevops/kafka-backup/releases/download/v0.14.0/kafka-backup-cli-installer.ps1 | iex"Install prebuilt binaries via Homebrew
brew install osodevops/tap/kafka-backupDownload kafka-backup-cli 0.14.0
| File | Platform | Checksum |
|---|---|---|
| kafka-backup-cli-aarch64-apple-darwin.tar.xz | Apple Silicon macOS | checksum |
| kafka-backup-cli-x86_64-apple-darwin.tar.xz | Intel macOS | checksum |
| kafka-backup-cli-x86_64-pc-windows-msvc.zip | x64 Windows | checksum |
| kafka-backup-cli-x86_64-unknown-linux-gnu.tar.xz | x64 Linux | checksum |
0.13.5 - 2026-04-16
What's Changed
Fixes
- fix: enable incremental one-shot backups via offset_storage config (#92)
Dependencies
- fix: enable incremental one-shot backups via offset_storage config (#92)
Full Changelog: v0.13.4...v0.13.5
Release Notes
Fixed
- Incremental one-shot backups now work — offset tracking was previously gated on
continuous: true,
making one-shot and snapshot backups always start fromearliest. Now, addingoffset_storageto the
config enables resume-from-last-offset in any backup mode.
Added
- Unit tests for
merge_manifests()function (previously untested) - Integration test for incremental one-shot backup resume behavior
Install kafka-backup-cli 0.13.5
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/osodevops/kafka-backup/releases/download/v0.13.5/kafka-backup-cli-installer.sh | shInstall prebuilt binaries via powershell script
powershell -ExecutionPolicy Bypass -c "irm https://github.com/osodevops/kafka-backup/releases/download/v0.13.5/kafka-backup-cli-installer.ps1 | iex"Install prebuilt binaries via Homebrew
brew install osodevops/tap/kafka-backupDownload kafka-backup-cli 0.13.5
| File | Platform | Checksum |
|---|---|---|
| kafka-backup-cli-aarch64-apple-darwin.tar.xz | Apple Silicon macOS | checksum |
| kafka-backup-cli-x86_64-apple-darwin.tar.xz | Intel macOS | checksum |
| kafka-backup-cli-x86_64-pc-windows-msvc.zip | x64 Windows | checksum |
| kafka-backup-cli-x86_64-unknown-linux-gnu.tar.xz | x64 Linux | checksum |
v0.13.4
What's Changed
Fixes
- fix: route OffsetFetch to group coordinator in consumer group snapshot (#91)
Dependencies
- fix: route OffsetFetch to group coordinator in consumer group snapshot (#91)
Full Changelog: v0.13.3...v0.13.4
Install kafka-backup-cli 0.13.4
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/osodevops/kafka-backup/releases/download/v0.13.4/kafka-backup-cli-installer.sh | shInstall prebuilt binaries via powershell script
powershell -ExecutionPolicy Bypass -c "irm https://github.com/osodevops/kafka-backup/releases/download/v0.13.4/kafka-backup-cli-installer.ps1 | iex"Install prebuilt binaries via Homebrew
brew install osodevops/tap/kafka-backupDownload kafka-backup-cli 0.13.4
| File | Platform | Checksum |
|---|---|---|
| kafka-backup-cli-aarch64-apple-darwin.tar.xz | Apple Silicon macOS | checksum |
| kafka-backup-cli-x86_64-apple-darwin.tar.xz | Intel macOS | checksum |
| kafka-backup-cli-x86_64-pc-windows-msvc.zip | x64 Windows | checksum |
| kafka-backup-cli-x86_64-unknown-linux-gnu.tar.xz | x64 Linux | checksum |
v0.13.3
What's Changed
Fixes
- fix: prevent consumer groups snapshot from being overwritten with empty list (#90)
Dependencies
- fix: prevent consumer groups snapshot from being overwritten with empty list (#90)
Full Changelog: v0.13.2...v0.13.3
Install kafka-backup-cli 0.13.3
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/osodevops/kafka-backup/releases/download/v0.13.3/kafka-backup-cli-installer.sh | shInstall prebuilt binaries via powershell script
powershell -ExecutionPolicy Bypass -c "irm https://github.com/osodevops/kafka-backup/releases/download/v0.13.3/kafka-backup-cli-installer.ps1 | iex"Install prebuilt binaries via Homebrew
brew install osodevops/tap/kafka-backupDownload kafka-backup-cli 0.13.3
| File | Platform | Checksum |
|---|---|---|
| kafka-backup-cli-aarch64-apple-darwin.tar.xz | Apple Silicon macOS | checksum |
| kafka-backup-cli-x86_64-apple-darwin.tar.xz | Intel macOS | checksum |
| kafka-backup-cli-x86_64-pc-windows-msvc.zip | x64 Windows | checksum |
| kafka-backup-cli-x86_64-unknown-linux-gnu.tar.xz | x64 Linux | checksum |
v0.13.2
What's Changed
Fixes
- fix: run schema migrations after loading offset DB from storage (#89)
Dependencies
- fix: run schema migrations after loading offset DB from storage (#89)
Full Changelog: v0.13.1...v0.13.2
Install kafka-backup-cli 0.13.2
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/osodevops/kafka-backup/releases/download/v0.13.2/kafka-backup-cli-installer.sh | shInstall prebuilt binaries via powershell script
powershell -ExecutionPolicy Bypass -c "irm https://github.com/osodevops/kafka-backup/releases/download/v0.13.2/kafka-backup-cli-installer.ps1 | iex"Install prebuilt binaries via Homebrew
brew install osodevops/tap/kafka-backupDownload kafka-backup-cli 0.13.2
| File | Platform | Checksum |
|---|---|---|
| kafka-backup-cli-aarch64-apple-darwin.tar.xz | Apple Silicon macOS | checksum |
| kafka-backup-cli-x86_64-apple-darwin.tar.xz | Intel macOS | checksum |
| kafka-backup-cli-x86_64-pc-windows-msvc.zip | x64 Windows | checksum |
| kafka-backup-cli-x86_64-unknown-linux-gnu.tar.xz | x64 Linux | checksum |
v0.13.1
What's Changed
Fixes
- fix: validation NOT_LEADER_FOR_PARTITION on multi-broker clusters (#88)
Dependencies
- fix: validation NOT_LEADER_FOR_PARTITION on multi-broker clusters (#88)
Other
- ci: dispatch release workflow after automated tag (#85)
Full Changelog: v0.13.0...v0.13.1
Install kafka-backup-cli 0.13.1
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/osodevops/kafka-backup/releases/download/v0.13.1/kafka-backup-cli-installer.sh | shInstall prebuilt binaries via powershell script
powershell -ExecutionPolicy Bypass -c "irm https://github.com/osodevops/kafka-backup/releases/download/v0.13.1/kafka-backup-cli-installer.ps1 | iex"Install prebuilt binaries via Homebrew
brew install osodevops/tap/kafka-backupDownload kafka-backup-cli 0.13.1
| File | Platform | Checksum |
|---|---|---|
| kafka-backup-cli-aarch64-apple-darwin.tar.xz | Apple Silicon macOS | checksum |
| kafka-backup-cli-x86_64-apple-darwin.tar.xz | Intel macOS | checksum |
| kafka-backup-cli-x86_64-pc-windows-msvc.zip | x64 Windows | checksum |
| kafka-backup-cli-x86_64-unknown-linux-gnu.tar.xz | x64 Linux | checksum |