-
Notifications
You must be signed in to change notification settings - Fork 73
drenv: Add automatic kustomize installation #2538
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
raaizik
wants to merge
1
commit into
RamenDR:main
Choose a base branch
from
raaizik:issue-1493-drenv-tools
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+430
−0
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| # SPDX-FileCopyrightText: The RamenDR authors | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| """ | ||
| Tool management for drenv. | ||
| This module provides functionality to check and install all tools required | ||
| by drenv. | ||
| This is a minimal implementation starting with kustomize only. | ||
| Other tools will be added incrementally. | ||
| """ | ||
|
|
||
| from .tools import ( | ||
| Kustomize, | ||
| Tool, | ||
| setup, | ||
| ) | ||
|
|
||
| __all__ = [ | ||
| "setup", | ||
| "Tool", | ||
| "Kustomize", | ||
| ] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,227 @@ | ||
| # SPDX-FileCopyrightText: The RamenDR authors | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| """ | ||
| Tool management for drenv. | ||
|
|
||
| This module provides functionality to check and install all tools required | ||
| by drenv. Each tool implements version() and install() methods. | ||
|
|
||
| This is a minimal implementation starting with kustomize only, addressing | ||
| PR #2538 review comments. Other tools will be added incrementally. | ||
| """ | ||
|
|
||
| import logging | ||
| import os | ||
| import platform | ||
| import shutil | ||
| import sys | ||
| import tarfile | ||
| import tempfile | ||
| import urllib.request | ||
|
|
||
| from packaging import version | ||
|
|
||
| from drenv import commands | ||
|
|
||
| # Platform detection - computed once at module import time | ||
| # All tools need the same values, so we cache them as module constants | ||
| _os_name = platform.system().lower() | ||
| _machine = platform.machine().lower() | ||
|
|
||
| # Normalize architecture names | ||
| if _machine in ("x86_64", "amd64"): | ||
| ARCH = "amd64" | ||
| elif _machine in ("aarch64", "arm64"): | ||
| ARCH = "arm64" | ||
| else: | ||
| ARCH = _machine | ||
|
|
||
| OS_NAME = _os_name | ||
|
|
||
|
|
||
| class Tool: | ||
| """Base class for all tools.""" | ||
|
|
||
| # Class attributes - override in subclasses | ||
| name = None | ||
| required_version = None | ||
| archive_binary_path = None # Exact path of binary in archive (e.g., "kustomize") | ||
|
|
||
| def __init__(self, install_dir=None): | ||
| """ | ||
| Initialize tool. | ||
|
|
||
| Args: | ||
| install_dir: Directory to install tool (default: venv bin directory) | ||
| """ | ||
| self._install_dir = install_dir | ||
|
|
||
| def install_dir(self): | ||
| """ | ||
| Return the directory where tools are installed. | ||
|
|
||
| The venv can be at different locations, so we cannot use a hardcoded | ||
| path. All tools are installed in the same location. | ||
| """ | ||
| if self._install_dir: | ||
| return self._install_dir | ||
| return os.path.join(sys.prefix, "bin") | ||
|
|
||
| def path(self): | ||
| """Return full path to tool binary in the venv.""" | ||
| return os.path.join(self.install_dir(), self.name) | ||
|
|
||
| def version(self): | ||
| """ | ||
| Return installed version or None if not installed. | ||
|
|
||
| Must be implemented by subclass. | ||
|
|
||
| Returns: | ||
| packaging.version.Version object or None | ||
| """ | ||
| raise NotImplementedError( | ||
| f"{self.__class__.__name__}.version() not implemented" | ||
| ) | ||
|
|
||
| def install(self): | ||
| """ | ||
| Install the tool. | ||
|
|
||
| Must be implemented by subclass. | ||
| """ | ||
| raise NotImplementedError( | ||
| f"{self.__class__.__name__}.install() not implemented" | ||
| ) | ||
|
|
||
| def verify(self): | ||
| """ | ||
| Check if tool is installed with correct version. | ||
|
|
||
| Returns: | ||
| True if tool is installed and version >= required_version | ||
| """ | ||
| current = self.version() | ||
| if current is None: | ||
| return False | ||
| return current >= version.parse(self.required_version) | ||
|
|
||
|
|
||
| # Helper functions | ||
|
|
||
|
|
||
| def download(url, target_path): | ||
| """ | ||
| Download file from URL to target path. | ||
|
|
||
| Args: | ||
| url: URL to download from | ||
| target_path: Local file path to save to | ||
| """ | ||
| logging.debug(f"Downloading {url} to {target_path}") | ||
| with urllib.request.urlopen(url) as response: | ||
| with open(target_path, "wb") as f: | ||
| shutil.copyfileobj(response, f) | ||
|
|
||
|
|
||
| def extract_tar(archive_path, target_dir, archive_binary_path): | ||
| """ | ||
| Extract binary from tar archive. | ||
|
|
||
| Args: | ||
| archive_path: Path to tar.gz file | ||
| target_dir: Directory to extract to | ||
| archive_binary_path: Exact path of binary in the archive | ||
| """ | ||
| logging.debug(f"Extracting {archive_binary_path} from {archive_path}") | ||
| with tarfile.open(archive_path, "r") as tar: | ||
| member = tar.getmember(archive_binary_path) | ||
| # Extract binary to target_dir root, removing any directory structure from archive | ||
| # This ensures the binary is placed directly in install_dir, not in a subdirectory | ||
| member.name = os.path.basename(member.name) | ||
| tar.extract(member, target_dir) | ||
|
|
||
|
|
||
| # Tool implementations | ||
|
|
||
|
|
||
| class Kustomize(Tool): | ||
| """Kubernetes native configuration management.""" | ||
|
|
||
| # Version from drenv requirements | ||
| name = "kustomize" | ||
| required_version = "5.7.0" | ||
| archive_binary_path = "kustomize" # Binary is at root of archive | ||
|
|
||
| # Download URL as class variable - resolved at import time | ||
| download_url = ( | ||
| f"https://github.com/kubernetes-sigs/kustomize/" | ||
| f"releases/download/kustomize%2Fv{required_version}/" | ||
| f"kustomize_v{required_version}_{OS_NAME}_{ARCH}.tar.gz" | ||
| ) | ||
|
|
||
| def version(self): | ||
| """Return installed version or None if not installed.""" | ||
| try: | ||
| out = commands.run(self.path(), "version") | ||
| except commands.Error as e: | ||
| if "No such file or directory" in str(e.error): | ||
| return None | ||
| # Provide context for other errors | ||
| raise RuntimeError(f"Failed to get {self.name} version: {e.error}") from e | ||
|
|
||
| try: | ||
| version_str = out.strip().lstrip("v") | ||
| return version.parse(version_str) | ||
| except Exception as e: | ||
| raise RuntimeError( | ||
| f"Failed to parse {self.name} version from output '{out}': {e}" | ||
| ) from e | ||
|
|
||
| def install(self): | ||
| """Install kustomize from GitHub releases.""" | ||
| with tempfile.TemporaryDirectory() as tmpdir: | ||
| archive_path = os.path.join(tmpdir, "kustomize.tar.gz") | ||
| download(self.download_url, archive_path) | ||
| extract_tar(archive_path, self.install_dir(), self.archive_binary_path) | ||
| os.chmod(self.path(), 0o755) | ||
|
|
||
|
|
||
| # Setup function | ||
|
|
||
|
|
||
| def setup(install_dir=None): | ||
| """ | ||
| Check and install all required tools. | ||
|
|
||
| Args: | ||
| install_dir: Directory to install tools (default: venv bin directory) | ||
| This parameter is provided for testing purposes. | ||
| """ | ||
| # List of all tools to install | ||
| tools = [ | ||
| Kustomize(install_dir=install_dir), | ||
| ] | ||
|
|
||
| for tool in tools: | ||
| current = tool.version() | ||
|
|
||
| if current is None: | ||
| logging.info(f"Installing {tool.name} {tool.required_version}") | ||
| try: | ||
| tool.install() | ||
| except Exception as e: | ||
| logging.error(f"Failed to install {tool.name}: {e}") | ||
| raise | ||
| elif current < version.parse(tool.required_version): | ||
| logging.info( | ||
| f"Updating {tool.name} from {current} to {tool.required_version}" | ||
| ) | ||
| try: | ||
| tool.install() | ||
| except Exception as e: | ||
| logging.error(f"Failed to update {tool.name}: {e}") | ||
| raise | ||
| else: | ||
| logging.info(f"Using {tool.name} {current}") | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🌐 Web query:
Python 3.12 3.13 tar.extract DeprecationWarning filter parameter💡 Result:
In Python 3.12 and 3.13, calling tarfile.TarFile.extract or extractall without the filter parameter (or with filter=None) triggers a DeprecationWarning: "Python 3.14 will, by default, filter extracted tar archives and reject files or modify their metadata. Use the filter argument to control this behavior." [1][2][3] This behavior was introduced in Python 3.12 via PEP 706 to address security issues like CVE-2007-4559 (path traversal in tar extraction). When no filter is specified, it falls back to the 'fully_trusted' filter (matching pre-3.12 behavior) but emits the warning. The default changes in Python 3.14 to 'data', which blocks dangerous features like absolute paths, symlinks outside the target directory, and certain metadata. [1][4][2][3] To silence the warning and prepare for 3.14: - Use filter='data' for safe extraction of data archives (recommended for most cases, ignores/block most UNIX-specific features). [1][3] - Use filter='fully_trusted' if you need full legacy behavior and trust the archive. [2] - Use filter='tar' to honor tar-specific features but block highly dangerous ones. [1] - Or set TarFile.extraction_filter globally/class-wide, e.g., TarFile.extraction_filter = 'data' [1][3] Example:
import tarfile with tarfile.open('archive.tar') as tf: tf.extractall(path='dest', filter='data') # No warning, safe for 3.14+Python 3.13 retains this exact deprecation behavior (no changes noted in 3.13 docs or changelog); the switch happens in 3.14. [1][5] Note: There was a CVE-2024-12718 in 3.12+ filters='data'/'tar' allowing limited metadata changes outside dest, but patches are available—upgrade Python. [6][7]Citations:
🌐 Web query:
Python 3.14 tarfile extract filter default behavior changes💡 Result:
In Python 3.14, the default behavior of tarfile.TarFile.extract and extractall changed significantly regarding the filter parameter. Previously (in 3.12 and 3.13), when the filter argument was None (the default), it would use the 'fully_trusted' filter after emitting a DeprecationWarning, allowing potentially dangerous operations like extracting to absolute paths or paths outside the destination directory [1][2]. In Python 3.14, the default filter is now 'data', which disallows such dangerous features including links to absolute paths or paths outside the destination, making extraction safer by default [1][3]. This implements PEP 706 [2][4]. The official documentation states: "Changed in version 3.14: The default filter is set to data, which disallows some dangerous features such as links to absolute paths or paths outside of the destination. Previously, the default was equivalent to fully_trusted." [1]. Developers are recommended to explicitly set the filter if specific behavior is needed, especially for compatibility across versions [1]. Note that even the 'data' filter does not prevent all security issues, as seen in later CVEs like CVE-2024-12718, but it addresses the most critical path traversal risks [5][6][3].
Citations:
🏁 Script executed:
Repository: RamenDR/ramen
Length of output: 1741
Add
filter='data'totar.extract()to silence DeprecationWarning and prepare for Python 3.14Without an explicit
filterargument,tar.extract()emits aDeprecationWarningon Python 3.12–3.13 and will use a safer default ('data') in Python 3.14. Since the archive is from a trusted GitHub release and contains only a single binary,filter='data'is the recommended approach.Fix
🤖 Prompt for AI Agents