Background
Currently, H2O-3 development Docker images are built and pushed to AWS ECR using a Jenkins pipeline. We want to migrate this process to GitHub Actions for better maintainability, transparency, and integration with the GitHub ecosystem.
Purpose
- Automate the build and push of all H2O-3 development Docker images to AWS ECR.
- Maintain image dependency order to ensure reproducibility.
- Enable optional caching, forced rebuilds, and selective publishing to ECR.
Images Built
Images are built in dependency order across multiple tiers:
Tier 1: dev-base (foundation image)
Tier 2: dev-jdk-8-base, dev-jdk-others-base
Tier 3: dev-python-base
Tier 4: dev-python-3.7 to dev-python-3.11, dev-r-base
Tier 5: dev-r-{3.4.1, 3.5.3, 4.0.2, 4.4.0, 4.5.2}, dev-jdk-{8,11,17}, dev-mojocompat
Tier 6: dev-build-base
Tier 7: dev-build-hadoop
Tier 8: dev-release
Tier 9: dev-python-{version}-jdk-{version}, dev-r-{version}-jdk-{version}
Workflow Features
- Branch input: Builds can be triggered for a specific H2O-3 branch.
- Force build: Option to rebuild images even if they already exist in ECR.
- Publish: Option to push built images to AWS ECR.
- No-cache: Option to build images without using Docker cache.
- AWS Roles: Separate roles for ECR and S3 downloads.
- Multi-version support: Python and JDK versions are handled via matrix builds.
Current Status
- GitHub Actions workflow has been drafted for all image tiers.
- ECR login, S3 download, and Docker build/push steps are implemented.
- Matrix strategies for Python and JDK versions are included.
- Conditional logic is in place to skip builds if images already exist and
force_build is false.
Background
Currently, H2O-3 development Docker images are built and pushed to AWS ECR using a Jenkins pipeline. We want to migrate this process to GitHub Actions for better maintainability, transparency, and integration with the GitHub ecosystem.
Purpose
Images Built
Images are built in dependency order across multiple tiers:
Tier 1:
dev-base(foundation image)Tier 2:
dev-jdk-8-base,dev-jdk-others-baseTier 3:
dev-python-baseTier 4:
dev-python-3.7todev-python-3.11,dev-r-baseTier 5:
dev-r-{3.4.1, 3.5.3, 4.0.2, 4.4.0, 4.5.2},dev-jdk-{8,11,17},dev-mojocompatTier 6:
dev-build-baseTier 7:
dev-build-hadoopTier 8:
dev-releaseTier 9:
dev-python-{version}-jdk-{version},dev-r-{version}-jdk-{version}Workflow Features
Current Status
force_buildisfalse.