Skip to content

Migrate the H2O-3 Docker Image Builds Automation to GitHub Actions #16764

@silvarohan18

Description

@silvarohan18

Background

Currently, H2O-3 development Docker images are built and pushed to AWS ECR using a Jenkins pipeline. We want to migrate this process to GitHub Actions for better maintainability, transparency, and integration with the GitHub ecosystem.

Purpose

  • Automate the build and push of all H2O-3 development Docker images to AWS ECR.
  • Maintain image dependency order to ensure reproducibility.
  • Enable optional caching, forced rebuilds, and selective publishing to ECR.

Images Built

Images are built in dependency order across multiple tiers:

Tier 1: dev-base (foundation image)
Tier 2: dev-jdk-8-base, dev-jdk-others-base
Tier 3: dev-python-base
Tier 4: dev-python-3.7 to dev-python-3.11, dev-r-base
Tier 5: dev-r-{3.4.1, 3.5.3, 4.0.2, 4.4.0, 4.5.2}, dev-jdk-{8,11,17}, dev-mojocompat
Tier 6: dev-build-base
Tier 7: dev-build-hadoop
Tier 8: dev-release
Tier 9: dev-python-{version}-jdk-{version}, dev-r-{version}-jdk-{version}

Workflow Features

  • Branch input: Builds can be triggered for a specific H2O-3 branch.
  • Force build: Option to rebuild images even if they already exist in ECR.
  • Publish: Option to push built images to AWS ECR.
  • No-cache: Option to build images without using Docker cache.
  • AWS Roles: Separate roles for ECR and S3 downloads.
  • Multi-version support: Python and JDK versions are handled via matrix builds.

Current Status

  • GitHub Actions workflow has been drafted for all image tiers.
  • ECR login, S3 download, and Docker build/push steps are implemented.
  • Matrix strategies for Python and JDK versions are included.
  • Conditional logic is in place to skip builds if images already exist and force_build is false.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions