Skip to content

Commit 122f35c

Browse files
paigecalvertpaigecalvert
committed
update instructions for creating ec v3 releases
1 parent d93c1f2 commit 122f35c

2 files changed

Lines changed: 218 additions & 24 deletions

File tree

‎embedded-cluster/embedded-using.mdx‎

Lines changed: 197 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,39 +1,220 @@
11
import UpdateOverview from "../docs/partials/embedded-cluster/v3/_update-overview.mdx"
22
import EcConfig from "../docs/partials/embedded-cluster/v3/_ec-config.mdx"
33
import ShellCommand from "../docs/partials/embedded-cluster/_shell-command.mdx"
4+
import RewriteHelmValues from "../docs/partials/proxy-service/_step-rewrite-helm-values.mdx"
5+
import DependencyYaml from "../docs//partials/replicated-sdk/_dependency-yaml.mdx"
46

57
# Configure Embedded Cluster (Beta)
68

7-
This topic describes how to configure your application releases to support installations with Replicated Embedded Cluster. For an introduction, see [Embedded Cluster Overview](embedded-overview).
9+
This topic describes how to configure and use Replicated Embedded Cluster with your application.
10+
For more information about Embedded Cluster, see [Embedded Cluster Overview](embedded-overview). For information about updating an existing release from Embedded Cluster v2 to v3, see [Migrate from Embedded Cluster v2](embedded-v3-migrate).
811

9-
## Add the Embedded Cluster Config resource
12+
## Create a release with Embedded Cluster v3
1013

11-
An [Embedded Cluster Config](embedded-config) must be present in the release to support installation with Embedded Cluster. The Embedded Cluster Config sets the version of Embedded Cluster to install, and lets you define additional characteristics about the cluster.
14+
To create an application release that supports installation with Embedded Cluster v3:
1215

13-
To add the Embedded Cluster Config:
16+
1. If you use the Replicated proxy registry, update all references to private or third-party images to use the Replicated proxy registry domain. See the _Embedded Cluster v3_ steps in [Configure your application to use the proxy registry](/vendor/private-images-kots#configure-v3).
1417

15-
1. Create a new release that includes your application and a unique [HelmChart v2](/reference/custom-resource-helmchart-v2) custom resource for each Helm chart in the release.
18+
1. In your application Helm chart `Chart.yaml` file, add the SDK as a dependency. If your application uses multiple charts, declare the SDK as a dependency of the chart that customers install first. Do not declare the SDK in more than one chart.
1619

17-
If you have not yet configured the HelmChart custom resource for your application, see [Onboard to the Replicated Platform](/vendor/replicated-onboarding). The onboarding guide provides detailed instructions for configuring releases that support installation with a Replicated installer.
20+
<DependencyYaml/>
1821

19-
1. In the release, add an [Embedded Cluster Config](embedded-config) manifest that specifies the Embedded Cluster version to use:
22+
1. Package each chart into a `.tgz` chart archive. See [Package a Helm chart for a release](/vendor/helm-install-release).
23+
24+
1. For each chart archive, add a unique HelmChart v2 custom resource (version `kots.io/v1beta2`).
25+
26+
```yaml
27+
# HelmChart custom resource
28+
apiVersion: kots.io/v1beta2
29+
kind: HelmChart
30+
metadata:
31+
name: samplechart
32+
spec:
33+
# chart identifies a matching chart from a .tgz
34+
chart:
35+
name: samplechart
36+
chartVersion: 3.1.7
37+
```
38+
39+
1. If you support air gap installations, update all image references so they resolve correctly in both online and air gap installations. See [Add support for air gap installations](#local-image-registry) on this page.
40+
41+
1. Add an [Embedded Cluster Config](embedded-config) manifest to the release. At minimum, the Config must specify the Embedded Cluster version to use.
2042
2143
<EcConfig/>
2244
23-
1. If your application requires that Embedded Cluster deploy certain components before the application and as part of the cluster itself, update the Embedded Cluster Config to add [extensions](embedded-config#extensions).
45+
1. If you use custom domains for the Replicated proxy registry or Replicated app service, add them to the Embedded Cluster Config `domains` key. See [Configure Embedded Cluster to use custom domains](/vendor/custom-domains-using#ec) in _Use custom domains_.
46+
47+
1. If you need Embedded Cluster to deploy certain components to the cluster before it deploys your application, add the Helm charts for those components to the Embedded Cluster Config `extensions` key. See [(Optional) Add Helm chart extensions](#add-extensions) on this page.
2448

2549
1. Save the release and promote it to the channel that you use for testing internally.
2650

2751
1. Install with Embedded Cluster in a development environment to test. See [Online installation with Embedded Cluster](installing-embedded) or [Air gap installation with Embedded Cluster](installing-embedded-air-gap).
2852

29-
1. After successfully installing your application with Embedded Cluster, customize the [Embedded Cluster Config](embedded-config) as desired:
30-
* Add your custom domain for the Replicated proxy registry and Replicated app service. See [domains](embedded-config#domains).
31-
* Add custom Helm extensions. Extensions allow you to provide Helm charts that Embedded Cluster deploys before your application. For example, you can add a Helm extension to ship an ingress controller. See [extensions](embedded-config#extensions).
32-
* Define roles to assign workloads to specific nodes in multi-node installations. See [roles](/embedded-cluster/v2/embedded-config#roles).
33-
34-
Replicated recommends that you work in small iterations and test your changes frequently in your development environment.
53+
## Add support for air gap installations {#local-image-registry}
54+
55+
To support air gap installations:
56+
57+
1. Configure each HelmChart custom resource's `builder` key. This ensures that all the required and optional images for your application are available in environments without internet access. See [`builder`](/reference/custom-resource-helmchart-v2#builder) in _HelmChart v2_.
58+
59+
1. Configure each HelmChart custom resource to ensure that all image reference resolve correctly in online and air gap installations. You do this in the HelmChart custom resource's [`values`](/reference/custom-resource-helmchart-v2#values) key using the [ReplicatedImageName](template-functions#replicatedimagename), [ReplicatedImageRegistry](template-functions#replicatedimageregistry), and [ReplicatedImageRepository](template-functions#replicatedimagerepository) template functions. See the following examples for more information:
60+
61+
<details>
62+
63+
<summary>Example (Single value for full image name)</summary>
64+
65+
For charts that expect the full image reference in a single field, use the [ReplicatedImageName](template-functions#replicatedimagename) template function in the HelmChart custom resource. ReplicatedImageName returns the full image name, including both the repository and registry.
66+
67+
For example:
68+
69+
```yaml
70+
# values.yaml
71+
initImage: proxy.replicated.com/proxy/my-app/docker.io/library/busybox:1.36
72+
```
73+
```yaml
74+
# HelmChart custom resource
75+
apiVersion: kots.io/v1beta2
76+
kind: HelmChart
77+
spec:
78+
values:
79+
initImage: '{{repl ReplicatedImageName (HelmValue ".initImage") true }}'
80+
```
81+
ReplicatedImageName sets `noProxy` to `true` because the image reference value in `values.yaml` already contains the proxy path prefix (`proxy.replicated.com/proxy/my-app/...`)
82+
</details>
83+
84+
<details>
85+
86+
<summary>Example (Separate values for image registry and repository)</summary>
87+
88+
If a chart uses separate registry and repository fields for image references, use the [ReplicatedImageRegistry](template-functions#replicatedimageregistry) and [ReplicatedImageRepository](template-functions#replicatedimagerepository) template functions in the HelmChart custom resource:
89+
90+
```yaml
91+
# values.yaml
92+
image:
93+
registry: proxy.replicated.com
94+
repository: proxy/my-app/ghcr.io/my-org/my-image
95+
tag: v1.0.0
96+
```
97+
98+
```yaml
99+
# HelmChart custom resource
100+
apiVersion: kots.io/v1beta2
101+
kind: HelmChart
102+
spec:
103+
values:
104+
image:
105+
registry: '{{repl ReplicatedImageRegistry (HelmValue ".image.registry") }}'
106+
repository: '{{repl ReplicatedImageRepository (HelmValue ".image.repository") true }}'
107+
```
108+
ReplicatedImageRepository sets `noProxy` to `true` because the repository value in `values.yaml` already contains the proxy path prefix (`proxy/my-app/...`). ReplicatedImageRegistry omits `noProxy` so the function returns the correct proxy domain for the installation, including any [custom domain](/vendor/custom-domains) configuration.
109+
</details>
110+
111+
<details>
112+
<summary>Example (References to public images)</summary>
113+
114+
For public images that don't go through the Replicated proxy registry, set the upstream reference directly in the chart's `values.yaml`. Use `noProxy` so the reference remains unchanged in online installations but is still rewritten for air gap.
115+
116+
```yaml
117+
# values.yaml
118+
publicImage: docker.io/library/busybox:1.36
119+
```
120+
121+
```yaml
122+
# HelmChart custom resource
123+
apiVersion: kots.io/v1beta2
124+
kind: HelmChart
125+
spec:
126+
values:
127+
publicImage: '{{repl ReplicatedImageName (HelmValue ".publicImage") true }}'
128+
```
129+
</details>
130+
131+
1. In the HelmChart resource that corresponds to the chart where you included the Replicated SDK as a dependency, rewrite the Replicated SDK image reference using the [ReplicatedImageRegistry](template-functions#replicatedimageregistry) and [ReplicatedImageRepository](template-functions#replicatedimagerepository) template functions:
132+
133+
```yaml
134+
# HelmChart custom resource
135+
apiVersion: kots.io/v1beta2
136+
kind: HelmChart
137+
spec:
138+
values:
139+
replicated:
140+
image:
141+
registry: '{{repl ReplicatedImageRegistry (HelmValue ".replicated.image.registry") }}'
142+
# The default location for the SDK image is
143+
# proxy.replicated.com/library/replicated-sdk-image
144+
repository: '{{repl ReplicatedImageRepository (HelmValue ".replicated.image.repository") true }}'
145+
```
146+
147+
1. If you added any Helm chart `extensions` in the Embedded Cluster Config, rewrite image references in each extension using either the [ReplicatedImageName](template-functions#replicatedimagename) template function (if the chart uses a single field for the full image reference) or the [ReplicatedImageRegistry](template-functions#replicatedimageregistry) and [ReplicatedImageRepository](template-functions#replicatedimagerepository) template functions (if the chart uses separate fields for the image registry and repository values).
148+
149+
<details>
150+
<summary>Example (Extension for a Helm chart that you own)</summary>
151+
152+
```yaml
153+
# Embedded Cluster Config
154+
apiVersion: embeddedcluster.replicated.com/v1beta1
155+
kind: Config
156+
spec:
157+
extensions:
158+
helmCharts:
159+
- chart:
160+
name: ingress
161+
chartVersion: "1.2.3"
162+
releaseName: ingress
163+
namespace: ingress
164+
values: |
165+
controller:
166+
image:
167+
registry: 'repl{{ ReplicatedImageRegistry (HelmValue ".controller.image.registry") }}'
168+
repository: 'repl{{ ReplicatedImageRepository (HelmValue ".controller.image.repository") true }}'
169+
```
170+
</details>
171+
172+
<details>
173+
<summary>Example (Extension for a third-party Helm chart)</summary>
174+
175+
```yaml
176+
# Embedded Cluster Config
177+
apiVersion: embeddedcluster.replicated.com/v1beta1
178+
kind: Config
179+
spec:
180+
extensions:
181+
helmCharts:
182+
- chart:
183+
name: ingress-nginx
184+
chartVersion: "4.11.3"
185+
releaseName: ingress-nginx
186+
namespace: ingress-nginx
187+
values: |
188+
controller:
189+
image:
190+
registry: 'repl{{ ReplicatedImageRegistry "registry.k8s.io" }}'
191+
repository: 'repl{{ ReplicatedImageRepository "registry.k8s.io/ingress-nginx/controller" }}'
192+
```
193+
The template functions will add the proxy prefix in online installations and rewrite to the local registry in air gap installations.
194+
195+
ReplicatedImageRepository requires the full upstream image path including the registry (for example, `registry.k8s.io/ingress-nginx/controller`), not just the repository.
196+
</details>
197+
198+
199+
1. Save the release and promote it to the channel that you use for testing internally.
200+
201+
1. Install with Embedded Cluster in a development environment to test. See [Air gap installation with Embedded Cluster](installing-embedded-air-gap).
202+
203+
## Add Helm chart extensions {#add-extensions}
204+
205+
If your application requires certain components deployed before the application and as part of the cluster itself, add them as [extensions](embedded-config#extensions) in the Embedded Cluster Config. For example, you can add a Helm extension to deploy an ingress controller. You can add extensions for Helm charts that you own or for third-party charts.
206+
207+
To add Helm extensions:
208+
209+
1. In the Embedded Cluster Config, add the Helm chart to the [`extensions`](embedded-config#extensions) key.
210+
211+
1. If you support air gap installations, configure each of your `extensions` so that they resolve correctly for both online and air gap installations. See [Add support for air gap installations](#local-image-registry) on this page.
212+
213+
1. Save the release and promote it to the channel that you use for testing internally.
214+
215+
1. Install with Embedded Cluster in a development environment to test. See [Online installation with Embedded Cluster](installing-embedded) or [Air gap installation with Embedded Cluster](installing-embedded-air-gap).
35216

36-
## (Optional) Serve installation assets using the Vendor API
217+
## Serve installation assets using the Vendor API
37218

38219
To install with Embedded Cluster, your end customers need to download the Embedded Cluster installer binary and their license. Air gap installations also require an air gap bundle. End customers can download all these installation assets using a curl command by following the installation steps available in the [Replicated Enterprise Portal](/vendor/enterprise-portal-about).
39220

@@ -92,6 +273,6 @@ Using the NVIDIA GPU Operator with Embedded Cluster requires configuring the con
92273

93274
When you configure the containerd options as shown earlier on this page, the NVIDIA GPU Operator automatically creates the required configurations in the `/etc/k0s/containerd.d/nvidia.toml` file. It is not necessary to create this file manually, or modify any other configuration on the hosts.
94275

95-
If you include the NVIDIA GPU Operator as a Helm extension, remove any existing containerd services running on the host (such as those deployed by Docker) before installing the release with Embedded Cluster. If any containerd services are present on the host, the NVIDIA GPU Operator will generate an invalid containerd config, causing the installation to fail. For more information, see [Installation failure when NVIDIA GPU Operator is included as Helm extension](#nvidia-gpu-operator) in _Troubleshooting Embedded Cluster_.
276+
If you include the NVIDIA GPU Operator as a Helm extension, remove any existing containerd services from the host before installing with Embedded Cluster. This includes services deployed by Docker. If any containerd services are present on the host, the NVIDIA GPU Operator will generate an invalid containerd config, causing the installation to fail. For more information, see [Installation failure when NVIDIA GPU Operator is included as Helm extension](#nvidia-gpu-operator) in _Troubleshooting Embedded Cluster_.
96277

97278
This is the result of a known issue with v24.9.x of the NVIDIA GPU Operator. For more information about the known issue, see [container-toolkit does not modify the containerd config correctly when there are multiple instances of the containerd binary](https://github.com/NVIDIA/nvidia-container-toolkit/issues/982) in the nvidia-container-toolkit repository in GitHub.

0 commit comments

Comments
 (0)