storage: add dynamic managed storage dispatcher

Add an optional "adapter" field in the ManagedVolumeConnection struct,
allowing dispatching to vendor managedvolume-helper adapters.

Drop the Python 2 workaround when running the helper, also dropping the
requirement for it (and thus the vendor-provided adapters) to be a
Python script.

Signed-off-by: Viktor Ivanov <viktor.ivanov@storpool.com>

Author: sp-viktori

doc/managed-volume-adapters.md

@@ -0,0 +1,52 @@
+<!--
+SPDX-FileCopyrightText: oVirt Developers
+SPDX-License-Identifier: GPL-2.0-or-later
+-->
+
+# Managed Volume Adapters
+
+By default the managed volumes in Vdsm are manipulated through an external
+helper that interfaces with the `os_brick` package (part of OpenStack).
+
+Similarly on the oVirt Engine side there is another external helper that
+interfaces with CinderLib.
+
+The two together can expose any supported Cinder storage driver to oVirt.
+
+- on the oVirt Engine side the helper deals with creating and managing volumes,
+  snapshots and preparing them to be attached to a VM;
+
+- on the Vdsm side the helper deals only with attaching and detaching volumes
+  that have been created by the oVirt Engine.
+
+The adapter mechanism redirects the execution to vendor-provided helper
+executables. To facilitate this redirection both the oVirt Engine and Vdsm have
+the notion of `adapter`.
+
+This allows storage vendors to integrate their managed storage directly in
+oVirt/Vdsm.
+
+## oVirt Engine
+
+Managed Storage Domains with adapter dispatch have an `adapter` field in their
+`driver_options` map that indicates the helper executable to run instead of
+the default `cinderlib-client.py` helper.
+
+The vendor packaging is expected to install a symlink in the Managed Block
+Storage data directory (defaults to `/usr/share/ovirt-engine/managedblock/`)
+named `{adapter}-adapter`.
+
+## Vdsm
+
+The `adapter` field is passed to Vdsm in the `connection_info` parameter (of
+type `ManagedVolumeConnection`).
+
+If the `adapter` field is not present, Vdsm uses the default OS-Brick helper
+`managedvolume-helper`.
+
+If the `adapter` field is present, Vdsm will instead execute
+`managedvolume-helper-{adapter}` when attaching and detaching volumes.
+
+The default install path for the helper (and where Vdsm would be looking for the
+adapter) is `/usr/libexec/vdsm`. Vendors should add a symlink in that directory
+to their own installation path.

lib/vdsm/api/vdsm-api.yml

@@ -1185,6 +1185,11 @@ types:
                 volume type.
             name: data
             type: *StringMap
+        -   defaultvalue: null
+            description: Adapter vendor
+            name: adapter
+            type: string
+            added: '4.5.8'
         type: object
 
     ManagedVolumeAttachement: &ManagedVolumeAttachement

lib/vdsm/storage/managedvolume.py

@@ -17,7 +17,6 @@
 import json
 import logging
 import os
-import sys
 
 from contextlib import closing
 
@@ -190,13 +189,21 @@ def run_helper(sub_cmd, vol_info=None):
         return supervdsm.getProxy().managedvolume_run_helper(
             sub_cmd, vol_info=vol_info)
     try:
+        adapter = None
         cmd_input = None
         if vol_info:
             cmd_input = json.dumps(vol_info).encode("utf-8")
-        # This is the only sane way to run python scripts that work with both
-        # python2 and python3 in the tests.
-        # TODO: Remove when we drop python 2.
-        cmd = [sys.executable, HELPER, sub_cmd]
+            adapter = vol_info.get("connection_info", {}).get("adapter")
+        helper = HELPER
+        if adapter:
+            helper = f"{HELPER}-{adapter}"
+            if not (
+                os.path.exists(helper) and os.access(helper, os.X_OK)
+            ):
+                raise se.ManagedVolumeHelperFailed(
+                    f"Helper for adapter '{adapter}' not found or"
+                    f" not executable at '{helper}'")
+        cmd = [helper, sub_cmd]
         result = commands.run(cmd, input=cmd_input)
     except cmdutils.Error as e:
         raise se.ManagedVolumeHelperFailed("Error executing helper: %s" % e)