[V5] CLI 2.0 - Pluggable Backends, Non-TTY Default, HTTP Dispatcher, Spec Files + Codegen

[deeleeramone]

This PR is a WIP. The goal is to reduce the overhead of running CLI by making it compatible with just the openbb-core package as the default, and expanding the base feature set to provide more utility.

Rewrites the CLI to be agent/CI-friendly out of the box and to talk to a remote openbb-platform-api server without importing openbb locally.

The legacy behavior is available via the -i flag with collection of ODP extensions installed.

For performant one-shots that are compatible with multi-user operation, it is recommended to generate a spec file from the running server and operate the CLI from a separate environment with only openbb-cli installed.

openbb --server http://127.0.0.1:6900 --generate-spec -o /tmp/openbb.spec

Now cold starts become instant.

openbb --spec /tmp/openbb.spec --describe economy.fred_series
{"id": null, "ok": true, "result": {"name": "economy.fred_series", "providers": {"fred": {"parameters": [{"name": "symbol", "in": "query", "type": "string", "required": true, "help": "Symbol to get data for. Multiple comma separated items allowed for provider(s): fred."}, {"name": "start_date", "in": "query", "type": "string", "help": "Start date of the data, in YYYY-MM-DD format."}, {"name": "end_date", "in": "query", "type": "string", "help": "End date of the data, in YYYY-MM-DD format."}, {"name": "limit", "in": "query", "type": "integer", "default": 100000, "help": "The number of data entries to return."}, {"name": "frequency", "in": "query", "type": "string", "choices": ["a", "q", "m", "w", "d", "wef", "weth", "wew", "wetu", "wem", "wesu", "wesa", "bwew", "bwem"], "help": "Frequency aggregation to convert high frequency data to lower frequency.\n    None = No change\n    a = Annual\n    q = Quarterly\n    m = Monthly\n    w = Weekly\n    d = Daily\n    wef = Weekly, Ending Friday\n    weth = Weekly, Ending Thursday\n    wew = Weekly, Ending Wednesday\n    wetu = Weekly, Ending Tuesday\n    wem = Weekly, Ending Monday\n    wesu = Weekly, Ending Sunday\n    wesa = Weekly, Ending Saturday\n    bwew = Biweekly, Ending Wednesday\n    bwem = Biweekly, Ending Monday"}, {"name": "aggregation_method", "in": "query", "type": "string", "default": "eop", "choices": ["avg", "sum", "eop"], "help": "A key that indicates the aggregation method used for frequency aggregation.\n        This parameter has no affect if the frequency parameter is not set.\n        avg = Average\n        sum = Sum\n        eop = End of Period"}, {"name": "transform", "in": "query", "type": "string", "choices": ["chg", "ch1", "pch", "pc1", "pca", "cch", "cca", "log"], "help": "Transformation type\n    None = No transformation\n    chg = Change\n    ch1 = Change from Year Ago\n    pch = Percent Change\n    pc1 = Percent Change from Year Ago\n    pca = Compounded Annual Rate of Change\n    cch = Continuously Compounded Rate of Change\n    cca = Continuously Compounded Annual Rate of Change\n    log = Natural Log"}], "output_schema": {"properties": {"date": {"type": "string", "format": "date", "title": "Date", "description": "The date of the data."}}, "additionalProperties": true, "type": "object", "required": ["date"], "title": "FredSeriesData", "description": "FRED Series Data."}}, "intrinio": {"parameters": [{"name": "symbol", "in": "query", "type": "string", "required": true, "help": "Symbol to get data for. Multiple comma separated items allowed for provider(s): fred."}, {"name": "start_date", "in": "query", "type": "string", "help": "Start date of the data, in YYYY-MM-DD format."}, {"name": "end_date", "in": "query", "type": "string", "help": "End date of the data, in YYYY-MM-DD format."}, {"name": "limit", "in": "query", "type": "integer", "default": 100000, "help": "The number of data entries to return."}, {"name": "all_pages", "in": "query", "type": "boolean", "default": false, "help": "Returns all pages of data from the API call at once."}, {"name": "sleep", "in": "query", "type": "number", "default": 1.0, "help": "Time to sleep between requests to avoid rate limiting."}], "output_schema": {"properties": {"date": {"type": "string", "format": "date", "title": "Date", "description": "The date of the data."}, "value": {"anyOf": [{"type": "number"}, {"type": "null"}], "title": "Value", "description": "Value of the index."}}, "additionalProperties": true, "type": "object", "required": ["date"], "title": "IntrinioFredSeriesData", "description": "Intrinio FRED Series Data."}}}}, "error": null}

Here is the new help text from openbb -h:

(obb5) darrenlee@Danglewood-i9 cli % openbb -h
usage: openbb [-h] [-i | --batch | --generate-spec | --generate-extension | --list-commands | --describe COMMAND | --print-config-template |
              --show-config] [--provider-name NAME] [--project-name NAME] [--package-name NAME] [--router-name NAME] [--config PATH]
              [--env-file PATH] [--batch-concurrency N] [--server URL] [--spec [NAME=]PATH] [--output PATH] [--openapi-path PATH]
              [--socrata-story URL_OR_PATH] [-H KEY=VALUE] [--header-file PATH] [-Q KEY=VALUE] [--query-param-file PATH] [--dev] [--debug]
              ...

OpenBB Platform CLI. Default mode is non-TTY.

positional arguments:
  command               Dotted command path followed by --key value pairs.

options:
  -h, --help            show this help message and exit
  -i, --interactive     Enter the interactive REPL with rich output and prompt-toolkit completion.
  --batch               Read NDJSON requests from stdin; emit NDJSON responses to stdout.
  --generate-spec       Fetch ``openapi.json`` from --server and write a precomputed .spec file to --output. Use --spec on subsequent invocations
                        to skip the OpenAPI fetch + parse on every call.
  --generate-extension  Generate a full installable OpenBB Platform extension package from a .spec file (--spec PATH). Each spec becomes one
                        provider with its own router. After generation: ``pip install -e <output>`` + ``openbb-build`` registers it. Configure
                        provider/project names via --provider-name / --project-name / --package-name / --router-name.
  --provider-name NAME  Snake-case provider identifier for --generate-extension (default: derived from --output).
  --project-name NAME   PyPI distribution name for --generate-extension (default: openbb-<provider-name>).
  --package-name NAME   Python package directory for --generate-extension (default: openbb_<provider-name>).
  --router-name NAME    Router identifier for --generate-extension (default: <provider-name>).
  --list-commands       Print every spec-declared command as a JSON list of ``{name, method, url_path, description}`` rows. Equivalent to
                        dispatching the reserved command ``__commands__``.
  --describe COMMAND    Print the full schema (parameters with type/required/default/choices/help, plus method/url_path/description) for one
                        command as JSON. Equivalent to dispatching ``__schema__`` with ``--name=COMMAND``.
  --print-config-template
                        Print a documented TOML template covering every supported config setting. Lines without a resolved value are commented out,
                        so dropping the output at ``~/.openbb_platform/openbb.toml`` is valid out of the box. Currently-resolved values from the
                        layered config are inlined as live values.
  --show-config         Print the merged TOML config — the result of layering ``pyproject.toml`` → user-global ``openbb.toml`` → project
                        ``openbb.toml`` → ``--config`` — as JSON. Useful for debugging which layer a given setting is coming from.
  --config PATH         Load CLI configuration from a TOML file. Layered atop ``[tool.openbb-cli]`` in the nearest ``pyproject.toml``,
                        ``~/.openbb_platform/openbb.toml`` (user-global), and any ``openbb.toml`` walking up from the working directory; CLI flags
                        and ``OPENBB_*`` env vars still win on conflict. Useful for swapping between ``congress.toml`` / ``platform.toml`` /
                        ``staging.toml`` without re-typing flags. (env: OPENBB_CLI_CONFIG)
  --env-file PATH       Path to a ``.env`` file to load into the process environment before argparse runs — so ``OPENBB_SERVER_URL``,
                        ``OPENBB_HTTP_QUERY_API_KEY``, etc. defined inside it populate the corresponding flags. ``~/.openbb_platform/.env`` is
                        always tried as well (the canonical openbb-platform .env location); real shell exports always beat both. (env:
                        OPENBB_CLI_ENV_FILE)
  --batch-concurrency N
                        Maximum number of concurrent in-flight dispatches in --batch mode. Higher values raise throughput against fast servers;
                        lower values give back-pressure when an upstream rate-limits. (env: OPENBB_CLI_BATCH_CONCURRENCY, default: 8)
  --server URL          Dispatch through an openbb-platform-api server at URL (env: OPENBB_SERVER_URL). Default: in-process LocalDispatcher.
  --spec [NAME=]PATH    Use a precomputed .spec file (built via --generate-spec) instead of fetching openapi.json. Repeatable: pass ``NAME=PATH``
                        to mount each spec under its own namespace (``congress.bill``, ``nyfed.markets.ambs``). A single unnamed ``--spec PATH``
                        keeps the flat (unprefixed) command surface. The spec carries the server URL it was generated against, so --server is not
                        required when using --spec. (env: OPENBB_SPEC_PATH)
  --output, -o PATH     Output path for --generate-spec (default: openbb.spec).
  --openapi-path PATH   Path (or full URL) to the OpenAPI document on the server. Defaults to /openapi.json. Servers that publish under a different
                        name (e.g. NY Fed at /static/docs/markets-api.yml) need this. JSON or YAML accepted.
  --socrata-story URL_OR_PATH
                        Generate a .spec from a Socrata story JSON. Walks the story for every ``datasetUid``, fetches each dataset column
                        metadata, and emits one router namespace per dataset with a ``query`` command wrapping ``/resource/{uid}.json`` and the
                        standard SoQL parameters (``$select`` / ``$where`` / ``$limit`` / ``$offset`` / ``$order`` / ``$group`` / ``$having`` /
                        ``$q``). Mutually exclusive with ``--server`` for ``--generate-spec``.
  -H, --header KEY=VALUE
                        Custom HTTP header to send on every dispatch and on the OpenAPI fetch. Repeatable: ``-H 'Authorization: Bearer xxx' -H
                        'X-Tenant: acme'``. Both ``KEY=VALUE`` and ``KEY: VALUE`` forms are accepted.
  --header-file PATH    Read additional headers from a JSON file (object of string values). Headers from --header take precedence on conflicts.
                        (env: OPENBB_HEADER_FILE)
  -Q, --query-param KEY=VALUE
                        Query parameter injected on every request (e.g. APIs that authenticate via ``?api_key=...`` like https://api.congress.gov).
                        Repeatable. Also auto-loaded from any env var prefixed ``OPENBB_HTTP_QUERY_`` — ``OPENBB_HTTP_QUERY_API_KEY=xxx`` becomes
                        ``?api_key=xxx``. CLI flag takes precedence over env.
  --query-param-file PATH
                        Read additional query params from a JSON file (object of string values). --query-param flags and ``OPENBB_HTTP_QUERY_*``
                        env vars take precedence on conflicts. (env: OPENBB_QUERY_PARAM_FILE)
  --dev                 Developer mode (verbose backend hooks).
  --debug               Debug logging.

Socrata Soda compatibility lets you point to a story, resource, view, or filter. The SoQL queries get bridged as OpenBB functions through a dedicated dispatcher, fully compatible with the .spec file. Essentially, creating an OpenAPI V3 client for the hosted service that can be used directly in the legacy TTY mode.

Codegen creates PEP-compliant ODP extension project code from a spec file. This makes it possible to instantly generate valid client code that works just like any other ODP provider and router extension.

Enter the terminal in interactive TTY mode by including the -i flag. No spec file required, loads into memory at runtime.

For Socrata APIs, use the --socrata-story argument:

openbb --socrata-story https://agtransport.usda.gov/resource/8bgf-5mdv -i

Generate a spec file, subsequent invocations are essentially zero-cost:

openbb --generate-spec \
       --server https://markets.newyorkfed.org \
       --openapi-path /static/docs/markets-api.yml \
       --output nyfed.spec

Generate extension code from a spec file:

openbb --generate-extension \
       --spec nyfed.spec \
       --provider-name nyfed \
       --output ./openbb-nyfed

Install and build:

pip install -e ./openbb-nyfed/openbb-nyfed
openbb-build

Run a command:

python -c "from openbb import obb; print(obb.nyfed.rates.all.latest().to_df())"

Spec files carry provenance and integrity checks:

{"version":5,"generated_at":"2026-05-03T17:27:22.112041+00:00","generator":"openbb-cli==2.0.0","source_url":"http://127.0.0.1:6900/openapi.json","openapi_version":"3.1.0","base_url":"http://127.0.0.1:6900","api_prefix":"/api/v1","commands":{ ... },"content_sha256":"b03be8fca58b031bd733202b2b1d329f1aed4de6335ffac6c000e578b8d9516d"}

Removed

• Hub routines (publish/fetch) and the account menu — surface no longer exists.

Packaging

Migrated to PEP 621 + Hatchling + uv.lock to match the V5 standard set by #7482. [tool.uv.sources] workspace-pins openbb-core, openbb-charting, and openbb-devtools to in-repo checkouts.

CI

Dedicated workflows under .github/workflows/:

• test-unit-cli.yml — full V5 platform matrix (Linux 3.10–3.15, macOS 3.10/3.15, Windows 3.10/3.14) with actions/cache on the uv wheel cache so source-built scientific deps (scipy/numpy/pandas) compile once per (os, python, dep-hash) tuple. Python 3.15 jobs install gfortran + OpenBLAS for the inevitable scipy source build.
• lint-openbb-cli.yml — ruff format/check + ty type-check on changed cli/**.py files. Cached uv install pulls the full [charting] surface so ty resolves every runtime import.

general-linting.yml is now scoped to openbb_platform/ only.

Coverage

100% test coverage

Notes for reviewers

• Default behavior change: openbb <cmd> no longer enters the REPL. Existing scripts that piped input to stdin must be migrated to openbb -i (REPL) or argv-style one-shot.
• Charting/interactive remain optional extras ([charting], [interactive], [all])