diff --git a/.agents/wiki/sources/whats-missing.md b/.agents/wiki/sources/whats-missing.md new file mode 100644 index 00000000..a3e6d057 --- /dev/null +++ b/.agents/wiki/sources/whats-missing.md @@ -0,0 +1,222 @@ +You have strong primitives. The gap is in the **composition layer** — the scripts that chain them into the higher-level flows you're describing. Here's what you can do today, what's missing, and where the daemon API itself has a gap. + +## Your Existing Primitives + +Every operation below is already available over JSON-RPC: + +| Primitive | What it gives you | Key metadata | +|---|---|---| +| `resolve` | Symbol identity at a position | `fqName`, `kind`, `visibility`, `supertypes`, `containingDeclaration` | +| `references` | All usages across workspace | `searchScope.exhaustive`, `searchScope.visibility`, per-reference `preview` | +| `call-hierarchy` (incoming) | Who calls this function | Bounded tree with `stats` + per-node `truncation` | +| `call-hierarchy` (outgoing) | What does this function call | Same bounded tree, outgoing direction | +| `type-hierarchy` | Supertypes + subtypes tree | Available on standalone backend only | +| `outline` | All declarations in a file | Nested tree: classes → members → nested classes | +| `workspace-symbol` | Find declarations by name | Substring or regex, filterable by `kind` | +| `rename` → `apply-edits` → `diagnostics` | Compiler-backed refactoring | Hash-based conflict detection, post-edit diagnostic validation | +| `semantic-insertion-point` | Where to insert new code | Offsets for class body start/end, file top/bottom, after imports | +| `imports/optimize` | Clean up imports after edits | Edit plan for unused/missing imports | [4-cite-0](#4-cite-0) + +## What's Missing: Three New Compound Scripts + +### 1. `kast-explore.sh` — Deep Symbol Navigation ("Codebase Navigation Sub-Agent") + +This is the "data flow mapping" flow. Given a symbol, build a complete picture of its role in the codebase. + +```mermaid +sequenceDiagram + participant Agent + participant Script as "kast-explore.sh" + participant Daemon + + Agent->>Script: --symbol=AnalysisServer + Script->>Daemon: resolve (file, offset) + Daemon-->>Script: symbol identity + Script->>Daemon: references (file, offset, includeDeclaration=true) + Daemon-->>Script: reference list + searchScope + Script->>Daemon: call-hierarchy (incoming, depth=2) + Daemon-->>Script: caller tree + Script->>Daemon: call-hierarchy (outgoing, depth=2) + Daemon-->>Script: callee tree + Script->>Daemon: type-hierarchy (both, depth=2) + Daemon-->>Script: supertype + subtype tree + Script->>Daemon: outline (declaration file) + Daemon-->>Script: sibling declarations + Script-->>Agent: Synthesized exploration result +``` + +**What it produces:** +```json +{ + "ok": true, + "symbol": { "fqName": "...", "kind": "CLASS", "visibility": "PUBLIC", "supertypes": [...] }, + "references": { "count": 12, "by_file": { "A.kt": 3, "B.kt": 9 }, "search_scope": {...} }, + "incoming_callers": { "root": {...}, "stats": {...} }, + "outgoing_callees": { "root": {...}, "stats": {...} }, + "type_hierarchy": { "supertypes": [...], "subtypes": [...] }, + "file_context": { "file": "AnalysisServer.kt", "sibling_declarations": [...] }, + "completeness": { + "references_exhaustive": true, + "callers_truncated": false, + "callees_truncated": false, + "type_hierarchy_truncated": false + } +} +``` + +This is a strict superset of what `kast-impact.sh` does today. `kast-impact.sh` does resolve + references + optional incoming callers. [4-cite-1](#4-cite-1) + +The new script adds: **outgoing callees**, **type hierarchy**, **file outline for context**, and a **completeness summary** that tells the LLM exactly which parts of the result are bounded. + +**What you can build this from today:** All five daemon calls already exist. The script is pure composition — no new daemon endpoints needed. The only caveat is that `type-hierarchy` is not available on the IntelliJ plugin backend (it throws `CapabilityNotSupportedException`), so the script should gracefully skip it when the capability is absent. [4-cite-2](#4-cite-2) + +### 2. `kast-module-profile.sh` — Module API Surface Mapping ("General Form") + +This is the "understand a module's reference shape" flow. Given a module path (e.g., `analysis-api`), enumerate its public API and compute the reference shape for each symbol. + +```mermaid +graph TD + A["Input: module path"] --> B["find *.kt files in module"] + B --> C["outline each file"] + C --> D["filter to public/internal symbols"] + D --> E["for each symbol: resolve + references"] + E --> F["aggregate: reference counts by source module"] + F --> G["emit module profile JSON"] +``` + +**What it produces:** +```json +{ + "ok": true, + "module_path": "analysis-api/src/main/kotlin", + "symbols": [ + { + "fqName": "io.github.amichne.kast.api.AnalysisBackend", + "kind": "INTERFACE", + "visibility": "PUBLIC", + "reference_count": 47, + "referenced_from_files": ["AnalysisDispatcher.kt", "CliService.kt", ...], + "referenced_from_modules": { "analysis-server": 12, "kast-cli": 8, "backend-standalone": 15, ... } + }, + ... + ], + "stats": { "total_symbols": 42, "total_references": 312, "files_scanned": 18 } +} +``` + +**The gap here:** There is no daemon endpoint to list workspace files or enumerate modules. The `workspace-symbol` command can find symbols by name, but it can't enumerate "all public symbols in module X." [4-cite-3](#4-cite-3) + +You have two options: +- **Client-side file enumeration:** The script uses `find` to list `.kt` files under a module path, then calls `outline` on each file. This works today — no daemon changes needed. The `kast-common.sh` already does client-side file scanning via `rglob("*.kt")` for candidate collection. [4-cite-4](#4-cite-4) +- **New daemon endpoint (future):** A `workspace/files` or `workspace/modules` endpoint that returns the file list and module structure the daemon already knows about internally (via `StandaloneAnalysisSession.allKtFiles()` and `sourceModuleSpecs`). [4-cite-5](#4-cite-5) + +### 3. `kast-progressive-refactor.sh` — Compiler-Backed Progressive Rename ("Observed Form → General Form") + +This is the "drive true refactors at a module level" flow. Given a list of rename operations, execute them sequentially with diagnostic gates between each step. + +```mermaid +sequenceDiagram + participant Agent + participant Script as "kast-progressive-refactor.sh" + participant Daemon + + Agent->>Script: --plan-file=refactor-plan.json + loop For each rename in plan + Script->>Daemon: rename (dry-run) + Daemon-->>Script: edit plan + fileHashes + Script->>Daemon: apply-edits + Daemon-->>Script: applied + Script->>Daemon: workspace/refresh + Daemon-->>Script: refreshed + Script->>Daemon: diagnostics (affected files) + Daemon-->>Script: diagnostic result + alt Diagnostics clean + Script->>Script: Record success, continue + else Errors found + Script->>Script: Record failure, halt or rollback + end + end + Script-->>Agent: Progressive refactor result +``` + +**What it produces:** +```json +{ + "ok": false, + "completed": 3, + "total": 5, + "steps": [ + { "symbol": "OldName1", "new_name": "NewName1", "ok": true, "edit_count": 4, "diagnostics": { "clean": true } }, + { "symbol": "OldName2", "new_name": "NewName2", "ok": true, "edit_count": 7, "diagnostics": { "clean": true } }, + { "symbol": "OldName3", "new_name": "NewName3", "ok": true, "edit_count": 2, "diagnostics": { "clean": true } }, + { "symbol": "OldName4", "new_name": "NewName4", "ok": false, "stage": "diagnostics", "error_count": 2 } + ], + "halted_at_step": 4, + "halt_reason": "ERROR-severity diagnostics after rename" +} +``` + +**What you can build this from today:** `kast-rename.sh` already does the full single-rename loop (resolve → plan → apply → diagnostics). The progressive script wraps N invocations with a `workspace/refresh` between each to ensure the daemon sees the updated files. The key addition is the **halt-on-error gate** and the **cumulative result** that tells the LLM exactly where the refactoring stopped and why. [4-cite-6](#4-cite-6) + +The `workspace/refresh` endpoint already exists and is critical here — after each `apply-edits`, the daemon needs to re-index the changed files before the next rename can resolve correctly. [4-cite-7](#4-cite-7) + +## The One Real API Gap + +The daemon has no way to answer "what files/modules exist in this workspace?" The session internally tracks this via `allKtFiles()` and `sourceModuleSpecs`, but it's not exposed over JSON-RPC. [4-cite-8](#4-cite-8) + +For the module-profile flow, you need to enumerate files. Today you can work around this with client-side `find` (which is what `kast-common.sh` already does). But a proper `workspace/files` endpoint would let the agent ask the daemon directly, which is more reliable because the daemon knows the actual resolved source roots and module boundaries — including Gradle-discovered source sets, test fixtures, and custom source sets. [4-cite-9](#4-cite-9) + +A minimal endpoint would return: +```json +{ + "modules": [ + { "name": ":app[main]", "sourceRoots": [...], "files": [...] }, + { "name": ":app[test]", "sourceRoots": [...], "files": [...] }, + { "name": ":lib[main]", "sourceRoots": [...], "files": [...] } + ] +} +``` + +This maps directly to what `StandaloneAnalysisSession` already computes — `sourceModuleSpecs` has the module names and source roots, and `allKtFiles()` has the file list. + +## How These Compose Into Your Two Use Cases + +### Codebase Navigation Sub-Agent + +The LLM uses `kast-explore.sh` as its primary tool. The flow: + +1. User says "understand how `AnalysisDispatcher` works" +2. Agent calls `kast-explore.sh --symbol=AnalysisDispatcher` +3. Gets back: symbol identity, all references, incoming callers, outgoing callees, type hierarchy, file context +4. Agent synthesizes: "AnalysisDispatcher is a class in analysis-server that routes JSON-RPC methods to AnalysisBackend. It's called from UnixDomainSocketRpcServer, StdioRpcServer, and TcpRpcServer. It calls all 15 AnalysisBackend methods. It has no subtypes." +5. If the agent needs to go deeper on a specific caller, it calls `kast-explore.sh` again on that caller + +This is the "IntelliJ data flow mapping" equivalent — the agent builds a mental model by recursively exploring the graph. + +### Module-Level Refactoring + +The LLM uses `kast-module-profile.sh` + `kast-progressive-refactor.sh`: + +1. Agent calls `kast-module-profile.sh --module-path=analysis-api/src/main/kotlin` → gets the "observed form" (current API surface + reference shape) +2. Agent compares against the "general form" (the desired pattern — e.g., "all query types should follow `XyzQuery` naming, all result types should follow `XyzResult` naming") +3. Agent identifies deviations (e.g., `ImportOptimizeQuery` should be `OptimizeImportsQuery`) +4. Agent builds a rename plan JSON and calls `kast-progressive-refactor.sh --plan-file=plan.json` +5. Each rename is applied with compiler validation between steps +6. If a rename breaks something, the flow halts and the agent gets a structured error to reason about + +## Priority Order + +1. **`kast-explore.sh`** — highest value, pure composition of existing primitives, no daemon changes needed. This is the "deep navigation" tool that makes the LLM genuinely useful for understanding code. + +2. **`kast-progressive-refactor.sh`** — second highest, also pure composition. This is the "drive true refactors" tool. The main subtlety is the `workspace/refresh` between steps. + +3. **`kast-module-profile.sh`** — requires either client-side file enumeration (works today) or a new `workspace/files` daemon endpoint (better long-term). This is the "general form vs observed form" tool. + +4. **`workspace/files` daemon endpoint** — the one actual API addition. Low complexity (the data already exists in `StandaloneAnalysisSession`), high leverage for module-level operations. + +## What You Can't Do Yet + +- **Cross-workspace analysis** — Kast is one daemon per workspace. If a refactor spans multiple repositories, you need multiple daemons. +- **Import-aware rename** — The standalone backend has a comment noting "A future import-aware rename pass can append import edits here once qualified-reference tracking is added." Today, renames don't update import statements that use the old name. [4-cite-10](#4-cite-10) +- **Arbitrary code edits with semantic validation** — You can rename and apply text edits, but you can't ask the daemon "generate the code for a new method that implements interface X." The daemon validates; it doesn't generate. +- **Type hierarchy on IntelliJ backend** — Currently throws `CapabilityNotSupportedException`. The explore flow needs to gracefully degrade. [4-cite-2](#4-cite-2) diff --git a/.github/scripts/smoke-installer.sh b/.github/scripts/smoke-installer.sh index 8632d026..56a7db74 100755 --- a/.github/scripts/smoke-installer.sh +++ b/.github/scripts/smoke-installer.sh @@ -83,19 +83,21 @@ trap cleanup EXIT cp "$portable_zip" "$asset_path" python3 - "$metadata_path" "$asset_path" <<'PY' +import hashlib import json import sys from pathlib import Path metadata_path = Path(sys.argv[1]) asset_path = Path(sys.argv[2]) +digest = hashlib.sha256(asset_path.read_bytes()).hexdigest() payload = { "tag_name": "v0.0.0-smoke", "assets": [ { "name": asset_path.name, "browser_download_url": asset_path.as_uri(), - "digest": "", + "digest": f"sha256:{digest}", } ], } diff --git a/.github/scripts/smoke-kast-cli.sh b/.github/scripts/smoke-kast-cli.sh index e19a69ae..1feb9b00 100755 --- a/.github/scripts/smoke-kast-cli.sh +++ b/.github/scripts/smoke-kast-cli.sh @@ -179,6 +179,44 @@ assert diagnostics["diagnostics"][0]["code"] == "UNRESOLVED_REFERENCE" edit_files = {Path(edit["filePath"]).name for edit in rename["edits"]} assert edit_files == {"Greeter.kt", "Use.kt", "SecondaryUse.kt"} assert set(Path(path).name for path in rename["affectedFiles"]) == edit_files + +# Build an ApplyEditsQuery from the rename result +apply_edits_query = { + "edits": rename["edits"], + "fileHashes": rename["fileHashes"], + "fileOperations": [], +} +(tmp_dir / "apply-edits-request.json").write_text( + json.dumps(apply_edits_query), encoding="utf-8" +) +PY + +KAST_CONFIG_HOME="$instance_dir" \ + "$KAST_CMD" apply-edits \ + --workspace-root="$workspace_dir" \ + --request-file="${tmp_dir}/apply-edits-request.json" \ + --wait-timeout-ms=180000 >"${tmp_dir}/apply-edits.json" + +python3 - "$tmp_dir" "$workspace_dir" <<'PY' +import json +import sys +from pathlib import Path + +tmp_dir = Path(sys.argv[1]) +workspace_dir = Path(sys.argv[2]) +source_root = workspace_dir / "src/main/kotlin/sample" + +apply_result = json.loads((tmp_dir / "apply-edits.json").read_text(encoding="utf-8")) +assert len(apply_result.get("applied", [])) > 0, f"expected edits to be applied: {apply_result}" + +greeter_text = (source_root / "Greeter.kt").read_text(encoding="utf-8") +use_text = (source_root / "Use.kt").read_text(encoding="utf-8") +secondary_text = (source_root / "SecondaryUse.kt").read_text(encoding="utf-8") + +assert "welcome" in greeter_text, f"Greeter.kt should contain 'welcome' after apply-edits: {greeter_text}" +assert "greet" not in greeter_text, f"Greeter.kt should not contain 'greet' after rename: {greeter_text}" +assert "welcome" in use_text, f"Use.kt should contain 'welcome' after apply-edits: {use_text}" +assert "welcome" in secondary_text, f"SecondaryUse.kt should contain 'welcome' after apply-edits: {secondary_text}" PY KAST_CONFIG_HOME="$instance_dir" \ diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 24afe79e..dee5f005 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -13,8 +13,8 @@ env: FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true jobs: - cli-smoke: - name: CLI smoke (${{ matrix.os }}) + build-and-test: + name: Build & test (${{ matrix.os }}) runs-on: ${{ matrix.os }} strategy: fail-fast: false @@ -25,12 +25,10 @@ jobs: steps: - uses: actions/checkout@v5 - - uses: graalvm/setup-graalvm@v1 + - uses: actions/setup-java@v4 with: - distribution: graalvm-community + distribution: temurin java-version: "21" - github-token: ${{ secrets.GITHUB_TOKEN }} - native-image-job-reports: "true" - uses: gradle/actions/setup-gradle@v4 @@ -39,18 +37,81 @@ jobs: path: ~/.gradle/kast/intellij-distributions key: idea-dist-${{ hashFiles('gradle/libs.versions.toml') }} - - name: Build and test Kast + - name: Test and build portable distribution run: > - ./gradlew + ./gradlew -PjvmOnly :analysis-api:test :analysis-server:test :backend-standalone:test + :kast-cli:test + :kast:test + :kast:portableDistZip + + - name: Upload portable distribution + uses: actions/upload-artifact@v4 + with: + name: kast-portable-${{ matrix.os }} + path: kast/build/distributions/kast-*-portable.zip + if-no-files-found: error + retention-days: 1 + + test-intellij-plugin: + name: IntelliJ plugin + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v5 + + - uses: actions/setup-java@v4 + with: + distribution: temurin + java-version: "21" + + - uses: gradle/actions/setup-gradle@v4 + + - uses: actions/cache@v4 + with: + path: ~/.gradle/kast/intellij-distributions + key: idea-dist-${{ hashFiles('gradle/libs.versions.toml') }} + + - name: Test and verify IntelliJ plugin + run: > + ./gradlew :backend-intellij:test :backend-intellij:buildPlugin :backend-intellij:verifyPluginStructure :backend-intellij:verifyPluginXmlPresent - :kast:test - :kast:portableDistZip + + - name: Upload IntelliJ plugin artifact + uses: actions/upload-artifact@v4 + with: + name: kast-intellij-plugin + path: backend-intellij/build/distributions/*.zip + if-no-files-found: error + retention-days: 1 + + smoke-kast-cli: + name: Smoke CLI (${{ matrix.os }}) + runs-on: ${{ matrix.os }} + needs: build-and-test + strategy: + fail-fast: false + matrix: + os: + - ubuntu-latest + - macos-latest + steps: + - uses: actions/checkout@v5 + + - uses: actions/setup-java@v4 + with: + distribution: temurin + java-version: "21" + + - name: Download portable distribution + uses: actions/download-artifact@v4 + with: + name: kast-portable-${{ matrix.os }} + path: kast/build/distributions - name: Smoke portable Kast distribution shell: bash @@ -77,7 +138,7 @@ jobs: eval-agent-routing: name: Eval agent routing runs-on: ubuntu-latest - needs: cli-smoke + needs: smoke-kast-cli steps: - uses: actions/checkout@v5 @@ -86,10 +147,21 @@ jobs: distribution: temurin java-version: "21" - - uses: gradle/actions/setup-gradle@v4 + - name: Download portable distribution + uses: actions/download-artifact@v4 + with: + name: kast-portable-ubuntu-latest + path: kast/build/distributions - - name: Build kast - run: ./gradlew --no-daemon :kast:installDist + - name: Prepare kast binary + shell: bash + run: | + set -euo pipefail + dist_dir="$RUNNER_TEMP/kast-dist" + rm -rf "$dist_dir" + mkdir -p "$dist_dir" + unzip -q kast/build/distributions/kast-*-portable.zip -d "$dist_dir" + echo "KAST_BIN=$dist_dir/kast/kast" >> "$GITHUB_ENV" - name: Run kast-routing evals run: bash evals/harness/run-evals.sh --suite=kast-routing --format=json diff --git a/backend-standalone/src/test/kotlin/io/github/amichne/kast/standalone/CacheManagerTest.kt b/backend-standalone/src/test/kotlin/io/github/amichne/kast/standalone/CacheManagerTest.kt index 13be2e53..accfc819 100644 --- a/backend-standalone/src/test/kotlin/io/github/amichne/kast/standalone/CacheManagerTest.kt +++ b/backend-standalone/src/test/kotlin/io/github/amichne/kast/standalone/CacheManagerTest.kt @@ -10,6 +10,7 @@ import org.junit.jupiter.api.Assertions.assertTrue import org.junit.jupiter.api.Test import org.junit.jupiter.api.io.TempDir import java.nio.file.Files +import java.nio.file.NoSuchFileException import java.nio.file.Path import java.util.concurrent.atomic.AtomicInteger import kotlin.concurrent.thread @@ -32,15 +33,16 @@ class CacheManagerTest { val cacheManager = CacheManager(workspaceRoot) val cacheFile = kastCacheDirectory(normalizeStandalonePath(workspaceRoot)).resolve("debounce.txt") val writeCount = AtomicInteger(0) + val debounceDelayMillis = 100L repeat(10) { - cacheManager.schedule(key = "debounce", delayMillis = 25) { + cacheManager.schedule(key = "debounce", delayMillis = debounceDelayMillis) { val nextCount = writeCount.incrementAndGet() writeCacheFileAtomically(cacheFile, nextCount.toString()) } } - waitUntil { writeCount.get() == 1 && Files.isRegularFile(cacheFile) } + waitUntil { writeCount.get() == 1 && readCacheFileIfPresent(cacheFile) == "1" } assertEquals("1", Files.readString(cacheFile)) cacheManager.close() } @@ -53,6 +55,7 @@ class CacheManagerTest { val newPayload = "new".repeat(8_192) val observedPayloads = linkedSetOf() writeCacheFileAtomically(cacheFile, oldPayload) + observedPayloads += Files.readString(cacheFile) val writer = thread(start = true) { cacheManager.runNow { @@ -61,11 +64,15 @@ class CacheManagerTest { } while (writer.isAlive) { - observedPayloads += Files.readString(cacheFile) + readCacheFileIfPresent(cacheFile)?.let(observedPayloads::add) + Thread.yield() } writer.join() + waitUntil { readCacheFileIfPresent(cacheFile) == newPayload } observedPayloads += Files.readString(cacheFile) + assertTrue(observedPayloads.contains(oldPayload)) + assertTrue(observedPayloads.contains(newPayload)) assertTrue(observedPayloads.all { payload -> payload == oldPayload || payload == newPayload }) cacheManager.close() } @@ -195,4 +202,10 @@ class CacheManagerTest { } error("Condition was not met within ${timeoutMillis}ms") } + + private fun readCacheFileIfPresent(path: Path): String? = try { + Files.readString(path) + } catch (_: NoSuchFileException) { + null + } } diff --git a/evals/harness/run-evals.sh b/evals/harness/run-evals.sh index 5b4aac85..ddb3d989 100755 --- a/evals/harness/run-evals.sh +++ b/evals/harness/run-evals.sh @@ -261,7 +261,7 @@ log "Workspace: ${WORKSPACE:-}" log "Transcript dir: $TRANSCRIPT_DIR" # ── Build kast if needed ───────────────────────────────────────────────── -KAST_BIN="$REPO_ROOT/kast/build/install/kast/bin/kast" +KAST_BIN="${KAST_BIN:-$REPO_ROOT/kast/build/install/kast/bin/kast}" if [[ ! -x "$KAST_BIN" ]]; then log_step "Building kast (installDist)…" (cd "$REPO_ROOT" && ./gradlew :kast:installDist --no-daemon --quiet) || die "kast build failed" diff --git a/response.md b/response.md new file mode 100644 index 00000000..7951720d --- /dev/null +++ b/response.md @@ -0,0 +1,119 @@ +Now I have enough to give a thorough review. Here it is: + +--- + +## CI & Smoke Review + +### The core architectural problem + +The `kast` module is the glue: it depends on both `:kast-cli` and `:backend-standalone` and produces the one portable zip that CI tests. This means the two units are **co-packaged and never independently validated**. Every smoke test exercises them as a single artifact. If you want atomic, independently releasable units, this coupling is the root issue to resolve — everything else follows from it. + +--- + +### Flaws in the smoke scripts + +**1. Installer smoke bypasses checksum verification (`smoke-installer.sh:99`)** + +```python +"digest": "", +``` + +An empty string means the installer's SHA-256 verification path is never exercised. The smoke gives false confidence: it proves the install *flow* works but not the *security-critical* checksum path. The fix is to compute the actual SHA-256 of the test zip and populate `digest` before running the installer: + +```python +import hashlib +digest = hashlib.sha256(asset_path.read_bytes()).hexdigest() +payload = { ..., "assets": [{ ..., "digest": f"sha256:{digest}" }] } +``` + +**2. Native binary is never tested in CI** + +`ci.yml` builds `:kast:portableDistZip` but **GraalVM native compilation is not in the CI build** — it's only in `release.yml`. The native binary (`bin/kast`) is therefore absent from the portable zip that CI smokes. What CI actually validates is the JVM fallback path in the wrapper script. You can ship a broken native binary and CI will never catch it. + +Mitigation options: add `:kast-cli:nativeCompile` to CI with `--no-fallback` disabled for speed, or have a separate CI job that at least confirms the native binary compiles and runs `--help`. The `kast` module's `syncPortableDist` already skips native if `jvmOnly` property is set — use that property explicitly in CI rather than implicitly relying on it. + +**3. `analysis-server` is never independently smoked** + +Every smoke call goes through the kast CLI → daemon subprocess path. There's zero validation that `backend-standalone` starts and responds to RPC in isolation. Since you want the server to be startable as a standalone background process, there should be a smoke that: + +1. Invokes `backend-standalone --transport=unix-socket --socket-path=` directly +2. Sends a `capabilities` RPC request against the socket +3. Asserts the response + +Without this, the "decoupled runnable" story is untestable in CI. + +**4. Rename smoke validates planning, not edit application** + +The smoke asserts `rename["edits"]` contains the right files but never calls `kast apply-edits` to verify those edits land on disk correctly. The mutation path is the most dangerous — if the apply-edits command regresses, CI won't catch it. + +**5. Stop-polling is fragile** + +```bash +for _ in $(seq 1 30); do + if ! find "$instance_dir" -name '*.json' -print -quit | grep -q .; then + break + fi + sleep 1 +done +``` + +This can either flake (daemon took 31 seconds to clean up) or silently pass when the daemon didn't actually stop (loop exits after 30s regardless). Replace the sleep loop with a proper wait — the `workspace stop` command should block until the descriptor is gone, or the smoke should fail immediately if descriptors remain after a timeout. + +--- + +### Flaws in `ci.yml` structure + +**6. Monolithic `cli-smoke` job** + +All of these run in a single sequential job: +``` +:analysis-api:test +:analysis-server:test +:backend-standalone:test +:backend-intellij:test +:backend-intellij:buildPlugin +:backend-intellij:verifyPluginStructure +:kast:test +:kast:portableDistZip +``` + +`backend-intellij:test` pulls the IntelliJ distribution and takes significant time. A `backend-intellij` test failure blocks the CLI smoke entirely. These should be split into independent parallel jobs with explicit `needs:` wiring: + +``` +┌─ test-analysis-api ─────────┐ +│ ├─> smoke-kast-cli +├─ test-analysis-server ──────┤ +│ │ +├─ build-intellij-plugin ─────┘ + +└─ eval-agent-routing (needs: smoke-kast-cli) +``` + +**7. GraalVM setup is wasted for the JVM-only smoke** + +CI sets up `graalvm/setup-graalvm` then builds a portable zip without the native binary (because `nativeCompile` isn't in the task list). GraalVM setup adds minutes for no benefit. Use `actions/setup-java` with Temurin in CI and reserve GraalVM for the release workflow. + +**8. `eval-agent-routing` builds `kast` redundantly** + +The eval job runs `./gradlew :kast:installDist` after `cli-smoke` already built the full portable zip. If the eval job received the built distribution as an artifact from `cli-smoke`, you'd avoid rebuilding from scratch. + +--- + +### What true atomic releases require + +| Unit | Today | Needed | +|------|-------|--------| +| `kast-cli` | Co-packaged with `backend-standalone` via `kast` module | Own release zip (native binary + embedded skill). Needs a way to locate/start a separately-installed backend. | +| `analysis-server` | Bundled inside `kast` distribution; never independently released | Own `backend-standalone--.zip` release asset. Own smoke. Own install path in `install.sh`. | +| `analysis-api` | `kast.kotlin-library` plugin only — no `maven-publish` visible in build file | Apply `maven-publish`, publish to GitHub Packages or Maven Central. | +| `backend-intellij` | Already effectively atomic (own zip) | Already correct. Verify the plugin zip release is independently installable via JetBrains toolbox/settings without `kast-cli` present. | + +The `kast` module today acts as a meta-package. To decouple: + +1. **`kast-cli` release**: Drop the `:backend-standalone` dependency from the `kast` module (or create a new `kast-cli-dist` module). The CLI already supports `internalDaemonRunner = null` and can discover an external daemon. Make the external backend path first-class rather than an afterthought. + +2. **`backend-standalone` release**: Add its own release job to `release.yml` (parallel with the CLI builds). Add a standalone smoke to `ci.yml`. The installer already has the infrastructure for multiple assets — extend it to handle `--components=standalone` independently. + +3. **`analysis-api` Maven publish**: Add `maven-publish` to the `kast.kotlin-library` plugin or specifically to `analysis-api/build.gradle.kts`. Coordinate versioning so the API version is independently consumable by plugin authors. + +The most impactful single change is **splitting `kast` module's dependencies** so `kast-cli` can be built and distributed without `backend-standalone`. That one structural change unlocks independent versioning, independent smoke testing, and independent release for both units. \ No newline at end of file