Skip to content

Commit 3c8569d

Browse files
amichnedevin-ai-integration[bot]CopilotCopilot
authored
intellij-backend: visibility-scoped search, parallel diagnostics, batched reads, telemetry (#106)
## Summary Eight performance and correctness improvements to the IntelliJ plugin backend: 1. **Visibility-scoped reference search** — `findReferences()` and `rename()` now narrow the `GlobalSearchScope` based on symbol visibility: `fileScope` for PRIVATE/LOCAL, `moduleScope` for INTERNAL, `projectScope` for PUBLIC/PROTECTED/UNKNOWN. The `SearchScopeKind` in the response reflects the actual scope used, and `candidateFileCount`/`searchedFileCount` report real file counts instead of result counts. 2. **Parallel diagnostics** — `diagnostics()` processes files concurrently via `coroutineScope { async(readDispatcher) { ... } }` instead of sequential `flatMap`. 3. **Batched workspaceSymbolSearch** — The three separate `timedReadAction` calls (classes, methods, fields) are combined into a single read action. 4. **Batched collectInShortReadActions** — Items are processed in batches of 50 per read action instead of one item per read action, reducing lock acquire/release overhead. 5. **Batched IntelliJTypeEdgeResolver.supertypeEdges** — Per-FQN read actions are combined into a single read action that resolves all supertypes at once. 6. **Shared ReadAccessScope** — A single `intellijReadAccess` field on `KastPluginBackend` replaces anonymous instances in `callHierarchy` and `typeHierarchy`. 7. **Missing telemetry scopes** — Added `TYPE_HIERARCHY`, `IMPLEMENTATIONS`, `COMPLETIONS`, `SEMANTIC_INSERTION_POINT`, `OPTIMIZE_IMPORTS`, and `WORKSPACE_SYMBOL_SEARCH` to the scope enum. Wrapped `typeHierarchy`, `implementations`, `completions`, `semanticInsertionPoint`, and `optimizeImports` with proper spans. Fixed `optimizeImports` and `workspaceSymbolSearch` which incorrectly used `DIAGNOSTICS` scope. 8. **Expanded performance test suite** — New `IntelliJBackendOperationPerformanceTest` exercises `findReferences` with private/public/internal symbols (validates scope narrowing), parallel `diagnostics` across multiple files, and `workspaceSymbolSearch` with timing budgets. All tests tagged `@Tag("performance")`. ## Review & Testing Checklist for Human - [ ] Verify `findReferences` for a private symbol returns `SearchScopeKind.FILE` and only searches the declaring file - [ ] Verify `findReferences` for an internal symbol returns `SearchScopeKind.MODULE` when a module can be resolved - [ ] Verify `rename` scope narrowing matches `findReferences` behavior - [ ] Run `./gradlew :backend-intellij:test -PincludeTags=performance` to validate the new performance baseline tests pass - [ ] Confirm parallel diagnostics produces identical results to sequential (no ordering or correctness regressions) ### Notes - The `collectInShortReadActions` function signature changed from `runPerItemReadAction` to `runBatchReadAction` to support batch processing. The `ReadActionBatchingTest` is updated accordingly. - Base branch is `devin/1777165956-remove-demo-code` (demo code removal). Link to Devin session: https://app.devin.ai/sessions/a368089e609f4a3bae92d4113b646f39 Requested by: @amichne --------- Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: amichne <22558698+amichne@users.noreply.github.com>
1 parent dcf9b09 commit 3c8569d

124 files changed

Lines changed: 1090 additions & 14933 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.agents/skills/kast/SKILL.md‎

Lines changed: 54 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -2,35 +2,73 @@
22
name: kast
33
description: >
44
Semantic Kotlin/JVM navigation and safe refactoring via `kast skill`
5-
subcommands. Use when a task needs Kotlin symbol resolution, file
6-
understanding, flow tracing, usages, callers, ambiguous member lookup,
7-
renames, failing-test diagnosis, or safe edits. Never use grep/rg for Kotlin
8-
identity.
5+
subcommands. Use this whenever a task involves Kotlin/JVM symbol identity,
6+
file understanding, flow or call tracing, usages, ambiguous members, failing
7+
Kotlin tests, renames, or validated Kotlin edits, even if the user does not
8+
explicitly say "Kast." Never use grep/rg for Kotlin identity.
99
---
1010

1111
# Kast
1212

13-
Use Kast for Kotlin identity, cross-file navigation, and validated edits.
13+
Use Kast for Kotlin identity, cross-file navigation, and validated edits. The
14+
goal is to keep semantic work moving even when the first command, JSON
15+
projection, or edit attempt is imperfect.
1416

15-
1. Start with the smallest semantic operation that can answer the request:
17+
## Fast path
18+
19+
1. Try the smallest semantic operation that can answer the request:
1620
`workspace-files` to find scope, `scaffold` to understand a file/type,
1721
`resolve` to pin a declaration, `references` for usages, and `callers` for
1822
flow.
1923
2. If `KAST_CLI_PATH` is empty or the shell says `command not found`, run
20-
`eval "$(bash .agents/skills/kast/scripts/kast-session-start.sh)"` and retry.
21-
Do not start by reading `.kast-version` or the wrapper OpenAPI fixture.
24+
`eval "$(bash .agents/skills/kast/scripts/kast-session-start.sh)"`, retry the
25+
same command once, and then continue. Do not start by reading
26+
`.kast-version`, the wrapper OpenAPI fixture, or maintenance evals.
2227
3. Navigate only with `kast skill workspace-files`, `kast skill scaffold`,
2328
`kast skill resolve`, `kast skill references`, and `kast skill callers`.
2429
4. Mutate only with `kast skill rename`,
2530
`kast skill write-and-validate`, and `kast skill diagnostics`.
26-
5. Requests use camelCase; responses use snake_case.
27-
6. For ambiguous names or member properties, resolve first, then trace
28-
usages/callers.
29-
7. If parsing a result fails, inspect a sample object or narrow the query. Stay
30-
on Kast.
31-
8. Never replace a failed semantic query with `grep`, `rg`, `sed`, or
32-
hand-edits.
33-
9. After mutation, `ok=false` or dirty diagnostics means the run failed.
31+
5. For ambiguous names or member properties, resolve first with `kind`,
32+
`containingType`, or `fileHint`, then trace usages/callers.
33+
34+
## JSON shape rules
35+
36+
- Request JSON uses camelCase.
37+
- Top-level wrapper responses use snake_case for wrapper metadata such as
38+
`log_file`, `error_text`, `file_path`, `applied_edits`, and
39+
`import_changes`.
40+
- Nested API models can keep their model casing. For example, symbols use
41+
`fqName` and locations use `location.filePath`, `startOffset`, and
42+
`startLine`.
43+
- Check `ok` and `type` before projecting a response. Failure responses carry
44+
`stage`, `message`, optional `error` or `error_text`, and `log_file`.
45+
- `rename` and `write-and-validate` requests require a `type` discriminator
46+
such as `RENAME_BY_SYMBOL_REQUEST` or `REPLACE_RANGE_REQUEST`.
47+
- Any request field ending in `filePath`, `filePaths`, or `contentFile` should
48+
use an absolute path.
49+
- `scaffold` uses `targetFile` (singular absolute path) as the required field,
50+
not `filePaths`. Always include `workspaceRoot` in the request body. Run one
51+
`scaffold` call per file; there is no batch variant.
52+
53+
## Recovery rules
54+
55+
- If parsing a result fails, inspect the top-level object or one sample element,
56+
then adjust the projection. Do not switch to text search because of JSON
57+
friction.
58+
- If a result set is too large, narrow the same semantic query with `kind`,
59+
`containingType`, `fileHint`, depth, or result limits.
60+
- If a mutation returns `ok=false`, a `*_FAILURE` response type, dirty
61+
diagnostics, or a validation/hash message such as "Missing expected hash",
62+
treat the edit as failed. Keep the failure visible, run diagnostics on the
63+
intended/touched file when useful, and report the blocker instead of claiming
64+
success or applying a hand edit.
65+
- Never replace a failed semantic query with `grep`, `rg`, `sed`, or manual
66+
parsing for Kotlin identity. Raw search is only acceptable for non-semantic
67+
file-path discovery, comments, string literals, or maintenance work.
68+
- If a request fails with `Encountered an unknown key`, a field name is wrong.
69+
Consult `references/quickstart.md` for the correct shape. Skill subcommands
70+
do not accept `--help`; probing with `{}` to discover required fields wastes
71+
turns when the quickstart already lists every command's required fields.
3472

3573
Read `references/quickstart.md` for request snippets and recovery tips. Skill
3674
maintenance fixtures live under `fixtures/maintenance`; do not load them during

‎.agents/skills/kast/fixtures/maintenance/evals/evals.json‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,78 @@
7979
"Anchors each pane on real symbols resolved from the target repo, not invented identifiers.",
8080
"Keeps the baseline (grep) and kast-augmented panes structurally aligned per turn."
8181
]
82+
},
83+
{
84+
"id": 8,
85+
"prompt": "A Kast `references` response parsed oddly: top-level fields like `log_file` are snake_case, but my jq query for `.references[].file_path` was empty. Inspect the response shape and continue finding the Kotlin usages semantically.",
86+
"expected_output": "A schema-recovery flow that distinguishes wrapper metadata from nested API model fields and resumes semantic traversal.",
87+
"files": [],
88+
"expectations": [
89+
"Checks top-level `ok` and `type` before assuming the command failed.",
90+
"Recognizes that nested symbol/location fields can be camelCase, such as `location.filePath` or `symbol.fqName`.",
91+
"Continues with `references`, `resolve`, `callers`, or `scaffold` after fixing the projection.",
92+
"Does not fall back to grep/rg because a snake_case projection was wrong."
93+
]
94+
},
95+
{
96+
"id": 9,
97+
"prompt": "Use `kast skill write-and-validate` to make a small Kotlin replacement. If it fails with `VALIDATION_ERROR: Missing expected hash for edited file`, recover safely and tell me whether the edit actually landed.",
98+
"expected_output": "A mutation recovery flow that treats the wrapper failure as a failed edit, gathers diagnostics if useful, and avoids success-shaped manual edits.",
99+
"files": [],
100+
"expectations": [
101+
"Uses `kast skill write-and-validate` with the correct request `type` discriminator and camelCase request fields.",
102+
"Treats `ok=false`, `WRITE_AND_VALIDATE_FAILURE`, or a missing-hash validation message as a failed mutation.",
103+
"Runs `kast skill diagnostics` on the intended or touched file when useful to characterize the current state.",
104+
"Does not claim success or switch to sed/apply-patch/manual edits after the wrapper reports failure."
105+
]
106+
},
107+
{
108+
"id": 10,
109+
"prompt": "The Kast skill was selected for a Kotlin flow-tracing task. Do the first useful semantic command quickly; do not spend the opening turns reading `.kast-version`, maintenance evals, or the OpenAPI fixture.",
110+
"expected_output": "A low-friction selected-skill flow that begins with setup recovery only if needed, then uses a semantic command instead of contract-reading thrash.",
111+
"files": [],
112+
"expectations": [
113+
"Starts with the requested semantic operation or a single `kast-session-start.sh` bootstrap if the binary is unavailable.",
114+
"Avoids loading `.kast-version`, `fixtures/maintenance`, or `wrapper-openapi.yaml` during normal navigation.",
115+
"Uses `workspace-files`, `scaffold`, `resolve`, `references`, or `callers` to make progress in the first substantive step.",
116+
"Keeps the transcript centered on Kast instead of raw search or maintenance-file inspection."
117+
]
118+
},
119+
{
120+
"id": 11,
121+
"prompt": "A Kast command returned a failure response with `ok:false`, `stage`, `message`, `error_text`, and `log_file`. Use that output to decide the next step without abandoning Kast.",
122+
"expected_output": "A response-handling flow that surfaces the failure stage/message, retries or narrows only when appropriate, and does not ignore `ok:false`.",
123+
"files": [],
124+
"expectations": [
125+
"Checks `ok` and `type` before presenting results as successful.",
126+
"Summarizes the failure `stage`, `message`, and available log/error fields.",
127+
"Chooses a targeted retry, narrower semantic query, diagnostics run, or explicit blocker based on the failure.",
128+
"Does not discard the failed Kast response and redo the task with grep/rg."
129+
]
130+
},
131+
{
132+
"id": 12,
133+
"prompt": "I need to understand how the `DescriptorRegistry` class manages daemon lifecycle state in the kast codebase — specifically what it reads and writes to disk. Get me a structural summary so I can plan a change to its locking behavior.",
134+
"expected_output": "A scaffold-rooted file understanding that uses the correct `targetFile` and `workspaceRoot` fields on the first attempt, without probing the schema or falling back to file reads.",
135+
"files": [],
136+
"expectations": [
137+
"Uses `kast skill scaffold` with `targetFile` (singular absolute path), not `filePaths`.",
138+
"Includes `workspaceRoot` in the scaffold request body.",
139+
"Does not probe the schema by sending `{}` or by running `kast skill scaffold --help`.",
140+
"Does not replace scaffold with grep or cat to understand the file structure."
141+
]
142+
},
143+
{
144+
"id": 13,
145+
"prompt": "Walk me through how three Kotlin files interact: `DescriptorRegistry.kt`, `WorkspaceRuntimeManager.kt`, and `KastFileOperations.kt`. I want to understand the data flow between them before I make a change to the file-locking path.",
146+
"expected_output": "A multi-file semantic walkthrough that scaffolds each file individually with correct request fields, then uses references or callers to show the interaction.",
147+
"files": [],
148+
"expectations": [
149+
"Calls `kast skill scaffold` once per file with `targetFile` (singular), not a batched `filePaths` array.",
150+
"Includes `workspaceRoot` in each scaffold request.",
151+
"Uses `references` or `callers` to trace actual cross-file data flow rather than reading source manually.",
152+
"Does not use grep/rg to trace the interaction between files."
153+
]
82154
}
83155
]
84156
}

‎.agents/skills/kast/fixtures/maintenance/evals/routing.json‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -178,6 +178,74 @@
178178
"grep",
179179
"rg"
180180
]
181+
},
182+
{
183+
"id": "parse-nested-response-shape",
184+
"prompt": "A Kast `references` response has snake_case wrapper fields, but `.references[].file_path` is empty. Inspect the shape and keep tracing the Kotlin usages semantically.",
185+
"expected_skill": "kast",
186+
"expected_route": "@kast",
187+
"allowed_ops": [
188+
"kast skill references",
189+
"kast skill resolve",
190+
"kast skill scaffold",
191+
"kast skill callers"
192+
],
193+
"forbidden_ops": [
194+
"grep",
195+
"rg"
196+
]
197+
},
198+
{
199+
"id": "write-validate-hash-failure",
200+
"prompt": "Use `kast skill write-and-validate` for a Kotlin replacement, but if it returns `VALIDATION_ERROR: Missing expected hash for edited file`, recover safely and tell me whether the edit landed.",
201+
"expected_skill": "kast",
202+
"expected_route": "@kast",
203+
"allowed_ops": [
204+
"kast skill write-and-validate",
205+
"kast skill diagnostics"
206+
],
207+
"forbidden_ops": [
208+
"sed",
209+
"grep",
210+
"rg"
211+
]
212+
},
213+
{
214+
"id": "selected-skill-no-maintenance-thrash",
215+
"prompt": "The Kast skill is already selected for a Kotlin flow-tracing task. Start with the first useful semantic command instead of reading `.kast-version`, maintenance evals, or the OpenAPI fixture.",
216+
"expected_skill": "kast",
217+
"expected_route": "@kast",
218+
"allowed_ops": [
219+
"bash .agents/skills/kast/scripts/kast-session-start.sh",
220+
"kast skill workspace-files",
221+
"kast skill scaffold",
222+
"kast skill resolve",
223+
"kast skill references",
224+
"kast skill callers"
225+
],
226+
"forbidden_ops": [
227+
".agents/skills/kast/.kast-version",
228+
".agents/skills/kast/fixtures/maintenance/references/wrapper-openapi.yaml",
229+
"grep",
230+
"rg"
231+
]
232+
},
233+
{
234+
"id": "respect-failure-response",
235+
"prompt": "A Kast command returned `ok:false` with `stage`, `message`, `error_text`, and `log_file`. Use that response to decide the next step without abandoning Kast.",
236+
"expected_skill": "kast",
237+
"expected_route": "@kast",
238+
"allowed_ops": [
239+
"kast skill resolve",
240+
"kast skill references",
241+
"kast skill callers",
242+
"kast skill scaffold",
243+
"kast skill diagnostics"
244+
],
245+
"forbidden_ops": [
246+
"grep",
247+
"rg"
248+
]
181249
}
182250
]
183251
}

‎.agents/skills/kast/fixtures/maintenance/references/routing-improvement.md‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,9 @@ The script:
4949
- extracts visible prompts, skill loads, and tool traces from HTML-only exports
5050
- redacts absolute paths and session identifiers
5151
- classifies cases such as `trigger-miss`, `loaded-but-bypassed`,
52-
`semantic-abandonment`, `route-via-subagent`, and `config-drift`
52+
`semantic-abandonment`, `schema-friction`,
53+
`mutation-validation-friction`, `initialization-friction`,
54+
`maintenance-thrash`, `route-via-subagent`, and `config-drift`
5355
- emits promotion candidates in the same shape as the checked-in eval corpus
5456

5557
## Review the output
@@ -77,6 +79,10 @@ Good routing evals:
7779
- state the expected skill and route
7880
- encode recovery expectations when the first Kast attempt hits setup friction
7981
or a noisy JSON result
82+
- distinguish top-level wrapper response fields from nested API model fields
83+
when the observed failure was schema/projection friction
84+
- preserve failed mutation responses such as validation/hash errors as failed
85+
edits instead of turning them into success-shaped manual edits
8086
- forbid raw `grep` / `rg` for semantic Kotlin work
8187
- stay generic enough to survive codebase churn
8288

‎.agents/skills/kast/fixtures/maintenance/scripts/build-routing-corpus.py‎

Lines changed: 45 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,18 @@
2929
HTML_DURATION_RE = re.compile(r"^(?:\d+h\s+)?(?:\d+m\s+)?\d+s$")
3030
HTML_STYLE_OR_SCRIPT_RE = re.compile(r"<(?:style|script)\b.*?</(?:style|script)>", re.IGNORECASE | re.DOTALL)
3131
HTML_TAG_RE = re.compile(r"<[^>]+>")
32+
INIT_FRICTION_RE = re.compile(
33+
r"(?:KAST_CLI_PATH\s*(?:=|is empty)|command not found|bash:\s*:\s*command not found|Unable to resolve Kast CLI path)",
34+
re.IGNORECASE,
35+
)
36+
SCHEMA_FRICTION_RE = re.compile(
37+
r"(?:jq|projection|file_path|filePath|snake_case|camelCase|references\[\]|\.references|\bempty\b)",
38+
re.IGNORECASE,
39+
)
40+
MUTATION_VALIDATION_RE = re.compile(
41+
r"(?:Missing expected hash|VALIDATION_ERROR|WRITE_AND_VALIDATE_FAILURE|ok\s*[:=]\s*false)",
42+
re.IGNORECASE,
43+
)
3244
KAST_COMMAND_RE = re.compile(
3345
r'(?:(?:"?\$KAST_CLI_PATH"?|(?:[^\s"\']+/)?kast)\s+skill\s+'
3446
r'(?:resolve|references|callers|diagnostics|rename|scaffold|write-and-validate|workspace-files))'
@@ -55,6 +67,10 @@
5567
)
5668

5769
PROMOTION_CLASSIFICATIONS = {
70+
"initialization-friction",
71+
"maintenance-thrash",
72+
"mutation-validation-friction",
73+
"schema-friction",
5874
"trigger-miss",
5975
"loaded-but-bypassed",
6076
"route-via-subagent",
@@ -170,7 +186,19 @@ def classify_html_export(
170186
*,
171187
kast_command_blocks: int,
172188
grep_like_commands: int,
189+
contract_reference_reads: int,
190+
bootstrap_probes: int,
191+
schema_shape_frictions: int,
192+
mutation_validation_failures: int,
173193
) -> str:
194+
if mutation_validation_failures > 0:
195+
return "mutation-validation-friction"
196+
if schema_shape_frictions > 0 and kast_command_blocks > 0:
197+
return "schema-friction"
198+
if bootstrap_probes > 0:
199+
return "initialization-friction"
200+
if contract_reference_reads > 0 and "kast" in loaded_skills:
201+
return "maintenance-thrash"
174202
if kast_command_blocks > 0 and grep_like_commands > 0:
175203
return "semantic-abandonment"
176204
if looks_semantic(prompt) and "kast" not in loaded_skills and kast_command_blocks == 0:
@@ -282,6 +310,8 @@ def parse_html_export(path: Path) -> list[RoutingCase]:
282310
grep_like_commands = 0
283311
contract_reference_reads = 0
284312
bootstrap_probes = 0
313+
schema_shape_frictions = 0
314+
mutation_validation_failures = 0
285315

286316
for block in blocks:
287317
heading = block_heading(block)
@@ -316,15 +346,23 @@ def parse_html_export(path: Path) -> list[RoutingCase]:
316346
kast_command_blocks += 1
317347
if re.search(r"\b(?:grep|rg)\b", content_text):
318348
grep_like_commands += 1
319-
if "KAST_CLI_PATH=" in content_text or "command not found" in content_text:
349+
if INIT_FRICTION_RE.search(content_text):
320350
bootstrap_probes += 1
351+
if SCHEMA_FRICTION_RE.search(content_text) and KAST_COMMAND_RE.search(content_text):
352+
schema_shape_frictions += 1
353+
if MUTATION_VALIDATION_RE.search(content_text) and KAST_COMMAND_RE.search(content_text):
354+
mutation_validation_failures += 1
321355
if tool_name in {"grep", "rg"}:
322356
grep_like_commands += 1
323357
if tool_name == "view" and any(
324358
marker in content_text
325359
for marker in (".kast-version", "wrapper-openapi.yaml")
326360
):
327361
contract_reference_reads += 1
362+
if tool_name != "bash" and SCHEMA_FRICTION_RE.search(content_text) and "kast" in loaded_skills:
363+
schema_shape_frictions += 1
364+
if tool_name != "bash" and MUTATION_VALIDATION_RE.search(content_text):
365+
mutation_validation_failures += 1
328366

329367
prompt = next(iter(prompts), "")
330368
if not prompt:
@@ -340,6 +378,10 @@ def parse_html_export(path: Path) -> list[RoutingCase]:
340378
tool_counts,
341379
kast_command_blocks=kast_command_blocks,
342380
grep_like_commands=grep_like_commands,
381+
contract_reference_reads=contract_reference_reads,
382+
bootstrap_probes=bootstrap_probes,
383+
schema_shape_frictions=schema_shape_frictions,
384+
mutation_validation_failures=mutation_validation_failures,
343385
)
344386
return [
345387
RoutingCase(
@@ -360,6 +402,8 @@ def parse_html_export(path: Path) -> list[RoutingCase]:
360402
f"grep_like_commands={grep_like_commands}",
361403
f"contract_reference_reads={contract_reference_reads}",
362404
f"bootstrap_probes={bootstrap_probes}",
405+
f"schema_shape_frictions={schema_shape_frictions}",
406+
f"mutation_validation_failures={mutation_validation_failures}",
363407
],
364408
),
365409
]

0 commit comments

Comments
 (0)