- Introduced the smart-completions module, including core classes such as SmartCompletions and SmartCompletion for managing completion requests and responses.

- Added various adapters for handling different completion scenarios, including `SmartCompletionTemplateAdapter`, `SmartCompletionContextAdapter`, and `ActionXmlCompletionAdapter`.
- Created a default configuration for smart completions and established a structure for handling user messages and system prompts.
- Included comprehensive documentation in `README.md` and `spec.md` to outline usage, key components, and integration details.
- Maintained existing comments and methods unrelated to the changes.

Author: brianpetro

smart-completions/README.md

@@ -0,0 +1,134 @@
+# Smart Completions
+
+Smart Completions is a **smart-collections**-based module for managing completion requests/responses in a structured way. Each completion is a **SmartCompletion** item that can run one or more request adapters to transform or enrich the data before sending it to a chat model. Responses are then stored, making it easy to track history or retrieve final output.
+
+## Key Components
+
+- **SmartCompletions**  
+    A collection (extending `Collection` from `smart-collections`) that manages multiple completion items.
+    
+    - Exposes a shared `chat_model` (if configured) for all items.
+    - Provides `completion_adapters` to transform item data prior to sending a request.
+- **SmartCompletion**  
+    A single completion item (extending `CollectionItem`), storing:
+    
+    - `completion.request`: Data in a chat-model-friendly format (e.g. OpenAI-like messages).
+    - `completion.responses[]`: The returned data from the model.
+    - `completion.chat_model`: (Optional) An override specifying a dedicated model instance.
+    - Adapters can modify the request or the response.
+- **Adapters** (in `adapters/`)  
+    Small classes extending `SmartCompletionAdapter`. Each checks for certain properties in `item.data` (e.g. `user_message`, `context_key`, etc.) and transforms `item.data.completion.request` or processes the response.
+    
+
+Examples:
+
+- `SmartCompletionUserAdapter`: Appends a user message to `request.messages`.
+- `SmartCompletionContextAdapter`: Merges ephemeral context into `request.messages`.
+- `SmartCompletionTemplateAdapter`: Injects template instructions.
+- `ThreadCompletionAdapter`: Appends conversation history.
+
+## Basic Usage
+
+1. **Register the collection** in your environment config:
+
+```
+{
+  collections: {
+    smart_completions: {
+      class: SmartCompletions
+      // optional data_adapter, etc.
+    }
+  },
+  modules: {
+    smart_chat_model: {
+      // chat model class & adapters
+    }
+  }
+}
+```
+
+1. **Create or update** a completion item:
+
+```
+const completions = env.smart_completions;
+const item = completions.create_or_update({
+  key: 'my_completion_1',
+  data: {
+    user_message: 'Hello AI, how are you?',
+    completion: {
+      request: {
+        messages: []
+      }
+      // optional chat_model overrides
+    }
+  }
+});
+```
+
+- Once created, the item calls its `init()` method, which:
+    - Runs each adapter (e.g. user adapter adds a user message).
+    - Calls the chat model to get a response.
+    - Stores the response in `completion.responses`.
+
+1. **Get the final text**:
+
+```
+console.log(item.response_text);
+```
+
+- This looks for the first choice returned by the model (e.g. `.choices[0].message.content`).
+
+## Adapters Flow
+
+Each adapter implements:
+
+- `to_request()`: Reads or transforms `item.data` → modifies `completion.request`.
+- `from_response()`: (Optional) Processes the final response. For example, parse out new properties or clean sensitive data.
+
+When `item.init()` runs, it calls `build_request()` which invokes `to_request()` in every relevant adapter. After calling the model, you can also trigger `from_response()` if needed (though most adapters simply rely on `to_request()`).
+
+## Custom Adapters
+
+To add your own adapter:
+
+1. Extend `SmartCompletionAdapter`.
+2. Implement `static get property_name()` returning the `item.data` property that triggers this adapter.
+3. Override `to_request()` (and optionally `from_response()`).
+
+Register it in your environment config:
+
+```
+{
+  collections: {
+    smart_completions: {
+      class: SmartCompletions,
+      completion_adapters: {
+        MyCustomAdapter
+      }
+    }
+  }
+}
+```
+
+## Data Storage
+
+By default, the code references `smart-collections/adapters/ajson_single_file.js` for storing items in an append-only JSON file. You can customize or swap out any data adapter.
+
+## Example
+
+```
+const completions = env.smart_completions;
+const item = completions.create_or_update({
+  data: {
+    user_message: "What's the weather?",
+    completion: { request: { messages: [] } }
+  }
+});
+console.log("Response text:", item.response_text);
+```
+
+No separate `new_completion` method is strictly required; you can just use `create_or_update` as shown. The item will handle constructing the request (via adapters) and fetching the response.
+
+## License
+
+MIT

smart-completions/adapters/_adapter.js

@@ -0,0 +1,46 @@
+import { insert_user_message } from "../utils/insert_user_message.js";
+
+/**
+ * @class SmartCompletionAdapter
+ * @extends SmartCompletionAdapter
+ *
+ * This adapter checks `item.data.user` and, if present, appends it as a user message
+ * to `completion.request.messages`.
+ */
+export class SmartCompletionAdapter {
+  constructor(item) {
+    this.item = item;
+  }
+  get data () {
+    return this.item.data;
+  }
+  get env () {
+    return this.item.env;
+  }
+  get completion () {
+    return this.data.completion;
+  }
+  get request () {
+    return this.item.data.completion.request;
+  }
+  get response () {
+    return this.item.response;
+  }
+  insert_user_message(user_message) {
+    insert_user_message(this.request, user_message);
+  }
+
+  // Override these methods in subclasses
+  static get property_name() {
+    return null;
+  }
+  /**
+   * @returns {Promise<void>}
+   */
+  async to_request() {}
+
+  /**
+   * @returns {Promise<void>}
+   */
+  async from_response() {}
+}

smart-completions/adapters/action.js

@@ -0,0 +1,181 @@
+import { SmartCompletionAdapter } from './_adapter.js';
+
+/**
+ * @class ActionCompletionAdapter
+ * @extends SmartCompletionAdapter
+ *
+ * This adapter checks `item.data.action` as a single SmartAction key,
+ * compiles ephemeral action text, and inserts it as a system message.
+ * Then, after response, if the assistant includes `tool_call` in `this.response`,
+ * it calls the corresponding `action_item.run(args)` and stores the result in
+ * `this.data.actions[action_key]`.
+ */
+export class ActionCompletionAdapter extends SmartCompletionAdapter {
+  static get property_name() {
+    return 'action_key';
+  }
+
+  /**
+   * @returns {Promise<void>}
+   */
+  async to_request() {
+    const action_key = this.data.action_key;
+    if (!action_key) return;
+
+    const action_opts = this.data.action_opts;
+
+    // Single context item only:
+    const action_collection = this.item.env.smart_actions;
+    if (!action_collection) {
+      console.warn("No 'smart_actions' collection found; skipping action adapter.");
+      return;
+    }
+    const action_item = action_collection.get(action_key);
+    if (!action_item) {
+      console.warn(`SmartAction not found for key '${action_key}'`);
+      return;
+    }
+
+    let tools;
+    try {
+      const action_module = action_item.module;
+      tools = action_module.tool ? [action_module.tool] : convert_openapi_to_tools(action_module.openapi);
+    } catch (err) {
+      console.warn('Error compiling ephemeral action', err);
+      return;
+    }
+
+    if (!tools) return;
+
+    // Mark that this action is being requested (before completion)
+    if (!this.data.actions) this.data.actions = {};
+    this.data.actions[action_key] = true;
+
+    this.insert_tools(tools, { force: true });
+  }
+
+  /**
+   * @returns {Promise<void>}
+   */
+  async from_response() {
+    console.log('ActionCompletionAdapter: from_response');
+
+    const tool_call = this.response.choices[0].message?.tool_calls?.[0];
+    if (!tool_call) return console.warn('No tool call found in response');
+    const action_key = tool_call?.function?.name;
+    const tool_arguments = tool_call?.function?.arguments;
+    if (!action_key) return;
+
+    // Use the same collection from env
+    const action_collection = this.item.env.smart_actions;
+    if (!action_collection) return;
+
+    const action_item = action_collection.get(action_key);
+    if (!action_item) return;
+
+    // Attempt to parse arguments if it's a string
+    let parsed_args = tool_arguments;
+    if (typeof parsed_args === 'string') {
+      try {
+        parsed_args = JSON.parse(parsed_args);
+      } catch (err) {
+        console.warn('Could not parse tool_call arguments', err);
+        return;
+      }
+    }
+
+    // Run the action
+    const result = await action_item.run_action(parsed_args);
+    console.log('ActionCompletionAdapter: result', result);
+    // If the tool returns an object with a 'final' key, treat it as the final assistant message
+    if (result && typeof result === 'object' && result.final) {
+      // Ensure we have a response object in completion
+      if (!this.item.data.completion.responses[0]) {
+        this.item.data.completion.responses[0] = { choices: [ { message: {} } ] };
+      } else if (!this.item.data.completion.responses[0].choices?.[0]) {
+        this.item.data.completion.responses[0].choices = [ { message: {} } ];
+      }
+
+      // Write final as the assistant's content
+      this.item.data.completion.responses[0].choices[0].message = {
+        ...this.item.data.completion.responses[0].choices[0].message,
+        role: "assistant",
+        content: result.final
+      };
+    }
+
+    // Store the result in `this.data.actions` under the same action key
+    if (!this.data.actions) this.data.actions = {};
+    this.data.actions[action_key] = result;
+  }
+
+  /**
+   * Insert the ephemeral tools into the request
+   * @param {Array<object>} tools
+   * @param {object} opts
+   * @returns {void}
+   */
+  insert_tools(tools, opts = {}) {
+    this.request.tools = tools;
+    if (opts.force) {
+      this.request.tool_choice = {
+        type: 'function',
+        function: {
+          name: tools[0].function.name
+        }
+      };
+    }
+  }
+}
+
+/**
+ * Converts OpenAPI specification into OpenAI's tool_call format
+ * @param {object} openapi_spec - Parsed OpenAPI JSON specification
+ * @returns {Array<object>} Array of tool_call formatted objects
+ */
+export function convert_openapi_to_tools(openapi_spec) {
+  const tools = [];
+
+  for (const path in openapi_spec.paths) {
+    const methods = openapi_spec.paths[path];
+
+    for (const method in methods) {
+      const endpoint = methods[method];
+
+      const parameters = endpoint.parameters || [];
+      const requestBody = endpoint.requestBody;
+
+      const properties = {};
+      const required = [];
+
+      parameters.forEach(param => {
+        properties[param.name] = {
+          type: param.schema.type,
+          description: param.description || ''
+        };
+        if (param.required) required.push(param.name);
+      });
+
+      if (requestBody) {
+        const schema = requestBody.content['application/json'].schema;
+        Object.assign(properties, schema.properties);
+        if (schema.required) required.push(...schema.required);
+      }
+
+      tools.push({
+        type: 'function',
+        function: {
+          name: endpoint.operationId || `${method}_${path.replace(/\//g, '_').replace(/[{}]/g, '')}`,
+          description: endpoint.summary || endpoint.description || '',
+          parameters: {
+            type: 'object',
+            properties,
+            required
+          }
+        }
+      });
+    }
+  }
+
+  return tools;
+}