traceway-cli application

[fred-drake]

I use LLMs to give me non-deterministic assessments wherever possible, and after self-hosting traceway, I built a CLI for it.

https://github.com/fred-drake/traceway-cli

The goal was for it to act similarly to other CLIs like gh , woodpecker-cli, etc and be a usable programmatic interface for either LLMs or shell pipe enthusiasts.

I've been using it internally with my projects, and want to open it up for others to try. I need to build the proper github actions, including cut releases and push to various repositories based on the distribution.

Take a look, and let me know if you would like to host it under the tracewayapp suite as an official CLI. Please let me know if you have questions!

[dusanstanojeviccs]

This is INCREDIBLE!

This can probably be connected with the alerting system and an LLM to automatically create PRs whenever an issue happens

[fred-drake]

Yes that is exactly why I built it. It was designed so a human could use it from the command line if they wanted (by default it displays human readable output when run interactively), but the real power comes from a power user either using it through LLM or someone who wants to script something and pipes it through jq to parse the downstream JSON output.

The JSON output is passed through just what your traceway API passes through.

For those who do use it with an LLM, I have a skill at .claude/skills/traceway.md that they can load in which gives the LLM instructions on how to use it so it doesn't waste tokens fumbling around.

Here's where I am with it: For my own homelab's purposes it works. Dogfooding so far is successful.

Here's where it still needs to go: Others need to try it out, and I need to set this up so it is in a clean pullable state for the user. Convenient packaging for flavors of Linux, homebrew, etc.

[dusanstanojeviccs]

The biggest question now is how do we include this as part of the offering and how do we give it enough info to be able to easily address issues

The big thing we're doing with this is locking the internal backend api as our public api that we will have to keep more or less the same over time (including the response types) - which might be fine

Let me know what your thoughts are @fred-drake I've included you into the repo collaborators should simplify contributing

[fred-drake]

Two schools of thought:

Option 1 -- include the CLI as part of the flagship traceway repo.

Pro: CI runs would build and test the CLI with any changes to traceway. You reduce the risk of drift on any API changes, and possibly even a breaking change if you modify something more fundamental like auth
Con: Having it as part of a mono repo can introduce spaghetti dependencies if you are not careful. Right now, the CLI is it own autonomous app and merely shares the API contract. Without careful separation, you'd run the risk of a server change breaking the CLI code.
If you go this route: Have a strong design decision as to no functional code being shared between the CLI and server portions. Sharing model structs should be just fine.

Option 2 -- run traceway-cli as its own repo

Pro: The CLI itself shares no code with the traceway server app, so you'd have a clean separation. No risk of a code change on the server breaking the CLI (excluding anything that adjusts the API, of course).
Con: Drift becomes more of a chore. Any time you modify the API, you need to separately ensure that the CLI behaves correctly. With a mono repo you can do this with far better coordination.
If you go this route: Consider breaking out the model into its own repo, and it can be automatically shared amongst the two apps.

My bias / opinion

While neither option is a bad choice, I think option 1 is stronger, if you keep logical code separation outside of shared model structs. There is not much opportunity for code sharing, but what you are buying is not having to separately maintain the CLI on every release -- any changes you make will ensure that the CLI works from github actions.

Let me know your thoughts and if you also lean to option 1, I can make a PR that moves the CLI into this repo. If option 2, then just make a blank traceway-cli repo and I will set it as the new origin and remove the one on my repo.

[dusanstanojeviccs]

@fred-drake Agreed, I like the mono repo for the cli so that we can release versions in sync, as any backend change will require a new cli version.

We could do a cli install script like I've done for the otel agent https://github.com/tracewayapp/traceway-otel-agent, so it's a single line install, do you think that would be a good distribution path?

[fred-drake]

@dusanstanojeviccs the biggest pro of the script install is that it can be a one-stop shop no matter what system you have (MacOS, Ubuntu, Arch, pretty much anything) and I definitely think we should have it so everybody has a clean option. The liability, though, is that they do not have an upgrade path. I think a built-in upgrade path is too complicated for this because every system has their own native preferences on upgrades, but we definitely can have a lookup that warns a user if a new version is available.

I would suggest we have an optional TRACEWAY_INSTALL_PATH environment variable so the user can install it wherever they want (e.g. if they have a system without privileged access they can store it in their own home directory).

Although shell install I think would be a priority, I also think adding native support for other systems will increase its professional pedigree.

[dusanstanojeviccs]

@fred-drake I completely agree. I'm ready to take this on, got the major issues wrapped up so I can put 100% of the focus on this and building a path towards integrating an AI with it automatically with like github actions.

What was the final decision we merge it under one mono repo for the cli to keep the versions locked?

[fred-drake]

@dusanstanojeviccs yes exactly! I will get a PR set up so we are at a stable point within the mono repo, then we can add features.