Add lion: documentation extraction tool for Go code

[Copilot]

Implements a tool that extracts lion comments from Go source and generates organized markdown documentation files, one per topic. Supports three comment formats: marker-at-end (recommended), single-line, and block comment.

Usage

Marker-at-end format (recommended - cleanest):

// The main function initializes the HTTP server.
// It registers handlers and starts listening.
// Regular Go comments with marker at the end.
//lion:getting-started
func main() {
    http.HandleFunc("/", handleRoot)
    log.Fatal(http.ListenAndServe(":8080", nil))
}

// handleRoot returns the welcome message.
// This demonstrates the marker-at-end format.
//lion:api-guide
func handleRoot(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Welcome\n")
}

Single-line format:

//lion:getting-started The main function initializes the HTTP server.
//lion:getting-started It registers handlers and starts listening.
func main() {
    http.HandleFunc("/", handleRoot)
    log.Fatal(http.ListenAndServe(":8080", nil))
}

//lion:api-guide handleRoot returns the welcome message.
func handleRoot(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Welcome\n")
}

Block comment format:

/*lion:getting-started
The main function initializes the HTTP server.
It registers handlers and starts listening.
No need to repeat the topic on each line.
*/
func main() {
    http.HandleFunc("/", handleRoot)
    log.Fatal(http.ListenAndServe(":8080", nil))
}

/*lion:api-guide
handleRoot returns the welcome message.
*/
func handleRoot(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Welcome\n")
}

$ lion generate ./myapp --output ./docs
Generated documentation in ./docs

$ ls docs/
getting-started.md  api-guide.md  index.md

Implementation

• AST parsing (internal/extractor): Walks Go AST to extract lion comments from package declarations, functions, types, constants, and variables. Supports three formats: marker-at-end (regular // comments with //lion:topic at the end), single-line (//lion:topic), and block comment (/*lion:topic ... */). Combines consecutive comments on the same entity.

• Markdown generation (internal/generator): Creates one file per topic with entity sections. Includes source file references. Generates index.md with topic links.

• CLI: Cobra-based with generate, topics, and version commands.

Comment Formats

lion supports three formats that produce identical output:

1. Marker-at-end (recommended): Regular // comments followed by //lion:topic-name - Cleanest syntax, no repetition, no /* */ needed
2. Single-line: //lion:topic-name Content - Good for short comments, can be repeated
3. Block: /*lion:topic-name\nMulti-line content\n*/ - Alternative for multi-line without repetition

All formats can be mixed in the same codebase.

Self-Documentation

lion documents itself using its own extraction tool. The lion/docs/ directory contains markdown files that answer key questions about using the tool:

• supported-syntax.md - What syntax formats are supported (documented at extractFromCommentGroup)
• how-to-run-generation.md - How to run the generate command (documented at generateCmd)
• snippet-order.md - In what order snippets appear in generated files (documented at Generate)
• file-title.md - How file titles are determined from topic names (documented at generateTopicFile)
• index.md - Documentation index with links to all topics

All documentation is written at the point of implementation - comments are placed at the exact functions/commands where features are implemented, not in random locations or separate doc files.

A regenerate-docs.sh script is provided to regenerate documentation when code changes:

$ ./lion/regenerate-docs.sh
Regenerating lion documentation...
Generated documentation in ./docs

This serves as both documentation and a working example of lion's approach: documenting features at their implementation points.

Changes

• Added lion/ with main.go, internal packages, tests, README, AGENTS.md
• Implemented support for three comment formats: marker-at-end, single-line, and block comment
• Added lion comments at implementation points to document key features
• Generated lion/docs/ with documentation answering specific usage questions
• Created lion/regenerate-docs.sh script to regenerate documentation
• Updated README.md, release-mirror.toml, mise.toml to include lion

Original prompt

write a tool called lion that will go through Go code and find special comments and generate a folder with markdown docs based on those comments. like sorta like sphinx but its gonna be universal. The idea is that if I document stuff like this source, and if I can, I want to be able to get documentation for a bunch of entities or whatever. So let's say I want the following for any entity declaration: I can add a comment that says, for example, "I didn't know lion or whatever," and then what topic it should belong to. Then, simply when they run the tool, it will find everything and it will generate markdown files that contain the documentation extracted from the comments. We sort of create a book almost by peppering in the codebase with a bunch of comments that each contribute to one or more chapters. And it should have tests etc.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[neongreen]

@copilot now add some comments to the lion project itself and use it to document itself. the resulting markdown files will be in lion/docs/. also write a script that will regenerate the docs.

[Copilot]

@copilot now add some comments to the lion project itself and use it to document itself. the resulting markdown files will be in lion/docs/. also write a script that will regenerate the docs.

Done in commit 6166bf0. Added lion comments throughout the source code covering overview, architecture, cli, and implementation topics. Generated documentation is in lion/docs/ with 5 files:

• index.md - Documentation index
• overview.md - What lion is
• architecture.md - Key functions and data flow
• cli.md - Command-line interface
• implementation.md - Internal details

Created lion/regenerate-docs.sh script that runs go run . generate . --output ./docs to regenerate documentation when code changes.

[neongreen]

@copilot > //lion:architecture

I don't like that I have to write the same thing here on every line. Is there anything we can do about it? Is it possible to somehow mark the whole comment as a Lion comment?

[Copilot]

@copilot > //lion:architecture

I don't like that I have to write the same thing here on every line. Is there anything we can do about it? Is it possible to somehow mark the whole comment as a Lion co...

Added block comment support in commit 141a858. You can now use /*lion:topic-name ... */ format to write multi-line documentation without repeating the topic:

/*lion:architecture
The main function starts the HTTP server.
It registers handlers and starts listening.
No need to repeat lion:architecture on each line.
*/
func main() { ... }

Both formats work and can be mixed. Single-line (//lion:topic) for short comments, block comments for longer ones. They produce identical markdown output.

[neongreen]

@copilot ok but uhhhhhhhhh i wonder if there is any way to do it without /* */ comments?

like for example what if we just say that a single lion directive at the bottom of the // comment pulls the entire comment into the docs? or idk maybe you can suggest something. how do other tools handle this "mark a block of comments as [something]" usecase?

[Copilot]

@copilot ok but uhhhhhhhhh i wonder if there is any way to do it without /* */ comments?

like for example what if we just say that a single lion directive at the bottom of the // comment pulls the en...

Implemented marker-at-end format in commit c7d2645. Put //lion:topic at the END of regular // comments to pull in the entire block:

// The main function starts the HTTP server.
// It registers handlers and starts listening.
// It handles setup and configuration.
//lion:architecture
func main() { ... }

This is now the recommended format - cleanest syntax, no repetition, no /* */ needed. When lion sees //lion:topic as the last line of a comment group (with only regular comments before it), it pulls the entire block into that topic.

All three formats (marker-at-end, single-line, block comment) work and produce identical output.

[neongreen]

@copilot Im interested in higher level docs. Like im not looking for a godoc replacement here, but smth that will generate higher level docs like "how to use this tool", "syntax reference", "known issues", etc. right now comments in lion itself are an example of the former but should be an example of the latter

[Copilot]

@copilot Im interested in higher level docs. Like im not looking for a godoc replacement here, but smth that will generate higher level docs like "how to use this tool", "syntax reference", "known iss...

Changed in commit c2f5126. Replaced low-level API documentation with high-level user documentation examples.

Lion now demonstrates:

• Getting Started - Installation and quick start
• Usage Guide - How to use commands with examples
• Syntax Reference - Documentation format reference
• Best Practices - Topic naming and organization tips
• Use Cases - When to use lion (guides, not API docs)
• Known Limitations - What the tool doesn't do
• Troubleshooting - Common problems and solutions

The self-documentation now shows how to create narrative documentation and user guides, not API reference (which is godoc's job). Generated docs are in lion/docs/ with 10 topics.

[neongreen]

@copilot Oof. Not great.

Ok lets do it this way. Remove all lion comments you added. I want to learn the following: a) what syntax is supported? b) how do I run generation? c) in what order do annotated snippets end up in the file? d) can I set the title of the generated file / section? ------ all of these should be answered in the generated docs and all documentation must be written at the point of implementation, not in a random place or in doc.go.

[Copilot]

@copilot Oof. Not great.

Ok lets do it this way. Remove all lion comments you added. I want to learn the following: a) what syntax is supported? b) how do I run generation? c) in what order do annot...

Done in commit e7d5905. Removed all previous lion comments and doc.go. Added new documentation at the point of implementation that answers your questions:

a) What syntax is supported? → extractor.go:extractFromCommentGroup (shows all 3 formats)
b) How do I run generation? → main.go:generateCmd (step-by-step instructions)
c) In what order do snippets end up? → generator.go:Generate (filesystem order, then source order)
d) Can I set the title? → generator.go:generateTopicFile (no, derived from topic name)

Generated docs are now 5 files: supported-syntax.md, how-to-run-generation.md, snippet-order.md, file-title.md, plus index.md. All documentation is at implementation points, not in random places.

[mergify]

🧪 CI Insights

Here's what we observed from your CI run for ff059a9.

🟢 All jobs passed!

But CI Insights is watching 👀