CXP Core

CXP (Context Exchange Protocol) is a universal, high-performance file format designed specifically for AI context management. It enables efficient storage, compression, and retrieval of contextual data for AI applications.

Why CXP?

Traditional formats like JSON or XML are inefficient for AI context:

Feature	JSON/XML	CXP
Compression	None	Zstd (up to 90% smaller)
Deduplication	None	Content-aware chunking
Binary support	Base64 (bloated)	Native binary
Streaming	Limited	Full streaming support
Token efficiency	Poor	Optimized for LLMs
Incremental updates	Full rewrite	Delta updates

CXP reduces token usage by 40-70% compared to raw text, saving costs and improving AI response quality.

Features

• Smart Compression: Zstd compression with content-aware chunking
• Deduplication: Automatic detection and elimination of duplicate content
• Hierarchical Structure: Recursive CXP files for organizing large codebases
• Binary Support: Native handling of images, PDFs, and other binary files
• Streaming API: Process large files without loading everything into memory
• Token Optimization: Designed to minimize token usage for LLM APIs
• Cross-Platform: Works on macOS, Windows, and Linux

Installation

Add to your Cargo.toml:

[dependencies]
cxp-core = { git = "https://github.com/endgegnerbert-tech/cxp-core" }

Quick Start

use cxp_core::{CxpBuilder, CxpReader};

// Create a CXP file
let cxp = CxpBuilder::new()
    .add_file("src/main.rs")?
    .add_directory("src/")?
    .build()?;

cxp.save("project.cxp")?;

// Read a CXP file
let reader = CxpReader::open("project.cxp")?;
for file in reader.files() {
    println!("{}: {} bytes", file.path, file.size);
}

Use Cases

AI Context Management

CXP is the backbone of ContextAI, providing:

• Efficient codebase indexing
• Smart file filtering and prioritization
• Optimized context windows for AI conversations

Codebase Archiving

Create compact, searchable archives of entire projects:

let cxp = CxpBuilder::new()
    .add_directory("./my-project")?
    .with_recursive(true)
    .build()?;

Incremental Backups

CXP's deduplication makes it ideal for backup systems:

let cxp = CxpBuilder::new()
    .from_existing("backup.cxp")?
    .add_changes_since(last_backup_time)?
    .build()?;

Architecture

┌─────────────────────────────────────────────┐
│                 CXP File                    │
├─────────────────────────────────────────────┤
│  Header (magic bytes, version, metadata)    │
├─────────────────────────────────────────────┤
│  Manifest (file index, checksums)           │
├─────────────────────────────────────────────┤
│  Chunks (deduplicated, compressed blocks)   │
├─────────────────────────────────────────────┤
│  Index (fast lookup tables)                 │
└─────────────────────────────────────────────┘

Feature Flags

[dependencies]
cxp-core = { version = "0.1", features = ["embeddings", "search"] }

Feature	Description
default	Core functionality
embeddings	Vector embeddings for semantic search
search	Full-text and semantic search
multimodal	Image and PDF processing
scanner	File system scanning utilities
contextai	ContextAI integration helpers

Performance

Benchmarks on a typical codebase (10,000 files, 500MB):

Operation	Time	Memory
Create CXP	2.3s	150MB
Read CXP	0.4s	50MB
Search	12ms	20MB
Compression ratio	73%	-

Specification

The CXP format specification is available in SPECIFICATION.md.

License

CXP Core is released under the CXP License.

Permitted:

• Personal and educational use
• Open-source projects (with attribution)
• Use in ContextAI

Requires permission:

• Commercial use
• Competing products

See LICENSE for full terms.

Attribution

If you use CXP in your project, please include attribution:

Powered by CXP Core - https://github.com/endgegnerbert-tech/cxp-core

Contributing

Contributions are welcome! Please read our contributing guidelines before submitting PRs. ##Ps Some Words from my Side i have in Mind to make it usable for every documetn type and make more out of it because there is a big problem in Ai because i isnt inteligent in it self ,everyone knows it but we could change it if we give the ai files maby files with human connection in it .Just for now this is what i am working on with Ai because i dont know what i am doing this is new but i have so many Ideas Thanks for reading my cxp and By

Links

• ContextAI - AI assistant powered by CXP
• Documentation (coming soon)
• Discord Community (coming soon)

---

Created by Einar Jaeger | GitHub