MAVFS — Model-agnostic virtual file system

Repository home: After you clone this project, the root of the repo is the MAVFS/ directory (the folder that contains this README.md, LICENSE, and the mafs/ package). If your editor opens mafs/ as the workspace, that folder is one level below the git root—commands like git status should be run from MAVFS/, not from inside mafs/ unless you only care about paths relative to the package.

MAVFS is a virtual file system (VFS) that presents two storage tiers behind one shell:

• /vram — hot, in-memory storage (pluggable backend; Redis or an in-process dict for development and tests).
• /vdisk — cold storage mapped to a real directory on the host disk.

The goal is fast reads and writes for working data in RAM, with explicit commands to move trees or files between RAM and disk without ad hoc scripts. The design keeps the RAM store swappable and the UI layer separate from storage so you can add backends or front ends without rewriting the core router.

---

Contents

• Concept
• Architecture
• Repository layout
• Requirements
• Installation
• How to run
• Shell commands
• Testing and benchmarks
• Protocol notes (MAFS)

---

Concept

MAVFS does not replace the kernel VFS. It is a user-space layer that:

1. Normalizes paths under two virtual roots: /vram and /vdisk.
2. Routes operations to a RAMBackend (abstract) or a DiskFSBackend (directory sandbox).
3. Implements cross-store copy for push / pull / cp / mv when source and destination live on different backends.

“In-memory” therefore means whatever implements RAMBackend (Redis by default in production-oriented setups; a pure-Python dict for CI and laptops without Redis).

---

Architecture

┌─────────────────────────────────────────────────────────────┐
│  Front ends (modular)                                        │
│  • default_shell.py  — interactive commands                │
│  • dashboard_tui.py  — Rich live RAM vs disk byte view       │
└───────────────────────────┬─────────────────────────────────┘
                            │ uses only MavfsCore API
┌───────────────────────────▼─────────────────────────────────┐
│  vfs_core.MavfsCore                                         │
│  pwd, cd, ls, mkdir, touch, rm, cat, cp, mv, push, pull     │
└───────────────┬─────────────────────────────┬───────────────┘
                │                               │
     ┌──────────▼──────────┐         ┌─────────▼──────────┐
     │  RAMBackend (ABC)    │         │  DiskFSBackend      │
     │  • DictRAMBackend    │         │  OS paths under     │
     │  • RedisRAMBackend   │         │  --disk-dir root    │
     └─────────────────────┘         └─────────────────────┘

Supporting modules (under mafs/):

Module	Role
path_util.py	Virtual path normalization, ..-safe joins, mount detection.
ram_backend.py	Abstract RAM API; dict and Redis implementations.
disk_backend.py	Maps /vdisk/... to a host directory; usage stats for the dashboard.
app.py	build_mavfs(...) — selects RAM backend kind and wires MavfsCore.
run_mafs.py	CLI: shell, dashboard, bench.

Architecture diagram (repo root): mavfs_main.png — same layout as the ASCII figure above.

---

Repository layout

MAVFS/
├── README.md                 ← this file
└── mafs/                     ← Python implementation
    ├── run_mafs.py           ← main entrypoint
    ├── app.py
    ├── vfs_core.py
    ├── ram_backend.py
    ├── disk_backend.py
    ├── path_util.py
    ├── frontends/
    │   ├── default_shell.py
    │   └── dashboard_tui.py
    ├── tests/
    ├── requirements.txt
    └── pytest.ini

Legacy experiment modules (managers.py, storage_utils.py) remain for reference; use run_mafs.py for the supported stack.

---

Requirements

• Python 3.10+ (tested on 3.11).
• Dependencies (see mafs/requirements.txt): redis, rich, pytest.
• Redis — optional for day-to-day use if you use --ram dict; required to exercise the Redis backend and the optional Redis integration test.

---

Installation

Use a virtual environment (recommended on Debian/Ubuntu PEP 668 setups). From the repository root (MAVFS/):

cd mafs
python3 -m venv .venv
.venv/bin/pip install -r requirements.txt

---

How to run

All examples assume you have already run cd mafs from the repo root, so your working directory is mafs/, and use .venv/bin/python as needed.

Interactive shell

In-memory tree with the dict backend (no Redis):

python run_mafs.py shell --ram dict --disk-dir block_dir

With Redis (default host/port; override as needed):

python run_mafs.py shell --ram redis --redis-host localhost --redis-port 6379 --redis-db 0 --disk-dir block_dir

Live dashboard (RAM vs disk bytes)

Uses Rich; refreshes on an interval (seconds):

python run_mafs.py dashboard --ram dict --disk-dir block_dir --dashboard-interval 1.0

Built-in speed benchmark

Compares sequential read/write latency for a blob on /vram vs /vdisk:

python run_mafs.py bench --ram dict --disk-dir block_dir --bench-size 65536 --bench-iters 200

---

Shell commands

Command	Description
pwd	Print current virtual working directory.
ls [path]	List directory (names with / suffix for dirs).
cd path	Change directory; .. stops at /vram or /vdisk root.
mkdir [-p] path	Create directory; -p creates parents.
touch path	Create empty file.
rm [-r] path	Remove file; -r for directory trees.
cp src dst	Copy; works across /vram and /vdisk.
mv src dst	Move/rename; cross-store uses copy + delete.
cat path	Print file contents (UTF-8 where possible).
push /vram/... /vdisk/...	Export RAM → disk.
pull /vdisk/... /vram/...	Import disk → RAM.
stats	Print approximate RAM and disk byte totals.
help / exit	Help or quit.

Paths may be absolute (/vram/..., /vdisk/...) or relative to the current directory.

---

Testing and benchmarks

Run the full suite from mafs/ (after cd mafs from the repo root):

.venv/bin/python -m pytest tests/ -v

By default, test_redis_backend_roundtrip is skipped unless you opt in (uses Redis DB 15 and deletes keys matching mafs:* before the test):

MAVFS_TEST_REDIS=1 .venv/bin/python -m pytest tests/ -v

Example result logs (for comparing dict-only vs Redis-enabled runs) are written as mafs/test_results_dict_only.txt and mafs/test_results_redis.txt when you capture pytest output.

---

Protocol notes (MAFS)

The broader MAFS idea is a protocol-style contract for unifying multiple storage media behind one UX: Unix-like commands, bidirectional data movement, and room for REST-style metadata APIs and mount scenarios. This repository implements a concrete, user-space MAVFS focused on:

• A small command set (see table above).
• A storage hypervisor role played by MavfsCore + backends.
• Backend flexibility (RAM implementation is not fixed to Redis).

POSIX compliance and network mount are not claimed by the current code; they remain future design space.

---

License / contributing

Add your preferred license and contribution guidelines here if the project is published.