Skip to content

Commit c541516

Browse files
authored
Merge pull request #5 from VisLab/repo_setup
Updated the README and working on the coverage setup
2 parents 9eeba3f + 40e7c0c commit c541516

8 files changed

Lines changed: 159 additions & 19 deletions

File tree

.coveragerc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
[run]
2+
source = remodel
23
branch = True
34
omit =
45
*/__init__.py

.github/workflows/ci_cov.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,5 +59,6 @@ jobs:
5959
- name: Upload coverage to qlty
6060
uses: qltysh/qlty-action/coverage@v2
6161
with:
62-
oidc: true
6362
files: coverage.xml
63+
env:
64+
QLTY_TOKEN: ${{ secrets.QLTY_COVERAGE_TOKEN }}

README.md

Lines changed: 25 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ Tabular file remodeling and reorganizing tools for event files and datasets.
1111

1212
`table-remodeler` provides a flexible, operation-based framework for transforming tabular data files through JSON-configurable pipelines. Originally extracted from the [hed-python](https://github.com/hed-standard/hed-python) remodeling tools, this package operates as a standalone tool while maintaining compatibility with HED (Hierarchical Event Descriptors) annotations.
1313

14-
**Key Features:**
14+
**Key features:**
1515
- Operation-based architecture for reproducible data transformations
1616
- JSON-configurable pipelines for batch processing
1717
- Support for HED-annotated event files
@@ -31,7 +31,7 @@ cd table-remodeler
3131
pip install -e .
3232
```
3333

34-
## Quick Start
34+
## Quick start
3535

3636
### Python API
3737

@@ -55,7 +55,7 @@ dispatcher = Dispatcher(operations, data_root="/path/to/dataset")
5555
dispatcher.run_operations()
5656
```
5757

58-
### Command Line
58+
### Command line
5959

6060
```bash
6161
# Run remodeling operations
@@ -68,7 +68,7 @@ run_remodel_backup /path/to/data --backup-name my_backup
6868
run_remodel_restore /path/to/data --backup-name my_backup
6969
```
7070

71-
## Available Operations
71+
## Available operations
7272

7373
### Data Transformation
7474
- `convert_columns` - Convert column data types
@@ -83,7 +83,7 @@ run_remodel_restore /path/to/data --backup-name my_backup
8383
- `reorder_columns` - Reorder columns
8484
- `split_rows` - Split rows based on criteria
8585

86-
### HED-Specific Operations
86+
### HED-specific operations
8787
- `factor_hed_tags` - Factor HED tags into separate columns
8888
- `factor_hed_type` - Factor by HED tag types
8989
- `summarize_hed_tags` - Summarize HED tag usage
@@ -92,13 +92,31 @@ run_remodel_restore /path/to/data --backup-name my_backup
9292
- `summarize_definitions` - Extract HED definitions
9393
- `summarize_sidecar_from_events` - Generate sidecar from events
9494

95-
### Analysis Operations
95+
### Analysis operations
9696
- `summarize_column_names` - List column names across files
9797
- `summarize_column_values` - Summarize unique values per column
9898

9999
## Documentation
100100

101-
Full documentation is available at [table-remodeler.readthedocs.io](https://table-remodeler.readthedocs.io/)
101+
Full API and developer documentation is available at [https://wwww.hedtags.org/table-remodeler]([https://wwww.hedtags.org/table-remodeler).
102+
103+
Users of the table-remodeler should look at the [HED remodeling quickstart](https://www.hedtags.org/hed-resources/HedRemodelingQuickstart.html) and [HED remodeling tools](https://www.hedtags.org/hed-resources/HedRemodelingTools.html).
104+
105+
### Building Documentation
106+
107+
To build the documentation locally:
108+
109+
1. Install documentation dependencies:
110+
```bash
111+
pip install -r docs/requirements.txt
112+
```
113+
114+
2. Build the HTML documentation:
115+
```bash
116+
sphinx-build -b html docs docs/_build/html
117+
```
118+
119+
3. View the documentation by opening `docs/_build/html/index.html` in your web browser.
102120

103121
## Requirements
104122

docs/_templates/quicklinks.html

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,10 @@
11
<div class="sidebar-quicklinks">
2-
<h3>Quick Links</h3>
2+
<h3>Quick links</h3>
33
<ul>
4-
<li><a href="https://github.com/hed-standard/table-remodeler" target="_blank">GitHub Repository</a></li>
5-
<li><a href="https://github.com/hed-standard/hed-python" target="_blank">HED Python Tools</a></li>
6-
<li><a href="https://www.hedtags.org/" target="_blank">HED Homepage</a></li>
4+
<li><a href="https://www.hedtags.org/" target="_blank">HED homepage</a></li>
5+
<li><a href="https://www.hedtags.org/hed-resources" target="_blank">HED resources</a></li>
6+
<li><a href="https://www.hedtags.org/hed-schema-browser" target="_blank">HED schema browser</a></li>
7+
<li><a href="https://www.hedtags.org/hed-specification" target="_blank">HED specification</a></li>
8+
<li><a href="https://hedtools.org/hed" target="_blank">HED online tools</a></li>
79
</ul>
810
</div>

docs/api/cli.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
Command-Line Interface
1+
Command-line interface
22
======================
33

44
Command-line tools for executing remodeling workflows.

docs/api/index.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ This section contains the complete API reference for Table Remodeler.
1010
operations
1111
cli
1212

13-
Package Overview
13+
Package overview
1414
----------------
1515

1616
The Table Remodeler package is organized into three main modules:

docs/introduction.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,11 @@ Table Remodeler is a Python package that provides flexible tools for transformin
66

77
## Key features
88

9-
- **Operation-based Architecture**: Apply transformations through a series of composable operations
10-
- **JSON Configuration**: Define remodeling pipelines in JSON for reproducible workflows
11-
- **HED Support**: Built-in operations for working with HED-annotated event files
12-
- **Backup & Restore**: Automatic backup management before applying changes
13-
- **Command-line Interface**: Easy-to-use CLI tools for batch processing
9+
- **Operation-based architecture**: Apply transformations through a series of composable operations
10+
- **JSON configuration**: Define remodeling pipelines in JSON for reproducible workflows
11+
- **HED support**: Built-in operations for working with HED-annotated event files
12+
- **Backup & restore**: Automatic backup management before applying changes
13+
- **Command-line interface**: Easy-to-use CLI tools for batch processing
1414
- **Extensible**: Create custom operations by extending the `BaseOp` class
1515

1616
## Installing table remodeler

tests/test_doc_consistency.py

Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
import unittest
2+
import os
3+
import re
4+
import glob
5+
import remodel
6+
from remodel.operations.valid_operations import valid_operations
7+
8+
9+
class TestDocConsistency(unittest.TestCase):
10+
def setUp(self):
11+
self.current_dir = os.path.dirname(__file__)
12+
self.docs_root = os.path.abspath(os.path.join(self.current_dir, "..", "docs", "api"))
13+
14+
def test_operations_are_documented(self):
15+
"""
16+
Ensure all operations in valid_operations are listed in docs/api/operations.rst.
17+
This preserves manual organization while ensuring completeness.
18+
"""
19+
docs_path = os.path.join(self.docs_root, "operations.rst")
20+
21+
# Read the documentation file
22+
with open(docs_path, "r", encoding="utf-8") as f:
23+
content = f.read()
24+
25+
# Extract all class names documented via autoclass
26+
# Matches: .. autoclass:: remodel.operations.some_module.ClassName
27+
documented_classes = set()
28+
pattern = r"\.\.\s+autoclass::\s+[\w\.]+\.(\w+)"
29+
30+
for match in re.finditer(pattern, content):
31+
documented_classes.add(match.group(1))
32+
33+
# Check against the source of truth
34+
missing_ops = []
35+
for op_name, op_class in valid_operations.items():
36+
class_name = op_class.__name__
37+
if class_name not in documented_classes:
38+
missing_ops.append(class_name)
39+
40+
# Fail if anything is missing
41+
if missing_ops:
42+
self.fail(
43+
f"\nThe following operations are defined in valid_operations but missing from operations.rst:\n"
44+
+ "\n".join(f"- {op}" for op in missing_ops)
45+
+ "\n\nPlease add them to docs/api/operations.rst under the appropriate section."
46+
)
47+
48+
def test_core_is_documented(self):
49+
"""
50+
Ensure all classes exported in remodel/__init__.py are documented in docs/api/core.rst.
51+
"""
52+
docs_path = os.path.join(self.docs_root, "core.rst")
53+
54+
# Read the documentation file
55+
with open(docs_path, "r", encoding="utf-8") as f:
56+
content = f.read()
57+
58+
# Get exported classes from remodel/__init__.py
59+
# We filter for classes that are actually defined in the package (not modules)
60+
exported_classes = [
61+
name for name in dir(remodel) if not name.startswith("_") and isinstance(getattr(remodel, name), type)
62+
]
63+
64+
# Extract documented classes
65+
documented_classes = set()
66+
pattern = r"\.\.\s+autoclass::\s+[\w\.]+\.(\w+)"
67+
68+
for match in re.finditer(pattern, content):
69+
documented_classes.add(match.group(1))
70+
71+
missing_classes = []
72+
for cls_name in exported_classes:
73+
if cls_name not in documented_classes:
74+
missing_classes.append(cls_name)
75+
76+
if missing_classes:
77+
self.fail(
78+
f"\nThe following core classes are exported by remodel but missing from core.rst:\n"
79+
+ "\n".join(f"- {cls}" for cls in missing_classes)
80+
+ "\n\nPlease add them to docs/api/core.rst."
81+
)
82+
83+
def test_cli_is_documented(self):
84+
"""
85+
Ensure all CLI scripts in remodel/cli/ are documented in docs/api/cli.rst.
86+
"""
87+
docs_path = os.path.join(self.docs_root, "cli.rst")
88+
cli_dir = os.path.abspath(os.path.join(self.current_dir, "..", "remodel", "cli"))
89+
90+
# Read the documentation file
91+
with open(docs_path, "r", encoding="utf-8") as f:
92+
content = f.read()
93+
94+
# Find all python files in cli directory, excluding __init__.py
95+
cli_files = glob.glob(os.path.join(cli_dir, "*.py"))
96+
cli_modules = [
97+
os.path.basename(f)[:-3] for f in cli_files if os.path.basename(f) != "__init__.py" # remove .py extension
98+
]
99+
100+
# Extract documented modules
101+
# Matches: .. automodule:: remodel.cli.module_name
102+
documented_modules = set()
103+
pattern = r"\.\.\s+automodule::\s+remodel\.cli\.(\w+)"
104+
105+
for match in re.finditer(pattern, content):
106+
documented_modules.add(match.group(1))
107+
108+
missing_modules = []
109+
for module in cli_modules:
110+
if module not in documented_modules:
111+
missing_modules.append(module)
112+
113+
if missing_modules:
114+
self.fail(
115+
f"\nThe following CLI modules are present in remodel/cli/ but missing from cli.rst:\n"
116+
+ "\n".join(f"- {mod}" for mod in missing_modules)
117+
+ "\n\nPlease add them to docs/api/cli.rst."
118+
)

0 commit comments

Comments
 (0)