Standardizing tabular spec for at-a-glance review of BIDS datasets, across tools

[christinerogers]

The following is a suggestion from @gdevenyi -- who is also willing to (help) drive this forward.

• from chat / BIDS outreach at the Mtl BrainHack Jan 2026, translated for further discussion by @christinerogers (Maintainer)

Several tools already exist to provide a tabular overview (CSV) of what's in a BIDS dataset:

• pyBIDS
• bids2table - cc @kaitj and the ChildMind team
• libBIDS.sh - used by bids2minc, Nf-Neuro(nextflow), by @gdevenyi

Given these tools, there is clearly a need to generate this kind of tabular overview of a dataset for selection/sorting/filtering.
Community convergence around a common format / spec would be ideal, as more people will be making their own tool.
Searching/filtering tooling development is also growing e.g. [AwesomeBids]https://github.com/bids-standard/awesome-bids/blob/main/README.md

Currently, generating (just) a tabular overview format with the major tools, e.g. PyBIDS, is slower and more costly than it needs to be due to a plethora of dependencies, so a standardized lightweight tool could be an asset, given enough community consensus on the tabular format.

Could also be highly useful for front-end review / export on platforms, e.g. open neuro, bids-examples (though cumbersome in the browser)

Specific Use cases:

• Indexing derivatives-only datasets intuitively and quickly would also be ideal to add at this point in BIDS evolution.

Standardization would resolve:

• current inconsistencies in some tools e.g. PyBIDS to df function outputs subject label as 01 not sub-01 whereas the task field is task-TaskName; consistency would be ideal
• column abbreviations e.g bids2table column rec short for reconstruction
• handling assumptions: e.g. blank columns: pybids doesn't output columns that don't have data; bids2table does and leaves them blank

Nice to have in / as a result of this spec :

a) a lightweight portable version (not python) that could run anywhere, e.g. a plugin on OpenNeuro, maybe even trigger from BIDS-examples

b) spec compliance with bids formatting / verification and BIDS common principles

c) a standardized (and/or configurable) set of fields: common convention for field format, labelling and order
e.g. from libBIDS

derivatives: Pipeline name if in derivatives folder
data_type: BIDS data type (anat, func, dwi, etc.)
BIDS entities: subject, session, sample, task, acquisition, etc.
suffix: File suffix (bold, T1w, dwi, etc.)
extension: File extension
path: Full file path

d) ideally could also be triggeered in the browser for quick review and CSV download e.g. GitHub integration on the BIDS-Examples browser, on OpenNeuro

e) supported by the community as part of the BIDS standard.

Still to do on this issue:

• compile existing tools, including examples/adjacent tools that are light / great / converging
• list out foreseeable challenges / configuration use cases
• add more links and keep making this issue clearer

Related(?) Neurostars discussion :

• [[https://neurostars.org/t/using-bids-schema-to-parse-datasets/31831]]

[gdevenyi]

I would propose perhaps defining the spec via
https://specs.frictionlessdata.io/table-schema/
which can potentially be programmatically derived from the existing schema files.

[gdevenyi]

https://ancpbids.readthedocs.io/en/latest/ has a very fast low-dependency parsing of BIDS datasets, but does not provide a tabular output, only pythonic/jsonic

[yarikoptic]

great discussion and pointers - subscribing with my comment: Besides standardizing within datasets, I would love to see the result coherent with BIDS principles themselves (see e.g #2283), and then extrapolate into extraction/summarizing across datasets - e.g. I started to do that for organizing OpenNeuro datasets and their derivatives (need to find more time to push on that): https://github.com/OpenNeuroStudies/OpenNeuroStudies - the overall goal, to have most valuable to that level extractions and summaries of metadata (e.g. which subject/sessions have all preprocessing done, which derivatives are complete and up to date; which studies are complete etc)

[gkiar]

We (b2t team) have had some conversation with @adelavega and @effigies about integrating bids2table into pybids more closely/officially, which in practice may mean a) exploring what doesn't currently exist in b2t that is regularly used/needed and adding it, before b) deprecating pybids (and optionally adding a lightweight interface layer, though I doubt this will be necessary).

This is primarily bottlenecked on our bandwidth to explore porting a pipeline (eg fmriprep) from pybids to b2t, though in our work with c-pac we did that migration last year and it was quite smooth and uncomplicated. Once we've got a reasonable proof of concept/a bit more experience with the migration process, I think we'll be in a better position to decide the path forward.

One thing that is worth noting is that b2t parses directory structures based on the updated spec directly, so if we want a tool that is literally interpreting the spec (and can be switched to do so with different versions of the schema), then it may be the closest. We specifically built it because of the lag in pybids and the fears of having dataset and tool versions be incompatible.

[julia-pfarr]

@surchs , I suggested to take a look at https://query.neurobagel.org/. Do you want to engage in this discussion? It seems to me that most of what is wanted here Neurobagel already does and maybe there is a chance for collaboration/help/extension?

[neuromechanist]

My immediate take is that this is a highly tool-related issue. Each tool develop their specifications based on the unique needs they are answering.

How ever the tabular specifications are designed, the next tool will need more or different structure, and will become non-compliant. Also, tools evolve over time and become noncompliant (but specs hardly evolve). Also^2, converting information from one tool to the another would merely need an adapter, tied to the source and destination versions.

FWIW, and based on the extensive MongoDB our team is generating for raw and processed data (see github.com/signalJourney, as a schema-focused process tracking), I am not sure if this standard would become useful, at least based on the currently stated needs.

[gkiar]

Totally agree with @neuromechanist — I don't think another spec in addition to the existing bids spec is actually needed, and the rest is "flavour"

[adelavega]

Agreed, @neuromechanist. I'm a weary to make another spec without a well defined downstream use-case.

The BIDS Spec and BIDS Schema is very strong and has a lot of work behind it. Ensuring existing tools are performant and spec compliant is key. I believe bids2table gets us most of the way there, but like @gkiar said, there's is more work to be done to test this in real-world existing tools such as fMRIPrep.

If someone wanted to make a js version of this tool that might be interesting. There's also been work previosuly to use pyodide to run pybids in the browser. Not sure if bids2table dependencies would allow that or not.

[adelavega]

It's a little far out but great project for OHBM Hackathon!

[gkiar]

Also, depending what your use case is for "at a glance review of bids datasets," @christinerogers , it's possible that CUBIDS is up your alley in terms of some of what you're hoping is covered? https://cubids.readthedocs.io/en/latest/