Compress features in database

[matlabbe]

• work-in-progress, related to [Feature Request] Improve Data Compression #1431

So I reprocessed a database of ~1000 SIFT features per node with version 0.24 (this PR) and version 0.23. Here the comparison of the database size on disk (ls -lh), which results in a ~15% reduction:

• 0.23.0: 130 MB (zipped = 103MB)
• 0.24.0: 111 MB

Details:

However, compressing the features has some CPU cost. On a desktop computer, it requires between 6 and 9 ms to compress up to 1000 SIFT features with keypoints and points3D.

Here some logs showing saving/loading features time from the database:

[ WARN] (2026-04-05 18:31:08.583) DBDriver.cpp:1635::deserializeFeatures() Uncompressed 268058 bytes in 1.042128 ms, deserialized 334564 bytes in 0.012875 ms
[ WARN] (2026-04-05 18:31:08.583) DBDriverSqlite3.cpp:4120::loadWordsQuery() totalTime=1.404 ms
Processed 168/178 nodes [id=168 map=1 graph=163 hyp=90]... 77ms Loop on 5 [0]
[ WARN] (2026-04-05 18:31:08.672) DBDriver.cpp:1559::serializeFeatures() Serialized 336772 bytes in 0.031948 ms, Compressed 269977 bytes in 9.194136 ms
[ WARN] (2026-04-05 18:31:08.672) DBDriverSqlite3.cpp:6087::stepNode() totalTime=9.391 ms

On 0.23.0, the corresponding logs:

[ WARN] (2026-04-05 19:02:02.705) DBDriverSqlite3.cpp:4122::loadWordsQuery() totalTime=1.400948 ms
Processed 168/178 nodes [id=168 map=1 graph=163 hyp=91]... 78ms Loop on 5 [0]
[ WARN] (2026-04-05 19:02:02.785) DBDriverSqlite3.cpp:6089::stepNode() totalTime=0.030994 ms

The loading time from the database is pretty much the same between the two versions (1.404 ms vs 1.401 ms for this case), but saving the features to the database takes significantly more time (9.391 ms vs 0.031 ms for this case). Based on the logs, most time spent is in zlib library, so not sure if that can be improved. One thing to note is that features are saved to database asynchronously while rtabmap is waiting for a new image, so in theory it doesn't add latency.

EDIT: a comparison with binary descriptors (GFTT/BRIEF)

0.24.0 57 MB (5% reduction):

[ WARN] (2026-04-05 19:09:49.605) DBDriver.cpp:1635::deserializeFeatures() Uncompressed 26570 bytes in 0.102997 ms, deserialized 36268 bytes in 0.003099 ms
[ WARN] (2026-04-05 19:09:49.605) DBDriverSqlite3.cpp:4122::loadWordsQuery() totalTime=0.327110 ms
Processed 168/178 nodes [id=168 map=1 graph=164 hyp=94]... 65ms Loop on 5 [0]
[ WARN] (2026-04-05 19:09:49.673) DBDriver.cpp:1559::serializeFeatures() Serialized 36268 bytes in 0.003099 ms, Compressed 26570 bytes in 0.633955 ms
[ WARN] (2026-04-05 19:09:49.673) DBDriverSqlite3.cpp:6089::stepNode() totalTime=0.689030 ms

0.23.0 60 MB:

[ WARN] (2026-04-05 19:09:27.096) DBDriverSqlite3.cpp:4122::loadWordsQuery() totalTime=1.039982 ms
Processed 168/178 nodes [id=168 map=1 graph=163 hyp=91]... 65ms Loop on 5 [0]
[ WARN] (2026-04-05 19:09:27.164) DBDriverSqlite3.cpp:6089::stepNode() totalTime=0.053167 ms

Comparing the timing results in this case is interesting: the loading time is faster with compressed features but saving time is slower, with the same order of magnitude in both cases.

TODO:

• Get results on less powerful computer (e.g., jetson)
• We could have an option to trigger or not the compression, in case we want less CPU usage at the cost of more disk usage.