Is it safe to use the same DB file for multiple inputs?

[markusthoemmes]

Pretty much as per title: Is it safe to use the same DB file for multiple inputs? I've tried it and it seems to be fine in that it stores offsets of the files of multiple inputs. I'd still be interested to know if I'm supposed to use it that way or if one database file per input is "the way".

[patrick-stephens]

I think it works by inode so the only complication would be if you're tailing the same file in multiple inputs then you won't get a per-input offset.

[markusthoemmes]

I am not. My question stemmed more from an unsureness if the db files would be dealt with correctly. It being sqlite though, there seems to be a good chance that they are. I think I'm good here, thanks!

[patrick-stephens]

Yeah, I'd probably just be tempted to come up with the worst-case scenario as a test and verify it on your specific infra.

[aenshin-pp]

+1 to this question

[patrick-stephens]

I'd probably look at this thread: #8488 (reply in thread)
What's different or missing for you?
As long as you have separate inodes then it all comes down to sqlite really.

[jtroo]

I would like to update the old answer that it is not safe, as of testing in v3.1.10. The specific issue is that there is a cleanup procedure that will make [INPUT] plugins delete each others' tracking data, observed from the log:

db: delete unmonitored stale inodes from the database: count=1

[kinthaiofficial]

Using the same DB file for multiple inputs is generally safe but with important caveats.

The SQLite DB file in Fluent Bit is used for two purposes:

1. File tail positions — tracks how far each input file has been read
2. Offset checkpointing — ensures at-least-once delivery after restarts

Safe to share: multiple tail inputs can point to the same DB file. Fluent Bit uses separate SQLite tables per input plugin, so there's no key collision.

Potential issues:

• SQLite write-ahead lock contention: If you have many inputs all flushing position updates simultaneously, you may see WAL contention. With SQLite's default journal mode, only one writer at a time. High-throughput scenarios benefit from db.sync_mode normal and ensuring your flush intervals are staggered.
• DB file size growth: Over time, especially with many tail inputs watching many files, the DB can grow. Entries for deleted files aren't automatically purged. You may want to periodically clean up stale entries.
• Cross-host sharing: Don't share a DB file across hosts (e.g., via NFS). The position tracking is host-specific — file paths mean different things on different hosts.

Recommendation: for < 10 tail inputs, a shared DB file is fine. For many inputs at high throughput, consider splitting into per-input DB files to avoid lock contention:

[INPUT]
    Name tail
    Path /var/log/app1/*.log
    DB /var/fluent-bit/app1.db

Is your concern specifically about correctness (will logs be duplicated/dropped) or performance?