Add --limit N flag to worker to limit the number of shards imported

[szabgab]

This can be especially useful for development where we would like to only load a few shards.
It would stop processing after N shards.

[straight-shoota]

I'm not opposed to this, but I'm also not convinced it's an important use case.

When importing the catalog, you either want to set up the entire database anyways, or you're just doing a test/dev setup. In that case, it's IMO preferable to use a smaller test catalog anyways (like we did in our session).

Perhaps it would be helpful to add a catalog with test data to this repo?

[szabgab]

That might be a better approach indeed.