Adds a new --max-downloads option by 265866 · Pull Request #8453 · mikf/gallery-dl

265866 · 2025-10-23T09:25:11Z

I added a new --max-downloads command line option to limit the number of files downloaded per job

I wanted to make it easier to stop downloads early without writing an external script or interrupting it manually, and I mainly used it for testing new extractors and previewing the first few files from various sources

The default behavior is unchanged when the option is omitted, and I tested it locally to make sure download halts cleanly after reaching the limit

(this is also my first open source contribution, feedback is very welcome!)

fixes regression introduced in 5747dbf

- provide 'search_tags' metadata for tag searches (mikf#8160) - support '/archive/tagged/' URLs (mikf#8160) - use self.groups - remove __init__ constructors & _init functions - remove "#category" test results

- only posts and creator posts work - 'home' and 'list' extractors do nothing atm - needs 'authorization' header as 'token' config value to access locked content - needs yt-dlp to download 1080p videos

and add 'lists' extractor to collect your account's lists

for filenames containing an apostrophe ' mikf#8150 (comment)

- fix 'list' URLs - allow 'id:…' as creator identifier - simplify API code

- add 'format' option - handle exceptions

* [vk] Added extractor for VK wall posts * update - fix flake8 - rename to 'wall-post' - remove __init__() / use self.groups - simplify 'description' extraction * add test * add to docs/supportedsites --------- Co-authored-by: Mike Fährmann <[email protected]>

… when possible / allowed by VK - album[name] - album[count] - user[name] - user[nick] - user[group]

fixes superfluous newlines in --version output

to be SPDX compliant and prevent a 'twine' warning

yt-dlp/yt-dlp@4e6a693

test for 'dt:…' to catch NullDatetime instances, which inherit from 'datetime.datetime' but are not exactly this class

…f#8429)

- use 'datetime.fromisoformat()' when possible (mikf#7671) - return a datetime-compatible object for invalid datetimes (instead of a 'str' value)

remove unnecessary 'util' imports

using None for 'fromlist' is slightly faster than an empty tuple

* Add configuration setting to control newline characters in metadata. * update docs/configuration * introduce 'open()' method * add 'newline' test

add 'format' option

265866 · 2025-10-23T09:34:01Z

The only thing I’m not sure about is whether there are any other places where I should update or add documentation

I added the option to gallery_dl/option.py, but I wasn’t sure if there are other files or docs that need changes

mikf and others added 30 commits August 30, 2025 23:01

[comick] detect broken chapters (mikf#8054)

75e7dc8

[chevereto] add 'video' extractor (mikf#8149)

8047aac

[jpgfish] update domain to 'jpg6.su'

7052463

[bunkr] fix albums with more than 100 files (mikf#8150)

12a6e04

[twitter] prevent "KeyError: 'name'" in '_transform_user()' (mikf#8154)

52c932a

fixes regression introduced in 5747dbf

[tumblr] fix pagination when using 'date-max'

ff147c2

[tumblr] update

951bf7c

- provide 'search_tags' metadata for tag searches (mikf#8160) - support '/archive/tagged/' URLs (mikf#8160) - use self.groups - remove __init__ constructors & _init functions - remove "#category" test results

[dl:ytdl] support '_ytdl_manifest_cookies'

1c9f4ff

[fansly] add initial support (mikf#4401)

e2ca64a

- only posts and creator posts work - 'home' and 'list' extractors do nothing atm - needs 'authorization' header as 'token' config value to access locked content - needs yt-dlp to download 1080p videos

[hentaifoundry:stories] fix 'src' & 'description' extraction (mikf#8163)

556af30

[fansly] implement 'home' extractor (mikf#4401)

1082d99

[fansly] implement 'list' extractor (mikf#4401)

0fcd34d

and add 'lists' extractor to collect your account's lists

[shimmie2] support 'co.llection.pics' (mikf#8166)

8967a6e

[scripts/supportedsites] convert subcategories with '-'

d30d42b

[bunkr] fix 'JSONDecodeError: Invalid \escape' (mikf#8150)

a094595

for filenames containing an apostrophe ' mikf#8150 (comment)

[fansly] update (mikf#4401)

b64eba1

- fix 'list' URLs - allow 'id:…' as creator identifier - simplify API code

[fansly] improve file extraction code (mikf#4401)

bfee2a3

- add 'format' option - handle exceptions

[kemono] add '.bin' archive files to archives list (mikf#8156)

97d4a76

[dl:ytdl] move 'http.cookiejar.Cookie' import to top of file

b9fd910

[vk] parse 'date' metadata (mikf#8029)

89e1c78

[vk] parse 'description' metadata (mikf#8029)

821f5c9

[vk:album] extract more metadata (mikf#8029)

752f82f

… when possible / allowed by VK - album[name] - album[count] - user[name] - user[nick] - user[group]

[options] remove custom Formatter.format_help()

35bf1a2

fixes superfluous newlines in --version output

release version 1.30.6

fbbf79a

[fansly] emit debug logging message when using a 'token' (mikf#4401)

41a7049

[fansly] provide fallback URL for manifest downloads (mikf#4401)

69f4036

[comick] handle redirects

2ebf1ad

[comick] update 'buildId' on 404 errors (mikf#8157)

67707a8

[build/setup.py] update license identifier to 'GPL-2.0-only'

ff0f5a4

to be SPDX compliant and prevent a 'twine' warning

mikf and others added 27 commits October 15, 2025 19:29

[workflows:tests] install yt-dlp from PyPI for PyPy 3.9

d0c1fd0

yt-dlp/yt-dlp@4e6a693

[dt] replace 'util' datetime functions

0eb3c8a

[dt] introduce 'NullDatetime' to represent invalid datetimes

21350c5

[dt] replace 'datetime' imports

69f7cfd

[twitter] fix 'KeyError' for "temporarily unavailable" users (mikf#8423)

e42030a

[tests/results] replace 'datetime' usage

d57dc48

test for 'dt:…' to catch NullDatetime instances, which inherit from 'datetime.datetime' but are not exactly this class

[dt] replace 'text.parse_datetime()' & 'text.parse_timestamp()'

085616e

[pixhost] force '.to' TLD (mikf#8428)

4072687

[civitai] add 'sort' & 'period' options (mikf#8426)

555ac2d

[dt] update 'parse_datetime' calls with one argument

6c71b27

[reddit] support comment share link (mikf#8434)

e2fb2c0

[twitter] fix "KeyError - 'source_id'" with disabled 'transform' (mik…

238d097

…f#8429)

[kemono] use f-strings

9ad937a

[schalenetwork][hdoujin] re-implement 'cbz' option (mikf#8431)

11ccadf

[weibo] add 'text' option (mikf#8422)

f9a8598

[dt] replace 'parse_compat' with compat workaround

989136b

[dt] update representation of invalid datetime objects

5802107

[dt] use 'parse_datetime_iso()' for ISO formats

c38856b

[tests/formatter] update results

ed8fe07

merge branch 'dt': move datetime utils into separate module

c8fc790

- use 'datetime.fromisoformat()' when possible (mikf#7671) - return a datetime-compatible object for invalid datetimes (instead of a 'str' value)

replace 'util.re()' with 'text.re()'

9bf76c1

remove unnecessary 'util' imports

use f-strings when building 'pattern'

d7c97d5

improve arguments for relative '__import__()'

3deb5a4

using None for 'fromlist' is slightly faster than an empty tuple

[pp:metadata] add 'newline' option (mikf#8439)

06e3126

* Add configuration setting to control newline characters in metadata. * update docs/configuration * introduce 'open()' method * add 'newline' test

[e621] add 'artist' & 'artist-search' extractors (mikf#8448)

df1b620

[wikimedia] add 'format=original' to file URLs (mikf#5512)

eefd8f9

add 'format' option

Added --max-downloads option

80c3ced

mikf force-pushed the master branch from ae7a315 to 325c64f Compare March 28, 2026 12:16

mikf force-pushed the master branch from 8137219 to 9f3d2b3 Compare April 5, 2026 18:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Adds a new --max-downloads option#8453

Adds a new --max-downloads option#8453
265866 wants to merge 7391 commits intomikf:masterfrom
265866:feature/max-downloads-arg

265866 commented Oct 23, 2025

Uh oh!

265866 commented Oct 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

Uh oh!

Conversation

265866 commented Oct 23, 2025

Uh oh!

265866 commented Oct 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants