Make s3.SinglepartWriter seekable#854
Merged
ddelange merged 6 commits intopiskvorky:developfrom Jul 1, 2025
Merged
Conversation
3 tasks
ddelange
commented
Mar 10, 2025
Comment on lines
-1146
to
-1138
| # This member is part of the io.BufferedIOBase interface. | ||
| # | ||
| self.raw = None |
Collaborator
Author
There was a problem hiding this comment.
_BufferedIOMixin (not used here) would expect self._raw which would be accessed via the raw property
ddelange
commented
Mar 10, 2025
|
|
||
| logger.debug("%s: direct upload finished", self) | ||
| self._buf = None | ||
| self._buf.close() |
Collaborator
Author
There was a problem hiding this comment.
such that calling seek, seekable etc at a later point in time will properly raise ValueError: I/O operation on closed file ref 2b31045
ddelange
commented
Mar 14, 2025
| if writebuffer is None: | ||
| self._buf = io.BytesIO() | ||
| elif not writebuffer.seekable(): | ||
| raise ValueError('writebuffer needs to be seekable') |
Collaborator
Author
There was a problem hiding this comment.
making this requirement explicit (needed in close)
Collaborator
Author
|
@mpenkov ready for review 👍 |
Collaborator
Author
|
@mpenkov kind reminder on this one and the other open PRs :) |
* 'develop' of https://github.com/ddelange/smart_open: Add help.txt linting step (piskvorky#853) Remove head_bucket call from s3.SinglepartWriter (piskvorky#857)
ddelange
force-pushed
the
s3-singlepart-seekable
branch
2 times, most recently
from
f3ad2f0 to
76ab3c5
Compare
ddelange
force-pushed
the
s3-singlepart-seekable
branch
from
76ab3c5 to
c606867
Compare
ddelange
added a commit
to ddelange/smart_open
that referenced
this pull request
Jul 1, 2025
…o pypa-publish * 'develop' of https://github.com/piskvorky/smart_open: Make s3.SinglepartWriter seekable (piskvorky#854)
ddelange
added a commit
that referenced
this pull request
Jul 1, 2025
* develop: Update CHANGELOG.md Modernize python packaging (#851) Make s3.SinglepartWriter seekable (#854) Fix try-except in ssh.py for missing port in ssh config (#863) Fix integration test CI rot (#864) Add buffer_size transport param to ssh.py (#861) Add help.txt linting step (#853) Remove head_bucket call from s3.SinglepartWriter (#857) bump version to 7.2.0.dev0
|
Released v7.3.0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Fixes #787
Enable single part upload use-cases (
open(..., transport_params={"multipart_upload": False})) that require seeking.Tests
Work in progress
Checklist
Workflow