Skip to content

Make sources report their ordering to Spark#212

Draft
EnricoMi wants to merge 16 commits intospark-3.3from
branch-report-ordering
Draft

Make sources report their ordering to Spark#212
EnricoMi wants to merge 16 commits intospark-3.3from
branch-report-ordering

Conversation

@EnricoMi
Copy link
Copy Markdown
Collaborator

Checklist before submitting

  • Did you read the contributing guide?
  • Did you update the docs?
  • Did you write any tests to validate this change?

Description

With release of Spark 3.4, data sources can report the existing ordering of read data. This allows Spark to avoid ordering data when the source satisfies that already.

Review process for approval

  1. All tests and other checks must succeed.
  2. At least one core contributors must review and approve.
  3. If a core contributor requests changes, they must be addressed.

@EnricoMi EnricoMi changed the title Make sources report their ordering Make sources report their ordering to Spark May 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant