Willingness to contribute
{"label"=>"Yes. I can contribute a fix for this bug independently.", "value"=>"independent"}
Feature Request Proposal
Currently HDFSStorage and LocalStorage set TableLocation without scheme, ie: /data/openhouse/db/tbl-UUID instead of full URI hdfs:///data/openhouse/db/tbl-UUID.
This was done to ensure backward compatibility, replication and migration.
Iceberg spec mandates a full-URI (containing scheme) as part of its metadata. source: https://iceberg.apache.org/spec/?ref=blog.min.io#manifests
As part of this bugFix, HDFSStorage and LocalStorage should start using fully-qualified URI.
Motivation
What is the use case for this feature?
Making this change will generalize tableLocation behavior for various Storages such as s3, hdfs, local etc.
What component(s) does this feature affect?
Details
No response
Willingness to contribute
{"label"=>"Yes. I can contribute a fix for this bug independently.", "value"=>"independent"}
Feature Request Proposal
Currently HDFSStorage and LocalStorage set TableLocation without scheme, ie:
/data/openhouse/db/tbl-UUIDinstead of full URIhdfs:///data/openhouse/db/tbl-UUID.This was done to ensure backward compatibility, replication and migration.
Iceberg spec mandates a full-URI (containing scheme) as part of its metadata. source: https://iceberg.apache.org/spec/?ref=blog.min.io#manifests
As part of this bugFix, HDFSStorage and LocalStorage should start using fully-qualified URI.
Motivation
Making this change will generalize tableLocation behavior for various Storages such as s3, hdfs, local etc.
What component(s) does this feature affect?
Table Service: This is the RESTful catalog service that stores table metadata.:services:tablesJobs Service: This is the job orchestrator that submits data services for table maintenance.:services:jobsData Services: This is the jobs that performs table maintenance.apps:sparkIceberg internal catalog: This is the internal Iceberg catalog for OpenHouse Catalog Service.:iceberg:openhouseSpark Client Integration: This is the Apache Spark integration for OpenHouse catalog.:integration:sparkDocumentation: This is the documentation for OpenHouse.docsLocal Docker: This is the local Docker environment for OpenHouse.infra/recipes/docker-composeOther: Please specify the component.Details
No response