Vector data... in glTF?

[lilleyse]

This is an idea I've been playing around with for a while. Can we use glTF for vector data? Specifically, large amounts of 3D cartesian vector data. Here are some initial ideas.

Points

• Use primitive mode: 0 (POINTS)
• Use EXT_mesh_features and EXT_structural_metadata to associate points with metadata
  • Feature ID by Index + property tables
• Styling is applied separately, e.g. with 3D Tiles Declarative Styling
  • This would include label rendering, anchor lines, etc.

Note

• How does this relate to KHR_billboard?

Lines

• Use primitive mode: 1 (LINES), mode: 2 (LINE_LOOP) or mode: 3 (LINE_STRIP)
  • For the latter two, use EXT_mesh_primitive_restart to batch multiple lines into a single primitive
• Use EXT_mesh_features and EXT_structural_metadata to associate lines with metadata.
  • Feature ID by Vertex + property tables.
• Styling is applied separately, e.g. with 3D Tiles Declarative Styling
  • This would include mitering, line width, dashed patterns, etc.

Polygons

• Use primitive mode: 4 (TRIANGLES) - polygons are pre-triangulated for performance reasons
• Use EXT_mesh_features and EXT_structural_metadata to associate polygons with metadata.
  • Feature ID by Vertex + property tables.
• Indicate which edges form the outline
  • Option 1: extension that requires positions to be contiguous. See the legacy .vctr format and FB_ngon_encoding
  • Option 2: extension that encodes edge indices separately, as in EXT_mesh_primitive_edge_visibility or CESIUM_primitive_outline
• Styling is applied separately, e.g. with 3D Tiles Declarative Styling
  • This would include outline, fill pattern, etc.

Note

• How to encode edges for polygons with holes?

Styling

• Styling features would happen client side. Some examples:
  • Points: labels, anchor lines, etc
  • Lines: mitering, line width, dashed patterns, etc
  • Polygons: outline, fill pattern, etc
• Styling would also define layering, ordering, and draping behavior
• 3D Tiles Declarative Styling files could be provided alongside a tileset.json

Compression

• From previous experiments, glTF + meshopt + gzip file size was on par with the legacy .vctr format + gzip
• Haven't compared against other formats like Mapbox Vector Tiles, MapLibre Tiles, or Geobuf

Demo

Here's a demo of some vector data (provided by Maxar) converted from GeoJSON to 3D Tiles - includes line and polygon features and runtime styling. No new extensions and running in stock CesiumJS.

glTF files are using the following extensions:

• EXT_mesh_features
• EXT_structural_metadata
• EXT_meshopt_compression

[donmccurdy]

On encoding polygon edges —

Option 1: extension that requires positions to be contiguous. See the legacy .vctr format and KhronosGroup/glTF#1620

This would present some tradeoffs in compression and performance:

Draco has two encoding modes, edgebreaker or sequential. Edgebreaker requires reordering vertices, and yields better compression ratios. Sequential encoding can preserve vertex order when combined with the "ExpertEncoder" API's deduplicate_values=false option. See google/draco#929 (comment).

Meshopt does benefit from reordering vertices, but sorting is a separate step from compression and never required, regardless of any other meshopt compression options. Sorting may be implemented to optimize compression ratio, vertex cache efficiency / runtime performance, or both.

Additionally I don't think FB_ngon_encoding can represent polygons with holes, as it uses something like a triangle fan topology to represent each n-gon. Did the legacy .vctr format support polygons with holes? It isn't clear to me from the spec.

So, I suspect we may need to encode the edge indices explicitly. But I haven't the two extensions mentioned for that option yet.

[lilleyse]

Additionally I don't think FB_ngon_encoding can represent polygons with holes, as it uses something like a triangle fan topology to represent each n-gon. Did the legacy .vctr format support polygons with holes? It isn't clear to me from the spec.

Yeah, I should have explicitly called that out above. Option 1 doesn't support holes and neither does the .vctr format. @xuanhuang1 and I were talking about this yesterday and were leaning towards Option 2 as well.

[xuanhuang1]

Additionally I don't think FB_ngon_encoding can represent polygons with holes, as it uses something like a triangle fan topology to represent each n-gon. Did the legacy .vctr format support polygons with holes? It isn't clear to me from the spec.

Yeah, I should have explicitly called that out above. Option 1 doesn't support holes and neither does the .vctr format. @xuanhuang1 and I were talking about this yesterday and were leaning towards Option 2 as well.

Yes separately storing (pairs of) indices with CESIUM_primitive_outline seems the way to go, as it's flexible enough for polygon with holes and for tiling boundary clipped information too.

[xuanhuang1]

Quick update on this. CESIUM_primitive_outline will not work directly given the case below:

We will have to distinguish edge vs outlines, otherwise clipped polygon [0,1,2,3,4] will look the same as [0,2,3,4]

There are several options going forward:

1. Modify CESIUM_primitive_outline
2. Use EXT_mesh_primitive_edge_visibility
3. Create an EXT_ extension

Option 2 seems favorable at this point as solution with the least amount of effort/friction based on discussion with @lilleyse @weegeekps?

[xuanhuang1]

Notes on tile boundary. Article referenced here

Thick outline rendering may suffer from tiling artefacts

and the common solution is to have a buffer region for each tile that serves as extra geometry padding at the boundaries:

which in our case may introduce a slight addition in spec to store the padding width with no extra tiling/rendering modification. The caveat though is that this padding size has to be determined beforehand at tiling time for all rendering and styling needs, the post above has some recommended (empirical) numbers, but still:

Would have to think about how that interacts with outline widths in screen space pixels and buffer widths in world-space units though..

A big buffer may just be OK, or a user specified option

[vincent-lecrubier-skydio]

This is great. That said I think it still leaves a lot of the spec of 2D Vector Tiles as undefined behavior.

For example the whole Text label layout and declutterring options are undefined.

[lilleyse]

@vincent-lecrubier-skydio that's true, we've mostly focused on the geometric representation here and should start a separate issue for styling and visualization.

[donmccurdy]

A few findings about rendering lines and polygons clipped across tile edges — We'd previously discussed using EXT_mesh_primitive_edge_visibility to handle polygon outlines, mentioned in this comment. The idea was to use the extension to indicate "real" edges of the polygon, as opposed to edges introduced along a tile boundary, or edges between two triangles within the same polygon. After investigation, we don't believe this approach with EXT_mesh_primitive_edge_visibility is necessary or sufficient, for two reasons:

1. As the polygon edge continues from tile A into tile B, the edge may not continue straight, affecting rendering of the tile outline back in tile A.
2. The same challenge affects polylines, not just outlines on polygons.

An illustrations of the problems, as this can present in MVT, here:

https://blog.cyclemap.link/2020-01-25-tilebuffer/

Our preferred solution, instead, is similar to what's non-normatively recommended by MVT: allow geometry to extend past tile boundaries, and recommend and/or require that runtimes clip tiles at the boundary (likely in the fragment shader), so clipped geometry still influences styling within the tile boundary, as expected.

The approach requires a suitable choice of "buffer", i.e. how far to extend geometry into the next tile. If the buffer is too small, relative to line widths or polygon outline widths applied in styling, then discontinuities become visible at tile edges. While we can recommend good defaults, users may occasionally need to adjust the buffer size when creating a tileset.

Importantly, this avoids requiring clients to reconstruct the source polygon at runtime, and allows us to draw each tile independently, which we believe to be critical for performance.

---

Aside — this comment explains only why EXT_mesh_primitive_edge_visibility does not solve discontinuities at tile boundaries, and why we need to extend geometry past the boundary. We may also want some explicit representation of the line or polygon topology, which EXT_mesh_primitive_edge_visibility could perhaps provide. We are leaning away from using this extension for this purpose too, but I'm still in the process of writing up alternatives.

[donmccurdy]

Draft specifications for geometric representation of vector data in 3D Tiles via glTF:

• 3D Tiles
  • Add 3DTILES_content_gltf_vector #838
• glTF
  • Add KHR_mesh_primitive_restart KhronosGroup/glTF#2569
  • Add EXT_mesh_polygon KhronosGroup/glTF#2570

The 3D Tiles specification provides a conceptual definition of "vector data" as applicable to 3D Tiles and geospatial data more broadly, and specifies a particular encoding of point, polyline, and polygon vector data in glTF. The encoding in glTF depends on the two proposed glTF extensions, KHR_mesh_primitive_restart and EXT_mesh_polygon.

Styling remains unspecified, but is expected to be a natural extension of the existing 3D Tiles styling language, with additional vector-specific target properties such as lineWidth (for polylines) and outlineWidth (for polygons).