Context
master used to support multipart: true for Ch.query/4. The rewrite removed it for now while simplifying the request path around NimblePool, eager Ch.query, default RowBinaryWithNamesAndTypes decoding, and explicit request/response compression.
Old behavior worth preserving:
multipart: true was an option on queries.
- Query params were encoded with the same ClickHouse escaped parameter encoding, but sent as multipart parts named
param_<name> / param_$<index> instead of URL query params.
- The SQL statement was sent as a multipart
query part.
x-clickhouse-format was still set on the request, defaulting to RowBinaryWithNamesAndTypes unless overridden.
- URL query string still carried ClickHouse settings.
Proposed restore scope
Restore multipart only as a transport encoding for query text + query params:
Ch.query(pool, "SELECT {value:String}", %{ "value" => large_value }, multipart: true)
Implementation shape:
- Keep public API as
Ch.query(pool, statement, params, opts).
- When
multipart: true, build a multipart body containing:
- one
query part with the SQL statement;
- one part per encoded query param, named
param_<name>.
- Keep
settings in the URL query string, not in the multipart body.
- Set
content-type: multipart/form-data; boundary=....
- Preserve the response behavior from the rewrite:
- default decoded
RowBinaryWithNamesAndTypes returns %Ch.Result{names, rows, headers, data};
- overridden formats return raw successful bodies;
- ClickHouse errors return
%Ch.Error{}.
Inserts
Keep inserts non-multipart for now.
For inserts, the current raw body model is clearer and works well with compression:
payload =
:zstd.compress([
"INSERT INTO table FORMAT RowBinaryWithNamesAndTypes\n",
Ch.RowBinary.encode_names_and_types(names, types),
Ch.RowBinary.encode_rows(rows, types)
])
Ch.query!(pool, payload, %{}, headers: [{"content-encoding", "zstd"}])
That keeps content-encoding scoped to the entire body ClickHouse parses after decompression. Multipart insert support can be considered separately if a concrete use case appears.
Compression interaction
If multipart: true is combined with content-encoding, compression should apply to the fully-built multipart body, not to individual parts. This is different from compressed inserts, where the caller can pass a precompressed complete insert body directly.
Tests to restore/add
- named params sent through multipart round-trip correctly;
- large string params that would be unsuitable for URL query strings;
- settings still go into the URL and work with multipart;
- custom response format with multipart still returns raw body;
content-encoding: zstd or gzip compresses the full multipart body, if we choose to support that combination;
- inserts continue to use the raw body path and are not affected by
multipart unless intentionally supported later.
Context
masterused to supportmultipart: trueforCh.query/4. The rewrite removed it for now while simplifying the request path aroundNimblePool, eagerCh.query, defaultRowBinaryWithNamesAndTypesdecoding, and explicit request/response compression.Old behavior worth preserving:
multipart: truewas an option on queries.param_<name>/param_$<index>instead of URL query params.querypart.x-clickhouse-formatwas still set on the request, defaulting toRowBinaryWithNamesAndTypesunless overridden.Proposed restore scope
Restore multipart only as a transport encoding for query text + query params:
Implementation shape:
Ch.query(pool, statement, params, opts).multipart: true, build a multipart body containing:querypart with the SQL statement;param_<name>.settingsin the URL query string, not in the multipart body.content-type: multipart/form-data; boundary=....RowBinaryWithNamesAndTypesreturns%Ch.Result{names, rows, headers, data};%Ch.Error{}.Inserts
Keep inserts non-multipart for now.
For inserts, the current raw body model is clearer and works well with compression:
That keeps
content-encodingscoped to the entire body ClickHouse parses after decompression. Multipart insert support can be considered separately if a concrete use case appears.Compression interaction
If
multipart: trueis combined withcontent-encoding, compression should apply to the fully-built multipart body, not to individual parts. This is different from compressed inserts, where the caller can pass a precompressed complete insert body directly.Tests to restore/add
content-encoding: zstdorgzipcompresses the full multipart body, if we choose to support that combination;multipartunless intentionally supported later.