I encountered an issue while trying to use TensorRT to accelerate the model, and I wanted to export the model to ONNX first. error:
torch.onnx.errors.SymbolicValueError: Unsupported: ONNX export of convolution for kernel of unknown shape.
Serialized File "code/__torch__/projects/edify_tokenizer/v1/module/patching/___torch_mangle_148011.py", line 43
hl2 = torch.to(hl1, 15)
_9 = [int(torch.sub(n0, CONSTANTS.c0)), int(torch.sub(n0, CONSTANTS.c1)), int(torch.sub(n0, CONSTANTS.c0)), int(torch.sub(n0, CONSTANTS.c1))]
x1 = torch.to(torch.pad(x0, _9, "reflect"), 15)
~~~~~~~~ <--- HERE
xl0 = torch._convolution(x1, torch.unsqueeze(hl2, 2), None, [1, 2], [0, 0], [1, 1], False, [0, 0], 12, True, False, True, True)
xh0 = torch._convolution(x1, torch.unsqueeze(hh2, 2), None, [1, 2], [0, 0], [1, 1], False, [0, 0], 12, True, False, True, True)
)
Inputs:
#0: 242 defined in (%242 : BFloat16(*, *, *, *, device=cpu) = onnx::Pad[mode="reflect"](%x0, %241), scope: testvae::/torch.nn.modules.container.Sequential::encoder/projects.edify_tokenizer.v1.module.layers2d.Encoder::encoder/projects.edify_tokenizer.v1.module.patching.Patcher::patcher # /usr/local/lib/python3.10/dist-packages/torch/nn/functional.py:4539:0
) (type 'Tensor')
Outputs:
#0: x1 defined in (%x1 : BFloat16(*, *, *, *, device=cpu) = onnx::Cast[to=16](%242), scope: testvae::/torch.nn.modules.container.Sequential::encoder/projects.edify_tokenizer.v1.module.layers2d.Encoder::encoder/projects.edify_tokenizer.v1.module.patching.Patcher::patcher # /lustre/fs3/portfolios/nvr/users/xianl/edify_tokenizer1/cosmos/CI1024_AE_cosmos_8x8_scratch_0920/projects/edify_tokenizer/v1/module/patching.py:73:0
) (type 'Tensor')
I believe that modifying the encoder model is necessary to solve this problem. However, the project seems to only provide a torch script module, which does not support modifications. Where can I find the original model?
Or could you provide an example of accelerating the model using TensorRT?
I encountered an issue while trying to use TensorRT to accelerate the model, and I wanted to export the model to ONNX first. error:
I believe that modifying the encoder model is necessary to solve this problem. However, the project seems to only provide a torch script module, which does not support modifications. Where can I find the original model?
Or could you provide an example of accelerating the model using TensorRT?