IAsyncEnumerable - JSONL/application/x-ndjson

[JohnGalt1717]

Is your feature request related to a problem? Please describe.
I have OpenAI outputting IAsyncEnumerable which my api is outputting IAsyncEnumerable. This is really single line json objects in IAsyncEnumerable. I need a way to annotate the openapi.json file to tell you this and then have the client actually be able to then query the IAsyncEnumerable properly and stream the response.

I have the endpoint tagged with content type of application/x-ndjson which would indicate that it's a stream.

Describe the solution you'd like
I'd like to see this generate IAsyncEnumerable response if the mime type of the response is application/x-ndjson. And then Refit would have to be generating a client from that that could stream the ndjson result.

Describe alternatives you've considered
The only thing I can think of is create a second partial interface and add the endpoint myself to the interface or resulting class library from refit that does this correctly, but then I have manual code to maintain.

Additional context
This is a pretty common issue now because of OpenAI.

[christianhelle]

@JohnGalt1717 Thanks for taking the time to report this.

I've never worked with the application/x-ndjson media type but if Refit supports and the OpenAPI specifications can describe it then it shouldn't be that hard to generate the interface.

Let me see what I can do.

[christianhelle]

@JohnGalt1717 I just noticed that you mentioned this from this Refit issue

Generating a Refit interface is one thing, but until Refit itself supports this, the generated code will not really work

Let's park this until Refit officially supports the application/x-ndjson media type

[JohnGalt1717]

It's more that Refit has to handle IAsyncEnumerable that you pass. Right now Refitter passes back ICollection instead of IAsyncEnumerable. I believe the only thing that you have to do on your end is when it sees application/x-ndjson is make the interface IAsyncEnumerable instead of ICollection and it would work from your end. Then the refit people need to then hand IAsyncEnumerable and there is a specific case of how to deserialize the stream of x-ndjson that they also need to handle.

[christianhelle]

@JohnGalt1717 Do you have an example OpenAPI spec that we can use to develop and test with?

[JohnGalt1717]

Here's what I have.

    "/question": {
      "post": {
        "tags": [
          "Answer Document Question"
        ],
        "requestBody": {
          "content": {
            "application/json": {
              "schema": {
                "$ref": "#/components/schemas/AnswerQuestionRequestDto"
              }
            }
          }
        },
        "responses": {
          "200": {
            "description": "Success",
            "content": {
              "application/x-ndjson": {
                "schema": {
                  "type": "array",
                  "items": {
                    "$ref": "#/components/schemas/StreamingResponseDto"
                  }
                }
              }
            }
          },
          "400": {
            "description": "Bad Request",
            "content": {
              "application/problem+json": {
                "schema": {
                  "$ref": "#/components/schemas/HttpValidationProblemDetails"
                }
              }
            }
          }
        },
        "security": [
          {
            "openidconnect": [ ]
          }
        ]
      }
    },

[christianhelle]

@JohnGalt1717 Thanks! Anything special in StreamingResponseDto? Or is this just a POCO?

[JohnGalt1717]

Just a poco. If you look at the link to the issue in refit, you can see how this will be used. The HttpClient also has an extension for get: GetFromJsonAsAsyncEnumerable

Basically every line is valid json and it's pulling the entire json object on readline and then decoding it to the TResponse type. In my case that's StreamingResponseDto but it could be whatever you want that can deserialize the result to json and then it gets pushed into the IAsyncenumerable to be used in an await foreach(...)

So for this, it just needs IAsyncEnumerable, but in Refit itself, it has to see IAsyncEnumerable and it has to assume (and probably doesn't matter about the content type attribute) that if it gets IAsyncEnumearble that it uses GetFromJsonAsAsyncEnumerable to stream it. Or if it's a post, uses my extension method to do so (I don't know why they didn't add one for post because OpenAI is a post, not a get...) which is a direct rip of the GetFromJsonAsAsyncEnumerable just converted to post.