[Question] Plan for prompt replay?

[binarycrayon]

I have a collection of prompts that are in text-to-text tasks, labeled with datetimes. Some queries are executed in linear, some are executed in parallel. I have the data that can recreate real workload in a game session, across multiple game features.

Would be interested to replay with genai-bench and do a cross comparison. Is replay feature on roadmap? I can share data in private channels.

[CatherineSue]

Hi @binarycrayon could you elaborate on the replay feature?

[binarycrayon]

Sure, with a list of request record in the past, imagine 2 columns datetime, prompts and ordered by datetime.
I would expect the replay feature offsets the earliest datetime in the record as start time, and playback the records to the same scale (# of requests) and optionally, with the same intervals (time difference to the first record) to simulate the LLM request traffic recorded in the past game sessions.

By optionally, I meant to be able to execute some requests as fast as possible.