Skip to content

[Question] can this be used to test against ST-WebAgentBench? #57

@zshanhui

Description

@zshanhui

Required prerequisites

  • I have searched the Issue Tracker that this hasn't already been reported. (+1 or comment there if it has.)

Questions

I am working on an issue for Camel-AI that requires integrating the benchmark for https://github.com/segev-shlomov/ST-WebAgentBench. Can the CRAB project be useful in benchmarking web agents such as ST-WebAgentBench? If so, i can perhaps integrate this benchmark here and use it to benchmark CamelAI. ST-WebAgentBench requires setting up a bunch of digital SaaS twins on docker or AWS AMI and I am looking for any ways to make the environment setup easier, cheaper to reproduce.

Metadata

Metadata

Assignees

Labels

questionFurther information is requested

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions