SimBA remarks

[aliotopal]

We observed a mismatch in ART between the documentation given for SimBA.generate(x, y) and its implementation. In this function, x is an array of clean images to be attacked and y is the array of target categories. While the documentation suggests that y can be provided as class indices (e.g., [[306], [204], ...]), the implementation of SimBA applies np.argmax(y, axis=1), and this latter function only works correctly if y is one-hot encoded or a probability distribution. As a result, using class indices causes defaulting to target class indices $[[0], [0], ...$. This, of course, affects the correctness of targeted attacks. We recommend that the ART library clarifies/corrects this feature in future versions.

Additionally, for the ImageNet dataset, the official SimBA GitHub repository recommends configurations that differ from those used in the ART library. Specifically, for targeted attacks, the settings recommended by the SimBa GitHub are max_iter=30,000, freq_dim=35, and stride=9; for untargeted attacks, these values are max_iter=10000, freq_dim=28, and stride=7.