How to bypass challenge for special urls (api entries)

[Francis-Nalta]

Hello Anubis community
First thanks to TecharoHQ and contributers fior this program :)

Just new to anubis :)
So at this time i dont know all about anubis configuration (bot policies, ...)

We tested anubis with a web site. All is ok, Anubis do the job well, bad crawlers and bot stopped :)

The setup is similar to : https://anubis.techaro.lol/docs/admin/environments/nginx

The problem :
The web site have some 'api' (specific urls with variable parameters) that will be called by external programs/web sites (some have specfic agent stings, others no agent string at all) to trigger or perform some actions in the web site.

But those external programs dont know anything about cookies, javascript, .... they just send requests to the web site on the api url.
They cannot response to the challenge send by Anubis.

Bypass by IP cannot be used (variable external ips, not constant).

Notice : Joomla and Wordpress also have somme fonctions or plugins that use external triggers (or loopback calls to themself this external url) to trigger crons, ..., the question is also valid for them too.

How to instruct anubis to not challenge those calls ?
(filter by request check with regex in my mind)

Francis

[Nickalus12]

The issue is that Anubis, by default, is "heavy-handed" and challenges any client that looks like a browser (typically those with Mozilla in the User-Agent). Since your external API callers and loopback triggers cannot solve JavaScript or Meta Refresh challenges, they get stuck.

To resolve this, you need to add an allowlist rule to your policy file that matches your API paths using a regex. In Anubis, rules are evaluated sequentially, so you must place your "allow" rules before the generic browser challenge rules.

Solution: Bypass API Paths

Add a rule using path_regex with the action ALLOW. This tells Anubis to skip all further checks and proxy the request directly to your backend.

bots:
  # 1. Bypass specific API endpoints (Place this first!)
  - name: bypass-api-endpoints
    path_regex: ^/(api|webhooks|cron-trigger)/.*
    action: ALLOW

  # 2. Bypass Joomla/Wordpress specific triggers
  - name: joomla-wp-loopback
    path_regex: ^/(index\.php\?option=com_ajax|wp-cron\.php)
    action: ALLOW

  # 3. Existing generic challenge rule (Keep this last)
  - name: generic-browser
    user_agent_regex: Mozilla|Opera
    action: CHALLENGE
    challenge:
      difficulty: 1
      algorithm: preact

Key Implementation Details

• Order Matters: Anubis stops at the first matching rule with a terminal action like ALLOW or CHALLENGE. If your generic-browser rule is at the top, it will challenge your API callers before it ever sees your bypass rule.
• Regex Syntax: Use standard regex. For example, ^/api/.* matches any path starting with /api/.
• Headers as a Fallback: If your API callers use a specific header (e.g., X-API-Key or a custom User-Agent), you can also match those to make the bypass more secure:

  - name: allow-internal-tools
    headers_regex:
      X-Custom-Tool: "InternalValue"
    action: ALLOW

Tip

If you are using the Nginx environment, ensure your POLICY_FILE environment variable is correctly pointing to this YAML configuration so Anubis picks up the changes on restart.

Do you have a specific URL pattern or a custom User-Agent for these external programs that we could use to make the regex more precise?

[Francis-Nalta]

Hi Nickalus

Thank you for your help :)
We will try asap your recommandations, and give results here

For URL pattern and custom User-Agent (if used) we have to request our devs for them.

Thank you again for your response.

[Nickalus12]

Sounds good! Once your devs share the URL patterns and User-Agent strings, you can make the rules more precise. Feel free to post the patterns here and I can help tighten up the regex so it only bypasses exactly what needs bypassing.