Captcha Protect

Traefik middleware to challenge individual IPs in a subnet when traffic spikes are detected from that subnet, using a captcha of your choice for the challenge (turnstile, recaptcha, or hcaptcha). Requires traefik v2.11.1 or above

You may have seen CAPTCHAs added to individual forms on the web to prevent bots from spamming submissions. This plugin extends that concept to your entire site (or specific routes on your site), effectively placing your entire site behind a CAPTCHA. However, the CAPTCHA is only triggered when a spike in traffic is detected from the same IP subnet. Once the CAPTCHA is successfully completed, that IP is no longer challenged, allowing uninterrupted browsing.

Copy
flowchart TD
    Client(Client accesses path on website) --> IP{Has client passed captcha challenge in the last 24h?}
    IP -- Yes --> Continue(Go to original destination)
    IP -- No --> IP_BYPASS{Is client IP excluded by captcha-protect config?}
    IP_BYPASS -- Yes --> Continue(Go to original destination)
    IP_BYPASS -- No --> GOOD_BOT{Is client IP hostname in allowed bot list?}
    GOOD_BOT -- No --> PROTECTED_ROUTE{Is this route protected?}
    GOOD_BOT -- Yes --> CANONICAL_URL_BOT{Are there URL parameters?}
    CANONICAL_URL_BOT -- Yes --> PROTECTED_ROUTE{Is this route prefix in protectRoutes?}
    CANONICAL_URL_BOT -- No --> Continue(Go to original destination)
    PROTECTED_ROUTE -- Yes --> RATE_LIMIT{Is this IP in a range seeing increased traffic?}
    PROTECTED_ROUTE -- No --> Continue(Go to original destination)
    RATE_LIMIT -- Yes --> REDIRECT(Redirect to /challenge)
    RATE_LIMIT -- No --> Continue(Go to original destination)
    REDIRECT --> CIRCUIT{Is circuit breaker open?}
    CIRCUIT -- Yes --> POJ_CHALLENGE{Proof-of-Javascript challenge}
    CIRCUIT -- No --> CAPTCHA_CHALLENGE{turnstile/recaptcha/hcaptcha challenge}
    POJ_CHALLENGE -- Pass --> Continue(Go to original destination)
    POJ_CHALLENGE -- Fail --> Stuck
    CAPTCHA_CHALLENGE -- Pass --> Continue(Go to original destination)
    CAPTCHA_CHALLENGE -- Fail --> Stuck

Config

Example

Below is an example docker-compose.yml with traefik as the frontend, and nginx as the backend. nginx is using this middleware to protect routes on the site that start with / (protectRoutes: "/")

Since the config values aren't specified, captcha-protect would use the default rateLimit: 20 and window: 86400 so any IPv4 in X.Y.0.0/16 (or ipv6 in /64) could only access the site 20 times before individual IPs in that subnet are required to pass a captcha to continue browsing.

Copy
networks:
    default:
services:
    nginx:
        image: nginx:${NGINX_TAG}
        labels:
            traefik.enable: true
            traefik.http.routers.nginx.entrypoints: http
            traefik.http.routers.nginx.service: nginx
            traefik.http.routers.nginx.rule: Host(`${DOMAIN}`)
            traefik.http.services.nginx.loadbalancer.server.port: 80
            traefik.http.routers.nginx.middlewares: captcha-protect@docker
            traefik.http.middlewares.captcha-protect.plugin.captcha-protect.rateLimit: 0
            traefik.http.middlewares.captcha-protect.plugin.captcha-protect.ipv4subnetMask: 8
            traefik.http.middlewares.captcha-protect.plugin.captcha-protect.window: 864000
            traefik.http.middlewares.captcha-protect.plugin.captcha-protect.protectRoutes: "/"
            traefik.http.middlewares.captcha-protect.plugin.captcha-protect.captchaProvider: turnstile
            traefik.http.middlewares.captcha-protect.plugin.captcha-protect.siteKey: ${TURNSTILE_SITE_KEY}
            traefik.http.middlewares.captcha-protect.plugin.captcha-protect.secretKey: ${TURNSTILE_SECRET_KEY}
            traefik.http.middlewares.captcha-protect.plugin.captcha-protect.goodBots: apple.com,archive.org,commoncrawl.org,duckduckgo.com,facebook.com,google.com,googlebot.com,googleusercontent.com,instagram.com,kagibot.org,linkedin.com,msn.com,openalex.org,twitter.com,x.com
            traefik.http.middlewares.captcha-protect.plugin.captcha-protect.persistentStateFile: /tmp/state.json
            traefik.http.middlewares.captcha-protect.plugin.captcha-protect.enableStateReconciliation: "false"
            traefik.http.middlewares.captcha-protect.plugin.captcha-protect.periodSeconds: 30
            traefik.http.middlewares.captcha-protect.plugin.captcha-protect.failureThreshold: 3
        networks:
            default:
                aliases:
                  - nginx
    traefik:
        image: traefik:${TRAEFIK_TAG}
        command: >-
            --api.insecure=false
            --api.dashboard=false
            --api.debug=false
            --ping=true
            --entryPoints.http.address=:80
            --providers.docker=true
            --providers.docker.network=default
            --experimental.plugins.captcha-protect.modulename=github.com/libops/captcha-protect
            --experimental.plugins.captcha-protect.version=v1.11.1
        volumes:
            - /var/run/docker.sock:/var/run/docker.sock:z
            - /CHANGEME/TO/A/HOST/PATH/FOR/STATE/FILE:/tmp/state.json:rw
        ports:
            - "80:80"
        networks:
            default:
                aliases:
                    - traefik
        healthcheck:
            test: traefik healthcheck --ping
        depends_on:
            nginx:
                condition: service_started

Config options

Circuit Breaker (failover if a captcha provider is unavailable)

The circuit breaker provides automatic failover when the primary captcha provider (Turnstile, reCAPTCHA, or hCaptcha) becomes unavailable. When enabled, it:

1. Enables a liveness probe on the captcha provider: Periodically sends HEAD requests to the provider's JavaScript file (every periodSeconds, default 30s). Also records 5xx errors during server side validation.
2. Detects failures: Counts consecutive health check failures
3. Opens circuit: After failureThreshold consecutive failures, switches to proof-of-javascript fallback
4. Falls back to PoJ: Ensures user is loading javascript. Requires revalidating in 1hr
5. Auto-recovery: Automatically returns to primary provider when health checks succeed

Proof-of-Javascript Fallback:

• Requires browsers to submit a form
• Self-contained (no external dependencies)

Configuration:

• Circuit breaker is enabled by setting two config settings: periodSeconds and failureThreshold
  • e.g. periodSeconds: 30 and failureThreshold: 3
• To disable: set both periodSeconds: 0 and failureThreshold: 0 (the default config)
• The poj provider can also be used directly as the primary provider (no circuit breaker needed)

Good Bots

To avoid having this middleware impact your SEO score, it's recommended to provide a value for goodBots. By default, no bots will be allowed to crawl your protected routes beyond the rate limit unless their second level domain (e.g. google.com) is configured as a good bot.

A good default value for goodBots would be:

goodBots: apple.com,archive.org,duckduckgo.com,facebook.com,google.com,googlebot.com,googleusercontent.com,instagram.com,kagibot.org,linkedin.com,msn.com,openalex.org,twitter.com,x.com

However if you set the config parameter protectParameters="true", even good bots won't be allowed to crawl protected routes if a URL parameter is on the request (e.g. /foo?bar=baz). This protectParameters feature is meant to help protect faceted search pages.

Overriding the challenge template file

You probably will want to theme the CAPTCHA challenge page to match the style of your site.

You can do that by copying the challenge.tmpl.html file in this repo into your docker compose project, mounting it into your traefik container

Copy
    traefik:
        volumes:
            - ./host/path/to/challenge.tmpl.html:/challenge.tmpl.html:ro

and pointing the middleware to your overridden template with

Copy
            traefik.http.middlewares.captcha-protect.plugin.captcha-protect.challengeTmpl: "/challenge.tmpl.html"

When you override the challenge template, the process probably looks like:

1. Copying some html file from your existing site (so the challenge looks like the rest of your site)
2. Replacing some <div> in the HTML body for the file copied in step 1 with the <form>...</form><script>...</script> HTML tags/contents in this repo's default challenge.tmpl.html. You must copy the form and script tags exactly as they are in the original challenge template. They use go's templating language to inject the proper site key and other variables into the HTML response when a challenge is presented
3. You must also be sure to have this in the <head> of your overridden template:

    <script src="{{ .FrontendJS }}" async defer referrerpolicy="no-referrer"></script>

Similar projects

• Traefik RateLimit middleware - the core traefik ratelimit middleware will start sending 429 responses based on individual IPs, which might not be good enough to protect against traffic coming from distributed networks. Also, this plugin (captcha-protect) allows not including files in your rate limiter to avoid static assets from being counted in the rate limit.
• crowdsec-bouncer-traefik-plugin has a captcha option, but requires integrating with crowdsec to verify individual IPs. This plugin (captcha-protect) instead just checks the traffic actually visiting your site and verifies the traffic is from a person only when the traffic exceeds some rate limit you configure.

Attribution

• the original implementation of this logic was a drupal module called turnstile_protect. This traefik plugin was made to make the challenge logic even more perfomant than that Drupal module, and also to provide this bot protection to non-Drupal websites
• making general captcha structs to support multiple providers was based on the work in crowdsec-bouncer-traefik-plugin
• in memory cache thanks to https://github.com/patrickmn/go-cache

When to enable regex

When possible, you want to keep regex disabled as seen in the example benchmark below.

However, when needed it can be enabled with mode: regex

$ go mod init bench
$ cat << EOF > bench_test.go
package main

import (
	"regexp"
	"strings"
	"testing"
)

var (
	testPath = "/api/v1/user/profile"
	prefix   = "/api/v1"
	regex    = regexp.MustCompile("^/api/v1")
)

func BenchmarkHasPrefix(b *testing.B) {
	for i := 0; i < b.N; i++ {
		_ = strings.HasPrefix(testPath, prefix)
	}
}

func BenchmarkRegexMatch(b *testing.B) {
	for i := 0; i < b.N; i++ {
		_ = regex.MatchString(testPath)
	}
}
EOF
$ go test -bench=. -benchmem

goos: darwin
goarch: amd64
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkHasPrefix-12     	340856451	         3.415 ns/op	       0 B/op	       0 allocs/op
BenchmarkRegexMatch-12    	27992568	        41.20 ns/op	       0 B/op	       0 allocs/op
PASS

How to monitor the rate limiter

If you set the enableStatsPage to true, it allows exemptIps to access /captcha-protect/stats to monitor the rate limiter. The key JSON key to look on the stats page is the top level "rate" key, which will list the subnets that are currently forced to be challenged based to request patterns and the captcha-protect configuration values used.

If you have use a computer within the exemptIps, and access to the command line tools curl and jq, here is a recipe for how to list the top 25 subnets being challenged...

Copy
curl -s https://example.com/captcha-protect/stats |   jq -r '.rate | to_entries | sort_by(.value) | .[] | "\(.key): \(.value)"' |   tail -25

This JSON state data is also found in the state.json file that you should have configured in your docker-compose.yml using the persistentStateFile setting and volume definition. NOTE: this file should only be changed by captcha-protect and not manually.

Troubleshooting

Here is a way to troubleshoot your captcha-protect set up.

Verify that your Turnstile site-key is configured properly

One reason that may cause captcha-protect to not work is that the Cloudflare Turnstile widget site-key or private-key are not properly set for captcha-protect to access. Below is a way to confirm if the Turnstile site-key is configured correctly. (WARNING: There is currently no easy way to check the private-key, since the secret-key should never be displayed on a webpage or shared.)

1. Visit the captcha-protect "challenge" URL which is set to https://example.com/challenge by default.
2. You should see a web page that says "Verifying connection"
   NOTE: If you customized the challengeTmpl configuration, the page may say something different.
3. Look at the HTML source code for the page https://example.com/challenge, by right-clicking on the page and selecting "View page source" (on Chrome).
4. In the HTML source view that opens up, look for a <Div> tag that has an attribute named data-sitekey, and check if its value matches your Cloudflare Turnstile widget sitekey value.
   TIP: You need to log in to your Cloudflare online account and go to the Turnstile section to see your site-key and secret-key values.
5. If the site-key value did not match in the HTML <Div> tag, then update the docker-compose.yml and/or .env file to correctly pass the site-key value. Also check if the Cloudflare Turnstile widget secret-key is set correctly.