Bug #25065
Updated by Raymond Hoh 7 days ago
Quick follow-up related to #24987 (thanks again for the WAF exception that fixed the wp-json case).
Noticed a related behavior on the public-facing side worth a look. Anonymous HTML fetches of my site's home page get captcha-walled by Cloudflare, even from non-suspicious user agents.
h2. Reproducer
<pre>
curl -A "Mozilla/5.0 (compatible; LinkPreview)" https://khatchad.commons.gc.cuny.edu/
</pre>
Response is a "Captcha Required" interstitial (HTML body containing the challenge, no actual page content).
h2. Two questions
# Is this anonymous-fetch challenge intentional, or is the WAF being aggressive with user agents Cloudflare doesn't recognize?
# Are there explicit exemptions configured for social-media preview bots (LinkedIn, Twitter/X, Slack, Mastodon, Facebook)? My main concern is link previews for shared posts coming up blank.
Major search-engine crawlers (Googlebot, Bingbot) typically have verified-IP allowlists that bypass these, so I'm less worried about indexing impact -- but worth confirming.
Thanks,
Raffi