Feature #24987
openAllow scriptable WordPress REST API access (Cloudflare bypass)
0%
Description
I'd like scriptable access to my pages on khatchad.commons.gc.cuny.edu via the WordPress REST API—course pages, research-related pages, and similar long-lived content. WordPress's wp-admin is excellent for interactive editing; what it doesn't support is programmatic operations on many pages at once or in unattended pipelines, which is exactly what the REST API was designed for. The Cloudflare layer in front of the site currently blocks REST API access entirely, even with valid authentication.
The Blocker¶
Authenticated REST API requests to https://khatchad.commons.gc.cuny.edu/wp-json/wp/v2/pages return HTTP/2 403 with cf-mitigated: challenge and a captcha HTML body, never reaching WordPress. I verified with curl using a valid Application Password and Authorization: Basic header; the request is blocked at the Cloudflare layer regardless of headers or User-Agent.
Request¶
Could the Cloudflare configuration be adjusted to skip Bot Management for authenticated REST API requests—either:
- a WAF custom rule to bypass the Managed Challenge when the request path matches
/wp-json/wp/v2/*and theAuthorizationheader is present, scoped to an opt-in list of accounts (starting with just mine if narrower scoping is helpful); or - an IP allow-list entry, if a per-account exception isn't workable.
What Scriptable Access Enables¶
- Batch updates. Apply a change (a course code rename, a citation update, a recurring footer fix) across many pages atomically rather than editing each one in wp-admin.
- CI checks on every change. Link validation, dead-image detection, HTML lint, accessibility audits before changes go live.
- Open Educational Resources workflows. Push course materials from a public CC-BY Git repository so other CUNY instructors can fork and reuse them, with updates flowing back into Commons via the same pipeline.
- Whole-site branching. Stage a coherent set of updates (e.g., a new semester's pages) and publish in one operation rather than as scattered per-post drafts.
These are operations the WordPress REST API was designed to support; the only thing missing is reachability past Cloudflare for authenticated callers.
Updated by Boone Gorges about 24 hours ago
Thanks for the ticket. I agree that it should be possible to communicate with Commons sites from external applications through the REST API. Your idea about bypassing the WAF rules if an Authorization header is present is a pretty good one. It's pretty conservative in that the vast majority of attack vectors and other problematic behavior associated with the REST API (brute-forcing, DDoS, content enumeration and scraping) would take place in an unauthenticated session. That being said, some of the off-the-shelf tools that interact with the REST API are likely to expect certain endpoints - say, /wp/v2/posts - to be publicly accessible, so the Authorization header trick may not be sufficient. Still, it's a good place to start, and I've sent a message to our hosts to see if they're willing to help us experiment by making the change.
IP-based whitelists are going to be nearly impossible to maintain, as the Commons is used by thousands of people, most of whom don't have static IP addresses available. And user-based whitelists are not feasible, because the WP user can only be determined by asking WP to parse the Authentication header, while the Cloudflare rules prevent the request from getting to WP in the first place. So let's see how our initial work with the Authentication header goes before we start considering other options.
Updated by Boone Gorges about 18 hours ago
Raffi, have you been doing tests of applications that talk to WP in this way, and hit the Cloudflare 500 or 403? If so, can you please share an IP address where the requests originated? Our host would like to examine some failed requests so that we can make principled decisions about how to loosen restrictions.
Updated by Raffi Khatchadourian about 15 hours ago
Boone Gorges wrote in #note-2:
Raffi, have you been doing tests of applications that talk to WP in this way, and hit the Cloudflare 500 or 403? If so, can you please share an IP address where the requests originated? Our host would like to examine some failed requests so that we can make principled decisions about how to loosen restrictions.
Yes! That was probably me. That's how I discovered the captcha block employed by Cloudflare. I think I did it from my apartment, but I don't want to put the IP address (if I can find it) on a public issue.
Updated by Boone Gorges about 4 hours ago
I don't think that the host has actually seen any failed requests yet - the tech asked for your IP so that he could search the logs. Perhaps you could email me your IP address? boone at { my last name with a dot between the fourth and fifth letters }