CUNY Academic Commons

Thanks for the ticket. I agree that it should be possible to communicate with Commons sites from external applications through the REST API. Your idea about bypassing the WAF rules if an Authorization header is present is a pretty good one. It's pretty conservative in that the vast majority of attack vectors and other problematic behavior associated with the REST API (brute-forcing, DDoS, content enumeration and scraping) would take place in an unauthenticated session. That being said, some of the off-the-shelf tools that interact with the REST API are likely to expect certain endpoints - say, /wp/v2/posts - to be publicly accessible, so the Authorization header trick may not be sufficient. Still, it's a good place to start, and I've sent a message to our hosts to see if they're willing to help us experiment by making the change.

IP-based whitelists are going to be nearly impossible to maintain, as the Commons is used by thousands of people, most of whom don't have static IP addresses available. And user-based whitelists are not feasible, because the WP user can only be determined by asking WP to parse the Authentication header, while the Cloudflare rules prevent the request from getting to WP in the first place. So let's see how our initial work with the Authentication header goes before we start considering other options.

Raffi, have you been doing tests of applications that talk to WP in this way, and hit the Cloudflare 500 or 403? If so, can you please share an IP address where the requests originated? Our host would like to examine some failed requests so that we can make principled decisions about how to loosen restrictions.

Boone Gorges wrote in #note-2:

Raffi, have you been doing tests of applications that talk to WP in this way, and hit the Cloudflare 500 or 403? If so, can you please share an IP address where the requests originated? Our host would like to examine some failed requests so that we can make principled decisions about how to loosen restrictions.

Yes! That was probably me. That's how I discovered the captcha block employed by Cloudflare. I think I did it from my apartment, but I don't want to put the IP address (if I can find it) on a public issue.

I don't think that the host has actually seen any failed requests yet - the tech asked for your IP so that he could search the logs. Perhaps you could email me your IP address? boone at { my last name with a dot between the fourth and fifth letters }

You can also make the issue private. Either way, I'll look up my IP tonight.

Boone Gorges wrote in #note-2:

Raffi, have you been doing tests of applications that talk to WP in this way, and hit the Cloudflare 500 or 403? If so, can you please share an IP address where the requests originated? Our host would like to examine some failed requests so that we can make principled decisions about how to loosen restrictions.

IP address sent via email.

I've sent a follow-up to our host with this information.

Raffi, the host wasn't able to find any blocked traffic in the logs related to the IP that you reported. I wonder if perhaps you are issued a dynamic IP by your ISP, and that it changed between your original tests and when you reported the IP to me. When you get a chance, perhaps you could run another test with your custom tools that use the REST API; assuming you hit the same 403 etc behavior, you can let us know your IP address and, perhaps, the specific URLs that you tried to hit. This will help us work with the host to understand the precise WAF rules that are being triggered.

For context: #25003 documents that the hosting provider has already authored Cloudflare WAF custom rules on this same site to let trusted authenticated REST API callers through Bot Management—first a rule allowing Jetpack to hit /wp-json/, then a follow-up scoped to Jetpack's ASN. The mechanism this ticket asks for is already in active use here; this request is the same shape, scoped narrower (a single account, opt-in, keyed on the Authorization header).

Thanks for passing along the details, Raffi. I'll send this to the host.

The mechanism this ticket asks for is already in active use here; this request is the same shape, scoped narrower (a single account, opt-in, keyed on the Authorization header).

To clarify, I would not consider this to be "narrower". Keying on the mere presence of the Authorization header is quite a bit broader than allowing an ASN or IP range. And we can't allow for "a single account" - account authorization and identification happens at the WP level, and the Cloudflare rules take place before the request hits WP; moreover, if we're going to make changes, ideally they'd be made in such a way as to benefit all Commons users.

This point aside, I'll work with the host to see what we can do.

Boone Gorges wrote in #note-10:

To clarify, I would not consider this to be "narrower". Keying on the mere presence of the Authorization header is quite a bit broader than allowing an ASN or IP range. And we can't allow for "a single account" - account authorization and identification happens at the WP level, and the Cloudflare rules take place before the request hits WP; moreover, if we're going to make changes, ideally they'd be made in such a way as to benefit all Commons users.

This point aside, I'll work with the host to see what we can do.

Thanks, Boone. Fair point—you're right that ASN scoping is tighter than auth-header presence. The WAF can't validate the password without letting the request through to WP, which is the reachability problem itself. A host-wide path sounds like a better outcome than a per-account exception anyway; happy to test from my end whenever the hosting provider has something to try.

Raffi, the host reports that he's made an exception in the Cloudflare firewall rule such that wp-json requests accompanied by an Authentication header are allowed through. Can you run another test and let me know what you find?

Boone Gorges wrote in #note-12:

Raffi, the host reports that he's made an exception in the Cloudflare firewall rule such that wp-json requests accompanied by an Authentication header are allowed through. Can you run another test and let me know what you find?

Verified—GET https://khatchad.commons.gc.cuny.edu/wp-json/wp/v2/pages with HTTP Basic + Application Password now returns HTTP/2 200 with content-type: application/json. The bypass is working.

For the record (not asking for any further change): GET .../wp-json/wp/v2/users/me still returns HTTP/2 403, but the response is now origin-side—x-turbo-charged-by: LiteSpeed, no cf-mitigated header, plain "403 Forbidden" body. Looks like the standard LiteSpeed/WP anti-enumeration hardening on /wp-json/wp/v2/users/*. Different layer from Cloudflare, and not relevant to my page-management workflow.

Thanks for the work on this one—the careful technical pushback in #note-10, the back-and-forth with the host to track down the right log entries, and pursuing a host-wide solution rather than a per-account exception. Please pass thanks along to the hosting provider as well. Happy to have this closed.