CUNY Academic Commons

Working on it.

Hey André - Thanks a million for all the work you've done on this. It's just about ready to go.

I do need to have the DNS changes in place before I can fully set up the environment, though (BuddyPress needs pretty URLs to work right). Do you have a rough timeline on that? Not a huge rush - though if you think it'll be a while, I'm going to set up a quick sandbox on my own server to share the first round of work with the team, as a stopgap until your solution is in place.

Thanks again!

Not a huge rush - though if you think it'll be a while, I'm going to set up a quick sandbox on my own server to share the first round of work with the team, as a stopgap until your solution is in place.

In other words: "No rush, but I need it now!" ;-)

Added the records to external DNS:

cbox                    IN      CNAME   cdev.gc.cuny.edu
dev.cbox                IN      CNAME   cdev.gc.cuny.edu.

...and added to internal GC domain controllers as well. Please test and let me know, thanks.

Later I'll migrate these records to AWS, but this will do for now.

In other words: "No rush, but I need it now!" ;-)

Ha ha. I didn't intend it this way. I just wasn't sure whether there was some bureaucratic reason that would mean a significant delay. It really does just take me a couple minutes to set up a space for temporary use. Thanks for fast-tracking this one :)

Just pulling your leg, but I say it with love.

:)

Can't seem to shell in:

$ ssh cboxdev@cdev.gc.cuny.edu
Connection closed by 146.96.128.210
$ ssh cboxdev@146.96.128.210
Connection closed by 146.96.128.210

Is cboxdev whitelisted for SSH? Or maybe my IP is blacklisted :)

Is cboxdev whitelisted for SSH?

This. Fixed now, my bad.

Also I added your public rsa key to authorized_keys.

Whoop, I'm in! Thanks for the key setup too. Almost there.

I put a dummy index.html file in both the www and dev directories, but can't access either through a browser. When I visit http://cbox.gc.cuny.edu, I get nothing - like the DNS isn't set up correctly. (Maybe it's not propagated yet?) When I visit http://dev.cdev.gc.cuny.edu, it resolves, but I get a generic 403 Forbidden Apache page. Would you mind visiting from an external IP to see if you can reproduce?

I put a dummy index.html file in both the www and dev directories, but can't access either through a browser.

SELinux. I forgot that was enabled there. Should be good now.

Awesome.

dev.cbox.gc.cuny.edu is 99% working. I am having a problem submitting new forum posts - I get a 403 when I send a POST request to http://dev.cbox.gc.cuny.edu/groups/foo-group/forum/. I can dig into the codebase to see if it's on my end, but I thought first I'd check to see whether there's anything on your end that jumps out.

http://cbox.gc.cuny.edu is still not resolving for me, but I'll just wait a day or two to see if it fixes itself.

Thanks to you both!

dev.cbox.gc.cuny.edu is 99% working. I am having a problem submitting new forum posts - I get a 403 when I send a POST request to http://dev.cbox.gc.cuny.edu/groups/foo-group/forum/. I can dig into the codebase to see if it's on my end, but I thought first I'd check to see whether there's anything on your end that jumps out.

ModSecurity. It thinks your form is a sql injection attack.

[Wed Jun 06 16:14:21 2012] [error] [client 67.243.140.73] ModSecurity: Rule execution error - PCRE limits exceeded (-8): (null). [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6HJJggNIAADJjNUcAAAAJ"]
[Wed Jun 06 16:14:21 2012] [error] [client 67.243.140.73] ModSecurity: Rule execution error - PCRE limits exceeded (-8): (null). [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6HJJggNIAADJjNUcAAAAJ"]
[Wed Jun 06 16:14:21 2012] [error] [client 67.243.140.73] ModSecurity: Rule execution error - PCRE limits exceeded (-8): (null). [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6HJJggNIAADJjNUcAAAAJ"]
[Wed Jun 06 16:14:21 2012] [error] [client 67.243.140.73] ModSecurity: Access denied with code 403 (phase 2). Match of "streq 0" against "TX:MSC_PCRE_LIMITS_EXCEEDED" required. [file "/etc/httpd/conf.d/mod_security.conf"] [line "93"] [msg "ModSecurity internal error flagged: TX:MSC_PCRE_LIMITS_EXCEEDED"] [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6HJJggNIAADJjNUcAAAAJ"]
[Wed Jun 06 16:14:25 2012] [error] [client 67.243.140.73] ModSecurity: Rule execution error - PCRE limits exceeded (-8): (null). [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6IZJggNIAAA4aWUgAAAAB"]
[Wed Jun 06 16:14:25 2012] [error] [client 67.243.140.73] ModSecurity: Rule execution error - PCRE limits exceeded (-8): (null). [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6IZJggNIAAA4aWUgAAAAB"]
[Wed Jun 06 16:14:25 2012] [error] [client 67.243.140.73] ModSecurity: Rule execution error - PCRE limits exceeded (-8): (null). [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6IZJggNIAAA4aWUgAAAAB"]
[Wed Jun 06 16:14:25 2012] [error] [client 67.243.140.73] ModSecurity: Access denied with code 403 (phase 2). Match of "streq 0" against "TX:MSC_PCRE_LIMITS_EXCEEDED" required. [file "/etc/httpd/conf.d/mod_security.conf"] [line "93"] [msg "ModSecurity internal error flagged: TX:MSC_PCRE_LIMITS_EXCEEDED"] [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6IZJggNIAAA4aWUgAAAAB"]

I can either:

a) track down and exempt the rule that this transaction is triggering.

b) disable mod_security completely and avoid the back-and-forth that could be needed for us to get everything to run.

Basically it boils down to an "executive decision" for if you all want the project to support mod_security and vice-versa. I personally favour option (a), but if that proves to be too expensive/too much hassle for development then, hey, whatever works...

http://cbox.gc.cuny.edu is still not resolving for me, but I'll just wait a day or two to see if it fixes itself.

I had a typo on that one (missing final dot) and it's fixed now.

I had a typo on that one (missing final dot) and it's fixed now.

Sweet, thanks.

I personally favour option (a)

No problem here. I'd rather do it right than do it fast. Let me know if there's anything you need on my end.

No problem here. I'd rather do it right than do it fast. Let me know if there's anything you need on my end.

Cool, let's roll. The upside of this approach is that we gain knowledge of what future "customers" will need to do if they want to run the software with mod_sec.

Let's start with the simplest. I'm trying this on /etc/httpd/conf/httpd.conf:

        SecPcreMatchLimit 15000
        SecPcreMatchLimitRecursion 15000

Wanna give it a try?

Worked like a charm.

Is this just a matter of too many internal redirects? I don't understand what these security params mean.

Worked like a charm.

Nice : )

Is this just a matter of too many internal redirects? I don't understand what these security params mean.

I believe the logic here is that PCRE is the regular expression engine used to search for patterns on loads, and the match limit relates to how many matches it is allowed to match before it stops looking. The rule limits to amount of resources used by the process.

I believe the POST load you were sending contained only lorem text, right? Perhaps there was "too much" of it? I wonder also if there was some hang up with HTML tags or similar.

We can test some variations of this to try and narrow it down. That would be cool.

Worked like a charm.

Nice : )

Is this just a matter of too many internal redirects? I don't understand what these security params mean.

I believe the logic here is that PCRE is the regular expression engine used to search for patterns on loads, and the match limit relates to how many matches it is allowed to match before it stops looking. The rule aims to limits the amount of resources used by the process.

I believe the POST load you were sending contained only lorem text, right? Perhaps there was "too much" of it? I wonder also if there was some hang up with HTML tags or similar.

We can test some variations of this to try and narrow it down. That would be cool.

This is from the mod_security list [http://comments.gmane.org/gmane.comp.apache.mod-security.user/7377]:

The payloads themselves do factor in, however this is really due to the translated php-ids
filters. There are some differences in pre-processing that phpids does to normalize
payloads before actually applying the filters/regexs. One of the normalizations is to
actually look for potential RegEx DoS payloads and then to strip out data (such as
repetitions of characters, etc...). This is critical to do this before the remainder of
the filters are used as the regular expressions are written assuming that these payloads
have been normalized. Since ModSecurity can accurately mimic this with our standard
transformation functions, there are many rules from the phpids filters conf file that will
trigger the new ModSecurity pcre limit error message when they inspect certain payloads.

We are thinking about a few options for correcting this. If upping the limits in the
config doesn't work, then you could review the debug log to see which specific phpids filters
are causing the error and then chose to disable it or skip it under certain circumstances.

Reading this I'm still not clear though if the problem is just a function of the load size or if there are actually a large number of matches or some combination of both.

Ah yes, my first test may have been a big lorem ipsum chunk. This probably won't be an issue in production, since people are generally, you know, writing actual text.

And actually, now that I test again using the same lorem ipsum, I get another 403 Forbidden.

Do you have very fine-grained control over the regexp that mod_sec is using? In this particular case, the only people sending POST requests to this URL are people who are authenticated against WordPress. I wonder if it would be worth it to check for the presence of a particular WP-generated piece of payload, such as '_wpnonce', which is only generated for cookie-authenticated users. If it's present, then loosen the restrictions significantly (I'm assuming that the mod_sec block in question is protection against DDOS, which we're not worried in the case of our logged in users).

And actually, in the particular case of dev.cbox, we are authenticating via .htaccess/.htpasswd too. So I would be OK simply disabling this particular DDOS protection measure for any request that passes this particular auth test. (In the case of production sites, we won't have this luxury - authentication happens in PHP, after mod_sec has done its work.)

I haven't had a chance to take another look at this yet. Haven't forgot it though, hang in there!