https://redmine.gc.cuny.edu/
https://redmine.gc.cuny.edu/favicon.ico
2012-06-04T12:40:55Z
CUNY Graduate Center - Project Tracking System
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7352
2012-06-04T12:40:55Z
local admin
admin@nothing.com
<ul></ul><p>Working on it.</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7354
2012-06-04T14:03:26Z
local admin
admin@nothing.com
<ul><li><strong>Priority name</strong> changed from <i>High</i> to <i>Normal</i></li></ul><p>1. Created local unix user:</p>
<pre>
[root@cdev ~]# useradd cboxdev
[root@cdev ~]# passwd cboxdev
Changing password for user cboxdev.
Enter new UNIX password:
Retype new UNIX password:
passwd: all authentication tokens updated successfully.
</pre>
<p>(will send chosen random password via secure channel)</p>
<p>2. Edited Apache configuration (<code>/etc/httpd/conf/httpd.conf</code>) with new virtual hosts:</p>
<pre>
<VirtualHost *:80>
DocumentRoot /home/cboxdev/dev
ServerName dev.cbox.gc.cuny.edu
CustomLog logs/cbox_log combined
CustomLog logs/cbox_access_log common
</VirtualHost>
<VirtualHost *:80>
DocumentRoot /home/cboxdev/www
ServerName cbox.gc.cuny.edu
CustomLog logs/cbox_log combined
CustomLog logs/cbox_access_log common
</VirtualHost>
</pre>
<p>(Notice that new virtual hosts log to a custom log file, <code>/var/log/cbox_log</code>. Also I've changed the URLs so that staging can just be accessed as cbox.gc)</p>
<p>3. Created web directories and set access permissions:</p>
<pre>
[root@cdev ~]# mkdir /home/cboxdev/www
[root@cdev ~]# mkdir /home/cboxdev/dev
[root@cdev ~]# chown -R cboxdev:apache /home/cboxdev/www/
[root@cdev ~]# chown -R cboxdev:apache /home/cboxdev/dev
[root@cdev ~]# chmod -R 755 /home/cboxdev/dev
[root@cdev ~]# chmod -R +s /home/cboxdev/dev
</pre>
<p>(Notice I've changed the naming slightly, to better conform with current naming standard)</p>
<p>4. Created DBs and granted full access to <code>cboxdev</code> user, with same password (omitted, obviously):</p>
<pre>
[root@cdev ~]# mysql -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 71415
Server version: 5.1.61 Source distribution
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> create database cboxdev;
Query OK, 1 row affected (0.00 sec)
mysql> create database cboxstage;
Query OK, 1 row affected (0.00 sec)
mysql> grant all on cboxdev.* to 'cboxdev'@'%' identified by 'XXXXXXXX';
Query OK, 0 rows affected (0.00 sec)
mysql> grant all on cboxstage.* to 'cboxdev'@'%' identified by 'XXXXXXXX';
Query OK, 0 rows affected (0.01 sec)
mysql> quit
Bye
</pre>
<p>5. Working on DNS records now....</p>
<p>Changing priority to normal, as I prefer to keep "high" and above to issues affecting production. I hope you don't mind : )</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7400
2012-06-06T15:52:54Z
Boone Gorges
boone@gorg.es
<ul></ul><p>Hey André - Thanks a million for all the work you've done on this. It's just about ready to go.</p>
<p>I do need to have the DNS changes in place before I can fully set up the environment, though (BuddyPress needs pretty URLs to work right). Do you have a rough timeline on that? Not a huge rush - though if you think it'll be a while, I'm going to set up a quick sandbox on my own server to share the first round of work with the team, as a stopgap until your solution is in place.</p>
<p>Thanks again!</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7401
2012-06-06T16:01:26Z
local admin
admin@nothing.com
<ul></ul><blockquote>
<p>Not a huge rush - though if you think it'll be a while, I'm going to set up a quick sandbox on my own server to share the first round of work with the team, as a stopgap until your solution is in place.</p>
</blockquote>
<p>In other words: "No rush, but I need it now!" ;-)</p>
<p>Added the records to external DNS:</p>
<pre>
cbox IN CNAME cdev.gc.cuny.edu
dev.cbox IN CNAME cdev.gc.cuny.edu.
</pre>
<p>...and added to internal GC domain controllers as well. Please test and let me know, thanks.</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7402
2012-06-06T16:02:14Z
local admin
admin@nothing.com
<ul></ul><p>Later I'll migrate these records to AWS, but this will do for now.</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7403
2012-06-06T16:12:40Z
Boone Gorges
boone@gorg.es
<ul></ul><blockquote>
<p>In other words: "No rush, but I need it now!" ;-)</p>
</blockquote>
<p>Ha ha. I didn't intend it this way. I just wasn't sure whether there was some bureaucratic reason that would mean a significant delay. It really does just take me a couple minutes to set up a space for temporary use. Thanks for fast-tracking this one :)</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7404
2012-06-06T16:19:36Z
local admin
admin@nothing.com
<ul></ul><p>Just pulling your leg, but I say it with love.</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7405
2012-06-06T17:12:35Z
Boone Gorges
boone@gorg.es
<ul></ul><p>:)</p>
<p>Can't seem to shell in:<br /><pre>
$ ssh cboxdev@cdev.gc.cuny.edu
Connection closed by 146.96.128.210
$ ssh cboxdev@146.96.128.210
Connection closed by 146.96.128.210
</pre></p>
<p>Is cboxdev whitelisted for SSH? Or maybe my IP is blacklisted :)</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7406
2012-06-06T18:11:39Z
local admin
admin@nothing.com
<ul></ul><blockquote>
<p>Is cboxdev whitelisted for SSH?</p>
</blockquote>
<p>This. Fixed now, my bad.</p>
<p>Also I added your public rsa key to <code>authorized_keys</code>.</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7407
2012-06-06T18:21:27Z
Boone Gorges
boone@gorg.es
<ul></ul><p>Whoop, I'm in! Thanks for the key setup too. Almost there.</p>
<p>I put a dummy index.html file in both the www and dev directories, but can't access either through a browser. When I visit <a class="external" href="http://cbox.gc.cuny.edu">http://cbox.gc.cuny.edu</a>, I get nothing - like the DNS isn't set up correctly. (Maybe it's not propagated yet?) When I visit <a class="external" href="http://dev.cdev.gc.cuny.edu">http://dev.cdev.gc.cuny.edu</a>, it resolves, but I get a generic 403 Forbidden Apache page. Would you mind visiting from an external IP to see if you can reproduce?</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7408
2012-06-06T18:24:09Z
local admin
admin@nothing.com
<ul></ul><blockquote>
<p>I put a dummy index.html file in both the www and dev directories, but can't access either through a browser.</p>
</blockquote>
<p>SELinux. I forgot that was enabled there. Should be good now.</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7409
2012-06-06T20:16:00Z
Boone Gorges
boone@gorg.es
<ul></ul><p>Awesome.</p>
<p>dev.cbox.gc.cuny.edu is 99% working. I am having a problem submitting new forum posts - I get a 403 when I send a POST request to <a class="external" href="http://dev.cbox.gc.cuny.edu/groups/foo-group/forum/">http://dev.cbox.gc.cuny.edu/groups/foo-group/forum/</a>. I can dig into the codebase to see if it's on my end, but I thought first I'd check to see whether there's anything on your end that jumps out.</p>
<p><a class="external" href="http://cbox.gc.cuny.edu">http://cbox.gc.cuny.edu</a> is still not resolving for me, but I'll just wait a day or two to see if it fixes itself.</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7410
2012-06-06T22:35:53Z
Matt Gold
mattgold@gmail.com
<ul></ul><p>Thanks to you both!</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7414
2012-06-07T13:17:47Z
local admin
admin@nothing.com
<ul></ul><blockquote>
<p>dev.cbox.gc.cuny.edu is 99% working. I am having a problem submitting new forum posts - I get a 403 when I send a POST request to <a class="external" href="http://dev.cbox.gc.cuny.edu/groups/foo-group/forum/">http://dev.cbox.gc.cuny.edu/groups/foo-group/forum/</a>. I can dig into the codebase to see if it's on my end, but I thought first I'd check to see whether there's anything on your end that jumps out.</p>
</blockquote>
<p>ModSecurity. It thinks your form is a sql injection attack.</p>
<pre>
[Wed Jun 06 16:14:21 2012] [error] [client 67.243.140.73] ModSecurity: Rule execution error - PCRE limits exceeded (-8): (null). [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6HJJggNIAADJjNUcAAAAJ"]
[Wed Jun 06 16:14:21 2012] [error] [client 67.243.140.73] ModSecurity: Rule execution error - PCRE limits exceeded (-8): (null). [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6HJJggNIAADJjNUcAAAAJ"]
[Wed Jun 06 16:14:21 2012] [error] [client 67.243.140.73] ModSecurity: Rule execution error - PCRE limits exceeded (-8): (null). [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6HJJggNIAADJjNUcAAAAJ"]
[Wed Jun 06 16:14:21 2012] [error] [client 67.243.140.73] ModSecurity: Access denied with code 403 (phase 2). Match of "streq 0" against "TX:MSC_PCRE_LIMITS_EXCEEDED" required. [file "/etc/httpd/conf.d/mod_security.conf"] [line "93"] [msg "ModSecurity internal error flagged: TX:MSC_PCRE_LIMITS_EXCEEDED"] [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6HJJggNIAADJjNUcAAAAJ"]
[Wed Jun 06 16:14:25 2012] [error] [client 67.243.140.73] ModSecurity: Rule execution error - PCRE limits exceeded (-8): (null). [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6IZJggNIAAA4aWUgAAAAB"]
[Wed Jun 06 16:14:25 2012] [error] [client 67.243.140.73] ModSecurity: Rule execution error - PCRE limits exceeded (-8): (null). [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6IZJggNIAAA4aWUgAAAAB"]
[Wed Jun 06 16:14:25 2012] [error] [client 67.243.140.73] ModSecurity: Rule execution error - PCRE limits exceeded (-8): (null). [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6IZJggNIAAA4aWUgAAAAB"]
[Wed Jun 06 16:14:25 2012] [error] [client 67.243.140.73] ModSecurity: Access denied with code 403 (phase 2). Match of "streq 0" against "TX:MSC_PCRE_LIMITS_EXCEEDED" required. [file "/etc/httpd/conf.d/mod_security.conf"] [line "93"] [msg "ModSecurity internal error flagged: TX:MSC_PCRE_LIMITS_EXCEEDED"] [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6IZJggNIAAA4aWUgAAAAB"]
</pre>
<p>I can either:</p>
<p>a) track down and exempt the rule that this transaction is triggering.</p>
<p>b) disable mod_security completely and avoid the back-and-forth that could be needed for us to get everything to run.</p>
<p>Basically it boils down to an "executive decision" for if you all want the project to support mod_security and vice-versa. I personally favour option (a), but if that proves to be too expensive/too much hassle for development then, hey, whatever works...</p>
<blockquote>
<p><a class="external" href="http://cbox.gc.cuny.edu">http://cbox.gc.cuny.edu</a> is still not resolving for me, but I'll just wait a day or two to see if it fixes itself.</p>
</blockquote>
<p>I had a typo on that one (missing final dot) and it's fixed now.</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7415
2012-06-07T13:44:12Z
Boone Gorges
boone@gorg.es
<ul></ul><blockquote>
<p>I had a typo on that one (missing final dot) and it's fixed now.</p>
</blockquote>
<p>Sweet, thanks.</p>
<blockquote>
<p>I personally favour option (a)</p>
</blockquote>
<p>No problem here. I'd rather do it right than do it fast. Let me know if there's anything you need on my end.</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7416
2012-06-07T13:48:17Z
local admin
admin@nothing.com
<ul></ul><blockquote>
<p>No problem here. I'd rather do it right than do it fast. Let me know if there's anything you need on my end.</p>
</blockquote>
<p>Cool, let's roll. The upside of this approach is that we gain knowledge of what future "customers" will need to do if they want to run the software with mod_sec.</p>
<p>Let's start with the simplest. I'm trying this on <code>/etc/httpd/conf/httpd.conf</code>:</p>
<pre>
SecPcreMatchLimit 15000
SecPcreMatchLimitRecursion 15000
</pre>
<p>Wanna give it a try?</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7417
2012-06-07T13:53:20Z
Boone Gorges
boone@gorg.es
<ul></ul><p>Worked like a charm.</p>
<p>Is this just a matter of too many internal redirects? I don't understand what these security params mean.</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7418
2012-06-07T14:14:21Z
local admin
admin@nothing.com
<ul></ul><blockquote>
<p>Worked like a charm.</p>
</blockquote>
<p>Nice : )</p>
<blockquote>
<p>Is this just a matter of too many internal redirects? I don't understand what these security params mean.</p>
</blockquote>
<p>I believe the logic here is that PCRE is the regular expression engine used to search for patterns on loads, and the match limit relates to how many matches it is allowed to match before it stops looking. The rule limits to amount of resources used by the process.</p>
<p>I believe the <code>POST</code> load you were sending contained only lorem text, right? Perhaps there was "too much" of it? I wonder also if there was some hang up with <code>HTML</code> tags or similar.</p>
<p>We can test some variations of this to try and narrow it down. That would be cool.</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7419
2012-06-07T14:15:17Z
local admin
admin@nothing.com
<ul></ul><blockquote>
<p>Worked like a charm.</p>
</blockquote>
<p>Nice : )</p>
<blockquote>
<p>Is this just a matter of too many internal redirects? I don't understand what these security params mean.</p>
</blockquote>
<p>I believe the logic here is that PCRE is the regular expression engine used to search for patterns on loads, and the match limit relates to how many matches it is allowed to match before it stops looking. The rule aims to limits the amount of resources used by the process.</p>
<p>I believe the <code>POST</code> load you were sending contained only lorem text, right? Perhaps there was "too much" of it? I wonder also if there was some hang up with <code>HTML</code> tags or similar.</p>
<p>We can test some variations of this to try and narrow it down. That would be cool.</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7420
2012-06-07T14:23:51Z
local admin
admin@nothing.com
<ul></ul><p>This is from the mod_security list [<a class="external" href="http://comments.gmane.org/gmane.comp.apache.mod-security.user/7377">http://comments.gmane.org/gmane.comp.apache.mod-security.user/7377</a>]:</p>
<blockquote>
<p>The payloads themselves do factor in, however this is really due to the translated php-ids <br />filters. There are some differences in pre-processing that phpids does to normalize <br />payloads before actually applying the filters/regexs. One of the normalizations is to <br />actually look for potential RegEx DoS payloads and then to strip out data (such as <br />repetitions of characters, etc...). This is critical to do this <strong>before</strong> the remainder of <br />the filters are used as the regular expressions are written assuming that these payloads <br />have been normalized. Since ModSecurity can accurately mimic this with our standard <br />transformation functions, there are many rules from the phpids filters conf file that will <br />trigger the new ModSecurity pcre limit error message when they inspect certain payloads.</p>
</blockquote>
<blockquote>
<p>We are thinking about a few options for correcting this. If upping the limits in the <br />config doesn't work, then you could review the debug log to see which specific phpids filters <br />are causing the error and then chose to disable it or skip it under certain circumstances.</p>
</blockquote>
<p>Reading this I'm still not clear though if the problem is just a function of the load size or if there are actually a large number of matches or some combination of both.</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7423
2012-06-07T14:40:28Z
Boone Gorges
boone@gorg.es
<ul></ul><p>Ah yes, my first test may have been a big lorem ipsum chunk. This probably won't be an issue in production, since people are generally, you know, writing actual text.</p>
<p>And actually, now that I test again using the same lorem ipsum, I get another 403 Forbidden.</p>
<p>Do you have very fine-grained control over the regexp that mod_sec is using? In this particular case, the only people sending POST requests to this URL are people who are authenticated against WordPress. I wonder if it would be worth it to check for the presence of a particular WP-generated piece of payload, such as '_wpnonce', which is only generated for cookie-authenticated users. If it's present, then loosen the restrictions significantly (I'm assuming that the mod_sec block in question is protection against DDOS, which we're not worried in the case of our logged in users).</p>
<p>And actually, in the particular case of dev.cbox, we are authenticating via .htaccess/.htpasswd too. So I would be OK simply disabling this particular DDOS protection measure for any request that passes this particular auth test. (In the case of production sites, we won't have this luxury - authentication happens in PHP, after mod_sec has done its work.)</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7483
2012-06-12T13:51:01Z
local admin
admin@nothing.com
<ul></ul><p>I haven't had a chance to take another look at this yet. Haven't forgot it though, hang in there!</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7852
2012-07-11T15:47:07Z
local admin
admin@nothing.com
<ul><li><strong>Status</strong> changed from <i>Assigned</i> to <i>Reporter Feedback</i></li></ul><p>Disabled mod_security for this vhost during development phase. we can always re-enable this vefore writing up documentation and test what exemptions need to be arranged.</p>
CUNY Academic Commons - Feature #1923: New staging environment(s) for Commons In A Box
https://redmine.gc.cuny.edu/issues/1923?journal_id=7853
2012-07-11T15:51:15Z
Boone Gorges
boone@gorg.es
<ul><li><strong>Status</strong> changed from <i>Reporter Feedback</i> to <i>Resolved</i></li></ul><p>Thanks a million, André. I'm closing this as Resolved. If I have specific other issues, I'll open new tickets.</p>