Feature #1923
closedNew staging environment(s) for Commons In A Box
Added by Boone Gorges almost 12 years ago. Updated almost 12 years ago.
0%
Description
Hi André,
As we start major development for the Commons In A Box software, we're going to need a common space or two where we can do collective bug fixing and acceptance testing with the non-technical members of the team. For that purpose, I was hoping that you could set up some new vhosts for us, perhaps on the same server that currently hosts cdev.
Here's my suggested setup. Of course, I'll defer to you on details.
1) A new ssh-enabled user, cboxdev, with a home folder at /home/cboxdev/
2) Two new vhosts: cboxdev.gc.cuny.edu (pointing at /home/cboxdev/public_html/cboxdev/) and cboxstage.gc.cuny.edu (pointing at /home/cboxdev/public_html/cboxstage)
3) Two new empty databases, cboxdev and cboxstage, as well as one new MySQL user with all privileges on both new DBs
These are just suggestions. The details don't matter that much. If you have a different preferred directory structure, or if it's easier for you to assign different kinds of URLs (eg cboxstage.cdev.gc.cuny.edu) that's totally fine.
I'm happy to discuss any of this in more detail if you have questions or suggestions for improvement. Thanks in advance for your help!
Updated by local admin almost 12 years ago
- Priority name changed from High to Normal
1. Created local unix user:
[root@cdev ~]# useradd cboxdev [root@cdev ~]# passwd cboxdev Changing password for user cboxdev. Enter new UNIX password: Retype new UNIX password: passwd: all authentication tokens updated successfully.
(will send chosen random password via secure channel)
2. Edited Apache configuration (/etc/httpd/conf/httpd.conf) with new virtual hosts:
<VirtualHost *:80>
DocumentRoot /home/cboxdev/dev
ServerName dev.cbox.gc.cuny.edu
CustomLog logs/cbox_log combined
CustomLog logs/cbox_access_log common
</VirtualHost>
<VirtualHost *:80>
DocumentRoot /home/cboxdev/www
ServerName cbox.gc.cuny.edu
CustomLog logs/cbox_log combined
CustomLog logs/cbox_access_log common
</VirtualHost>
(Notice that new virtual hosts log to a custom log file, /var/log/cbox_log. Also I've changed the URLs so that staging can just be accessed as cbox.gc)
3. Created web directories and set access permissions:
[root@cdev ~]# mkdir /home/cboxdev/www [root@cdev ~]# mkdir /home/cboxdev/dev [root@cdev ~]# chown -R cboxdev:apache /home/cboxdev/www/ [root@cdev ~]# chown -R cboxdev:apache /home/cboxdev/dev [root@cdev ~]# chmod -R 755 /home/cboxdev/dev [root@cdev ~]# chmod -R +s /home/cboxdev/dev
(Notice I've changed the naming slightly, to better conform with current naming standard)
4. Created DBs and granted full access to cboxdev user, with same password (omitted, obviously):
[root@cdev ~]# mysql -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 71415 Server version: 5.1.61 Source distribution Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> create database cboxdev; Query OK, 1 row affected (0.00 sec) mysql> create database cboxstage; Query OK, 1 row affected (0.00 sec) mysql> grant all on cboxdev.* to 'cboxdev'@'%' identified by 'XXXXXXXX'; Query OK, 0 rows affected (0.00 sec) mysql> grant all on cboxstage.* to 'cboxdev'@'%' identified by 'XXXXXXXX'; Query OK, 0 rows affected (0.01 sec) mysql> quit Bye
5. Working on DNS records now....
Changing priority to normal, as I prefer to keep "high" and above to issues affecting production. I hope you don't mind : )
Updated by Boone Gorges almost 12 years ago
Hey André - Thanks a million for all the work you've done on this. It's just about ready to go.
I do need to have the DNS changes in place before I can fully set up the environment, though (BuddyPress needs pretty URLs to work right). Do you have a rough timeline on that? Not a huge rush - though if you think it'll be a while, I'm going to set up a quick sandbox on my own server to share the first round of work with the team, as a stopgap until your solution is in place.
Thanks again!
Updated by local admin almost 12 years ago
Not a huge rush - though if you think it'll be a while, I'm going to set up a quick sandbox on my own server to share the first round of work with the team, as a stopgap until your solution is in place.
In other words: "No rush, but I need it now!" ;-)
Added the records to external DNS:
cbox IN CNAME cdev.gc.cuny.edu dev.cbox IN CNAME cdev.gc.cuny.edu.
...and added to internal GC domain controllers as well. Please test and let me know, thanks.
Updated by local admin almost 12 years ago
Later I'll migrate these records to AWS, but this will do for now.
Updated by Boone Gorges almost 12 years ago
In other words: "No rush, but I need it now!" ;-)
Ha ha. I didn't intend it this way. I just wasn't sure whether there was some bureaucratic reason that would mean a significant delay. It really does just take me a couple minutes to set up a space for temporary use. Thanks for fast-tracking this one :)
Updated by local admin almost 12 years ago
Just pulling your leg, but I say it with love.
Updated by Boone Gorges almost 12 years ago
:)
Can't seem to shell in:
$ ssh cboxdev@cdev.gc.cuny.edu Connection closed by 146.96.128.210 $ ssh cboxdev@146.96.128.210 Connection closed by 146.96.128.210
Is cboxdev whitelisted for SSH? Or maybe my IP is blacklisted :)
Updated by local admin almost 12 years ago
Is cboxdev whitelisted for SSH?
This. Fixed now, my bad.
Also I added your public rsa key to authorized_keys.
Updated by Boone Gorges almost 12 years ago
Whoop, I'm in! Thanks for the key setup too. Almost there.
I put a dummy index.html file in both the www and dev directories, but can't access either through a browser. When I visit http://cbox.gc.cuny.edu, I get nothing - like the DNS isn't set up correctly. (Maybe it's not propagated yet?) When I visit http://dev.cdev.gc.cuny.edu, it resolves, but I get a generic 403 Forbidden Apache page. Would you mind visiting from an external IP to see if you can reproduce?
Updated by local admin almost 12 years ago
I put a dummy index.html file in both the www and dev directories, but can't access either through a browser.
SELinux. I forgot that was enabled there. Should be good now.
Updated by Boone Gorges almost 12 years ago
Awesome.
dev.cbox.gc.cuny.edu is 99% working. I am having a problem submitting new forum posts - I get a 403 when I send a POST request to http://dev.cbox.gc.cuny.edu/groups/foo-group/forum/. I can dig into the codebase to see if it's on my end, but I thought first I'd check to see whether there's anything on your end that jumps out.
http://cbox.gc.cuny.edu is still not resolving for me, but I'll just wait a day or two to see if it fixes itself.
Updated by local admin almost 12 years ago
dev.cbox.gc.cuny.edu is 99% working. I am having a problem submitting new forum posts - I get a 403 when I send a POST request to http://dev.cbox.gc.cuny.edu/groups/foo-group/forum/. I can dig into the codebase to see if it's on my end, but I thought first I'd check to see whether there's anything on your end that jumps out.
ModSecurity. It thinks your form is a sql injection attack.
[Wed Jun 06 16:14:21 2012] [error] [client 67.243.140.73] ModSecurity: Rule execution error - PCRE limits exceeded (-8): (null). [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6HJJggNIAADJjNUcAAAAJ"] [Wed Jun 06 16:14:21 2012] [error] [client 67.243.140.73] ModSecurity: Rule execution error - PCRE limits exceeded (-8): (null). [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6HJJggNIAADJjNUcAAAAJ"] [Wed Jun 06 16:14:21 2012] [error] [client 67.243.140.73] ModSecurity: Rule execution error - PCRE limits exceeded (-8): (null). [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6HJJggNIAADJjNUcAAAAJ"] [Wed Jun 06 16:14:21 2012] [error] [client 67.243.140.73] ModSecurity: Access denied with code 403 (phase 2). Match of "streq 0" against "TX:MSC_PCRE_LIMITS_EXCEEDED" required. [file "/etc/httpd/conf.d/mod_security.conf"] [line "93"] [msg "ModSecurity internal error flagged: TX:MSC_PCRE_LIMITS_EXCEEDED"] [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6HJJggNIAADJjNUcAAAAJ"] [Wed Jun 06 16:14:25 2012] [error] [client 67.243.140.73] ModSecurity: Rule execution error - PCRE limits exceeded (-8): (null). [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6IZJggNIAAA4aWUgAAAAB"] [Wed Jun 06 16:14:25 2012] [error] [client 67.243.140.73] ModSecurity: Rule execution error - PCRE limits exceeded (-8): (null). [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6IZJggNIAAA4aWUgAAAAB"] [Wed Jun 06 16:14:25 2012] [error] [client 67.243.140.73] ModSecurity: Rule execution error - PCRE limits exceeded (-8): (null). [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6IZJggNIAAA4aWUgAAAAB"] [Wed Jun 06 16:14:25 2012] [error] [client 67.243.140.73] ModSecurity: Access denied with code 403 (phase 2). Match of "streq 0" against "TX:MSC_PCRE_LIMITS_EXCEEDED" required. [file "/etc/httpd/conf.d/mod_security.conf"] [line "93"] [msg "ModSecurity internal error flagged: TX:MSC_PCRE_LIMITS_EXCEEDED"] [hostname "dev.cbox.gc.cuny.edu"] [uri "/groups/foo-group/forum/"] [unique_id "T8@6IZJggNIAAA4aWUgAAAAB"]
I can either:
a) track down and exempt the rule that this transaction is triggering.
b) disable mod_security completely and avoid the back-and-forth that could be needed for us to get everything to run.
Basically it boils down to an "executive decision" for if you all want the project to support mod_security and vice-versa. I personally favour option (a), but if that proves to be too expensive/too much hassle for development then, hey, whatever works...
http://cbox.gc.cuny.edu is still not resolving for me, but I'll just wait a day or two to see if it fixes itself.
I had a typo on that one (missing final dot) and it's fixed now.
Updated by Boone Gorges almost 12 years ago
I had a typo on that one (missing final dot) and it's fixed now.
Sweet, thanks.
I personally favour option (a)
No problem here. I'd rather do it right than do it fast. Let me know if there's anything you need on my end.
Updated by local admin almost 12 years ago
No problem here. I'd rather do it right than do it fast. Let me know if there's anything you need on my end.
Cool, let's roll. The upside of this approach is that we gain knowledge of what future "customers" will need to do if they want to run the software with mod_sec.
Let's start with the simplest. I'm trying this on /etc/httpd/conf/httpd.conf:
SecPcreMatchLimit 15000
SecPcreMatchLimitRecursion 15000
Wanna give it a try?
Updated by Boone Gorges almost 12 years ago
Worked like a charm.
Is this just a matter of too many internal redirects? I don't understand what these security params mean.
Updated by local admin almost 12 years ago
Worked like a charm.
Nice : )
Is this just a matter of too many internal redirects? I don't understand what these security params mean.
I believe the logic here is that PCRE is the regular expression engine used to search for patterns on loads, and the match limit relates to how many matches it is allowed to match before it stops looking. The rule limits to amount of resources used by the process.
I believe the POST load you were sending contained only lorem text, right? Perhaps there was "too much" of it? I wonder also if there was some hang up with HTML tags or similar.
We can test some variations of this to try and narrow it down. That would be cool.
Updated by local admin almost 12 years ago
Worked like a charm.
Nice : )
Is this just a matter of too many internal redirects? I don't understand what these security params mean.
I believe the logic here is that PCRE is the regular expression engine used to search for patterns on loads, and the match limit relates to how many matches it is allowed to match before it stops looking. The rule aims to limits the amount of resources used by the process.
I believe the POST load you were sending contained only lorem text, right? Perhaps there was "too much" of it? I wonder also if there was some hang up with HTML tags or similar.
We can test some variations of this to try and narrow it down. That would be cool.
Updated by local admin almost 12 years ago
This is from the mod_security list [http://comments.gmane.org/gmane.comp.apache.mod-security.user/7377]:
The payloads themselves do factor in, however this is really due to the translated php-ids
filters. There are some differences in pre-processing that phpids does to normalize
payloads before actually applying the filters/regexs. One of the normalizations is to
actually look for potential RegEx DoS payloads and then to strip out data (such as
repetitions of characters, etc...). This is critical to do this before the remainder of
the filters are used as the regular expressions are written assuming that these payloads
have been normalized. Since ModSecurity can accurately mimic this with our standard
transformation functions, there are many rules from the phpids filters conf file that will
trigger the new ModSecurity pcre limit error message when they inspect certain payloads.
We are thinking about a few options for correcting this. If upping the limits in the
config doesn't work, then you could review the debug log to see which specific phpids filters
are causing the error and then chose to disable it or skip it under certain circumstances.
Reading this I'm still not clear though if the problem is just a function of the load size or if there are actually a large number of matches or some combination of both.
Updated by Boone Gorges almost 12 years ago
Ah yes, my first test may have been a big lorem ipsum chunk. This probably won't be an issue in production, since people are generally, you know, writing actual text.
And actually, now that I test again using the same lorem ipsum, I get another 403 Forbidden.
Do you have very fine-grained control over the regexp that mod_sec is using? In this particular case, the only people sending POST requests to this URL are people who are authenticated against WordPress. I wonder if it would be worth it to check for the presence of a particular WP-generated piece of payload, such as '_wpnonce', which is only generated for cookie-authenticated users. If it's present, then loosen the restrictions significantly (I'm assuming that the mod_sec block in question is protection against DDOS, which we're not worried in the case of our logged in users).
And actually, in the particular case of dev.cbox, we are authenticating via .htaccess/.htpasswd too. So I would be OK simply disabling this particular DDOS protection measure for any request that passes this particular auth test. (In the case of production sites, we won't have this luxury - authentication happens in PHP, after mod_sec has done its work.)
Updated by local admin almost 12 years ago
I haven't had a chance to take another look at this yet. Haven't forgot it though, hang in there!
Updated by local admin almost 12 years ago
- Status changed from Assigned to Reporter Feedback
Disabled mod_security for this vhost during development phase. we can always re-enable this vefore writing up documentation and test what exemptions need to be arranged.
Updated by Boone Gorges almost 12 years ago
- Status changed from Reporter Feedback to Resolved
Thanks a million, André. I'm closing this as Resolved. If I have specific other issues, I'll open new tickets.