CUNY Academic Commons

Thanks. FWIW, I can tell you that Zach used Chef for Debates in DH, so you could check in with him on that. And my DH students used Ansible for DH Box (http://twitter.com/dh_box). Here's what they wrote about that:

Stephen then turned to a set of tools explicitly designed for mass provisioning of cloud servers. He evaluated several different possibilities: Chef, Puppet, Salt, and Ansible. Each is an ‘IT automation’ library/toolset. Eventually Stephen chose Ansible as his solution, for both objective and subjective reasons: Ansible allows for SSH-ing into a client computer (a DH Box instance in this case) and watching the provisioning as it happens, live. It is comparatively easy to see when a build fails, and to see what failed in the install script. And Ansible was written in Python, which Stephen felt comfortable with.

Thanks, Matt. The use case of DH Box is pretty different from ours, so we don't need that particular set of features, but it's a good data point.

I would actually enjoy working on this. Boone and I have had a few conversations about the intricacies of our local setups and how an automated process would remove some of the initial pain in getting a development env up and running.

Not sure chef/puppet style tools are the right way to go since they are designed to provision remote cloud based servers - this ticket is talking about provisioning local environments. But I will research them, I'm thinking bash might be the simplest solution. Although using tools like Vagrant or Docker might lend themselves to working more efficiently with a chef/puppet style provisioning setup.

Will report back when I know a little more.

Great, Dom. Glad you're interested.

Bit more background on my motivation here: there's a lot to be gained from doing development in a VM, which will allow for greater sandboxing, better compatibility with the production server, etc. And using local VMs is not all that different from provisioning remote servers.

See https://github.com/Varying-Vagrant-Vagrants/VVV for an example of a project that uses Vagrant for provisioning local WP dev environments. They use Bash for the provisioning process itself, but not for a principled reason. Here's another similar project using Salty https://github.com/humanmade/Salty-WordPress

I agree, a local VM setup would be ideal. I'll check out those links as soon as I can.

I mentioned Docker in my last reply only because I've heard and read that developers who used to use Vagrant find Docker to be a better alternative. I do not know if this is true - I will read up on both (Docker and Vagrant).

I have experimented with Vagrant in the past. I've built custom VM OS images using both Ubuntu LTS and Arch linux. I know nothing about RHEL, I think using Ubuntu wouldn't pose a problem, but if anyone monitoring this ticket wants me to investigate another FOOS OS please chime in.

CentOS is based on RHEL, and is no-cost, so it's probably the best place to start in terms of OS. A little Github poking shows that there might be plenty of starting places: https://github.com/search?q=centos+vagrant&ref=cmdform

Boone,

I've started researching this and looking into the links you provided above. Would you let me know when you have a few minutes to talk? I'd just like to make sure that you and I are on the same page regarding this project since this is not just a feature that is being added to the commons in general; this is going to be a very interactive project that all of the commons developers are going to use. I don't want to start going off in one direction while you are thinking in another. The call (or Skype chat?) doesn't have to be long, just a quick couple of minutes to make sure I'm understanding and researching correctly to help build what you have in mind.

Dom - I've got a couple online meetings this afternoon, but you can ping me on Skype to see if you can catch me with a few minutes of downtime. Otherwise let's do something Friday.

sounds good, I've got Skype open now and will leave it open all day. I'm going to try and split today between some commons and littlstar stuff so will be in front of my laptop almost continually.

Am I one of your Skype contacts? You are in my list. If I'm not in yours let me know and I'll send a contact request.

This sounds like a great project. One other thing you might want to check out is Boxen, by the folks at Github. They wrote Boxen to help set up development environments for people. It uses Puppet. I think the downside is it's Mac only right now but it is probably at least worth a look.
https://boxen.github.com/
https://github.com/blog/1345-introducing-boxen

Here is someone who did something similar to Boxen with and Ansible he calls Battleschool (works on Linux too)
http://spencer.gibb.us/blog/2014/02/03/introducing-battleschool/
https://github.com/spencergibb/battleschool

and another similar idea with Vagrant and Ansible
http://blog.jcalonso.com/2013/11/01/your-development-environment-with-vagrant-and-ansible/

I don't really know much about this stuff but I'm interested because I want the IT folks at BMCC to do something similar to set up our lab computers and servers for the students, so I'll be following along what you come up with. I did also talk with the DH Box folks and based on that conversation and some blog posts I've ready Ansible does seem to be more approachable and easier to get into than Chef or Puppet.

Dom - I've pinged you on Skype.

Chris - Thanks for the links - this is helpful background. We'll do our best to document changes to our workflow so that others can benefit.

Chris, thanks for the links and thoughts.

My plan right now is to keep this whole process as simple and plain vanilla as possible. To that end I envision a first attempt simply using a custom VirtualBox Vagrant "Base Box" built on CentOS 6.5 per Boone's suggestion to stay as close to the RHEL environment used on the GC's production servers. Any additional custom config will probably come right out of Bash so no other developer (or non-developer) need learn any crazy new provisioning tool. The whole point of this thing is to make development easier across the group.

My belief is always start simple and add complexity only when needed. If the building of this virtualized container requires advanced provisioning i will probably look at Ansible first. Earlier comments plus your links seem to shed a favorable light on it. I've tried Puppet and Chef and always found them to be more work to configure than writing a few lines of Bash script.

I'll keep in touch as to my progress.

Hey Boone,

Just wanted to let you know that I'll be working from the GC tomorrow. I've got some local stuff going in a topic branch that I'll try and push up to our repo at some point today/tonight. If your schedule permits maybe we can discuss on the dev call or afterwards on Skype.

Thanks, Dom. Let's make sure to touch on this during tomorrow's call, and then we can plan to stay on at 12:30 to talk any additional details.

Boone (and anyone else interested in this issue),

I've push the topic branch with the start to the Vagrant work. Here's a summary:

1.) base CentOS 6.5 64-bit box created from clean install. Currently hosted in my public Dropbox folder for now.
2.) basic Vagrantfile generated to use affore-mentioned box
3.) vagrant-config folder to store custom configurations and a basic Bash provisioning script

As I mentioned on the call I am currently working out networking and config files so I can keep adding more to the provisioning script. In its current state it's still pretty useless but I wanted to get you something to look at incase you've got feedback.

If you want to test the current setup in your own env install Virtualbox and Vagrant then run these commands:

git fetch
git checkout -b vagrant origin/vagrant
vagrant up

That's about all you can do right now. You can run vagrant ssh after provisioning completes to ssh in. Here are the user details:

system user:
username: vagrant
password: vagrant

System user has passwordless sudo access and you can su to the superuser using the password: vagrant

I rescheduled my GC development to tomorrow so I'll be there for 5-6 hours tomorrow working on this and any other issues I can get to.

Awesome, Dom. This looks like a great start.

Let me know if you need any specific info about how the production server is configured as you continue to work on this.

Will do, I'm sure I'll have questions/updates tomorrow.

Boone,

I've pushed up some new code to the vagrant topic branch

If you'd like to test ensure you've installed VirtualBox/Vagrant which I linked to in comment 15 above and then pull the latest code from the vagrant branch.

I was having trouble with the networking in the base Cent box I created so for now I am just using an official "clean and minimal" CentOS 6.5 64-bit from the official Vagrant box repositories. I don't think this is a problem for now since eventually we want to install the whole thing from scratch during provisioning anyways.

All you need to do is run vagrant up from the root of our repo and it will go get the base box, launch it and then provision it. Right now the provisioning just installs the LAMP components and I've set Apache's DocumentRoot to point to the index.php in the vagrant-config directory I've added to the repo. Once the box is up and running you can visit 192.168.167.9 in your browser and you should see output from phpinfo().

It's still very basic but coming along. I decided to stop here because it's time to configure MySql and that's going to be a decent hurdle to get over. If you'd like to set aside some time to go over the current state of the project let me know and we'll pick a time to chat. If you're not ready to test it yet, then I will update here once I've got new info regarding MySql. As soon as the DB is properly setup and installed it shouldn't be too hard to actually load our site; probably just a few config tweaks.

Awesome, thanks. I'll give this a thorough testing in the next few days, but it looks like it's headed in the right direction.