CUNY Academic Commons

Some resources: http://codex.wordpress.org/Blog_by_Email

Still no update on the core plugin, though I know it is planned.

Ron, do you know the status of the core post-by-email plugin? I had a link to the svn repo at one point but I lost it. That could probably do a lot of the heavy lifting. The hash stuff doesn't seem too hard to write on the BP end.

If this is tagged 1.2 then 1.2 will be pushed back indefinitely. I don't have a problem making this a top priority, but making it work right could take months.

Is 1.3 okay? I'd just like to keep it away from "Future Release," which has a ring of "some day" to me. We can always punt it to a later version if we need to.

Fine by me.

Sorry, I posted this in the wrong ticket:

I don't think the post by email core plugin had much traction. The codex page says it needs updating, but that may be because some filters and constants were added in 3.0.

http://codex.wordpress.org/Post_to_your_blog_using_email

Ron, I think that codex page is referring to a different feature from the core plugin. I found the svn link btw https://plugins.svn.wordpress.org/post-by-email/branches/alpha/

I'm going to start messing with this in another branch if anyone wants to play along: https://github.com/castiron/cac/tree/post-by-mail

D'oh, I just installed the plugin and I'm wrong - it does work like the codex page above describes. We might be able to use the parser, but we'll have to get the mail listener from somewhere else.

Interesting problem that will have to be kept in mind when setting up the mail server to pick up the replies: http://openacs.org/forums/message-view?message_id=539009

With google mail you can use addresses like rrrennick+anything@gmail.com and it lands in the rrennick@gmail.com mailbox. If the mail server could be set up to do that, then the reply to address could be a unique hash (something+hash@...) that validated the post being responded to & user making the response. Doing that would virtually eliminate phishing/spoofing of email addresses. Each outbound email would have to be hashed but that wouldn't be too much coding.

Boone, since we have the Road Map set up, what do you think of assigning tickets like this to milestones that correspond to versions on that road map? I bring this up because "Future release" seems like a kind of hazy purgatory for tickets and ideas that we might or might not get to; having these tickets assigned instead, even provisionally, to future milestones would make them more visible and would help us plan the project. And, of course, we could always bump things up or down as needed.

What do you think?

If it would make you feel better to do so, that's fine. Otherwise I will just pick through the Future Release tickets at the beginning of each cycle to make sure that the items on the public roadmap are represented in the milestone. It is all the same to me.

I was going to start work on a project similar to this for group forums only, but that project fell through.

Was going to use an approach similar to Ron's that would use an IMAP inbox, which supported address tags where we could extrapolate variables from the email address.

eg. forum+g3t1@example.com - would grab group ID 3 and reply to forum topic 1.

I think IMAP is the easiest way to go, but if we want to support POP3, a good plugin to reference is Postie

Wow --- awesome!

As Ray comes back on board, this is going to be one of the first things he tackles.

Ray, I know you have to make some modifications in order to ensure compatibility with BP 1.5. Once you have done so, I think the best way forward will be to do some extensive testing using the cdev environment. Our cdev has been overhauled since you last worked on the project - it's now a separate box, totally identical to the production environment - so it will be ideal for testing things like cron, etc. When the time comes for you to start working on it in the cdev environment, let's talk - I'll have to get you up to speed about the new cdev.

Excellent. Really excited about this.

I'm going to spend some time this week to try and get this working under BP 1.5.

Hi Ray. I have had a chance to look this over in a couple of different environments. Overall, it looks really great. Feels like magic.

I have a couple of questions.

- I've been using a test Gmail account for this purpose. Ideally, we'd use a local account. I've added André as a watcher on this ticket so that he can chime in with some thoughts. André: We need an IMAP inbox that supports "plus addressing", so that the following addresses all go to the same place: commons@foo.com, commons+bar@foo.com, commons+baz@foo.com, etc

- Ray, what do you think about cropping the last line above the pointer "--- Reply ABOVE THIS LINE to add a comment ---". On many email clients, this will be a catch-all for cutting off the "On Thursday, March 31, Boone wrote:". And even on clients where the format is a bit different, it's like that there will at least be a blank line above this line. What do you think? I know that we can accomplish this with a plugin, but I'm thinking that for the vast majority of mail clients, it'll be good enough.

- Ray, can I ask you to go ahead and set it up on cdev so that the team can start to play with it a bit? We can use a dummy Gmail account until we have a more permanent solution. (That'll also give us a bigger sample of email clients.)

Exciting.

André: We need an IMAP inbox that supports "plus addressing", so that the following addresses all go to the same place: commons@foo.com, commons+bar@foo.com, commons+baz@foo.com, etc

For what address? commons@gc.cuny.edu? For all users?

For what address? commons@gc.cuny.edu? For all users?

Not for all users - we just need a single inbox. As for what address: Probably not commons@gc.cuny.edu, which is currently used for support request (we should have an inbox just for this purpose). As for what the address is, I guess I'm hoping you can weigh in with your best judgment. If the current gc.cuny.edu setup supports such a scheme, and if the GC mail folks won't mind that we keep a persistent IMAP connection and receive potentially hundreds of emails a day, then something like commonsreply@gc.cuny.edu would probably be fine. If not, maybe you could give us a sense of how hard it would be to have a single inbox on the commons.gc.cuny.edu domain for this purpose.

maybe you could give us a sense of how hard it would be to have a single inbox on the commons.gc.cuny.edu domain for this purpose.

Does it actually have to go into (and potentially create new) named folders/subfolders or is just delivering to the inbox enough?

I think just delivering it to the inbox, though we would probably need some sort of cleanup routine to make sure that the inbox doesn't fill up. Ray can probably say more about the necessary IMAP mechanics.

I think just delivering it to the inbox, though we would probably need some sort of cleanup routine to make sure that the inbox doesn't fill up. Ray can probably say more about the necessary IMAP mechanics.

Ok, so in this case it would require a small change to postfix, which would be configured to deliver messages addressed to both, say, commonsreply+SOMETHING@commons.gc.cuny.edu and commonsreply+ANYTHING@commons.gc.cuny.edu to the mailbox of commonsreply@commons.gc.cuny.edu.

But we would still have to run dovecot on the server to allow IMAP access, and I guess run some anti-spam filter as well.

It's not a light order, but I look forward to learning more about the scheme before making a recommendation on the best path forward.

Boone Gorges wrote:

- I've been using a test Gmail account for this purpose. Ideally, we'd use a local account. I've added André as a watcher on this ticket so that he can chime in with some thoughts. André: We need an IMAP inbox that supports "plus addressing", so that the following addresses all go to the same place: commons@foo.com, commons+bar@foo.com, commons+baz@foo.com, etc

As an alternative, I know that CUNY has the cuny.is domain. If we're comfortable with the idea of using cuny.is and Google Apps, perhaps we can put Google Apps Mail on there and let GMail handle IMAP instead?

- Ray, what do you think about cropping the last line above the pointer "--- Reply ABOVE THIS LINE to add a comment ---". On many email clients, this will be a catch-all for cutting off the "On Thursday, March 31, Boone wrote:". And even on clients where the format is a bit different, it's like that there will at least be a blank line above this line. What do you think? I know that we can accomplish this with a plugin, but I'm thinking that for the vast majority of mail clients, it'll be good enough.

In my opinion, I would refrain against cropping the last line; I used Basecamp as a model and they don't do this. (Test on Basecamp and see.) I guess it depends on how we educate the users on CAC.

I do like stripping any trailing blank lines though; I'll test this out and implement it.

- Ray, can I ask you to go ahead and set it up on cdev so that the team can start to play with it a bit? We can use a dummy Gmail account until we have a more permanent solution. (That'll also give us a bigger sample of email clients.)

Do you just want me to add it to the 1.3.x branch or some other branch, Boone?

As an alternative, I know that CUNY has the cuny.is domain. If we're comfortable with the idea of using cuny.is and Google Apps, perhaps we can put Google Apps Mail on there and let GMail handle IMAP instead?

If we decide to go that route we can actually still use @commons.gc.cuny.edu, by changing the MX records for the subdomain. I don't think it's being use for anything else currently.

Do you know, at this juncture, if the creation/routing of IMAP subfolders is needed for this use case, or can we just deliver all the messages to the same mailbox?

If we decide to go that route we can actually still use @commons.gc.cuny.edu, by changing the MX records for the subdomain. I don't think it's being use for anything else currently.

Cool! Would there be an issue with migrating existing email accounts on @commons.gc.cuny.edu though (if there are any)?

Do you know, at this juncture, if the creation/routing of IMAP subfolders is needed for this use case, or can we just deliver all the messages to the same mailbox?

If we're talking about implementing IMAP on the existing CAC email server, then we can just deliver all messages to the same mailbox. If we want to leave this to Gmail, then we don't have to do anything. Let me know if I'm not being clear!

In the plugin, I don't do anything with IMAP subfolders, I just check the main inbox folder and delete / expunge the email when done.

Do you just want me to add it to the 1.3.x branch or some other branch, Boone?

The feature is slated for the 1.4 release (ie the next feature release), so please add it to the master branch.

Reply By Email plugin is added to the master branch:
https://github.com/castiron/cac/commit/206454ee90b5bf39cac422606dd0f1e267e0405f

Let's start testing! :)

Boone, do you have a dummy Gmail account to test?

Nice! By the way, I'm running the imap request through the "channels" and will "circle back" about it shortly ...

By the way, I'm running the imap request through the "channels" and will "circle back" about it shortly ...

Cool, thanks André!

Boone, do you have a dummy Gmail account to test?

I have some laying around, but they're used for other stuff that might throw the system off. Could you set one up? And you should have the creds to shell into cdev - they're on Basecamp. Webroot is /var/www/html/. cdev is already running the master branch (I think) so you should just be able to pull the latest, and use SFTP to do any live debugging. Let us know when you've verified it working on cdev, and I can help to write up a testing procedure. Thanks, Ray!

I've activated the plugin on cdev, however the IMAP extension needs to be installed for PHP:
http://www.php.net/manual/en/imap.setup.php

André, can you install this for us?

Installed now.

André Pitanga wrote:

Installed now.

That was quick, André! Thanks!

I created a new dummy Gmail account for testing (cunyimap@gmail.com) and tested the plugin and it works!

Proof of email reply:
http://cdev.gc.cuny.edu/members/raytester/activity/

The whole thread:
http://cdev.gc.cuny.edu/members/ray/activity/32475

As we're approaching version 1.4 of CAC, it's time to revisit if we want to continue using GMail as the "Reply-To" email address in the BP Reply By Email plugin.

If not, we'll have to look into implementing this locally. André mentions possibly using postfix and Dovecot here.

Talk about reading my mind - I opened this ticket in a tab to ask the very question that Ray was about to ask.

Ideally we would use a *.gc.cuny.edu address. However, this is not a must-have for rev 1 - it should be easy enough to start off using Gmail, and swap out at any time. André, what are our options?

Hi André,

Our goal is to try to finish up this ticket today, if possible, so if you could let us know your thoughts, we'd appreciate it. Many thanks!

Best,

Matt

OK, I'm going to make the executive decision to use Gmail for the moment. Let's open a new account just for this purpose. How about something like cuny_academic_commons?

Sounds good to me.

André mentions possibly using postfix and Dovecot here.

If it's all the same from the application POV I say go with gmail.

Boone Gorges wrote:

OK, I'm going to make the executive decision to use Gmail for the moment. Let's open a new account just for this purpose. How about something like cuny_academic_commons?

I'd probably go with something shorter like cac.reply@gmail.com.

w00t!!!