Project

General

Profile

Bug #9979

Reports of slow email activation emails

Added by Matt Gold 8 months ago. Updated 6 months ago.

Status:
Reporter Feedback
Priority name:
Normal
Assignee:
Category name:
Email Notifications
Target version:
Start date:
2018-07-03
Due date:
% Done:

0%

Estimated time:

Description

Some members of the team were running a workshop for BMCC faculty members last week (on Thursday 6/28), and some BMCC faculty were registering for the CAC during the workshop. What they experienced is that it could take as long as 15 minutes for the email verification emails to be received, while other notifications, like password resets, came through immediately.

Could this delay be related to recent changes we've made to cron jobs or is it more likely that it was a BMCC email server issue?

bmcc.JPG (134 KB) bmcc.JPG Marilyn Weber, 2018-08-26 10:32 PM

Related issues

Related to CUNY Academic Commons - Feature #10230: Reject login.cuny.edu registrationsResolved2018-08-27

History

#1 Updated by Boone Gorges 8 months ago

  • Status changed from Assigned to Reporter Feedback
  • Target version set to Not tracked

Activation emails are sent using the same mechanism as password resets and other emails. Cron is not involved.

My money is definitely on your BMCC email server hypothesis. The WordPress activation email in particular has a history of getting caught up in enterprise filters. https://core.trac.wordpress.org/ticket/25350

I've just tested with a Graduate Center email account, and can confirm that activation emails are being sent immediately.

If it's possible to provide more info about the user accounts, I can run some checks on the registration time, and the corresponding times in the mail logs.

#2 Updated by Laurie Hurson 8 months ago

Hi Boone,

Thanks for this info. The faculty member who never received the registration confirmation email was Allan Leibman (). This occurred around 1pm last Thursday 6/28. Just so I can inform faculty-- is there a particualr reason the commons emails are getting stopped by the BMCC Servers? Is there anything we can do to push the Commons emails through the BMCC servers?

Thanks,
Laurie

#3 Updated by Boone Gorges 8 months ago

Thanks, Laurie.

The system shows that the registration took place at 18:51:24 UTC (2:51pm EDT). I'm unsure why this doesn't match the 1pm timeframe that you've cited. In any case, the mail log then shows that an email was sent to that address at the exact same time - 18:51:24 - which is, presumably, the activation email. The account was then activated at 19:08:34 (about 17 minutes later), and a second email was sent at this time, which is likely our standard "you have activated your account" email.

So, there's no indication that anything went wrong on the Commons end. The right emails appear to have been sent at the right times.

is there a particualr reason the commons emails are getting stopped by the BMCC Servers? Is there anything we can do to push the Commons emails through the BMCC servers?

To be clear, it doesn't seem as if they're being blocked altogether, just delayed. As above, my guess is that something in the content of the email triggered BMCC's security scanning service to flag the email, at which point it needed to process the email and deem it safe before delivering it to the user. I'm surprised that this would take 15+ minutes, but it's possible that there's a lag due to the complexity of the scan, or how busy the email server is, or something like that. The fact that our emails are subject to scanning is not in itself a problem, and is in any case something we'd be unlikely to change. If the delay turns out to recur for many different users, and turns out to be a source of annoyance or other problems, then we could probably contact BMCC IT to investigate. But that's likely to be a complex and difficult conversation (BMCC itself is likely just a client of a third-party vendor, and I'm guessing the vendor doesn't give out details on their scanning algorithms), so I'd suggest we don't undertake it unless the problem becomes widespread.

#4 Updated by Matt Gold 8 months ago

Thanks, Boone. Laurie, can you please pass this info on to Gina and Jean? It seems like something they should be aware of.

#5 Updated by Marilyn Weber 7 months ago

Everyone -

This sounded familiar so I looked at Zendesk. He wrote to me there at 2:08 pm which seems to be before the time "system shows that the registration took place at 18:51:24 UTC (2:51pm EDT)."

It's a moot issue with this particular guy, but it seems to point to something odd with how registration times are recorded.

#6 Updated by Boone Gorges 7 months ago

  • Status changed from Reporter Feedback to Resolved

Hm. I've checked the timezone settings in a bunch of places (within WP, within PHP, on the web server, on the database server) and they all seem to be as expected. WP/PHP/Apache read the current local (Eastern) time, while MySQL records dates in UTC. And I'm pretty sure that UTC 18:51 on June 28 is 14:51 EDT, or 2:51pm, but it's possible that my math is wrong? Relative dates and times elsewhere on the server appear to be displayed correctly, so I don't think there's a broader problem. The fact that there was a Zendesk email at 2:08pm - just before the account was activated - suggests that system dates were off by one hour, but it's not clear how that would happen.

Anyway, as Marilyn notes, it's probably moot in this case - it's not a 1-hour offset that's causing problems, but instead some 10-15 minute limbo for some emails, as imposed by BMCC. In the interest of cleaning up our inbox, I'm going to close this ticket, but feel free to resurrect if there are further steps we should be taking on our end.

#7 Updated by Marilyn Weber 6 months ago

I have two more reports of slow to confirm registrations from BMCC. Screenshot attached showing that all the unconfirmed on the first page are BMCC (or that road to nowhere login.cuny.edu email.)

#8 Updated by Marilyn Weber 6 months ago

Here's that screenshot, sorry!

#9 Updated by Boone Gorges 6 months ago

  • Status changed from Resolved to Reporter Feedback

Could anyone tell me about login.cuny.edu email addresses? It appears that these are linked to CUNYFirst logins, but are they also linked to actual email inboxes? If users are entering these addresses, but are unable to check those specific inboxes, then they'd never receive the activation emails.

In addition - Marilyn, do you have any evidence that these login.cuny.edu addresses are linked to BMCC? The BMCC-specific reports that you've received, do they include any of the login.cuny users you see on this list? I have a feeling that the login.cuny registrations are an unrelated issue.

As for the BMCC-specific items, can you confirm for me that the problem is that the emails are slow to arrive, and not that they haven't arrived at all? The fact that you're seeing the emails on this list suggests that: a long enough time passed after the registration that the user got confused; the user found documentation on how to ask for help, and managed to send a Zendesk message; you read the Zendesk message; and you checked the Unconfirmed list - all of this before they had a chance to confirm their registration. This suggests a delay of much more than 10-15 minutes. Can you give details about when you received the Zendesk message?

As a piece of background info, I do see that many BMCC registrations have been successfully activated in the past few days - some of them as quickly as 60 seconds after the Register button was hit.

#10 Updated by Matt Gold 6 months ago

Hi Boone -- My understanding is that the login.cuny.edu addresses are NOT linked to actual inboxes. I believe that they were created to make a single sign-on option for various central IT services such as CUNYFirst and Blackboard.

#11 Updated by Boone Gorges 6 months ago

  • Related to Feature #10230: Reject login.cuny.edu registrations added

#12 Updated by Boone Gorges 6 months ago

Thanks, Matt. I've opened #10230 to track the login.cuny.edu issue, which I'm pretty sure is separate from this one.

#13 Updated by Marilyn Weber 6 months ago

Boone, sorry I didn't see this before. The BMCC emails seem to be slow to arrive. Will report with more info when I have it.

#14 Updated by Marilyn Weber 6 months ago

The registered times I see on the unconfirmed screen are incorrect. Attached is a screenshot showing that people have registered this evening, and it's currently only 3:42.

I wouldn't mention it but it makes it impossible for me if someone just registered 5 minutes ago (in which case I'd advise that they give the emails a chance to work through the spam filters.

#15 Updated by Boone Gorges 6 months ago

Hi Marilyn - I don't see a screenshot, but I'm looking at the Unconfirmed page now. It appears that the times there are in UTC. NYC is currently UTC-4, which means that, eg, 2018-08-29 20:18:53 UTC is 2018-08-29 16:18:53 EDT, or 4:18pm. I don't see a reason to change this behavior as long as the team (mainly you, since you're the one looking at it!) understand that you need to make the UTC conversion.

This does help me to understand the delay, though, so thanks for that. If you can provide more information on the length of the delay in as many cases as you reasonably can, we may have enough information to submit to the powers-that-be at BMCC. And if they can't help us, we may be forced to add some disclaimers to our own registration process to the effect that some campuses may impose several hours' delay on activation emails.

#16 Updated by Matt Gold 6 months ago

Luke, Tom, Laurie -- please take a look at this ticket. I just looked through the Unconfirmed listings on the Admin backend, and there are a lot of BMCC addresses there. Tom and Laurie, we might want to touch base with BMCC faculty from our summer group to check in.

Separately, please see this ticket, as well: https://redmine.gc.cuny.edu/issues/10230 . Though that issue won't happen for new registrations, there are a number of login.cuny.edu addresses in the Unconfirmed list, and we should let faculty know that if students entered the login.cuny.edu address at login, they won't get a confirmation message and we will have to activate/change email manually (or ask them to re-register with the correct address)

Also available in: Atom PDF