Bug #10678
openNewsletter Plugin Not Sending Out Newsletters
Added by Mark Webb about 6 years ago. Updated about 5 years ago.
0%
Description
Hi,
The Newletter plugin for the CPCP website (https://cpcp.commons.gc.cuny.edu) is again having issues. Newsletters are not sending out. You then have to go in and manually run the queue but this takes forever and stops on its own unless you go back and run again. Any idea what is happening? Is it related to the WP Cron? How can we fix it? Thanks,
Mark
Files
Newsletter problem.JPG (65.8 KB) Newsletter problem.JPG | Marilyn Weber, 2019-04-28 12:06 PM | ||
test 2.JPG (117 KB) test 2.JPG | Marilyn Weber, 2019-05-10 04:59 PM | ||
test 1.JPG (95.6 KB) test 1.JPG | Marilyn Weber, 2019-05-10 04:59 PM | ||
Newsletter queue.JPG (100 KB) Newsletter queue.JPG | Marilyn Weber, 2019-05-10 05:03 PM | ||
CPCP subscribers.JPG (40.8 KB) CPCP subscribers.JPG | Marilyn Weber, 2019-05-10 08:10 PM | ||
CPCP newsletters.JPG (103 KB) CPCP newsletters.JPG | Marilyn Weber, 2019-05-10 08:10 PM | ||
CPCP sends.JPG (29.6 KB) CPCP sends.JPG | Marilyn Weber, 2019-05-10 08:16 PM | ||
CPCP sent and draft.JPG (82.1 KB) CPCP sent and draft.JPG | Marilyn Weber, 2019-05-10 08:29 PM | ||
CPCP post-run now.JPG (20.6 KB) CPCP post-run now.JPG | Marilyn Weber, 2019-05-10 09:05 PM | ||
Screenshot_2019-05-23_12-39-20.png (116 KB) Screenshot_2019-05-23_12-39-20.png | Boone Gorges, 2019-05-23 01:39 PM | ||
RSCP newsletter scheduling.JPG (80.8 KB) RSCP newsletter scheduling.JPG | Marilyn Weber, 2019-05-23 01:51 PM | ||
cpcp-delivery.png (80.3 KB) cpcp-delivery.png | Colin McDonald, 2019-09-16 07:22 PM |
Updated by Boone Gorges about 6 years ago
- Status changed from New to Reporter Feedback
Hi Mark - I am unfamiliar with the way that this Newsletter plugin works. Could you please provide me with more specific details about what you're doing, and how it's supposed to work?
I see https://cpcp.commons.gc.cuny.edu/wp-admin/admin.php?page=newsletters-queue but I don't understand whether these are items that have already sent. When you say you have to "go in and manually run the queue", what exactly are you doing? And when you say it "takes forever and stops on its own", exactly what is happening? Does "takes forever" mean that the page hangs after you click a button? Or does it mean that emails send in the background, but only one at a time, and there's a long pause between? Or perhaps an initial batch (of 20 or whatever) is sent right away, but the next one never happens? When you say "again having issues", do you mean that you've had this specific problem in the past? If it's just started, could you please indicate when was the last time it worked as intended, and when you noticed it stopped working? (We had some performance issues throughout the day today, which may be related.)
The more details you can provide, the easier it will be for me to try pinpointing the issue.
Updated by Mark Webb about 6 years ago
When I say "again" I am simply referring to the fact that my last bug report was in reference to the same plugin.
So, I actually don't know how this plugin in normally works as I am working on behalf of the CPCP who sends these newsletters out and asked me to look into what the problem is now. Now when you go to send a newsletter the option to click at the bottom right is "queue newsletter." You click on that and it takes you to the "email queue" page where you have the option to "run now" which presumably would send the newsletter to all of the emails in the queue. But when you click "run now" it takes you to a blank page with a url that ends in "scheduled tasks." It is unclear what is happening on this page. It seems that the first few batches of emails send but then it stops. You then have to go back and hit "run now" again but even that eventually stops doing anything (as in you can't even hit "run now" to manually send all of the batches). I understand that when you click "queue newsletter" it should automatically start sending out batches. I am told the problem started a little over a week ago and was noticed after 3-4 newsletters did not go out and were just left in the queue.
Updated by Boone Gorges about 6 years ago
Hi Mark - Thanks very much for the additional info, which helps me narrow down what's happening.
We do not use WP's built-in cron system - which is why you see the notice through the Newsletters admin about 'DISABLE_WP_CRON' - but we have a replacement system service called Cavalcade that is responsible for asynchronous tasks. I can see from my logs that the most recent scheduled queue-send event was marked "failed", but this could mean many things, and my guess is that it means that the process started successfully but didn't complete - as in, perhaps it ran for too long and couldn't be shut down. This meshes with your comment that some emails seem to be sent, but then the process dies for one reason or other.
The first thing I'd suggest trying is to reduce the batch size. See "Emails per interval" at https://cpcp.commons.gc.cuny.edu/wp-admin/admin.php?page=newsletters-settings. Try something low, like 20, and then trigger another batch. This might be enough to allow the Newsletters plugin to complete its task, and then properly schedule the next one.
If this doesn't work, I'll need to do some more active debugging, which means I'll need to be able to press the "go" button on an email queue batch. Before I'm comfortable doing that, I need to have a better understanding of the plugin interface. This page https://cpcp.commons.gc.cuny.edu/wp-admin/admin.php?page=newsletters-settings has almost 4000 items in the "queue", but the "Date" column suggests that they've already been processed/sent. Do you have a sense of what's happening here? The timestamps (from last night and this morning) suggest that they correspond to times when you, or another member of the team, has manually triggered email sending. Is that correct? Can you point me toward a specific URL and a specific button I can hit, without being fearful of double-sending an email?
Updated by Mark Webb almost 6 years ago
Hi Boone,
This issue has popped up again. For whatever reason the queue starting backing up again and was not sending out emails automatically. I am currently manually hitting "run now" on the queue and have been playing around with different queue sizes, but I have no idea what the source of the problem is. We did recently switch the template we are using in newsletter but I don't know why that would make a difference. Thanks for the help!
Updated by Boone Gorges almost 6 years ago
Hi Mark - Could you provide some more detail on what's happened recently?
- You mentioned that it "popped up again" - does that mean it'd been working as expected?
- When did you first notice that it wasn't working again?
- Does the queue contain items from more than one newsletter, or just a single one?
- Does the plugin appear to have sent any items automatically?
- When you hit "run now", does it process a single batch, or multiple batches before stopping?
- When did you switch the template?
Updated by Mark Webb almost 6 years ago
Thanks Boone,
Things seem to be working again. Perhaps something triggers when the queue gets backed up preventing things from functioning smoothly? Why the queue got backed up in the first place is still a mystery to me.
We noticed the problem when we tried to send out a newsletter a couple of weeks ago. We just switched the template before the holidays and this was the second newsletter sent with the new template. I don't do the newsletters, I am just working on smoothing out the hitches with the site and newsletter, so I was contacted when the newsletter queue was already backed up with 4,000 emails and it was not sending out anything unless I ran it manually, one batch at a time (it would send one or half of one, depending on the size, and then stop).
thanks!
Mark
Updated by Marilyn Weber over 5 years ago
Esra Padgett reports via ZenDesk:
We have been having some major issues with our newsletter and have not been able to figure out what is going on. Essentially, the automatic queue has slowed down to the point of sending one newsletter over three days--despite the fact that it is set to release a batch every minute. . . We have tried adjusting the time/amount of batches in the past, but it seems every couple of months the whole thing just stops working again.
Updated by Marilyn Weber over 5 years ago
- Status changed from Reporter Feedback to New
I've changed the status to New just in case this hasn't caught anyone's eye while in Reporter Feedback. Very sorry to be a nudge - I leave for a week on Friday so I'm hoping there's an easy fix here.
I don't know if it's related, but I think this is a domain mapped site.
Updated by Matt Gold over 5 years ago
- Status changed from New to Assigned
- Assignee set to Boone Gorges
- Priority name changed from Normal to High
Boone -- I want to make sure you see this
Updated by Boone Gorges over 5 years ago
- Status changed from Assigned to Reporter Feedback
Thanks for the nudges.
As in the previous instances of this issue, it's difficult to debug without information about the intended behavior. Marilyn, could you please ask the reporter for the following information:
1. The URL of the site where the problem is being experienced, and the name of the plugin being used (it's not specified in the report)
2. The report indicates that things were once working properly but are no longer doing so; more information about when the change took place would help
3. A description of exactly how the newsletters are created and the sending is triggered
4. Information about how the user is verifying that "one newsletter over three three days" is being sent - where does this info live, and how can I look at it?
As an aside, more for our team than for the reporter, the use of the Commons and WordPress for the sending of newsletters to large mailing lists, is a stretch of what the platform is designed to do. It sounds like there may be a specific bug preventing proper functioning in this case, but it's also probably the case that there's an upper bound to the kind of detailed support we can provide for these use cases.
Updated by Marilyn Weber over 5 years ago
Thanks! Asking now.
(Is there a way to add watchers at this point? I'd like to add myself, Scott and Colin. Thank you.)
Updated by Boone Gorges over 5 years ago
Thanks, Marilyn! I've added the watchers.
Updated by Marilyn Weber over 5 years ago
I just noticed that this is connected to https://redmine.gc.cuny.edu/issues/8924
Thanks for adding us!
Updated by Matt Gold over 5 years ago
Boone Gorges wrote:
As an aside, more for our team than for the reporter, the use of the Commons and WordPress for the sending of newsletters to large mailing lists, is a stretch of what the platform is designed to do. It sounds like there may be a specific bug preventing proper functioning in this case, but it's also probably the case that there's an upper bound to the kind of detailed support we can provide for these use cases.
Thanks for this note, Boone. There are several groups using the Commons for just this purpose. I wonder if there is another model we can suggest to them -- perhaps some kind of integration with MailChimp? My question for you is whether the issue is with the sending out of newsletters, period, or the sending of them out to large groups of people. If it's the latter, perhaps these users could design a workflow where the email is sent to one address and then forwarded through another system (Mailchimp, email, etc). Also, I wonder whether a group could be leveraged for this purpose.
Updated by Boone Gorges over 5 years ago
I wonder if there is another model we can suggest to them -- perhaps some kind of integration with MailChimp? My question for you is whether the issue is with the sending out of newsletters, period, or the sending of them out to large groups of people. If it's the latter, perhaps these users could design a workflow where the email is sent to one address and then forwarded through another system (Mailchimp, email, etc). Also, I wonder whether a group could be leveraged for this purpose.
I believe the problem is the sending of large numbers of emails to a large number of people. The Tribulant Newsletters plugin batches its email sending and uses WP's cron system to trigger the batches. These batches run once per hour, and the logs don't show any of these batches as "failed", but the logs don't provide anything approaching fine-grained data. I'm guessing that the newsletter-sending process occasionally hits a snag due to some sort of malformed subscriber (?) and can't get past it, but it's hard to debug without more info.
Close integration with something like Mailchimp seems promising to me. CCing Ray in case he's got experience with how this process works.
Updated by Marilyn Weber over 5 years ago
Esra has replied:
1. https://cpcp.commons.gc.cuny.edu/wp-admin/admin.php?page=newsletters-create&method=history&id=321. I'm not sure what the plug-in but I think it might be jetpack?
2. When the change took place: this is a difficult question to answer because it stops and starts--right now for instance it seems to be working again, but on 4/11-4/14 it stopped again. This problem began sometime last year--it does not seem to have any regularity in its re-occurrence. For instance, I often send the same newsletter several times, and the problem will sometimes effect one of the letters going out but not times before or after, so I don't think it is related to content.
3. The newsletters are created in the commons- I don't know what the name of this plug in is though. There are text boxes for the main content and the subject line, and then on the side there is a box which says "subscribers" where you select the list of people to send to. We have our mailing list there, so once the box is checked there is button which says "Queue newsletter". This begins the sending process and takes you to a page where the email queue is visible. I think it has been set to send 100 emails every 2 minutes, but we had someone working with us that may have changed this slightly. When we first had the experience we tried changing the amount and frequency but it never seemed to effect whether the sending was working or not. It seems like the automatic queue process is just on hold and we have no way to reset it. In times when it has stopped, I have been able to send batches of emails manually via the email queue page, but there have also been times when I was unsure whether the batches were really being sent out.
4. When you look at the sent emails there are analytics about how many emails were sent per day and how many read etc. In the most recent time when it stopped working you can see that the emails are being sent over two days, despite the fact that they should all have been out on the first day. The one that spanned three days seems to have been deleted, I think because it still had not finished sending and I had to take if off the list.
This newsletter is here:
https://cpcp.commons.gc.cuny.edu/wp-admin/admin.php?page=newsletters-history&method=view&id=325
Updated by Marilyn Weber over 5 years ago
- File Newsletter problem.JPG Newsletter problem.JPG added
IDK if it will help, but I've attached a screenshot of the Scheduled Tasks under the Configuation settings. It seems to say it's using Cron. I noticed above you wrote "We do not use WP's built-in cron system - which is why you see the notice through the Newsletters admin about 'DISABLE_WP_CRON' - but we have a replacement system service called Cavalcade that is responsible for asynchronous tasks."
Updated by Boone Gorges over 5 years ago
Marilyn, please pass along my thanks to Esra for the detailed response. This is enough for me to have a better sense of how the newsletters are being triggered.
The indications that certain tasks "are automatically run using the WordPress cron" is almost certainly a red herring. There may be a problem related to background tasks, but it's clearly not a direct result of using our Cavalcade system rather than WP's built-in system, since the newsletters appear to often (usually?) work. So there's something more specific causing batches to fail in some cases.
Could you please ask Esra to be on the lookout for another case where newsletters aren't properly sending? If I could be informed while the delay is in progress, I would be better equipped to debug where the system is hanging up.
Updated by Marilyn Weber over 5 years ago
This is happening on another site that runs Newsletter as well. Boone, would you prefer to debug or can we install another plugin? Perhaps this one, which was just updated 5 hours ago - https://wordpress.org/plugins/newsletter/
Updated by Boone Gorges over 5 years ago
Thanks for the report, Marilyn. I'd love to move to another plugin, if I thought that another plugin would work perfectly. But (a) I don't know that this'll be the case, and (b) asking users to migrate from one plugin to another is a very large ask (we have to migrate their existing content, their existing subscribers, their existing appearance settings, their existing subscription front-end UI, etc), so it's not trivial to undertake.
As noted above, it'd be great if I could get detailed information about the behavior while an incident is taking place. This information needs to be as detailed as possible: exactly what the user did (links that were clicked, URLs to reference, etc), what the user expected to see ("when viewing such-and-such a log, I expect to see that all addresses have received notifications", etc), and what actually happened ("only the first two pages of users on such-and-such a log/URL show as having received the newsletter"). I know it's onerous for users to provide this info, but without it, I am unable to do the necessary debugging - "some users didn't get the email" doesn't provide me with enough context.
Updated by Marilyn Weber over 5 years ago
I will pass on the specific questions you ask, but the CPCP user seems to only check the ticket every once in a while. Now that I see that it's happening on another site - one where I may ask for permission to send the newsletter myself, as I'm friends with the admin - I'll try to do one myself and will let you know how it goes.
I'd be curious to see how the other plugin works myself. We used to have Mailpoet but that's gone. Can we add this new, also blandly name one even if it's not what we suggest as a fix?
Updated by Boone Gorges over 5 years ago
Sounds like a plan, Marilyn. Thank you.
I would strongly prefer NOT to set up any new plugins at this time. The current one causes such problems that I'm not eager for the potential to multiply. Moreover, these kinds of problems have a tendency not to arise until the plugin's in real-life use, with many subscribers - so issues may not arise until the new plugin is in heavy use and difficult to remove.
Updated by Marilyn Weber over 5 years ago
I support your no new plugin plan.
Esra just replied: "This issue is currently still happening."
He's not very detailed, despite my cutting and pasting what you asked above. Can I put him directly in touch with you via email? Or can I add you to the ZenDesk ticket?
I'm actually not at the GC today as I'm expecting repair guys at home any minute.
Updated by Matt Gold over 5 years ago
Hi Marilyn -- we're not going to be able to solve his problems if he is not able to give us the information we are asking for. I'd prefer not to put Boone in the position of dealing directly with this user if we can avoid that. If you don't mind, please reephasize to him that we need answers to the questions in order to troubleshoot
Updated by Marilyn Weber over 5 years ago
Ok, emphasized again and asked some of my own questions:
Esra - I see you're on your phone and I know it's tricy to be detailed from there. The dev team is telling me they need more information from you. When you convenient, can you answer:
1) exactly what you've done (links that were clicked, URLs to reference, etc),
2) what you expected to see ("when viewing such-and-such a log, I expect to see that all addresses have received notifications", etc), and
3) what actually happened ("only the first two pages of users on such-and-such a log/URL show as having received the newsletter").
And please also let me know this:
1) Would you consider having people subscribe, rather than sending a newsletter? Subscribers can control how often they receive updates, so it could function like a newsletter where they only get one email a week with everything.
OR
2) Should I remove the bad subscriber emails?
Updated by Marilyn Weber over 5 years ago
Boone and Matt -
Rukshana Jalil runs the Medieval Studoies newsletter and has let me be an admin. I'm going to try to answer your questions by sending a test newsletter. I'm doing this in several posts so you can follow along.
1) I created a newsletter and pressed "queue newsletter" (which seems to be the equivalent of Publish)
Updated by Marilyn Weber over 5 years ago
- File test 2.JPG test 2.JPG added
- File test 1.JPG test 1.JPG added
I selected Medieval Studies Subscribers (124 active)and chose Queue immediately.
Updated by Marilyn Weber over 5 years ago
- File Newsletter queue.JPG Newsletter queue.JPG added
It goes to queue, but doesn't send them all at once. So I hit send queue.
Updated by Marilyn Weber over 5 years ago
Aaaand . . . that works. But I don't think it's working for CPCP. Not the result I expected!
I'm going to send a test from CPCP this weekend and will do another play by play. Sorry for the bother.
Updated by Boone Gorges over 5 years ago
This is extremely helpful, Marilyn - no apologies necessary! Thank you so much for the clear documentation. Please keep this thread updated with all the gory details :)
Updated by Boone Gorges over 5 years ago
Also, my impression is that the CPCP folks are having a hard time with the automated sending of newsletter batches, so once it's launched, it may be worth letting it sit for a while to see what happens.
Updated by Marilyn Weber over 5 years ago
- File CPCP newsletters.JPG CPCP newsletters.JPG added
- File CPCP subscribers.JPG CPCP subscribers.JPG added
- File CPCP sends.JPG CPCP sends.JPG added
OK, more gore for you!
CPCP has 2500 subscribers (see attached)
They've sent 3 different newsletter in the last 4 days (see attached)
The largest number of emails sent in a day is 1286 (see attached)
So yes, totally, I think the fault is in the automation. But what's the solution?
I have suggested that they ditch the newsletters and replace it with Jetpack's subscribe function, which in my mind works the same but with a greate degree of control. Getting digests once a week or once a day is de facto a newsletter to me.
Updated by Marilyn Weber over 5 years ago
Esra's response to the digest idea is:
I spoke to Mary about the Subscriber option and it seems like that won't fit our needs, we use the newsletter to remind people about events, so its important that we can send these out within a day of the event, etc. We also use the newsletter like a listserv for circulating other people's events that are not published on our site, but are related to our community. If we can't get through this issue on the site, it's more likely we would need to return to a paid option like mailchimp.
Updated by Marilyn Weber over 5 years ago
- File CPCP sent and draft.JPG CPCP sent and draft.JPG added
Now I stumble across this "sent and draft" page, which seems utterly inconsistent with the chart (see attached) - to read this you'd think there are more subscribers than listed, and fewer sends.
Updated by Marilyn Weber over 5 years ago
The queue page says there are 1271 emails to send. I'm going to hit Run Queue now.
It definitely isn't working well - I made myself a subscriber on Monday. I received the Capital City reminder yesterday, 5/9, at 8:10 pm - even though I see you had sent it on May 6th.
Let's see what happens after "Run Queue" - the email called "Alejandra Bronfman: The Aesthetics of Toxicity, Vieques" was sent today, and I certainly haven't received it.__
Updated by Marilyn Weber over 5 years ago
- File CPCP post-run now.JPG CPCP post-run now.JPG added
Oh, now I'm feeling their pain! I clicked Run Now and it sent 72 emails. Only 1199 to go! Good grief!
Updated by Marilyn Weber over 5 years ago
I stopped clicking, and many were sent without my prompting. There are now 63 in the queue. Let's see where it stands tomorrow.
Updated by Boone Gorges over 5 years ago
Marilyn, this continues to be extremely helpful. Thank you.
From the above, I gather that the emails are in fact being sent - https://cpcp.commons.gc.cuny.edu/wp-admin/admin.php?page=newsletters-history&method=view&id=333&_wpnonce=969889c110&_wp_http_referer=%2Fwp-admin%2Fadmin.php%3Fpage%3Dnewsletters-history%26method%3Dview%26id%3D333&status=unsent&read=all&clicked=all&bounced=all&filter=1#emailssent is empty - but that the batches are running at a pace that's difficult to predict, sometimes very slowly.
For the next steps, I'll probably need to do some live debugging during a send routine. This will require me to set up some logging tools, and then watch in real time. Do you think that you and I might try to coordinate an upcoming send? The PCP team would let you know when their next item is set to be sent, and then you and I would be available at that specific time? Then, you'll be responsible for clicking the buttons, while I watch the logs and see if I can figure out what's going on. Would you mind chatting with their team to see whether this might be possible in the upcoming weeks?
Updated by Marilyn Weber over 5 years ago
Sounds good to me. I'll let you know how they respond.
Updated by Marilyn Weber over 5 years ago
We've agreed that I can send a test newsletter - which is a great relief. All you and I have to do is coordinate times. Wednesday is crazy, but I could do it tomorrow or Thursday.
Updated by Boone Gorges over 5 years ago
Awesome - are you available around noon or 1pm EDT on Thursday? This'll give me a few days to figure out what the logging tools might look like.
Updated by Marilyn Weber over 5 years ago
I think you wrote that you'd like to Skype during this. That can't be done in the office I'm in, but I can be on speaker phone. The number here is 212-817-8449 or I can call you. Sound good?
Updated by Boone Gorges over 5 years ago
Thanks, Marilyn. I'll call you shortly to get started.
Updated by Marilyn Weber over 5 years ago
Great! I'm attaching a view of the newsletter interface (top and bottom)
Updated by Marilyn Weber over 5 years ago
Boone, the CPCP team is wondering where this now stands. LMK how I can help. Thanks!
Updated by Boone Gorges over 5 years ago
Summarizing for posterity:
- Marilyn and I had a confab on 2019-05-16 where we sent a test newsletter and I debugged in real time
- I was able to understand better how the plugin works. It uses one of its three "queue processes" (the unsuffixed one) to send queued batches. It doesn't use pingbacks, but instead schedules a new cron event at the end of each batch. This is likely where the disconnect is happening.
- Our test newsletter mostly sent properly, but there was minor lag. I didn't have enough info to see why.
I've put some debugging code in place, which will write to a log when batches are triggered by the plugin. Please let the CPCP team that I'm standing by to do more in-depth analysis the next time they send and have problems.
Updated by Marilyn Weber over 5 years ago
They'd like to do it again. Neither of the admins received our test - I didn't either. Alas!
Or we could just try a different plugin
Updated by Boone Gorges over 5 years ago
I stopped the test email after a few batches ("Clear queue") so as not to spam the group. That's why you didn't receive the test emails.
It's OK with me if you want to trigger another test email. If you do so, be sure to keep track of expected behavior vs actually observed behavior, especially any large pauses between batches (including specific times). I'll then be able to compare against my logs.
If we have another newsletter plugin and the CPCP team would like to try it out, it's OK with me. However, I do not want to try throwing a brand new plugin at the problem at this time, as it's possible or even likely that it will hit the exact same set of mysterious problems as this one.
Updated by Marilyn Weber over 5 years ago
They sent out a newsletter yesterday (May 20, at 16:31, "CALL FOR CUNY FACULTY APPLICATIONS for the Seminar on Public Engagement and Collaborative Research (Deadline to apply: September 30, 2019")
There are 768 emails in the queue today, May 21st at 18:03.
Updated by Marilyn Weber over 5 years ago
When I search the installed plugins, this is all I see:
"MailPoet Newsletters Premium: Extended functionalities to the free version"
If memory serves, MailPoet was terrible and so it was removed. So this is is tag-on plugin that should probably be removed.
We have nothing else to try.
Updated by Marilyn Weber over 5 years ago
Here's where the decision was made to tell CPCP to no longer use Mailpoet - https://redmine.gc.cuny.edu/issues/6821
I don't disagree that a new plugin might have the same issues. But Tribulant Newsletter is a plugin with years of complaints:
https://redmine.gc.cuny.edu/issues/11175
https://redmine.gc.cuny.edu/issues/10571
https://redmine.gc.cuny.edu/issues/10290
Updated by Boone Gorges over 5 years ago
Thanks, Marilyn. I can see in the log files where the process stopped, and these log entries help to narrow down the possible issues. I'll take some more time tomorrow to dig deeper and try some things.
In the meantime, I must continue to push back on the idea of a new plugin. Throwing another questionable plugin on top of a bad one has a tendency to multiply problems, not reduce them.
Updated by Boone Gorges over 5 years ago
I've spent a bit of time this morning working on the pending queue. The issue is at least partly due to the following sets of facts:
a. The plugin triggers each batch using a cron job https://redmine.gc.cuny.edu/issues/10678#note-48 (though it appears that sometimes it's not the unsuffixed one - in my current testing, it's queue_3 - this means IMO that the developer was grasping at straws when building the queue system)
b. In this specific case, there were a couple of defunct cron jobs from April 3, 2019.
c. Because these cron jobs were so old, either they weren't being run, or when they ran, the plugin was finding an empty "batch" corresponding to the cron job (couldn't tell exactly which)
d. As a result, the old cron job was not properly cleared, and when the plugin tried to set up a new (future) cron job, it didn't work
It's still not clear to me why it appears to fail periodically and then self-start. There appears to be a sort of health-check mechanism built in, but I don't understand how it works.
In any case, I cleared out the old cron jobs and re-triggered the queue. This triggered the remaining eight or ten batches.
I have not identified the root cause for the old/duplicate cron jobs. But I believe I now have enough information to jump-start a stalled queue (ie, by deleting the old cron jobs).
Marilyn, feel free to pass on some appropriate summary of the above ("we are working on it and have made progress" may be enough :) ) and tell them that they should let us know the next time they trigger a send, so that we can monitor.
Updated by Marilyn Weber over 5 years ago
Excellent! There are 0 emails in the queue today, May 22nd.
Are you up for me sending a test email tomorrow? I'll report on it every hour or so.
Updated by Marilyn Weber over 5 years ago
Problems right away. I created a newsletter at 13:09 (according to the internal time) and ask for it to be scheduled for 13:11. Nothing has been send, and when I look to Settings for the time I see this "Universal time (UTC) is 2019-05-23 15:02:54. Local time is 2019-05-23 11:02:54"
I still see this (clicking View from the Overview page - https://cpcp.commons.gc.cuny.edu/wp-admin/admin.php?page=newsletters-history&method=view&id=338)
Scheduled Yes - 2019-05-23 13:11:00
And this (on the Send& Edit page:
Scheduled
(May 23, 2019 1:11 pm)
So the first problem is that it wants to follow the actual time although it seems like it should follow its own time.
Updated by Marilyn Weber over 5 years ago
I have now scheduled a Second Test for May23 which should go out in a minute.
Updated by Marilyn Weber over 5 years ago
It didn't. Neither letter is sending, even when I go in and click Schedule again. I have a error message at the top of the page which simply says "Newsletter could not be sent" - no explanation.
Updated by Marilyn Weber over 5 years ago
Tried again - all three are showing as Draft.
Updated by Boone Gorges over 5 years ago
Thanks, Marilyn. I'm unsure how this plugin is supposed to work, but I'm seeing that your emails are in "Draft" status - see screenshot. Could that be part of the issue?
Updated by Marilyn Weber over 5 years ago
I do not understand why they are in Draft - there is no Publish option.
Updated by Marilyn Weber over 5 years ago
Rukshana Jalil uses this on two of her sites. On the American Studies site, she sees "Queue Newsletter" on the Create page, and can send it out right away.
On the Renaissance Studies site, she sees "Schedule Newsletter". As you can see in the attached screenshot, here last scheduled one didn't go out. She also noticed that there was one from April 25th that remained as Draft just as ours are now.
Updated by Marilyn Weber over 5 years ago
Now one of them (the original, "Test for May23") shows as Scheduled at 5:26 pm. The other two are still Draft I have no idea why.
Matt, you had asked about numbers. I see that the most read newsletter has these stats:
612 read (28.90%), 0 unsubscribes (0.00%), bounces (0.00%) and 8 clicks out of 2118 emails sent out
Updated by Matt Gold over 5 years ago
Thanks, Marilyn. That's significant usage/readership and speaks to the value the Commons is serving to quite a large community.
Updated by Boone Gorges over 5 years ago
Matt, could you please chime in with what you see as the next steps? Here's the situation as I understand it: There was originally a report on this ticket that emails were not being sent out in batches. Marilyn has helped me to do some testing and to come closer to an understanding of how this particular feature is not working properly. In the process, Marilyn has found a couple of other ways in which the plugin is either confusing or seems not to work at all. While I'm willing to spend some time to investigate why specific parts of the plugin don't work properly in the Commons environment, it's not possible or prudent for me to analyze or rewrite every substandard part of the plugin. As things stand, the time that Marilyn and I have sunk into support is way out of proportion to the relative popularity and importance of this tool vis-a-vis the rest of the Commons.
My personal inclination is to deprecate this tool (hide it from new users) and to urge existing sites that are unhappy with it to migrate to a third-party service that is intended for the use of creating and sending newsletters. That being said, you've indicated here and elsewhere that you think it's important for the Commons to provide the newsletter service to CPCP and to others. So I'd like to have some guidance from you about the resources that we should sink into newsletter support on the Commons, and especially how you envision these resources being prioritized alongside other short and long term goals.
Updated by Matt Gold over 5 years ago
Hi Boone -- thanks for your response, and I do hear your frustration with this issue and your sense that this feature moves beyond the scope of the CAC.
I do think that these are important use cases that involve high-profile groups using the Commons as their home base for content. The integration of newsletter functionality into that environment is clearly important to them. And I would say that the key feature here is not just the sending of newsletters, but the way that these newsletters are harvested from their CAC content.
It has seemed to me that up until now, you have been reluctant to install an alternate plugin, arguing that doing so would be likely to cause additional/other problems. Is that still your opinion? I would love to see whether we can find a way to serve these users.
Updated by Marilyn Weber over 5 years ago
After the initial confusion about time and draft mode, it seems that the newsletter sent - "1 times and 2130 emails." That's mostly good news but I see two problems:
1) how can I get a newsletter to send immediately, to everyone?
2) why did I, as a subscriber, not get it?
Thanks Boone and Matt! Sorry for all the bother!
I deleted the other two test newsletters.
Updated by Boone Gorges over 5 years ago
Marilyn - I'm afraid I don't have access to outgoing mail logs (after a recent change to the Commons server) so I don't have the ability to verify whether an email was sent to you. My guess is that it was sent, but perhaps it got caught by a spam filter, since it was just a test notice. I notice at https://cpcp.commons.gc.cuny.edu/wp-admin/admin.php?page=newsletters-history&method=view&id=338 that many of the emails were read, which indicates to me that the send actually took place.
I don't know about the "immediate" part. My initial thought is that it's a timezone issue: when you go to schedule an email at 8am EDT (12pm UTC), it won't let you schedule for 8:01am, because it's looking at UTC and thinks that 8:01am is four hours in the past; so it forces you to schedule for 12pm; but that 12pm ends up being 12pm EDT, which is 4 hours in the future. This is just an educated guess. I'm unsure how to fix this without a deep dive into the plugin. In any event, it sounds like this specific bug report comes from your observations, not from the users?
Matt, I'd be glad to install a plugin that works well in our environment. The problem is that I don't know what plugin that would be. And it's not necessarily something I can discover through study, or even through sandbox testing. And I have reason to believe that some version of the problems we experience with this plugin would be replicated in any other plugin that does something similar. To adopt something new, we'd need both the dev/design resources for initial research/implementation/troubleshooting, but also one or more willing partners with mailing lists that are large enough for meaningful testing who are willing to be our guinea pigs, which means significant communication and support resources.
Updated by Matt Gold over 5 years ago
Thanks, Boone. I hear you on the difficulty on this and the potential cost. here is my thinking:
-- I would like to know more about the features that are important to the groups using this plugin. Marilyn, you've been working with a few of them. Can you please ask them what they find valuable about it? Is it the automated movement of content from the website to a newsletter? the automated sending? the subscription options? something else? Please also feel them out on whether they are willing to work with us on alternatives -- thinking about how a system like Mailchimp, which is designed to send messages to large groups, could be used to ease this problem; talking to them about other ways to address this issue.
-- Boone, I would love to know from you:
1) are there plugins that would export content in such a way that it could be easily ingested into a platform like Mailchimp?
2) would Mailchimp plugins/integrations be something that we should explore here? If so, how would we help people address the problems outline in #1 above, where users would need website content automagically inserted into a newsletter?
-- if, Boone, you'd like a member of the community team to do the research and then share it with you to discuss technical feasibility, we can also do that
Updated by Colin McDonald over 5 years ago
I have a couple scattered and hopefully helpful thoughts to offer:
- I don't think Wordpress's core Jetpack subscription functionality helps us much here, because they don't really let you import subscribers from a prior list. They're very particular about permissions, so it's more about accepting subscribers anew once you set it up. I get why Jetpack looks attractive, because it's shepherded by Wordpress and less likely to get screwy, but it might not fit needs with these prior lists of 2500 or what have you.
- Is anyone familiar with a service called Tinyletter? It is an offshoot of Mailchimp designed expressly for small, no-frills email newsletters and is free for up to 5000 subscribers. Importing looks pretty easy. You can't pull a Wordpress post into a Tinyletter email directly as you might with a plugin, but Tinyletter is so straightforward that it would just be a few more copy/pastes and clicks to set up something similar when they want to distribute a post or make an announcement. Here are some documentation links:
https://tinyletter.zendesk.com/hc/en-us/articles/360005529833-TinyLetter-subscriber-limits
https://tinyletter.zendesk.com/hc/en-us/articles/360006201734-Designing-your-newsletter
And this might be overkill, but I wonder if you could rig up a de facto automated email newsletter by having Wordpress send an email notification to your "secret" Tinyletter email as described here, which would in turn send that email out to the subscriber list:
- In trying to think about the user first, I hate to be asking CPCP and others to migrate to an external service for newsletters. It will be different, and an adjustment for the people sending the newsletter and the people reading them. Also, there's always the chance that Mailchimp decides to shutter Tinyletter or that any external service will suddenly disappear. But it may be worth the risk. We know that Tinyletter works, as email is its sole purpose, and it takes us away from trying to make Wordpress cover that function with plugins. Like us, it seems like the CPCP staff is investing a lot of time in trying to salvage the current solution, and the effort to migrate would only be a fraction of the current headaches. Hopefully they'd be willing to do their part with us to make the transition work.
Updated by Marilyn Weber over 5 years ago
Still no luck. On May 28th, I sent out a "Mobilizations and Migrations" newsletter in the hopes that the name would beat the filter. It took a long time to get it sent out, but they seemed to all send by May 30th. I never received it.
Updated by Colin McDonald about 5 years ago
- File cpcp-delivery.png cpcp-delivery.png added
It seems like the summer slowdown didn't help fix issues with the Newsletter plugin delivering to the CPCP's subscribers. Marilyn reports that recently it is only sending to about 300 of their list of 2100 and isn't sure why (see screenshot). Let's try to refresh ourselves on this ticket and find a path forward on the dev call tomorrow, perhaps involving an external service if we've exhausted our potential fixes here.
Updated by Marilyn Weber about 5 years ago
Here's a screenshot. As you can see, my last battle in July was successful. Esra called me back in for the most recent one ("09/16: Towards the City") and I cannot get it to send to more than 476. It is not in the queue.
If you want to suggest Tinyletter, that's fine by me, but I'd need some help understanding it myself.