Project

General

Profile

Feature #11795

Import Events to group calendars

Added by Matt Gold over 2 years ago. Updated about 2 years ago.

Status:
Resolved
Priority name:
Normal
Assignee:
Category name:
Events
Target version:
Start date:
2019-08-26
Due date:
% Done:

0%

Estimated time:

Description

Boone -- is it currently possible to import events into a group calendar via an ics file? If not, I'd like to suggest it as a feature. It's currently an issue for some groups that are already using multiple calendars in multiple systems, and this would make it easier to replicate events across calendars

event-organiser-ical-sync.png (27.2 KB) event-organiser-ical-sync.png Raymond Hoh, 2019-11-11 03:06 PM
group-ical-manage.png (23.3 KB) group-ical-manage.png Raymond Hoh, 2019-11-11 03:06 PM
group-ical-feed-meta.png (37.2 KB) group-ical-feed-meta.png Raymond Hoh, 2019-11-11 03:06 PM
group-ical-filter.png (42.5 KB) group-ical-filter.png Raymond Hoh, 2019-11-11 03:06 PM
event-privacy.png (12 KB) event-privacy.png Raymond Hoh, 2019-11-11 03:06 PM
screen1 - test import feed.png (401 KB) screen1 - test import feed.png Laurie Hurson, 2019-11-13 11:47 AM
screen2 - test3import with ics file.png (569 KB) screen2 - test3import with ics file.png Laurie Hurson, 2019-11-13 11:47 AM
subscribed-calendars.png (13.5 KB) subscribed-calendars.png Raymond Hoh, 2019-11-15 07:15 PM

Related issues

Related to CUNY Academic Commons - Feature #5226: One-way sync group events using calendar feed?Duplicate2016-02-15

History

#1 Updated by Matt Gold over 2 years ago

  • Subject changed from Import iCal to group calendars to Import Events to group calendars

#2 Updated by Raymond Hoh over 2 years ago

Duplicate of #5226?

#3 Updated by Boone Gorges over 2 years ago

  • Related to Feature #5226: One-way sync group events using calendar feed? added

#4 Updated by Boone Gorges over 2 years ago

  • Target version set to 1.16

Maybe a duplicate. I think that what's being requested here the ability to run a one-time import of calendar events from an .ics file. #5226 appears to be about subscribing to an external calendar; I think the underlying mechanism there is an RSS feed. I'm pretty sure that these are different operations, given the way the file formats work; see the "Difference" section of https://support.office.com/en-us/article/Import-or-subscribe-to-a-calendar-in-Outlook-com-cff1429c-5af6-41ec-a5b4-74f2c278e98c. Confusingly, though, it appears that the EO extension may "subscribe" to a calendar by doing periodic ics imports. See http://docs.wp-event-organiser.com/importing-events/importing-events-from-an-ical-feed/.

I'd say our next step is to get the EO license up to date and to get a recent version of the EO iCal sync plugin. Matt, can you please look into this?

Then, I think we just need a bit of UI to expose the functionality in group contexts. This could probably be built into bp-event-organiser. Ray, once we have the license, could I ask you to take the lead on this? Maybe it'd work as a subpanel of Manage?

#5 Updated by Raymond Hoh over 2 years ago

  • Assignee changed from Boone Gorges to Raymond Hoh

Ray, once we have the license, could I ask you to take the lead on this? Maybe it'd work as a subpanel of Manage?

I'd be happy to work on this. Subpanel under the "Manage" tab seems like the most logical place.

#6 Updated by Raymond Hoh over 2 years ago

I was just looking at the purchase price for the individual iCal sync plugin for Event Organiser and it is £10 pounds for one site license (our multisite instance counts as one site):
https://wp-event-organiser.com/extensions/event-organiser-ical-sync/

Whereas an all-encompassing developer or pro license, which includes the iCal sync plugin, ranges from £90-120 pounds:
https://wp-event-organiser.com/pricing/

Just listing all the pricing options. I think we could just go for the individual plugin and then I can build out the frontend functionality.

#7 Updated by Matt Gold over 2 years ago

I'm ready to make a purchase whenever you tell me what is needed.

The question of how to subscribe TO a calendar feed came up yesterday in a digital fellows meeting. This would not help that, right -- it would just allow one to pull events in to a calendar from an external source

#8 Updated by Raymond Hoh over 2 years ago

I'm ready to make a purchase whenever you tell me what is needed.

I think we can just go with the £10 pound purchase, located at the bottom of this page:
https://wp-event-organiser.com/extensions/event-organiser-ical-sync/

Matt, can you forward or attach a copy of that plugin to me when you've made the purchase? Thanks!

The question of how to subscribe TO a calendar feed came up yesterday in a digital fellows meeting

If you're talking about subscribing to a group or personal calendar, use the Subscribe link located at the bottom of the events calendar. You should be able to use that URL to subscribe using your calendar application of choice.

This was introduced in #4524. See https://redmine.gc.cuny.edu/issues/4524#note-6 for an example.

I tested this briefly in Thunderbird with the Lightning addon and it worked.

#9 Updated by Matt Gold over 2 years ago

Whoa. So if one uses the ICS to add a Commons calendar to, say, a google calendar, the Google calendar will be updated as new events are added to the Commons calendar? That's fantastic if so!!!

#10 Updated by Raymond Hoh over 2 years ago

Matt Gold wrote:

Whoa. So if one uses the ICS to add a Commons calendar to, say, a google calendar, the Google calendar will be updated as new events are added to the Commons calendar? That's fantastic if so!!!

In theory, that should work. I haven't really tested this in-depth, so if you encounter any bugs, let me know. Things like synchronizing added/deleted events for a specific calendar would be good to test.

#11 Updated by Raymond Hoh about 2 years ago

An update on my end. I've got the new iCalendar importing features working for our group calendar.

First of all, I'm going to apologize ahead of time for the length of this post :)

Secondly, here's a screenshot of the import features that the Event Organiser plugin provides:

As you can see, there is an "iCal Feeds" section, which has a table to manage existing feeds as well as an "Add New Feed" block in the left sidebar. The "Import Events" section allows for manual uploading of an ICS file.

We are going to copy these elements from this Event Organiser admin page to a group's admin area. The ICS import feature comes with the default plugin, while the iCalendar URL import feed comes from the premium extension.


This is what it looks like when you navigate to "Manage > Events" within a group:

There are a couple of additions and modifications I have made for our needs. I'll list them here:

1. The default cronjob for Event Organiser fetches all iCalendar feeds on a singular job. This will not work for our needs due to our usage of the horizontally-scaled cronjob system, Cavalcade (#8987). What I've done is remove Event Organiser's cronjob and set up an individual cronjob for each group. The cronjob for the group is only created when the group admin changes the "Sync Schedule" to something other than the default of "Sync Off".

2. Imported events from an iCalendar feed uses a different background color than the default blue. I thought that this would be a neat feature to distinguish between a regular-created event in the group calendar from an imported one. Currently, the background color for an iCalendar feed is randomly generated and isn't customizable. Let me know if you'd rather I remove this feature.

Also, when you are on an imported event's page, you will see an "Imported from" field. Clicking on the link will filter the group's calendar by that feed.

See the following attachments for an example:

- https://redmine.gc.cuny.edu/attachments/12930/group-ical-feed-meta.png
- https://redmine.gc.cuny.edu/attachments/12931/group-ical-filter.png

3. Activity items for imported events are not created. I made this choice because for large iCalendar feeds, this would flood the group activity stream and potentially send out a bunch of emails. Also, during my testing, activity items would be created for older events, but published at the time of import, which doesn't really make sense for us.

4. I chose to disable the creation of event categories and event venues during iCalendar import. By default, Event Organiser automatically creates event categories and venues if they do not previously exist on the Commons when importing an iCalendar feed.

For the Commons, we decided to disable event categories early in our calendar implementation, so that functionality has been disabled. Venues are a different matter, but since we can't really validate the venue information, I decided to remove this functionality entirely. For example, in the sample Canadian Holidays feed in the screenshots, the created venue is "Canada". That isn't really specific enough.


Some questions I need answers to:

- Deleting an iCalendar feed does not delete the previous imported events from that feed. Do we want to delete these events when deleting an iCalendar feed?

- In the original implementation (see first screenshot), the "Assign events to" field shows all users from a site. This obviously doesn't work within the context of a group, so I changed this to only show group administrators. If there is only one group admin, this field is hidden. Do we also want to list group moderators and/or group members? Or would we want to remove this field entirely?

- In the original implementation (see first screenshot), the "Event Status" field allows the admin to set the status of the event. I've changed this to "Event Privacy" and have removed various options that do not make sense for a group admin to see like draft and pending-review. I did add a default option called "Inherit group privacy", which would make the event public or private depending on the group's privacy setting. However, is this "Event Status" field even needed? Would there be an instance where a private group would want to make an iCalendar feed public to the entire Commons? I'm leaning towards removing this field entirely. Let me know what you think.


This is ready for testing on the development site.

You can access these features when navigating to any group's "Manage > Events" page. Only group admins can access this page.

To test the iCalendar feed feature, you'll need a link to an iCalendar file. If you don't have a link handy, you can get an iCalendar URL from https://www.calendarlabs.com/ical-calendar/. Navigate to a feed you want and right-click Download to copy the URL and paste it in the "Source" field. By default, the feed will not be automatically fetched, but you can either set the "Sync Schedule" to something other than "Sync Off" or manually fetch the feed by hovering your mouse over the feed and clicking on the "Fetch Now" link.

To test the ICS import feature, just upload an existing ICS file from your computer. You can also just download an ICS file from the calendarlabs.com site as well if you don't have one.

Let me know what you think once you've had a chance to play around with it.

(Side note: Boone, I noticed that group pages on the dev site are loading unusually slow, around 10 seconds and more. Could use your help in debugging.)

#12 Updated by Boone Gorges about 2 years ago

Amazing work, Ray! I love the way you've borrowed from the existing plugins but made thoughtful mods to work within the Commons context.

- Deleting an iCalendar feed does not delete the previous imported events from that feed. Do we want to delete these events when deleting an iCalendar feed?

I'd say no. We definitely would not want to link feed deletion with event deletion without providing an opt-out mechanism - this would leave users without a way to unsubscribe while keeping older events from the calendar. In some perfect world we might want an interstitial "Do you also want to delete existing events imported from this feed?", but IMO this is not a dealbreaker and is probably not worth the lift at the moment. It's always possible to manually delete unwanted entries.

- In the original implementation (see first screenshot), the "Assign events to" field shows all users from a site.

I'd say remove. In 99% of cases it will be intended behavior to attribute (a better word than "assign" IMO - "assign" suggests it's a task) to the user performing the import. Actually, in 99% of cases I bet no one will notice who the "author" is anyway :) In any case, in the remaining 1% of cases, this can be changed after the fact on a per-event basis with the 'Organizer' dropdown when editing an event. (Side note: the 'Organizer' dropdown appears to contain usernames that it shouldn't. It's probably pulling from the entire site, rather than group membership.)

- In the original implementation (see first screenshot), the "Event Status" field allows the admin to set the status of the event.

IMO it's not needed. We explicitly decided when first building group calendars that we'd inherit event visibility from the group, disallowing public events in private groups. I think we can continue that for imported items.

(Side note: Boone, I noticed that group pages on the dev site are loading unusually slow, around 10 seconds and more. Could use your help in debugging.)

I'm seeing similar performance but it's pretty inconsistent. Much of the time is TTFB which suggests it's a network problem that we probably can't fix ourselves. I'll keep poking around.

#13 Updated by Laurie Hurson about 2 years ago

Hi All,

I have been testing this today on the dev site here: https://commons.gc.cuny.edu/groups/testing-123-2088349790/events/

A caveat- I have never used any the of the calendar tools so I could be doing this wrong...

- First, I cannot see or do not know where to find the screen that is shown in the "screenshot of the import features that the Event Organiser plugin", This looks more like the backend of a site and plugin and I cannot seem to figure out how this connects or where this would be in the group area. From reading through these tickets, I think maybe this is just a plugin whose features were pulled into the group manage area? If so, okay I get it.

- When I go to use the Manage>Events I cannot get the feed and ics file to relate to one another. By this I mean that when create a feed there is a random "source URL" that becomes associated with it that pulls nothing. Or, if I upload an ics file it works and pulls my iCal events into the group calendar but there no feed asdsociated with it and there is no difference in color between ical and group events, and there is no way to view just the events from the ical file since it was never associated with a feed.
- Test import was trying to create a feed but no ics file attached and there is no option to associate feed with ics file screenshot1
- I made Test 3 import attaching ics file at the same time and ical events are imported but no feed is created. screenshot 2

Ray wrote: "Deleting an iCalendar feed does not delete the previous imported events from that feed. Do we want to delete these events when deleting an iCalendar feed?"
- After doing this multiple times and having to manually delete everything multiple times, yes I think it would be helpful to have a mechanism to bulk delete events. I can imagine that people may want to wipe the calendar clean (so to speak) if they mess up or if they want to update the calendar. Deleting a feed should delete the events associated with it (assuming the feed and ical file/events are synced, which i was unable to achieve)

I am going to wait for clarification on these issues or what I am doing wrong before testing further.
i
Sidenote: I was able to download/import the Commons Group calendar into my ical using the private URL (ticket #5526) no problem.

#14 Updated by Raymond Hoh about 2 years ago

In some perfect world we might want an interstitial "Do you also want to delete existing events imported from this feed?", but IMO this is not a dealbreaker and is probably not worth the lift at the moment. It's always possible to manually delete unwanted entries.

I was thinking about something similar, or maybe adding a "Delete events" link next to the existing "Edit / Delete / Fetch Now" inline links. Based on Laurie's feedback, this looks like something I should look into. I'll see what I can do.

First, I cannot see or do not know where to find the screen that is shown in the "screenshot of the import features that the Event Organiser plugin"

Hi Laurie, this was just to reference what the Event Organiser plugin does and how I was going to use those existing tools for the group's "Manage > Events" page. So you're correct that this admin page can only be accessed by a site administrator.

When I go to use the Manage>Events I cannot get the feed and ics file to relate to one another. By this I mean that when create a feed there is a random "source URL" that becomes associated with it that pulls nothing.

Sorry for not making this clear, but the iCalendar feed import and the ICS file import are two separate features. I just bundled them together in this ticket since work on both features were concurrent.

An iCalendar feed is a URL to an existing iCalendar on the internet. In my screenshot, you can see that my "Canadian Holidays" feed is a link to an iCalendar - https://www.calendarlabs.com/ical-calendar/ics/39/Canada_Holidays.ics. It must be a link to an iCalendar file and not a link that displays events.

iCalendar feeds can be synced to automatically fetch event updates, while an ICS import is a one-time import.

Based on your screenshot, it looks like the Event Organiser plugin does not validate the iCalendar feed's source when you add an invalid link. Looks like I'll have to modify the Event Organiser iCal Sync plugin to fix this.

Or, if I upload an ics file it works and pulls my iCal events into the group calendar but there no feed asdsociated with it and there is no difference in color between ical and group events, and there is no way to view just the events from the ical file since it was never associated with a feed.

Something I forgot to mention is when you upload an ICS file, it does not get a different background color like an iCalendar feed feature.

The reason here is two people can upload an ICS file with the same filename and both could be totally different. One way to figure out if an ICS file is unique is by creating a hash, then I could target that specific ICS file. If others are interested in having a different background color and a filter link for ICS file imports, let me know and I'll look into it.

#15 Updated by Boone Gorges about 2 years ago

Following up on today's meeting: I think we'd like to squeeze in some text/inline-doc changes before we ship this for 1.16.

Ray, I started working on a patch, but I realize now that you're calling the display_feeds() method from the EO plugin, to avoid unnecessary duplication. I wonder if we may simply have to reproduce this method in our own plugin in order to do the necessary modification. Do you think this is a reasonable approach, or should we try doing more gettext juggling?

I'd like to suggest the following changes, which I think will help to clarify the difference between subscribing to a feed URL and performing a one-time .ics import.

1. The page should have two sections (h3): 'Subscribed Calendars' and 'Event Import'.
2. Under 'Subscribed Calendars' appears (a) the feed list table (probably don't need a subheading to explain this, but if others disagree, it could be an h4 "Manage feeds" or something like that), followed by (b) an h4 'Add New Feed' and the add-new-feed form
3. Under the 'Add New Feed' h4 we add an explanatory paragraph along the lines of: "Subscribe to an external calendar by entering the URL of its iCalendar (iCal) feed. Recent events from that calendar will be imported to your group calendar. New events in the feed will be automatically imported every 60 minutes."
4. Remove/hide the sync section, and hardcode it to 60 minutes. I don't see the benefit of allowing users to change this.
5. In addition to the h3/h4 header changes, which should clearly split the page into its two separate parts, maybe add a horizontal rule or something above 'Event Import'
6. Change the text under the 'Event Import' header to something like 'Perform a one-time import of one or more events into your group's calendar. Select an .ics file to get started.'.

Ray, do you think it's doable to make these kinds of changes? Do others agree with the general suggestions?

#16 Updated by Raymond Hoh about 2 years ago

I've made the text and stylistic changes. I've also removed the Organiser, Categories, Venues and Privacy fields for the "Add New Feed" block to simplify the interface.

Here's an updated screenshot:

The following is on my to-do list in order of importance:
- Remove "Sync Schedule" and automate cronjobs for groups to 60 minutes. I'll add the blurb under the "Add New Feed" heading once I've automated the group cronjobs.
- Add "Imported from" field on a single event page for ICS file imports, as well as the ability to view the calendar by ICS file import.
- Validate iCalendar feeds before they are added to the database. Might be tricky with the way I'm piggybacking off the existing EO plugin.
- Add "Delete events" functionality when deleting a feed (this also might be tricky...)

Is there anything I'm missing?

#17 Updated by Boone Gorges about 2 years ago

Ray, these minor changes are hugely beneficial in terms of usability. Thanks so much for tackling them so quickly.

We're right at the point where we should be stopping feature development in order to begin team testing for 1.16 in earnest. I wonder if we might draw a line between the remaining items on your to-do list that can be done very soon - let's say, by the end of the day tomorrow (Tuesday)? - and then opening tickets to handle the other improvements for a future release.

#18 Updated by Raymond Hoh about 2 years ago

- Remove "Sync Schedule" and automate cronjobs for groups to 60 minutes. I'll add the blurb under the "Add New Feed" heading once I've automated the group cronjobs.
- Add "Imported from" field on a single event page for ICS file imports, as well as the ability to view the calendar by ICS file import.

These two items are done.

Depending on if I have some time today, I'll see if I can get to the last two.

#19 Updated by Boone Gorges about 2 years ago

  • Status changed from Testing Required to Resolved

Also available in: Atom PDF