Bug #8930
closed
Throttle Jetpack batch routines
Added by Boone Gorges about 7 years ago.
Updated over 6 years ago.
Category name:
WordPress (misc)
Description
Previously: #6731.
I've been doing some logging after recent performance problems, and have found a couple different kinds of runaway requests involving Jetpack.
1. Jetpack takes a really long time to generate a sitemap, especially when the site has lots of images. This happens a lot on historyprogram, for instance.
2. Jetpack indexes site posts to wordpress.com on certain setups, which does not play nicely with plugins that generate huge amounts of posts. The cases I've seen are related to the use of wp-rss-multi-importer on gcenglish.
To mitigate the first item, I've decreased Jetpack's sitemap batch sizes. The following lines are now in cac-env-config.php (untracked):
// Limit size of Jetpack sitemap generation routine.
define( 'JP_SITEMAP_UPDATE_SIZE', 10 );
define( 'JP_SITEMAP_BATCH_SIZE', 100 );
define( 'JP_SITEMAP_INTERVAL', 24 * 60 * 60 );
- Related to Bug #6731: Out-of-control queries on production server added
- Related to Bug #6737: wp-rss-multi-importer cron job requires huge number of DB queries added
Jetpack is supposed to be skipping the syncing of rssmi_feed_item post type, but is not for some reason. I've set up a debugging tool to figure this out.
- Target version changed from 1.12.3 to 1.13
The debugging tool suggests that Jetpack is not, in fact, indexing the items. Moving to a future release for further debugging.
- Assignee set to Boone Gorges
Maybe not totally related, but Jetpack is causing problems with the introduction of a new cron system. See #8987.
It's possible that Jetpack is the resource hog for cron jobs that's at the root of recent performance problems. As an experiment, I've temporarily disabled Jetpack's cron-based syncing system: https://wordpress.org/support/topic/jetpack-sync-killing-the-db-2/ I'll check back in a day to see if there's a difference.
So, about a day later, and there have been no outages on the Commons. This is not conclusive - sometimes we go several days without - but it suggests we may be onto something.
Ray, I'm adding you to this ticket because I'm going to leave my disabling code in place, and I'm going away. See wp-content/mu-plugins/bbg-toolbox.php, at the very top. To be honest, I'm not 100% sure what it means when we disable sync-via-cron. The wording makes me think that Jetpack continues to sync, but only at the time that new content is created. (For example, the sync-modules-posts.php file hooks to transition_post_status.) So, maybe this is something we can leave turned off indefinitely. But if there are reports from Commons users that wordpress.com integration is broken in some way, it might be necessary to reenable cron-sync. In either case, let's monitor over the next week or two.
- Status changed from New to Resolved
- Target version changed from 1.13 to 1.12.13
A brief update that, based on my observations, outages have been much less frequent since this change was implemented. I can recall maybe half a dozen instances in the last month, rather than the several-per-week that was typical of the last few months. At the same time, I haven't heard reports that Jetpack/wordpress.com connections have been adversely affected. As such, I'm going to go ahead and add my off-toggle to the codebase: https://github.com/cuny-academic-commons/cac/commit/702a173e00d35a539d46b031deea3307d1641af1 This has been deployed immediately as a hotfix. This should make moot the question of "throttling", since we're now throttling it down to zero.
- Related to Bug #10051: Tweets and email notifications are not being generated from posts on OpenAtCUNY site added
Also available in: Atom
PDF