Project

General

Profile

Bug #14908

Stale object cache on cdev

Added by Raymond Hoh about 1 month ago. Updated about 4 hours ago.

Status:
New
Priority name:
Normal
Assignee:
Category name:
Performance
Target version:
-
Start date:
2021-10-27
Due date:
% Done:

0%

Estimated time:

Description

This is a note for Boone that I've experienced some weird instances on cdev with stale object cache.

I noticed this in the admin area yesterday because the BuddyPress update routine kept running on each page load. The value for bp_get_option( '_bp_db_version' ) kept returning the version before BuddyPress v8.0 (12385), whereas the version in the DB was the latest one (12850). As a result, the BP emails kept on installing on each admin page load. See https://commons.gc.cuny.edu/wp-admin/edit.php?post_status=trash&post_type=bp-email where I've cleaned up all instances this has occurred.

Deleting the options object cache fixed this -- wp_cache_delete( 'alloptions', 'options' ).

This also fixed an issue just now where the homepage was throwing a fatal error because the CAC Home Creation plugin was not activated because the object cache was referencing the older 'active_plugins' option.

Has something changed to our memcached set up that might explain the stale object cache?


Related issues

Related to CUNY Academic Commons - Bug #14986: Commons Site Down?Resolved2021-11-23

Related to CUNY Academic Commons - Support #14994: Clear Cache on CDEVIn Progress2021-11-28

History

#1 Updated by Boone Gorges about 1 month ago

Weird. I recall noticing some oddness in the past with cdev's Memcached behavior, but I can't recall if it's something we ever pinned down.

I just looked over the WP cache config for cdev (cac-env-config.php) and it looks like we're set up to use the same Memcached servers as the production site. And the cache is basically working - I can see in Query Monitor that it's hitting the cache. So there must be something more fine-grained happening. But I don't have any guess what it is.

Let's use this ticket to collect more info if and when we see this happen again.

#2 Updated by Raymond Hoh 13 days ago

#3 Updated by Raymond Hoh 8 days ago

#4 Updated by Raymond Hoh about 4 hours ago

I decided to do some more research about this and came across this documentation link about the size of the alloptions cache while working with memcached: https://docs.wpvip.com/technical-references/code-quality-and-best-practices/working-with-wp_options/#h-identify-and-resolve-problems-with-alloptions

The link outlines that the alloptions cache should be relatively small (around 1MB) to prevent performance issues. On cdev, the wp_1_options table is 13MB (see wp db size --tables --size_format=mb). However on production, the size is 16MB, but we haven't seen many reports of cache problems on production.

Anyway on cdev, I've deleted some older options that were prefixed with 'bbpress_live_cache_' (from an older version of bbPress), 'jpsq_sync' (from Jetpack), and 'bp_system_report' and this dropped the size of the wp_1_options table to 8MB. Still some more work to do.

#5 Updated by Raymond Hoh about 4 hours ago

Update: running OPTIMIZE table wp_1_options; in MySQL after clearing the older options dropped our wp_1_options DB size to 1MB.

I think we should purge the older options on production as well.

Update 2: just did the same thing on production and the wp_1_options DB size is down to 2MB.

Also available in: Atom PDF