Project

General

Profile

Actions

Bug #6734

closed

Bug #6731: Out-of-control queries on production server

PressForward RSS fetching requires very large number of queries

Added by Boone Gorges over 7 years ago. Updated almost 7 years ago.

Status:
Rejected
Priority name:
Normal
Assignee:
Category name:
WordPress Plugins
Target version:
-
Start date:
2016-11-15
Due date:
% Done:

0%

Estimated time:
Deployment actions:

Description

The PressForward plugin is built around an RSS aggregator that fetches feed items on a wp-cron job. The fetching process is very resource-intensive. I've tracked a large number of requests that have required more than 1000 database queries. One such query log is attached (it stops at 1000 queries because this is what my logging tool is built to do - I don't know how much longer it goes).

A few preliminary thoughts:
- This does not appear to be a case of a query loop. The queries are generally distinct.
- The process of fetching an RSS item and inserting it into the WP database is extremely resource-intensive. By my count, each item requires about 46 db queries, about half of which are INSERT/SELECTs
- I seem to remember (from time spent working on earlier versions of PF) that feeds were processed in batches/chunks, but the code now suggests that this may not be the case - it looks like all subscribed feeds are checked during every cron job
- The last update to PF on the Commons was on Oct 21 (4.0.5 > 4.1.2). It's possible that there was something in that update that increased overhead, or perhaps this was the version where PF stopped processing in batches
- Some recent events at the GC have pushed the use of PF, so it's possible that this was a latent issue that became evident only through more extensive use

I'm going to reach out to Aram, the lead developer of PF, to see if anything comes to mind that might help us narrow in on any potential problems.


Files

pf.txt (1.34 MB) pf.txt Boone Gorges, 2016-11-15 09:46 PM
Actions #1

Updated by Boone Gorges over 7 years ago

  • Description updated (diff)
Actions #2

Updated by Boone Gorges over 7 years ago

Actions #3

Updated by Boone Gorges over 7 years ago

  • Target version set to 1.11

Aram let me know that PF ought to be processing in chunks. I'll continue to investigate.

Actions #4

Updated by Boone Gorges almost 7 years ago

  • Status changed from New to Rejected
  • Target version deleted (1.11)

I spent some more time with this a couple weeks ago, and I'm afraid it's beyond the scope of what I can do for the Commons to cut this number down. Since it's a relative edge case for us, I'm rejecting the task.

Actions

Also available in: Atom PDF