Magento Partial Index Process Visibility
Magento Enterprise 1.13+ uses a "partial index" process that runs in the background as part of a Magento cron job. I am currently struggling to understand what this process is doing.
I understand that a partial indexer works by keeping track of objects for re-indexing through the "changelog" tables in the database ( catalog_category_flat_cl
, catalog_category_product_cat_cl
etc) and then re-indexing those specific items when cron runs.
Magento cron is now running on my server (the same cron process runs for about 3 hours). The table cron_schedule
indicates what is running enterprise_refresh_index
and running concurrently with the output of the above cron process.
However, when I count the rows in the change tables, the values ββare always the same; the number of rows in these tables does not decrease (what would I expect if the partial indexer was processing them?). The n98-magerun tool lists the indices, but none of them are currently being processed:
$ n98-magerun.phar index:list
+------------------------------+-----------------+------+
| code | status | time |
+------------------------------+-----------------+------+
| catalog_product_attribute | require_reindex | 28m |
| catalog_product_price | pending | 0 |
| catalog_url | pending | 0 |
| catalog_product_flat | pending | 0 |
| catalog_category_flat | pending | 0 |
| catalog_category_product | pending | 0 |
| catalogsearch_fulltext | pending | 0 |
| cataloginventory_stock | pending | 0 |
| tag_summary | require_reindex | 1s |
| url_redirect | pending | 0 |
| catalog_url_category | pending | 0 |
| catalog_url_product | pending | 0 |
| catalog_category_product_cat | pending | 0 |
| targetrule | pending | 0 |
+------------------------------+-----------------+------+
There var/locks
are active locks in the directory for each indexer, which doesn't make sense (of course, the lock files have to be removed when the index process ends?
$ ls -lah
total 64K
drwxrws--- 2 www-data www-data 4.0K Sep 23 22:56 .
drwxrws--- 8 www-data www-data 4.0K Sep 10 07:12 ..
-rw-rw-r-- 1 www-data www-data 31 Sep 24 00:10 index_process_10.lock
-rw-rw-r-- 1 www-data www-data 31 Sep 24 00:10 index_process_11.lock
-rw-rw-r-- 1 www-data www-data 31 Sep 24 00:10 index_process_12.lock
-rw-rw-r-- 1 www-data www-data 31 Sep 24 00:10 index_process_13.lock
-rw-rw-r-- 1 www-data www-data 31 Sep 24 00:10 index_process_14.lock
-rw-rw-r-- 1 www-data www-data 31 Sep 23 22:20 index_process_1.lock
-rw-rw-r-- 1 www-data www-data 31 Sep 23 17:53 index_process_2.lock
-rw-rw-r-- 1 www-data www-data 31 Sep 24 00:10 index_process_3.lock
-rw-rw-r-- 1 www-data www-data 31 Sep 24 00:10 index_process_4.lock
-rw-rw-r-- 1 www-data www-data 31 Sep 24 00:10 index_process_5.lock
-rw-rw-r-- 1 www-data www-data 31 Sep 24 00:10 index_process_6.lock
-rw-rw-r-- 1 www-data www-data 31 Sep 24 00:10 index_process_7.lock
-rw-rw-r-- 1 www-data www-data 31 Sep 24 00:10 index_process_8.lock
-rw-rw-r-- 1 www-data www-data 31 Sep 24 00:10 index_process_9.lock
Any suggestions on how I can get more visibility into the partial index process, what it is doing at the moment, and what progress it has made / what remains to be invested would be much appreciated!
source to share
There are also some other tables, check enterprise_mview_*
. Haven't worked with enterprise indexers for a while, but the last time I configured XDebug for the console and analyzed what it does. Or just Mage::log()
everywhere, but I don't recommend it for a real debugger.
Hope it helps you on the right track.
source to share