Critical OpenSearch Pagination Bug in Magento 2.4.x: Unpacking Performance Hits and Error Logs

As e-commerce migration experts at Shopping Mover, we constantly monitor the Magento ecosystem for critical insights that can impact store performance, stability, and development efforts. A recent GitHub issue (#41770) has brought to light a significant bug within Magento 2.4.x's OpenSearch integration, specifically concerning pagination for large product catalogs. This issue, identified in production environments, can lead to misleading search results, critical error logs, and substantial performance degradation.

The Problem: Paging Beyond the Limits

The core of the issue manifests when a user or an application (via GraphQL) attempts to access a search result page that is far beyond the actual last available page. For instance, if a search term yields 10,800 products, resulting in 540 pages (with 20 items per page), requesting currentPage: 1001 correctly triggers an error indicating the page is out of bounds. However, requesting currentPage: 1002 or any significantly larger value (e.g., 100000000) results in unexpected and problematic behavior:

  • Misleading Client Response: The GraphQL API returns {"products":{"total_count":0,"page_info":{"total_pages":0},"items":[]}}, effectively telling the client that no results were found, which is incorrect and confusing.
  • Critical Error Logging: Magento's exception.log is flooded with CRITICAL entries: Exception: Warning: Trying to access array offset on false in app/code/Magento/OpenSearch/SearchAdapter/Adapter.php on line 127. This can obscure genuine issues and consume valuable logging resources.
  • Performance Degradation: Before failing, the system unnecessarily executes multiple OpenSearch queries. For 10,800 hits, it runs three queries, each fetching up to 10,000 hits. With a catalog of 1 million products, this could escalate to approximately 100 such queries for a single out-of-bounds request, severely impacting server and OpenSearch performance. The storefront search also triggers this behavior, logging the CRITICAL entry multiple times per request.

Unpacking the Technical Root Cause

The issue stems from the logic within Magento\OpenSearch\SearchAdapter\Adapter::query(). This method is designed to handle large result sets by using OpenSearch's search_after mechanism, effectively "walking" through results in chunks (up to maxPageSize, which is 10,000 by default) to reach the requested offset ($request->getFrom()).

The critical flaw lies in the while loop responsible for this paging:

while ($processed < $request->getFrom()) {
    $query['body']['size'] = min($request->getFrom() - $processed, $maxPageSize);
    $processed += $query['body']['size'];
    $rawResp>query($query);
    $lastHit = end($rawResponse['hits']['hits']);
    $query['body']['search_after'] = $lastHit['sort'];
}

As the issue author, lbajsarowicz, meticulously explains, this loop fails to check if a chunk returned by OpenSearch is empty. When the $request->getFrom() offset extends beyond the total number of available hits, $rawResponse['hits']['hits'] becomes empty, causing end($rawResponse['hits']['hits']) to return false. Subsequently, the attempt to access $lastHit['sort'] (i.e., false['sort']) triggers the "Trying to access array offset on false" warning, which Magento's error handler converts into a CRITICAL exception.

Compounding the problem, the loop continues to page through the entire result set even after it has passed the last available hit, leading to the excessive OpenSearch queries.

Impact on Adobe Commerce and Open Source Merchants

For merchants, especially those with extensive product catalogs, this bug presents a dual threat: performance bottlenecks and obscured error monitoring. Unnecessary OpenSearch queries consume valuable resources, slowing down legitimate search requests and potentially impacting overall site responsiveness. The deluge of CRITICAL errors in logs can make it difficult for development teams to identify and address genuine, pressing issues.

This bug was initially identified in a production environment via New Relic, highlighting its real-world impact on live Magento stores. It affects Magento 2.4-develop, 2.4.8, and later versions, making it a relevant concern for many Adobe Commerce and Open Source users.

Proposed Solution and Next Steps

The issue author suggests a straightforward fix: stopping the loop as soon as an OpenSearch chunk returns fewer hits than requested. This would correctly indicate that the offset is out of range, allowing the system to return an empty result with the accurate total count, and crucially, preventing both the critical error and the wasteful additional queries.

As of the time of this insight, the issue is marked "ready for confirmation" on GitHub, with no further comments or official patches released. We encourage Magento developers and merchants to monitor this issue closely on the official Magento 2 GitHub repository for updates and official solutions. Understanding and addressing such core integration bugs is vital for maintaining robust and high-performing Magento stores.

Start with the tools

Explore migration tools

See options, compare methods, and pick the path that fits your store.

Explore migration tools