Critical Bug Alert: Magento 2's Message Queue Locks Vanish Hourly, Leading to Duplicate Processing

Critical Bug Alert: Magento 2's Message Queue Locks Vanish Hourly, Leading to Duplicate Processing

As e-commerce migration experts at Shopping Mover, we constantly monitor the Magento ecosystem for critical updates and insights that impact store performance and data integrity. Recently, a significant bug report on the Magento 2 GitHub repository (Issue #41599) caught our attention, revealing a long-standing flaw that could be silently affecting numerous Magento Open Source and Adobe Commerce installations.

The Silent Threat: Message Queue Locks Deleted Prematurely

The issue revolves around the messagequeue_clean_outdated_locks cron job, a crucial component designed to maintain the health and efficiency of Magento's message queue system. This job is intended to periodically clear out message locks that are older than a specified interval (defaulting to 24 hours), ensuring that stale locks don't clog the system while active messages are still protected from duplicate processing.

However, the bug report highlights a critical miscalculation. Instead of deleting locks older than "now minus 24 hours," the cron job incorrectly computes the cutoff date as "now plus 24 hours." This seemingly minor error has a catastrophic consequence: every hour, the job deletes ALL existing entries in the queue_lock table, regardless of their age.

Why This Matters: The Risk of Duplicate Operations

The queue_lock table plays a vital role in Magento's message queue framework, particularly for ensuring "at-least-once" delivery with deduplication. When a message consumer starts processing a message, it creates a lock. If the same message is redelivered (e.g., due to a consumer crashing, a timeout, or a network interruption before acknowledgment), the existing lock prevents it from being processed again, throwing a MessageLockException.

With this bug, however, that protection is severely compromised. If a message is redelivered even a minute after the hourly cleanup cron job runs, its original lock would have been prematurely deleted. This means the message will be processed a second time, leading to silent duplicate operations. Imagine the implications for:

• Asynchronous/Bulk REST API Operations: Duplicate order creation, product updates, or customer data imports.
• Product Attribute Updates: Incorrectly applied changes or redundant processing.
• Inventory Management: Potential discrepancies if stock updates are processed twice.
• Export Processes: Repeated data exports.

These duplicate operations can lead to data inconsistencies, performance bottlenecks, and a general lack of reliability in critical business processes.

The Root Cause: A Single Line of Code

The author of the GitHub issue, iranimij, meticulously traced the problem to a single line of code in app/code/Magento/MessageQueue/Model/ResourceModel/Lock.php within the releaseOutdatedLocks() method:

public function releaseOutdatedLocks()
{
    $date = (new DateTime())->setTimestamp($this->dateTime->gmtTimestamp());
    $date->add(new DateInterval('PT' . $this->interval . 'S'));   // <-- The culprit
    $this->getConnection()->delete($this->getTable(self::QUEUE_LOCK_TABLE), ['created_at <= ?' => $date]);
}

The method uses $date->add() instead of $date->sub(). By adding the interval to the current timestamp, the cutoff date is always in the future, causing the SQL condition created_at <= 'future_date' to match every single row in the table, leading to a full deletion. The bug has been present since the module was open-sourced, with even a unit test incorrectly validating the flawed logic.

Proposed Fix and Impacted Versions

The proposed fix is straightforward, involving a change from add() to sub():

-        $date->add(new DateInterval('PT' . $this->interval . 'S'));
+        $date->sub(new DateInterval('PT' . $this->interval . 'S'));

This bug has been reproduced on Magento Open Source 2.4.9, 2.4.7-p2, and is confirmed to exist in the 2.4-develop branch, indicating its widespread presence across various recent Magento 2 versions.

Community Insight and What's Next

While the GitHub issue currently stands as a detailed bug report without extensive community discussion (as of our review), the clarity and depth provided by the author are exceptional. The issue is marked with a Severity S1, underscoring its critical impact on functionality and data integrity. We anticipate that this well-documented report will lead to a swift resolution in an upcoming Magento patch release.

For merchants and developers, this highlights the importance of staying updated with Magento's core code and being aware of potential silent data integrity issues. If your store heavily relies on message queues for critical asynchronous operations, monitoring for this fix in future Magento updates is paramount. In the interim, understanding this vulnerability can help in diagnosing unexpected duplicate processing issues.