Recently, some customers with large deployments [30k or more users] saw problems after updating to AM 8.9 Patch 1.

The readme for AM 8.9 P2 listed a single Jira bug identifying the fix to these performance problems.

AM-66252: Fixed an issue where a misconfiguration in the jndi.properties file due to a third party library upgrade caused performance issues.

The performance issues associated with AM-66252 include the following;

1. Security Console slow performance or stops working [Invalid Request] 

2. RADIUS Service stopped accepting authentications on some AM servers, with loss of radius logging. This RADUIS failure was also not flagged by Critical Event Notification emails

Symptoms included; 'Stuck Threads', "work queue full", 'deadlock detected', Connection reset by peer

and '500 Internal Server Error'

===biztier.log===

[STUCK] ExecuteThread: 'weblogic.kernel.Default (self-tuning)' has been busy for "670" seconds working on the request "Http Request Information: weblogic.servlet.internal.ServletRequestImpl@56aabec7[POST /mfa/v1_1//authn/initialize]

===System Log Report===

WARN 26023 Socket Connection Error Error occurred while communicating with remote host “work queue full. Maximum”:“200”. Socket is locally bound to “nothing”: “Could not accept new work.”

===ReplicaReplication.log===

ERROR [ApplyP2R latestAppliedSweepId:

ApplyP2R.executeBatchUpdate(202) | ,,,unable to execute batch update when applying primary changes

was aborted: ERROR: deadlock detected

while deleting tuple (153,5) in relation "am_token_dg"

===PrimaryReplication.log===

FATAL [ApplyR2P 

Last Received (from replica) Chgset Id: <ID_#>

Next Chgset Id To Apply: <ID_#+1>

... We have encountered a deadlock which cannot be resolved

 Deadlock detected in an apply2p function. ERROR: deadlock detected

===postgres_<date>.log===

<date> 03:22:46.655 GMT could not receive data from client: Connection reset by peer

<date> 03:22:47.748 GMT received fast shutdown request

<date> 03:22:47.773 GMT aborting any active transactions

===rsaserv.logs===

<date> 6:26:05 AM UTC RSA RADIUS Server is running

<date> 6:25:50 AM UTC RSA RADIUS Server Operations Console is running

<date> 6:24:58 AM UTC RSA RADIUS Server not running

<date> 6:24:39 AM UTC RSA RADIUS Server Operations Console is running

===radius.log===

<date> 00:14:40 2026 Error: rlm_perl: Exception when calling rsa_securid_mfa_call: Exception in rsa_securid_mfa_first_step_process_initialize when calling UserApi->initialize: API Exception(500): Internal Server Error 10.5.1 500 Internal Server Error

The Root Cause Analysis focused on four issues, 

 1. The legacy and EOPS Authentication Agent for Windows, AAWin ver. 7.4.x, was using the auto-registration service on TCP port 5500. 

 2. There were also a lot of RADIUS Authentications

 3. A misconfiguration in the jndi.properties file

 4. Opportunistic row locking failure under load with "Delayed Response Manager", 'DRM' 

The situation was fixed by a manual restart of the RADIUS Service. Sometimes RADIUS restart needed to be done more than once. 

AM 8.9 Patch 2 prevents the problem from coming back.

Root Cause: 

After AM 8.9 P1 update there was a deadlock in the database [ERROR: deadlock detected while deleting tuple] while applying replication updates [ApplyP2R..unable to execute batch update, FATAL [ApplyR2P]. Likely cause was from "Delayed Response Manager", 'DRM' Opportunistic locking row failure under load, originally reported in bug AM-66252.

This left RADIUS unable to update the database through its MFA agent database connection and resulted in RADIUS 'fast shutdown request'. A previous RADIUS fix auto-Restarts RADIUS when it is down. But when RADIUS restarted this again lead to another 'fast shutdown request' with RADIUS thrashing up and down until a manual restart of RADIUS broke the cycle.

Engineering found at least 1 contributing cause was mis-configuration in jndi.properties file because of a third party library upgrade, aggravated by too low of a time out setting in third party library. Engineering also observed some socket leakage, but that was not the root cause of the system hanging and being slow.

Fix: is to update to Authentication Manager, AM 8.9 P2 or later.

https://community.rsa.com/s/product-download/a9GPO000000Tl5C2AS/rsa-authentication-manager-89-patch-2-update-download

https://community.rsa.com/s/product-download/a9GPO000000mX3J2AU/rsa-authentication-manager-89-patch-2-hotfix-1-update-download

Engineering will continue investigate the timeout issue, socket leak and DRM failure and provide fix in later patches if needed.

Work-arounds, Request for Enhancements, RFEs and other options

Issue: Critical Events only sent for RADIUS change failures

1. Manually restart RADIUS in Operations console. 

2. Request for Enhancements, RADIUS RFE AM-50321 - RADIUS service not responding, not up. 

3. Open a Support Case and ask for the Internal Knowledge Base, KB 'How to create custom SNMP objects showing the status of the services on RSA Authentication Manager 8.1'