Authentication agent for Windows, AAWin autoregistration fails after update to RSA Authentication Manager 8.4 Patch 14

[#1]

Article Number

000039529

Applies To

RSA Product Set: RSA SecurID
RSA Product/Service Type: Authentication Agent for Windows
RSA Version/Condition: AAWin 7.4, 7.4.4, AM 8.4 P14, AM 8.5 P1
Platform: Windows, Linux
 

Issue

Windows Agent Auto-registration fails after upgrade to Authentication Manager, AM 8.4 P14 or AM 8.5 P1, using newly downloaded Authentication Manager Server Certificate (server.cer) from new version of AM

Symptoms:
===autoreg log from agent====
Handshake failed ssl_error  
Handshake failed: SSL Protocol Failure File
Handshake failed sdErr <1> 
errCliUtlOpenServerConnection(): 
SDSSLPerformHandshake failed with error code SD_ERR_SSL_HANDSHAKE_FAILED <20023>

===imsTrace.log===
Failover list: |AAAAAgAAAAFyc2FyZXBsaWNhLm1ja3NkYy5jb20AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA...

WARN, rsa.domain.com,,,,SSL handshake fails for the socket [ServerMode...
CipherSuite: SSL_NULL_WITH_NULL_NULL, Protocol: SSLv3]...
javax.net.ssl.SSLException: Fatal Alert received: Certificate Unknown...

Cause

Corrupted text in the server.cer files from AM 8.4 P14 or AM 8.5 P1,

Resolution

Update to AM 8.5 P3, when it is released.
Or 
Instead of downloading server.cer from Security console, use WinSCP to copy it from Linux 
/opt/rsa/am/config/src/resources/certs/server.cer from the appliance.
Or
Use the older version of server.cer, downloaded from AM 8.4 P13 or earlier, making sure that this server.cer is equivalent and not from an earlier license or different instance.
 

Workaround

Edit the AM 8.4 P14 server.cer file (e.g. with NotePad++) to remove specific text starting with: "<meta http-equiv" and going all the way through to the end of the server.cer file.

Image description