Authentication Agent for Windows v. 7.4.x Challenge lookup fails with "Cannot open challenge cache data key for user <UserID>" and "The server is not operational"
4 years ago
Originally Published: 2021-03-15
Article Number
000044396
Applies To
RSA Product Set: RSA SecurID
RSA Product/Service Type: Authentication Agent for Windows
RSA Version/Condition: 7.4
Platform: Windows
Platform (Other): challenge group for users across 2 Domains
O/S Version: Server 2012 R2
Issue
Authentication Agent for Windows v. 7.4.x Challenge lookup fails with 
"ADsOpenObject failed",
"Failed to open IADsGroup
"Returning: The server is not operational"
[ADSIHelper::getAdsiBindingFlags] Policy: No binding {noformat}"
"Cannot open challenge cache data key for user"

===SIDAuthenticator(LogonUI).log===
2020-01-30 16:11:51.790 9380.9052 [I] [ADSIHelper::getAdsiBindingFlags] Policy: SSL binding
2020-01-30 16:11:51.790 9380.9052 [V] [ADSIHelper::getAdsiBindingFlags] Return
2020-01-30 16:11:51.806 9380.9052 [E] [ADSIHelper::openLdapADsObject<IADsGroup>] ADsOpenObject failed.
2020-01-30 16:11:51.806 9380.9052 [I] [ADSIHelper::openLdapADsObject<IADsGroup>] Returning: The server is not operational.
Object path: LDAP://CN=<Windows_Name>,OU=<ou>,OU=People,DC=<domain>,DC=<org> 
User: LDAP://CN=<UserID>,OU=<ou>,OU=People,DC=<domain>,DC=<org>


 
Cause
All these errors indicate that the group lookup failed before it even started.  The User challenge lookup failed to connect to Active Directory, AD because the Security Policy required an LDAPS secure connection but the Windows platform did not have a Security Certificate.. 
Therefore:
If the Fail open option set, users can logon without a Passcode, with just a Password. 
If the Fail close option set, users can only logon with a Password. 
 
Resolution
If "SSL Binding" is configured for Active Directory, AD connections, ensure that the Windows platform has a Security Certificate, either self-signed or from a Public Certificate Authority, CA.
For SSL binding with AAWin 7.4.x
Two of the key takeaways:
1. The need to import the SSL cert into the Server’s Service Account NTDS\Personal store
2. Use of the ldp.exe tool as a test utility to confirm the setup.

Another possibility from RSA Engineering:
If the “SSL Binding” is set to "Kerberos" option for the AD binding, the AD traffic is encrypted with a key that is derived from Kerberos credentials rather than with the key in the SSL cert, which protects the AD payload against network sniffing without PKI or Certificates.
Workaround
Try a local challenge group with the exclusion for local Admin, and no nested groups especially AD groups, and challenge everyone else.  This would avoid the need to connect to Active Directory, AD in order to lookup Challenged users.
Notes
You should refer to Microsoft documents in order to configure SSL over LDAP (LDAPS) but this link might help you start your search
https://social.technet.microsoft.com/wiki/contents/articles/2979.event-id-1220-ldap-over-ssl-ldaps.aspx