Mitigator Memory Increasing Daily in RSA Web Threat Detection 5.1
2 years ago
Originally Published: 2016-03-24
Article Number
000058740
Applies To
RSA Product Set: Web Threat Detection
RSA Product/Service Type: Mitigator
RSA Version/Condition: 5.1
Platform: CentOS
 
Issue
Customer is complaining that the Mitigator memory usage is too high and not being released, they have to resort to restarts of the service to free up the memory. 

Example customer statement:
Consumption of machine memory is increasing daily and causing alerts issue occurs when it is longer than 2 weeks v5.1.1.5
Error Message: none Recent
Changes: none
Business Impact: SEV 2, partner requested, unknown impact 
Tasks
Gather some information from the Customer.  It might be best to have a Webex session and do the following:
1. View the Varz Graph, looking at memory utilization.
2. Go to the Schema and look at the 'Mitigator' configuration, look for WindowSize and check the settings.
    If they are not seen, push 'Edit' to see if the default setting is there. (This would only appear on Edit if it was never changed from default in the past.)
3. Go to rules, and ask the Customer if they are using a lot of rules with wildcards ' * '.
   This tends to cause extra memory consumption due to the need to keep all pages in memory for each attribute, i.e., for each click.   
4. Ask the Customer if they have a lot of testing going on in their environment that may cause spikes of many hits on only one or two IP addresses. 
 
Resolution
1.  The Mitigator has a sliding window for memory and this is set by default to a 24 hour window size and one 'pane'.
     Depending on what is seen for the schema in the configuration manager, they may have the default setting of 24 hours and 1 'pane'.

     Our R&D research has shown that this default setting can be 'tuned' for improved response,
     e.g., seeing memory being released and not growing as large day to day. 

     (Take a look at the VARZ graph for the Mitigator memory and look for steady trends in increased memory. This would indicate
     that default settings are in place as not enough memory is being released with the current setting. There may be sharp decreases
     when the service is restarted, which releases memory, but it is still followed by a steady rise, as just release the memory does not resolve the issue.) 

     If a change is needed, it should be gradual, made in small increments and observed over several days. Recommend a window change to 12 hours
     and keep the setting of 1 pane. Tell the Customer that they should see more memory being released after 3 or 4 days and steadier,
     more even day to day utilization, rather than a sharply rising graph.

     2. Ask the Customer to continue to work on the Rules and IP Filters.

     These steps will take time for the Customer to research, change and observe for improvement.  
     You may be able to close the case at this time, and have them reopen if needed. 

Related Articles

Trending Articles

Don't see what you're looking for?