Add ctl:auditengine action support to libmodsecurity

[alekravch]

Hello.
Suggest to make work “ctl:auditengine” option.
Attached patch implements this.
ctl_auditengine2_patch.txt

Feature ctl:auditengine is not yet implemented, but IMHO it’s quite important and useful feature for logging and debugging, mentioned even in ModSecutiry Handbook (Chapter 4. Logging. Selective Audit Logging)

This example from the book will not work.

4 Logging

Selective Audit Logging

SecRule HIGHEST_SEVERITY "@lt 4"
id:9000,phase:5,pass,nolog,setvar:ip.logflag=1,expirevar:ip.logflag=3600
Finally, we add a rule that detects the flag and forces logging for all the requests from the flagged IP address:
SecRule IP:logflag "@gt 0"
id:9001,phase:5,pass,nolog,ctl:auditEngine=On

Implementation Proposal

This feature should operate on a transaction scope, not interfering with the global rule set.
So we store AuditLogStatus in the transaction and then use function AuditLog::getActualStatus to get AuditLogStatus either from the transaction scope (if set), or from the global rule set (unless it’s not set on the transaction scope). This function is used in AuditLog::init and AuditLog::saveIfRelevant.

Aleksey Kravchuk

[martinhsv]

Hi @alekravch ,

You are correct that ctl:auditengine has not been implemented in v3 thus far.

Note that a change in the 3.1 development branch was made to produce an error message if that action is used rather than simply failing silently ( 0466a5a ). That may or may not eventually proceed to v3/master.

It is certainly possible for ctl:auditengine to become a supported action in v3. However, there has been a deliberate approach in v3 to not always implement functionality from v2 just because it exists in that version. In at least some cases, the use case for some functionality is dubious at best (although it may have been useful years ago) -- see for example SecArgumentSeparator.

So, out of curiosity, do you have particular use case(s) where you have wanted to use ctl:auditEngine, and where other tools like 'log/nolog', and 'ctl:auditLogParts' have proven inadequate?

[alekravch]

Hi,

The problem with ctl:auditLogParts is that it can't disable auditlog altogether - A and Z parts are mandatory and can't be turned off.

Downsides (in my opinion) are:

• Littering audit log with extra unneeded info, when log only for the specific situation is necessary.
• Possible performance impact on heavy load system - audit engine is init and A and Z parts can be logged for all requests.
• Extra disk storage usage.

It seems to be more straightforward and reliable to disable auditengine globaly and turn it on conditionally if one wants to perform selective audit logging for specific situations only.

Agree that this feature is not absolute necessity, but it seems to be useful and easy enough to implement.
This approach looks logical and practical (as for me)

[martinhsv]

Agreed that the implementation is not particularly complex. I guess I'm still wondering about the usefulness. In other words, what are the underlying use cases for which this is considered desirable by current users of ModSecurity.

A couple of examples:
A) An automated health check HTTP request to my server is causing false positives and filling the log. If the IP address is that single known address, I want to turn off the audit log for requests from that IP address.
B) I trust a particular user's IP address, and want to turn off audit logging for those requests
C) Some other use case?

In the case of (A), I would normally suggest that it is safer to disable only the particular rules that are causing false positives. For either case, one could also check for the IP address and the disable the engine (ctl:ruleEngine=Off) for those requests -- unless of course there is a good reason to want to still execute the rules but to not have any audit logging from those executed rules.

Just wanting to better understand the need ...

[airween]

C) Some other use case?

I don't have any useful instance, therefore I can't check it, but as I remember, once we ran into this problem:

• Nginx as HTTP proxy
• libmodsecurity3
• back end server sends the response with 4XX code (eg. 404)

The proxy logged these requests, we couldn't turn off the auditlog engine, and those log entries filled the disk (without any ModSecurity warning). In that case, that would have been nice this feature.

Then I made a very minimalist patch (partially implemented the auditEngine), that worked as well: we could make a rule which used the RESPONSE_STATUS, that solved our problem.

[alekravch]

Hi,

For example I don't need audit log feature for normal requests and I have to keep audit log turned off due to the reason of performance or disk storage.

Then I have some client experiencing problem and to debug this problem I want to capture all activities (via audit log) for this specific client only. So it would be nice to have a feature to turn audit log on locally for the specific condition only.

Or I have some suspicious activity and I want to make a detailed investigation, capturing appropriate traffic. So again it would be nice to turn audit log on for the specific condition only.

[martinhsv]

All right; I'm sufficiently convinced that this would be useful enough.