Added Response Phase for Nginx by chaizhenhua · Pull Request #34 · owasp-modsecurity/ModSecurity

chaizhenhua · 2013-01-26T14:52:58Z

Added Response Phase for Nginx

brenosilva · 2013-01-26T19:24:38Z

Hello,

Request body looks good. I'm doing some tests with response body and headers all looks fine. Just notice two stuffs:

1 - When the response body is compressed we use SecDisableBackendCompression On to decompress the data to the backend in reverse proxy. This is a feature for apache and i don't know if we can do something similar to Nginx ?

2 - When we finish the transaction and logging to audit log i'm seeing:

[24/Jan/2013:02:58:00 --0800] [standalone/sid#87528e0][rid#87a2808][/acao.php][4] Audit log: Logging this transaction.
[24/Jan/2013:02:58:00 --0800] [standalone/sid#87528e0][rid#87a2808][/acao.php][1] Audit log: Failed to lock global mutex: Permission denied
[24/Jan/2013:02:58:00 --0800] [standalone/sid#87528e0][rid#87a2808][/acao.php][1] Audit log: Failed to unlock global mutex: Permission denied

Any idea to fix it ?

Topics 1 and 2 should go to another pull. I will apply and close this.

Thanks

Breno

Added Response Phase for Nginx

brenosilva · 2013-01-27T14:32:15Z

Hello,

Deny action is not working in request body phase. Looks like modsecProcessRequestBody is called but not sure if its return values is being checked ?

Thanks

Breno

brenosilva · 2013-01-27T15:07:56Z

Also in ngx_http_modsecurity_cody_filter. Looks like rc value is not right in the end of the functions. It is just a guess because looks like we should expect a HTTP code but it looks like not a HTTP code in the end.

If it is true and fixed it.. we just need to add something like:
if (rc < NGX_HTTP_SPECIAL_RESPONSE || rc >= 600) {
rc = NGX_HTTP_INTERNAL_SERVER_ERROR;
} else if (rc == NGX_HTTP_FORBIDDEN) {
rc = NGX_HTTP_FORBIDDEN;
}

I'm printing the rc value in debug log and it is 20014. Is it expected ?

and get deny action working for response body.

brenosilva · 2013-01-27T16:00:06Z

Just tested without the patch and the deny action worked fine in request body:

2013/01/24 23:55:07 [error] 3081#0: [client 192.168.0.102] ModSecurity: Access denied with code 403 (phase 2). Pattern match "price" at REQUEST_BODY. [file "/usr/local/nginx/conf/modsecurity.conf"] [line "223"] [id "113"] [hostname "standalone"] [uri "/acao.php"] [unique_id "12345"]
2013/01/24 23:55:07 [debug] 3081#0: *1 ModSecurity: status: 403, need action
2013/01/24 23:55:07 [debug] 3081#0: *1 http finalize request: 403, "/acao.php?" a:1, c:2
2013/01/24 23:55:07 [debug] 3081#0: *1 http special response: 403, "/acao.php?"
2013/01/24 23:55:07 [debug] 3081#0: *1 http set discard body
2013/01/24 23:55:07 [debug] 3081#0: *1 HTTP/1.1 403 Forbidden^M

Looks like something in the patch in not OK. My guess is something related to return codes (rc).

chaizhenhua · 2013-03-05T12:35:11Z

hi, brenosilva.

did the problem solved?

brenosilva · 2013-03-05T12:39:48Z

Hello chaizhenhua,

Looks like "yes". I was planning to release 2.7.3 soon. However i will be on vacation next week and i was contacted to release it just when i come back from vacation at 21th Feb. So we need to wait a little bit more.

Thanks

chaizhenhua · 2013-03-06T05:03:16Z

hi, brenosilva

I have reprodue the error, with following config:
'''
SecRequestBodyAccess Off
SecRule REQUEST_BODY "@contains attack" "phase:2,log,deny,id:1"
'''
but i can't get rc 20014 when debug. did your config look like this? did you print it in ngx_http_modsecurity_body_handler? 20014 means APR_EOF. if rc == APR_EOF we should correct it then return.

chaizhenhua · 2013-03-06T05:10:51Z

-rc = modsecProcessRequestBody(ctx->req);
+
+if(modsecIsRequestBodyAccessEnabled(ctx->req))
+rc = modsecProcessRequestBody(ctx->req);

may be we can move this if condition to ngx_http_modsecurity_handler like this so that we can skip read client request body phase.

-if (r->method == NGX_HTTP_POST) {
+if (modsecIsRequestBodyAccessEnabled(ctx->req)
+&&r->method == NGX_HTTP_POST) {
rc = ngx_http_read_client_request_body(r, ngx_http_modsecurity_body_handler);

chaizhenhua · 2013-03-06T09:47:47Z

1 - When the response body is compressed we use SecDisableBackendCompression On to decompress the data to the backend in reverse proxy. This is a feature for apache and i don't know if we can do something similar to Nginx ?
if we copy request_rec's out_headers to ngx_http_headers_out_t, i think it might work.

brenosilva · 2013-03-06T14:05:06Z

Just added a patch. Looks like fixed the issue when RequestBodyAccess Off.

Related to SecDisableBackendCompression On I still don't know how to do it in Nginx. Did you test this ngx_http_headers_out_ idea ?

chaizhenhua · 2013-03-07T01:08:52Z

if SecDisableBackendCompression set to On,
modsecurity unset "Accept-Encoding" and "TE" header in after request body phase, and then send to backend,
when recevied from backend, modsecurity recover "Accept-Encoding" and "TE" header. and nginx will compress body for us. is it right?

brenosilva · 2013-03-07T02:12:47Z

Yes. This is how it works with Apache. However i think we need to work with request_rec nginx version struct.

brenosilva · 2013-04-04T13:00:59Z

Hello chaizhenhua,

Looks like we have an issue #56
The user said it is seeing a PCRE version warn message. So we still don't know if the 100% cpu issue can be caused by some pcre rule or not. I ask this info, but we will have it later.

I cannot reproduce the issue. Let me know if you can.

Thanks

Breno

brenosilva · 2013-04-04T14:07:07Z

Looks like loading all CRS it trigger the issue

brenosilva · 2013-04-04T14:16:09Z

I can reproduce it just one time. Looking at error.log ... a few rules was executed in a kind a loop (960017 and 960032):

2013/04/04 07:09:26 [error] 2608#0: [client 192.168.0.101] ModSecurity: Warning. Pattern match "^[\d.:]+$" at REQUEST_HEADERS:Host. [file "/usr/local/nginx/conf/owasp-modsecurity-crs-2.2.6/base_rules/modsecurity_crs_21_protocol_anomalies.conf"] [line "98"] [id "960017"] [rev "2"] [msg "Host header is a numeric IP address"] [data "192.168.0.102"] [severity "WARNING"] [ver "OWASP_CRS/2.2.6"] [maturity "9"] [accuracy "9"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/IP_HOST"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"] [tag "http://technet.microsoft.com/en-us/magazine/2005.01.hackerbasher.aspx"] [hostname ""] [uri "/index.html"] [unique_id "12345"]
2013/04/04 07:09:26 [error] 2609#0: [client 192.168.0.101] ModSecurity: Warning. Pattern match "^[\d.:]+$" at REQUEST_HEADERS:Host. [file "/usr/local/nginx/conf/owasp-modsecurity-crs-2.2.6/base_rules/modsecurity_crs_21_protocol_anomalies.conf"] [line "98"] [id "960017"] [rev "2"] [msg "Host header is a numeric IP address"] [data "192.168.0.102"] [severity "WARNING"] [ver "OWASP_CRS/2.2.6"] [maturity "9"] [accuracy "9"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/IP_HOST"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"] [tag "http://technet.microsoft.com/en-us/magazine/2005.01.hackerbasher.aspx"] [hostname ""] [uri "/index.html"] [unique_id "12345"]

chaizhenhua · 2013-04-04T14:16:51Z

did it relate to some rules in response headers/body phase?

brenosilva · 2013-04-04T14:23:06Z

I'm not sure any more. Looks like the multiple execution of OK because it is in different uris. So Shouldn't be the problem. And those rules are not in phase 3 and 4.

brenosilva · 2013-04-04T14:30:17Z

It is hard to reproduce. I cannot reproduce it anymore. Not sure if it could be a uninitialized variable problem ?

brenosilva · 2013-04-04T14:40:58Z

Ignore the slow request info. I notice my backend disk was full. Maybe this is the reason

brenosilva · 2013-04-05T20:49:23Z

Hello chaizhenhua,

Are you be able to reproduce the 100% cpu issue ?

Thanks

chaizhenhua · 2013-04-06T01:55:38Z

I can not reproduce and have no idea about it.

chaizhenhua added 2 commits January 26, 2013 22:17

dos2unix standalone/api.c

48bd099

Added Response Phase for Nginx

972d9e2

chaizhenhua mentioned this pull request Jan 26, 2013

Improved split Request to headers and body phase, use apr_brigade to deal with request body #29

Closed

brenosilva added a commit that referenced this pull request Jan 26, 2013

Merge pull request #34 from chaizhenhua/remotes/trunk

ae0bee0

Added Response Phase for Nginx

brenosilva merged commit ae0bee0 into owasp-modsecurity:remotes/trunk Jan 26, 2013

majordaw mentioned this pull request Sep 26, 2017

double free in ModSecurity3 'reading_logs_via_rule_message' example and OWASP CRS #1573

Closed

Uh oh!

Conversation

chaizhenhua commented Jan 26, 2013

Uh oh!

brenosilva commented Jan 26, 2013

Uh oh!

brenosilva commented Jan 27, 2013

Uh oh!

brenosilva commented Jan 27, 2013

Uh oh!

brenosilva commented Jan 27, 2013

Uh oh!

chaizhenhua commented Mar 5, 2013

Uh oh!

brenosilva commented Mar 5, 2013

Uh oh!

chaizhenhua commented Mar 6, 2013

Uh oh!

chaizhenhua commented Mar 6, 2013

Uh oh!

chaizhenhua commented Mar 6, 2013

Uh oh!

brenosilva commented Mar 6, 2013

Uh oh!

chaizhenhua commented Mar 7, 2013

Uh oh!

brenosilva commented Mar 7, 2013

Uh oh!

brenosilva commented Apr 4, 2013

Uh oh!

brenosilva commented Apr 4, 2013

Uh oh!

brenosilva commented Apr 4, 2013

Uh oh!

chaizhenhua commented Apr 4, 2013

Uh oh!

brenosilva commented Apr 4, 2013

Uh oh!

brenosilva commented Apr 4, 2013

Uh oh!

brenosilva commented Apr 4, 2013

Uh oh!

brenosilva commented Apr 5, 2013

Uh oh!

chaizhenhua commented Apr 6, 2013

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants