Oracle Cloud Infrastructure Documentation

Details for Load Balancer Logs

Logging details for load balancer logs.

Also see Logging for Load Balancers for more information.

Resources

  • load balancer

Log Categories

API value (ID): Console (Display Name) Description
Access Access Logs Load balancer access logs.
Error Error Logs Load balancer error logs.

Availability

LBaaS access logs are available in all the regions of the commercial realms.

Load Balancer Access Log

Load balancer access logs capture detailed information about requests sent to the load balancer. Each access log entry contains:
  • The time the request was received.
  • Client and intermediate HTTP proxy IP addresses.
  • Time taken at the load balancer and backend to process the request.
Limitations and Considerations

Some traffic might not be logged during a capture window because of capacity issues or system errors. In such cases, the following error log message is logged.

{"timestamp":"2020-08-05T00:12:39+00:00","errorLog":{"type":"General","errorDetails": "Missed 100 access logs" }}

For traffic destined to the public IP of a load balancer, access logs record the corresponding private IP.

Contents of an Access Log

Access logs appear as a value in the Log Data field. This value is a JSON-formatted data with the following fields.

Field Example Description
timestamp 2020-04-20T00:56:18+00:00 Log entry generation time in ISO-8601 format.
clientAdd 192.168.0.33:7870 IP address and port number of the requesting client.
forwardedForAddr 192.168.0.33 IP address of the client and http proxies between client and load balancer.
host a.com Domain name which resolves to VIP address assigned to the load balancer.
backendAddr 192.168.0.34:8080 IP address and port number of the backend server, which processed the client request.
requestProcessingTime 0.003 Total time taken (in seconds, with millisecond precision), from when the load balancer starts receiving request from the client, until it completes sending responses to the client.
backendConnectTime 0.00 Time spent (in seconds, with millisecond precision), to establish backend server connection.
backendProcessingTime 0.002 Total time taken from the load balancer establishing a connection to a backend, until it completes receiving the response from the backend.
lbStatusCode 200 Status code of the response from the load balancer.
backendStatusCode 200 Status code of the response from the target.
receivedBytes 150 Total size of the request (in bytes), received from the client.
sentBytes 450 Total size of the response (in bytes), sent to the client from the load balancer.
request "GET / HTTP/1.1" Request line received from the client.
sslCipher ECDHE-RSA-AES256-GCM-SHA384 Negotiated SSL cipher between the client and the load balancer.
listenerName http_listener_1 Listener which received the incoming traffic request on the load balancer's IP address.
sslProtocol TLSv1.2 Negotiated SSL protocol between the client and the load balancer.
userAgent curl/7.29.0 User Agent, which sent the request to the load balancer.
routingRulesMatchedRule RoutingPolicy_test_rule1 Routing policy rule name, which is matched for this specific client request.
routingRulesRuleHits 1 Number of routing rules getting matched for the request. Either 1 (for a match) or 0 (for no match).
routingRulesRuleMisses 4 Number of routing rules evaluated to false for the request.
routingRulesEngineErrors 0 Routing rule engine error during policy evaluation for the request. Either 0 (no error) or 1 when any error occurs. If an error occurs, requests are forwarded to the default backend set attached to the listener.
requestId 0123456789abcdef0123456789abcdef Unique request identifier in hexadecimal format (16 bytes).
responseProvider 192.168.0.34:8080

Information about where the response is coming from. May contain:

  • Backend IP address and port when response is coming from a backend.

  • Load Balancer rule type (one of 'Load Balancer HTTP method rule', 'Load Balancer IP ACL rule', 'Load Balancer redirect rule') when response is generated by a Load Balancer rule.

  • WAF rule module and name when Load Balancer has WAF attached and request is blocked by WAF.

  • Empty string when request processing stops in Load Balancer because of error (for example, 400 Bad request).

Load Balancer Error Log

Load balancer error logs capture detailed information about requests related to troubleshooting and monitoring. Each Error Log entry contains information such as the time the request was received, error type, and extra details related to the specific error.

Contents of an Error Log

This value is JSON-formatted data with the following fields.

Field Example Description
timestamp 2020-08-04T21:25:27+00:00 Log entry generation time in ISO-8601 format.
type frontDoor Error log Category.
errorDetails Access for client 160.34.88.6 forbidden by ACL rule Detailed description of the error message.
Sample Error Logs
Type Scenario Description Example
healthChecker Connection Timeout Backend health check failure because of connection timeout. {"timestamp":"2020-08-05T00:12:39+00:00","errorLog":{"type":"healthChecker","errorDetails":{"healthStatus":"Healthy to Unhealthy",

"backendSetName":"newtest","backend":"10.10.100.7:80","details":{"date":1596586352368,"failures":3,"successes":6,"skips":0,"message":
{"msg":"connect timed out","elapsed":3000}}}}}
healthChecker RespCode Mismatch Backend health check failure because of RespCode mismatch. {"timestamp":"2020-08-04T23:08:07+00:00","errorLog":{"type":"healthChecker","errorDetails":{"healthStatus":"Healthy to Unhealthy",

"backendSetName":"newtest","backend":"10.10.100.7:80","details":{"date":1596582477049,"failures":3,"successes":0,"skips":0,"message":
{"msg":"invalid statusCode","statusCode":404,"expected":"200"}}}}}
healthChecker RegEx Mismatch Backend health check failure because of RegEx mismatch. {"timestamp":"2020-08-04T23:28:52+00:00","errorLog":{"type":"healthChecker","errorDetails":{"healthStatus":"Healthy to Unhealthy",

"backendSetName":"newtest","backend":"10.10.100.7:80","details":{"date":1596583722793,"failures":3,"successes":0,"skips":0,"message":{"expectedRegex":"^notexist$","msg":"response match result: failed",
"base641kData":"CjwhRE9DVFlQRSBodG1sIFBVQkxJQyAiLS8vVzNDLy9EVEQgWEhUTUwgMS4wIFRyYW5zaXRpb25hbC8vRU4iICJodHRwOi8vd3d3LnczLm9yZy9UUi94aHRtbDEvRFREL3hodG1s
MS10cmFuc2l0aW9uYWwuZHRkIj4KPGh0bWwgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkveGh0bWwiPgogIDwhLS0KICAgIE1vZGlmaWVkIGZyb20gdGhlIERlYmlhbiBvcmlnaW5hbCBmb3Ig
VWJ1bnR1CiAgICBMYXN0IHVwZGF0ZWQ6IDIwMTYtMTEtMTYKICAgIFNlZTogaHR0cHM6Ly9sYXVuY2hwYWQubmV0L2J1Z3MvMTI4ODY5MAogIC0tPgogIDxoZWFkPgogICAgPG1ldGEgaH
R0cC1lcXVpdj0iQ29udGVudC1UeXBlIiBjb250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9VVRGLTgiIC8+CiAgICA8dGl0bGU+QXBhY2hlMiBVYnVudHUgRGVmYXVsdCBQYWdlOiBJdCB3b3JrczwvdG
l0bGU+CiAgICA8c3R5bGUgdHlwZT0i"}}}}}
healthChecker Connection Refused Backend health check failure because of connection RST. {"timestamp":"2020-08-04T19:53:29+00:00","errorLog":{"type":"healthChecker","errorDetails":{"healthStatus":"Healthy to Unhealthy",

"backendSetName":"newtest","backend":"10.10.100.7:80","details":{"date":1596570799950,"failures":3,"successes":0,"skips":0,"message":
{"msg":"error","error":{"code":"ECONNREFUSED","errno":"ECONNREFUSED",
"syscall":"connect","address":"10.10.100.7","port":80}}}}}}

healthChecker

 Host Unreachable Backend health check failure because of unreachable host. {"timestamp":"2020-08-06T22:29:10+00:00","errorLog":{"type":"healthChecker","errorDetails":{"healthStatus":"Healthy to Unhealthy","backendSetName":"http-pool","backend":"10.10.100.8:80","details":{"date":1596752940288,"failures":3,"successes":0,"skips":0,"message":{"msg":"error","error":{"code":"EHOSTUNREACH","errno":"EHOSTUNREACH","syscall":"connect","address":"10.10.100.8","port":80}}}}}}
healthChecker Success (Unhealthy to healthy) Unhealthy backend became healthy. {"timestamp":"2020-08-04T20:00:20+00:00","errorLog":{"type":"healthChecker","errorDetails":{"healthStatus":"Unhealthy to Healthy","backendSetName":"newtest","backend":"10.10.100.7:80","details":{"date":1596571210421,"failures":41,"successes":3,"skips":0,"message":{"expectedRegex":"(?:)","msg":"response match result: ok","base641kData":"CjwhRE9DVFlQRSBodG1sIFBVQkxJQyAiLS8vVzNDLy9EVEQgWEhUTUwgMS4wIFRyYW5zaXRpb25hbC8vRU4iICJodHRwOi8vd3d3LnczLm9yZy9UUi94aHRtbDEvRFREL3h

odG1sMS10cmFuc2l0aW9uYWwuZHRkIj4KPGh0bWwgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkveGh0bWwiPgogIDwhLS0KICAgIE1vZGlmaWVkIGZyb20gdGhlIERlYmlhbiBvcmlnaW5
hbCBmb3IgVWJ1bnR1CiAgICBMYXN0IHVwZGF0ZWQ6IDIwMTYtMTEtMTYKICAgIFNlZTogaHR0cHM6Ly9sYXVuY2hwYWQubmV0L2J1Z3MvMTI4ODY5MAogIC0tPgogIDxoZWFkPgogI
CAgPG1ldGEgaHR0cC1lcXVpdj0iQ29udGVudC1UeXBlIiBjb250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9VVRGLTgiIC8+CiAgICA8dGl0bGU+QXBhY2hlMiBVYnVudHUgRGVmYXVsdCBQYW
dlOiBJdCB3b3JrczwvdGl0bGU+CiAgICA8c3R5bGUgdHlwZT0idGV4dC9j"}}}}}
healthChecker SSL Backend, with some SSL error Backend health check failure because of SSL error. {"timestamp":"2020-08-06T06:45:23+00:00","errorLog":{"type":"healthChecker","errorDetails":{"healthStatus":"Healthy to Unhealthy",

"backendSetName":"ssl-pool","backend":"10.10.100.7:443","details":{"date":1596696313688,"failures":3,"successes":0,"skips":0,"message":
{"msg":"error","error":{"code":"EPROTO","errno":"EPROTO","syscall":"write"}}}}}}
frontDoor ACL Drop (TCP) Client connection drop to ACL rules. {"timestamp":"2020-08-04T21:25:27+00:00","errorLog":{"type":"frontDoor","errorDetails":"Access for client 160.34.88.6 forbidden by TCP ACL rule"}}
sslClient SSL error as client did not send cert SSL handshake failure because of no certificate. {"timestamp":"2020-08-06T06:19:36+00:00","errorLog":{"type":"sslClient","errorDetails":"Client 160.34.89.107 sent no required SSL certificate"}}
backEnd All backends down (HTTP) Connection failure when all backends are unhealthy. {"timestamp":"2020-08-06T06:48:54+00:00","errorLog":{"type":"backEnd","errorDetails":"No healthy backends available"}}
backEnd Connection failure when no backend configured (TCP) Connection failure when all backends are unhealthy. {"timestamp":"2020-08-04T19:59:03+00:00","errorLog":{"type":"backEnd","errorDetails":"No healthy backends available"}}
backEnd IOError.(http)Connection closed in middle Connection failure when backend closes connection in middle of transaction. {"timestamp":"2020-08-06T22:40:29+00:00","errorLog":{"type":"backEnd","errorDetails":"Backend 10.10.100.7 closed connection abruptly"}}
backEnd Backend max connection limit reached Connection failure when backend already has max_connection connections. {"timestamp":"2023-11-13T23:21:13+00:00", "errorLog":{"type":"backEnd","errorDetails" "Backend 10.0.4.176 closed connection abruptly"}}
backend Backend max connection limit reached Connection failure when all backends already have max_connection connections. {"timestamp":"2023-11-13T23:21:13+00:00", "errorLog":{"type":"backEnd","errorDetails":"No healthy backends available in associated backendSet bs_lb_2023-1107-1228"}}
general Listener IP max connections limit reached Client connection drop due to IP_BASED_MAX_CONNECTIONS rule limit being reached. {"timestamp":"2023-11-08T20:56:55+00:00","errorLog":{"type":"general","errorDetails":"Connection to 10.0.4.232:80 was abruptly closed by 139.87.112.128:49938"}}