When regular filters just aren’t enough… there’s always CEL!

Mastering Envoy’s CEL Filters: Advanced Access Log Control

In our previous post about Envoy logging optimization, we explored basic filtering techniques to reduce logging volume and improve proxy performance.

But what happens when these basic filters aren’t enough? What if you need to filter based on backend latency? Or create complex conditions combining multiple headers and response codes?

This is where CEL (Common Expression Language) shines. It’s a powerful tool for crafting sophisticated rules.

What is CEL?

“CEL makes it easy to write sophisticated access log filtering rules while maintaining performance”

The Common Expression Language (CEL) isn’t just another filtering mechanism — it’s a game-changer for Envoy configurations. Here’s why it matters:

1. Speed: Designed for linear-time execution.
2. Extensibility: Built to be embedded in applications.
3. Developer-Friendly: Familiar syntax for anyone who knows C++/Java/JavaScript

The Power of Expressive Filtering

“CEL simplifies what would otherwise require multiple nested filters in Envoy’s native configuration”

Before diving into examples, here’s what you can access with CEL expressions:

1. Request Data: Path, headers, method, size, duration, protocol
2. Response Data: Status codes, latency, size, headers, gRPC status
3. Connection Details: TLS info, addresses, ports, certificates
4. Upstream Info: Connection details, latency, attempt counts
   Metadata: Dynamic request metadata, filter states

See the complete list of available attributes in the Envoy documentation.

Now, let’s look at some real-world examples that showcase CEL’s capabilities:

// Catch slow error responses
response.backend_latency >= duration("2s") && response.code >= 500

// Monitor large API responses
response.size >= 1048576 && request.path.startsWith("/api/v2")

// Track specific error conditions
request.headers["x-custom-header"] == "special-value" && 
    (response.code == 429 || response.grpc_status == 14)

Implementing CEL in Practice

With CEL, you can craft rules for critical use-cases like performance monitoring (tracking backend latency), security auditing (logging suspicious TLS patterns), debugging (capturing retry attempts), and traffic analysis (monitoring API versions and payload sizes).

Let’s see how you can use CEL to implement a logging filter in your Envoy configuration.

Before we dive in, it’s worth noting that CEL is part of an Extension Filter. Envoy provides some built-in filters and also allows you to create your own pluggable extensions to customize access log filtering. We’ll discuss how to build your own filters in an upcoming post. For now, just remember that CEL (Expression Filter) is one of these extensions and comes bundled with Envoy.

Here’s a configuration that logs requests only when the backend response time exceeds 2 seconds:

access_log:
- name: envoy.access_loggers.file
  filter:
    extension_filter:
      name: envoy.filters.accesslog.cel
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.access_loggers.filters.cel.v3.ExpressionFilter
        expression: "(response.backend_latency >= duration('2s'))"
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
    path: /dev/stdout
    log_format:
      json_format:
        listener_name: https_ingress
        upstream_connection_id: "%UPSTREAM_CONNECTION_ID%"
        downstream_connection_id: "%CONNECTION_ID%"
        time: "%START_TIME%"
        request_protocol: "%PROTOCOL%"
        method: "%REQ(:METHOD)%"
        path: "%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%"
        scheme: "%REQ(:SCHEME)%"
        upstream_request_protocol: "%UPSTREAM_PROTOCOL%"
        status: "%RESPONSE_CODE%"
        response_code_details: "%RESPONSE_CODE_DETAILS%"
        request_time_msec: "%DURATION%"
        request_ttfb_msec: "%RESPONSE_DURATION%"
        response_flags: "%RESPONSE_FLAGS%"
        connection_termination_details: "%CONNECTION_TERMINATION_DETAILS%"
        user_agent: "%REQ(USER-AGENT)%"
        request_body_length: "%BYTES_RECEIVED%"
        bytes_sent: "%BYTES_SENT%"
        remote_addr: "%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%"
        remote_port: "%DOWNSTREAM_REMOTE_PORT%"
        local_addr: "%DOWNSTREAM_LOCAL_ADDRESS_WITHOUT_PORT%"
        local_port: "%DOWNSTREAM_LOCAL_PORT%"
        upstream_cluster: "%UPSTREAM_CLUSTER%"
        upstream_addr: "%UPSTREAM_HOST%"
        upstream_request_attempt_count: "%UPSTREAM_REQUEST_ATTEMPT_COUNT%"
        upstream_transport_failure_reason: "%UPSTREAM_TRANSPORT_FAILURE_REASON%"
        downstream_transport_failure_reason: "%DOWNSTREAM_TRANSPORT_FAILURE_REASON%"
        envoy_log_type: HTTP
        hostname: "%REQUESTED_SERVER_NAME%"

Notice the CEL expression being used in the filter,

response.backend_latency >= duration("2s")

You can extend this pattern to create more complex filters by combining multiple conditions. For example:

expression: >
 response.backend_latency >= duration("2s") && 
 (response.code >= 500 || request.headers["x-req-debug"] == "true")

This would log requests that either took more than 2 seconds AND resulted in a 5xx error, or had a debug header set.

Key Benefits of CEL

1. Simplified Configuration: Replace complex nested filters with single expressions.
2. Enhanced Access: Reach metrics and attributes not available in standard filters.
3. Maintainable Code: Easier updates and changes to filtering logic
   Performance: Linear-time evaluation for efficient processing.

Caveats

When implementing CEL filters in Envoy, there are a few important considerations:

1. Production Readiness: While CEL filters are fully functional, they have not undergone extensive production testing at scale. Consider gradual rollout and thorough testing in your environment.
2. Security Considerations: The security model of CEL extensions is still evolving. Only deploy in environments where you have full trust in both downstream clients and upstream servers.
3. Trust Boundaries: CEL expressions have access to sensitive request and response data. Ensure your deployment environment maintains proper trust boundaries between services.

Beyond Access Logging

“CEL is not just for access logging — it’s a versatile tool for many Envoy configurations”

CEL’s utility extends far beyond just access logging. Through Envoy’s Unified Matchers, you can use CEL expressions for:

1. Filter chain selection: Select filter chains dynamically at connection time based on network-level inputs. The matching occurs once per connection, with automatic connection draining triggered by configuration changes but not by matcher updates.
2. Route selection: Configure routing using a generic match tree that enables sub-linear matching on any header field. This provides significantly more flexibility and performance compared to traditional routing engines that were limited to :authority header based matching.
3. RBAC rule definition: Define access control policies using both network and HTTP-level inputs. Network inputs are available across all RBAC implementations (both network and HTTP filters), while HTTP-specific inputs are exclusively available when using HTTP filters.
4. Conditional filter activation: Enable dynamic filter behavior in HTTP environments using a wrapper protocol buffer. This allows you to associate matching rules directly with filter configurations for context-aware filter activation and behavior.

Looking Ahead

Stay tuned for upcoming posts where we’ll dive into:

1. Using CEL with Unified Matchers
2. Advanced RBAC configurations
3. Dynamic routing with CEL
4. Real-world use cases and patterns

Key Takeaways

1. CEL provides a powerful, flexible way to create complex filtering rules.
2. Single expressions can replace multiple nested filters.
3. Linear-time evaluation ensures performance at scale.
4. CEL’s utility extends beyond just access logging.

Are you using CEL in your Envoy setup? I’d love to hear about your use cases and experiences! If you need additional matchers or have specific scenarios you’re trying to solve, let me know in the comments. I’m actively working on extending our CEL matchers and your input would help prioritize what to build next.

This post is part of our ongoing series about Envoy proxy optimization and configuration. Follow along as we dive into topics like RBAC configurations, dynamic routing patterns, and more advanced use cases of Envoy’s powerful features. Stay tuned!