GitHub

Advanced Topics

This page covers production-ready features for gRPCServer.jl: interceptors, TLS, compression, and more.

Ordinary servers are built on the codegen interface: one protojl run emits the message types and per-service registration functions, and you register handlers with the generated register_<Service>! / register_<Service>_<Rpc>! functions. The last section of this page, "The runtime interface beneath the codegen", explains the descriptor-based layer those generated functions sit on — reach for it only when you need to build or inspect services manually.

Interceptors

Interceptors allow you to add cross-cutting concerns like logging, authentication, and rate limiting. They work the same way whether the service was registered through the generated register_*! functions or the runtime interface.

Logging Interceptor

host = "127.0.0.1"port = 50051server = GRPCServer(host, port)add_interceptor!(server, LoggingInterceptor(    log_requests = true,    log_responses = true,    log_errors = true))

Metrics Interceptor

request_counter = Ref(0)latencies = Float64[]add_interceptor!(server, MetricsInterceptor(    on_request = (method, size) -> begin        request_counter[] += 1    end,    on_response = (method, status, duration_ms, size) -> begin        push!(latencies, duration_ms)    end))

Custom Authentication Interceptor

struct AuthInterceptor <: Interceptor    valid_tokens::Set{String}endfunction (auth::AuthInterceptor)(    ctx::ServerContext,    request::Any,    info::MethodInfo,    next::Function)    token = get_metadata_string(ctx, "authorization")    if token === nothing || !(token in auth.valid_tokens)        throw(GRPCError(            StatusCode.UNAUTHENTICATED,            "Invalid or missing authentication token"        ))    end    return next(ctx, request)endadd_interceptor!(server, AuthInterceptor(Set(["token123", "token456"])))

Rate Limiting Interceptor

mutable struct RateLimitInterceptor <: Interceptor    requests_per_second::Int    window_start::Float64    request_count::IntendRateLimitInterceptor(rps::Int) = RateLimitInterceptor(rps, time(), 0)function (rl::RateLimitInterceptor)(    ctx::ServerContext,    request::Any,    info::MethodInfo,    next::Function)    now = time()    if now - rl.window_start >= 1.0        rl.window_start = now        rl.request_count = 0    end    rl.request_count += 1    if rl.request_count > rl.requests_per_second        throw(GRPCError(            StatusCode.RESOURCE_EXHAUSTED,            "Rate limit exceeded"        ))    end    return next(ctx, request)endadd_interceptor!(server, RateLimitInterceptor(100))

Health Checking

Health checks allow load balancers and orchestrators to monitor server status.

server = GRPCServer(host, port;    enable_health_check = true)# Set overall server healthset_health!(server, HealthStatus.SERVING)# Set health for specific serviceset_health!(server, "my.Service", HealthStatus.SERVING)# Mark service as not ready (e.g., during maintenance)set_health!(server, "my.Service", HealthStatus.NOT_SERVING)

Testing Health Check

grpcurl -plaintext -d '{"service": ""}' localhost:50051 grpc.health.v1.Health/Check

Expected output:

{
  "status": "SERVING"
}

TLS Configuration

Basic TLS

tls_config = TLSConfig(    cert_chain = "server.crt",    private_key = "server.key")server = GRPCServer(host, port; tls = tls_config)

Mutual TLS (mTLS)

tls_config = TLSConfig(    cert_chain = "server.crt",    private_key = "server.key",    client_ca = "ca.crt",    require_client_cert = true)server = GRPCServer(host, port; tls = tls_config)

Hot Reloading Certificates

# Reload certificates without restarting serverreload_tls!(server)

See the TLS page for a full walkthrough.

Compression

Server-side Compression (not yet implemented)

Send-side response compression is not yet implemented on any backend: responses are always sent with grpc-encoding: identity. The configuration keywords for it (compression_enabled=true, compression_threshold, supported_codecs) are recognized but inert, so setting them explicitly now raises UnsupportedFeatureError at construction instead of being silently ignored:

# Raises UnsupportedFeatureError on every backend — send-side compression# is not implemented yet.server = GRPCServer(host, port;    supported_codecs = [CompressionCodec.GZIP, CompressionCodec.DEFLATE])

Omit those keywords for now. Receive-side decompression works: the server accepts grpc-encoding: gzip/deflate requests and decompresses them (strictly on HTTPjlBackend, leniently on PureHTTP2Backend), so compressed clients are fully supported.

Manual Compression

using gRPCServer: compress, decompress, CompressionCodecdata = Vector{UInt8}("Large data to compress...")# Compress with GZIPcompressed = compress(data, CompressionCodec.GZIP)# Decompressoriginal = decompress(compressed, CompressionCodec.GZIP)

Error Handling

Returning Specific Status Codes

function my_handler(ctx::ServerContext, request)    if request.id < 0        throw(GRPCError(            StatusCode.INVALID_ARGUMENT,            "ID must be non-negative"        ))    end    item = find_item(request.id)    if item === nothing        throw(GRPCError(            StatusCode.NOT_FOUND,            "Item not found: $(request.id)"        ))    end    if !has_permission(ctx, item)        throw(GRPCError(            StatusCode.PERMISSION_DENIED,            "Access denied to item $(request.id)"        ))    end    return itemend

Error Details

throw(GRPCError(    StatusCode.INVALID_ARGUMENT,    "Multiple validation errors",    Any[        Dict("field" => "email", "error" => "Invalid email format"),        Dict("field" => "age", "error" => "Must be positive")    ]))

Context Usage

Accessing Metadata

function my_handler(ctx::ServerContext, request)    # Get string metadata    auth = get_metadata_string(ctx, "authorization")    # Get binary metadata (keys ending in -bin)    trace = get_metadata_binary(ctx, "x-trace-bin")    # Check remaining time before deadline    remaining = remaining_time(ctx)    if remaining !== nothing && remaining < 1.0        @warn "Less than 1 second remaining"    end    return responseend

Setting Response Headers and Trailers

function my_handler(ctx::ServerContext, request)    # Set response header    set_header!(ctx, "x-request-id", string(ctx.request_id))    # Set trailer (sent at end of response)    set_trailer!(ctx, "x-processing-time", "50ms")    return responseend

Client Streaming

A service that receives multiple requests and returns a single response, registered with the generated codegen function:

function sum_numbers(ctx::ServerContext, stream::ClientStream{NumberRequest})    total = 0    for request in stream        total += request.value    end    return SumResponse(total = total)end# do-block per-RPC formregister_Math_Sum!(server) do ctx, stream    total = 0    for request in stream        total += request.value    end    SumResponse(total = total)end

Bidirectional Streaming

A chat-like service with two-way streaming:

function chat(ctx::ServerContext, stream::BidiStream{ChatMessage, ChatMessage})    for message in stream        if is_cancelled(ctx)            break        end        # Echo back with prefix        response = ChatMessage(            user = "Server",            text = "You said: $(message.text)"        )        send!(stream, response)    end    close!(stream)    return nothingend# do-block per-RPC formregister_Chat_Chat!(server) do ctx, stream    for message in stream        send!(stream, ChatMessage("Server", "You said: $(message.text)"))    end    close!(stream)    return nothingend

Passing Application State

Attach application state to the server and read it from any handler through ctx.payload:

struct AppState    dbendserver = GRPCServer("127.0.0.1", 50051; context = AppState(open_db()))register_MyService_GetThing!(server) do ctx, req    Thing(query(ctx.payload.db, req.id))end

The context keyword accepts any value; it is threaded untouched into every request's ServerContext.payload. Handlers never construct the state themselves, which makes them easy to test in isolation.

Concurrency Model

The server is built on HTTP.jl, which spawns one task per inbound HTTP/2 stream. That per-stream task is where your handler runs. Because each connection and each stream is independent, many RPCs are served concurrently as a matter of course.

The max_concurrent_requests keyword on GRPCServer (default 1024) caps how many RPCs run at once. When the cap is reached, additional requests are shed immediately with a trailers-only StatusCode.RESOURCE_EXHAUSTED status — there is no queue and no waiting. Pass nothing or 0 for an unlimited cap (the pre-1.0 default).

server = GRPCServer("127.0.0.1", 50051;    max_concurrent_requests = 256)

Handler thread safety is the application's responsibility: two handlers can execute simultaneously on different threads and share whatever you attached as context. Guard mutable shared state (database connection pools, caches, counters) with the appropriate locks or atomics. For CPU-bound work, launch Julia with --threads=auto; with a single thread, tasks still interleave cooperatively at I/O boundaries, but CPU-bound handlers will not run in parallel.

Production Hardening

Authentication and authorization

Authentication and authorization are the application's responsibility. The server does not authenticate callers. A handler reads credentials from request metadata and rejects the call with the appropriate status:

function my_handler(ctx::ServerContext, request)    token = get_metadata_string(ctx, "authorization")    if token === nothing || !is_valid(token)        throw(GRPCError(StatusCode.UNAUTHENTICATED, "Missing or invalid credentials"))    end    if !authorized(token, request)        throw(GRPCError(StatusCode.PERMISSION_DENIED, "Access denied"))    end    return responseend

Transport

Use TLS in production. The default is cleartext HTTP/2 (h2c); pass tls = TLSConfig(...) to serve h2. Cleartext should only be used behind a trusted boundary, for example a localhost sidecar or a TLS-terminating proxy. See TLS.

Concurrency cap

max_concurrent_requests ships enabled with a conservative default of 1024 (HTTP.jl allows 100 concurrent streams per connection, so without a cap N connections imply 100·N concurrent handler tasks). Still size it explicitly to the host's memory and the configured max_message_size in production; a cap that is too small sheds legitimate load with RESOURCE_EXHAUSTED, while one that is too large lets a flood consume all available memory.

Connection timeouts

Connection timeouts default to read_header_timeout = 30 seconds, which reaps slow-header connections without disturbing established streams, and idle_timeout = 300 seconds, which closes connections that stop sending bytes (including one holding a partial request body, bounding slow-body memory accrual). read_timeout and write_timeout are disabled by default; enabling them defends against a peer that trickles or never finishes a request or response body, but a non-zero read_timeout also terminates legitimately idle long-lived streaming RPCs, so set it only for unary or short-lived workloads.

Deadlines and cancellation

The grpc-timeout header is parsed strictly into ctx.deadline (a malformed value fails the call with INVALID_ARGUMENT). The deadline is enforced at two points, and never while your handler is running:

  • Pre-dispatch: if the deadline has already passed when the request reaches

the handler (a zero timeout, or queueing delay past the deadline), the call fails fast with a trailers-only DEADLINE_EXCEEDED and your handler is never invoked.

  • Post-return: once your handler returns, a deadline that has since passed

maps the result to DEADLINE_EXCEEDED (a status your handler already produced as DEADLINE_EXCEEDED/CANCELLED is left untouched).

A handler that runs past its deadline is not interrupted — it runs to completion, and only then is its result mapped. There is no watchdog-based cancellation yet. Handlers that must bound their own runtime should check remaining_time(ctx) / is_cancelled(ctx) cooperatively (the gRPC interceptors work here too — see TimeoutInterceptor, which is also pre-check-only):

function my_handler(ctx::ServerContext, request)    remaining = remaining_time(ctx)    if remaining !== nothing && remaining <= 0        throw(GRPCError(StatusCode.DEADLINE_EXCEEDED, "Deadline exceeded"))    end    # ... work, checking is_cancelled(ctx) between long stepsend

A long-running handler that ignores the deadline is a resource-exhaustion vector: it ties up a stream (and memory) for as long as it runs. Pair cooperative deadline checks with a max_concurrent_requests cap sized to your workload, and keep idle_timeout enabled, so that a flood of slow or never-completing calls cannot pile up unbounded handlers.

The Runtime Interface Beneath the Codegen

The generated register_<Service>_<Rpc>! functions build a MethodDescriptor and call register_method! on the server's dispatcher. You normally never do this by hand, but the pieces are public so custom registration flows are possible.

Descriptor builders

Each *_Method builder creates a MethodDescriptor for one RPC:

method = MyService_GetThing_Method((ctx, req) -> Thing(req.id))# raw variants: receive/return undecoded Vector{UInt8} payloadsmethod = MyService_GetThing_Method((ctx, raw) -> raw; raw_request = true, raw_response = true)

A raw response is the protobuf message body only — the framing layer still applies — so whatever bytes a raw handler returns must decode as a well-formed protobuf of the response type on the client. An echo handler like the one above only works when the client expects exactly those bytes (for example a raw client); a typed client decoding a different message type will fail.

Manual service registration

Build a ServiceDescriptor and register it with register!, or register a single method with register_method!:

struct MyService endfunction gRPCServer.service_descriptor(::MyService)    ServiceDescriptor(        "pkg.MyService",        Dict(            "GetThing" => MyService_GetThing_Method((ctx, req) -> Thing(req.id))        ),        nothing    )endregister!(server, MyService())# or, for a single method:register_method!(server.dispatcher, "pkg.MyService",    MyService_GetThing_Method((ctx, req) -> Thing(req.id)))

register_method! is what the generated register_*! functions call internally. See the API Reference for the full surface (MethodDescriptor, ServiceDescriptor, register!, register_method!).

Graceful Shutdown

server = GRPCServer(host, port)register_Greeter!(server; SayHello = say_hello)# Run in background taskserver_task = @async run(server; block = true)# Later, initiate graceful shutdown@info "Shutting down..."stop!(server; timeout = 30.0)# Wait for server to stopwait(server_task)@info "Server stopped"

Next Steps