Request Batching

Learn how to batch multiple requests into fewer external calls using RequestResolver and Request.Class

Overview

Request batching is a powerful pattern for optimizing external API calls. Instead of making individual requests for each piece of data, Effect’s RequestResolver automatically batches multiple concurrent requests into a single call, reducing network overhead and improving performance.

This is particularly useful when:

• Fetching data from external APIs that support batch queries
• Loading multiple database records by ID
• Calling any service that benefits from bulk operations

Defining Request Classes

Use Request.Class to model a single lookup operation. The request class defines:

• The input parameters (e.g., an ID)
• The success type (what the request returns)
• The error type (what can go wrong)
• Any service requirements (optional)

import { Request, Schema } from "effect"

export class User extends Schema.Class<User>("User")({
  id: Schema.Number,
  name: Schema.String,
  email: Schema.String
}) {}

export class UserNotFound extends Schema.TaggedErrorClass<UserNotFound>()("UserNotFound", {
  id: Schema.Number
}) {}

class GetUserById extends Request.Class<
  { readonly id: number },
  User,          // Success type
  UserNotFound,  // Error type
  never          // Requirements (use never if none)
> {}

Creating a RequestResolver

A RequestResolver receives a batch of requests and completes each one with either success or failure:

import { Effect, Exit, RequestResolver } from "effect"

const usersTable = new Map<number, User>([
  [1, new User({ id: 1, name: "Ada Lovelace", email: "[e-mail]" })],
  [2, new User({ id: 2, name: "Alan Turing", email: "[e-mail]" })],
  [3, new User({ id: 3, name: "Grace Hopper", email: "[e-mail]" })]
])

const resolver = yield* RequestResolver.make<GetUserById>(
  Effect.fnUntraced(function*(entries) {
    for (const entry of entries) {
      const user = usersTable.get(entry.request.id)
      
      if (user) {
        // Complete with success
        entry.completeUnsafe(Exit.succeed(user))
      } else {
        // Complete with error
        entry.completeUnsafe(Exit.fail(new UserNotFound({ id: entry.request.id })))
      }
    }
  })
).pipe(
  // Allow 10ms for requests to accumulate before executing
  RequestResolver.setDelay("10 millis"),
  // Add distributed tracing spans
  RequestResolver.withSpan("Users.getUserById.resolver"),
  // Cache results to avoid duplicate lookups
  RequestResolver.withCache({ capacity: 1024 })
)

Using the Resolver

Wrap the resolver in a service method to make it easy to use:

import { Effect, Layer, ServiceMap } from "effect"

export class Users extends ServiceMap.Service<Users, {
  getUserById(id: number): Effect.Effect<User, UserNotFound>
}>()("app/Users") {
  static readonly layer = Layer.effect(
    Users,
    Effect.gen(function*() {
      // ... create resolver as shown above ...
      
      const getUserById = (id: number) =>
        Effect.request(new GetUserById({ id }), resolver).pipe(
          Effect.withSpan("Users.getUserById", { attributes: { userId: id } })
        )
      
      return { getUserById } as const
    })
  )
}

Batching in Action

When you make multiple concurrent requests, the resolver automatically batches them:

import { Effect } from "effect"

export const batchedLookupExample = Effect.gen(function*() {
  const { getUserById } = yield* Users
  
  // This triggers only ONE call to the resolver with unique IDs [1, 2, 3]
  // Duplicate IDs are automatically deduplicated
  yield* Effect.forEach([1, 2, 1, 3, 2], getUserById, {
    concurrency: "unbounded"
  })
})

Configuration Options

Batching Delay

Control how long the resolver waits before executing to allow more requests to accumulate:

RequestResolver.setDelay("10 millis")  // Short delay for real-time APIs
RequestResolver.setDelay("100 millis") // Longer delay for batch-optimized systems

Caching

Add a cache to avoid repeated lookups for the same data:

RequestResolver.withCache({ capacity: 1024 })

Tracing

Add distributed tracing spans to monitor resolver performance:

RequestResolver.withSpan("resolver-name")

Best Practices

1. Use batching for external APIs: Databases, REST APIs, and GraphQL endpoints that support batch queries
2. Set appropriate delays: Balance latency (shorter delay) vs. batch size (longer delay)
3. Enable caching: Avoid redundant requests within the same operation
4. Add tracing spans: Monitor how many requests are batched together
5. Handle errors per-request: Use entry.completeUnsafe() to handle success/failure individually

Real-World Example

Here’s a complete service using request batching for a user lookup system:

import { Effect, Exit, Layer, Request, RequestResolver, Schema, ServiceMap } from "effect"

export class User extends Schema.Class<User>("User")({
  id: Schema.Number,
  name: Schema.String,
  email: Schema.String
}) {}

export class UserNotFound extends Schema.TaggedErrorClass<UserNotFound>()("UserNotFound", {
  id: Schema.Number
}) {}

export class Users extends ServiceMap.Service<Users, {
  getUserById(id: number): Effect.Effect<User, UserNotFound>
}>()("app/Users") {
  static readonly layer = Layer.effect(
    Users,
    Effect.gen(function*() {
      class GetUserById extends Request.Class<
        { readonly id: number },
        User,
        UserNotFound,
        never
      > {}
      
      const usersTable = new Map<number, User>([
        [1, new User({ id: 1, name: "Ada Lovelace", email: "[e-mail]" })],
        [2, new User({ id: 2, name: "Alan Turing", email: "[e-mail]" })],
        [3, new User({ id: 3, name: "Grace Hopper", email: "[e-mail]" })]
      ])
      
      const resolver = yield* RequestResolver.make<GetUserById>(
        Effect.fnUntraced(function*(entries) {
          for (const entry of entries) {
            const user = usersTable.get(entry.request.id)
            
            if (user) {
              entry.completeUnsafe(Exit.succeed(user))
            } else {
              entry.completeUnsafe(Exit.fail(new UserNotFound({ id: entry.request.id })))
            }
          }
        })
      ).pipe(
        RequestResolver.setDelay("10 millis"),
        RequestResolver.withSpan("Users.getUserById.resolver"),
        RequestResolver.withCache({ capacity: 1024 })
      )
      
      const getUserById = (id: number) =>
        Effect.request(new GetUserById({ id }), resolver).pipe(
          Effect.withSpan("Users.getUserById", { attributes: { userId: id } })
        )
      
      return { getUserById } as const
    })
  )
}