@nest-native/kafka

Decorator-first NestJS Kafka integration built on Confluent's officially supported @confluentinc/kafka-javascript client.

Note

Status: 0.1.1 (0.x). Functional and fully tested (100% coverage), and usable today — but the public API may still change before 1.0. Per semver, 0.x minor releases can include breaking changes, so pin a version. See the support policy. The initial 0.x release covers the module (KafkaModule.forRoot() / forRootAsync() / forFeature()), the KafkaProducerService (send, sendBatch, transactional), @InjectKafkaProducer(), the consumer decorators (@KafkaConsumer, @KafkaHandler) with the full Nest enhancer pipeline, the parameter decorators (@KafkaMessage, @KafkaHeaders, @KafkaCtx, @KafkaBatch), error mapping, graceful shutdown, batch consumption, per-topic concurrency, backpressure, the testing utilities (KafkaTestModule, InMemoryKafkaBroker, createMockKafkaProducer), the sample catalog, and the documentation site. The published package keeps "dependencies": {}.

What This Is

@nest-native/kafka is a community NestJS integration for Kafka consumers and producers built on Confluent's officially supported @confluentinc/kafka-javascript client. The goal is a decorator-first, Nest-native transport that preserves the @MessagePattern / @EventPattern ergonomics of @nestjs/microservices so teams can migrate — while keeping the full Nest enhancer pipeline (guards, pipes, interceptors, filters) intact on handler methods.

It is a transport-only integration. It wraps the Confluent client; it does not re-implement or hide it.

Why

The official @nestjs/microservices Kafka transport is built on kafkajs, which the community widely treats as effectively unmaintained (see nestjs/nest#13223, where Confluent staff offered their client as the replacement). The official transport also carries unresolved correctness issues:

• Sequential per-topic processing (#12703)
• Rebalance hangs (#12355)
• Exception swallowing (#9679)

This package's headline differentiators:

• Confluent client, not kafkajs: built on the officially supported, actively maintained @confluentinc/kafka-javascript (v1.9+).
• Rebalance-safe consumption: offsets commit only after a successful handler return; in-flight messages complete or are explicitly aborted.
• Per-topic concurrency: addresses #12703 with a documented default and an opt-out, instead of forced sequential processing.
• A documented migration path from the @nestjs/microservices Kafka transport.

Compatibility

Runtime	Supported line
Node.js	>=20
NestJS	11.x
@confluentinc/kafka-javascript	^1.9 (pin major; tracks librdkafka)
Validation	class-validator and Zod, both app-owned

The published package keeps "dependencies": {}. The Confluent client and the NestJS packages are declared as peerDependencies, so applications install only the ecosystems they actually use.

Repository Layout

This repository contains:

• packages/kafka: the @nest-native/kafka integration package
• sample: the runnable sample catalog (producer basics, consumer enhancers, headers/context/errors, batch + concurrency, transactions, and the @nestjs/microservices migration)
• website: the documentation site source
• scripts: quality, coverage, complexity, and release-check helpers
• CONTRIBUTING.md: contributor workflow, including the sample/library PR separation rule
• CHANGELOG.md: release history and unreleased changes
• SECURITY.md: vulnerability reporting and project security boundaries
• GUIDELINES_NEST_KAFKA.md: the project constitution

Installation

npm i @nest-native/kafka @confluentinc/kafka-javascript

Required peers:

npm i @nestjs/common @nestjs/core @nestjs/microservices reflect-metadata rxjs

Usage

Wire the module with your broker connection:

import { Module } from '@nestjs/common';
import { KafkaModule } from '@nest-native/kafka';

@Module({
  imports: [
    KafkaModule.forRoot({
      clientId: 'orders-service',
      client: { brokers: ['localhost:9092'] },
    }),
  ],
})
export class AppModule {}

Async configuration is supported through KafkaModule.forRootAsync():

KafkaModule.forRootAsync({
  inject: [ConfigService],
  useFactory: (config: ConfigService) => ({
    clientId: config.getOrThrow('KAFKA_CLIENT_ID'),
    client: { brokers: config.getOrThrow('KAFKA_BROKERS').split(',') },
  }),
});

Publish messages with the injected KafkaProducerService, and consume them with @KafkaConsumer / @KafkaHandler classes that run through the full Nest enhancer pipeline (guards, interceptors, pipes, filters):

import { Injectable } from '@nestjs/common';
import {
  KafkaConsumer,
  KafkaHandler,
  KafkaMessage,
  KafkaProducerService,
} from '@nest-native/kafka';

@Injectable()
export class OrdersService {
  constructor(private readonly producer: KafkaProducerService) {}

  placeOrder(id: string) {
    return this.producer.send({
      topic: 'orders.placed',
      messages: [{ key: id, value: JSON.stringify({ id }) }],
    });
  }
}

@Injectable()
@KafkaConsumer('orders.placed', { groupId: 'orders-service' })
export class OrdersConsumer {
  @KafkaHandler()
  handle(@KafkaMessage() order: { id: string }) {
    // runs after guards, interceptors, and pipes; exception filters wrap it
  }
}

Feature modules register their consumer/handler classes through KafkaModule.forFeature():

@Module({
  imports: [KafkaModule.forFeature([OrdersConsumer])],
})
export class OrdersModule {}

forRoot and forRootAsync return a global DynamicModule by default — pass isGlobal: false to scope them to a single module boundary. forFeature returns a non-global module that registers and exports the supplied handlers.

The full API — transactions, batch consumption, per-topic concurrency, error mapping, graceful shutdown, the parameter decorators, and the testing utilities — is covered in the package README and the documentation site.

Quality Gates

The repository ships the same review posture as its sibling @nest-native packages, using node:test and c8:

• package build, typecheck, and coverage on Node.js 20 and 22
• coverage with c8, enforced at 100% for statements, branches, functions, and lines
• sticky PR comments for coverage, test performance, and cognitive complexity
• cognitive complexity enforcement with SonarJS threshold 15
• package tarball validation and README link validation
• supply-chain audit for high-severity issues
• a real-broker integration job that runs a produce → consume round-trip, a transactional commit, and per-topic concurrency against a single-node KRaft Kafka (skipped locally unless KAFKA_BROKERS is set)

Run the local gate with:

npm run ci

Two optional, local-only layers sit on top (neither runs in CI, and forks work without them):

• Full mode — npm run infra:up && npm run test:full runs the gated real-broker integration suite against a disposable single-node KRaft Kafka (compose.yaml, host port 127.0.0.1:19094); npm run infra:down cleans up.
• Mutation testing — npm run test:mutation (incremental Stryker run; test:mutation:full re-tests everything). Scope with STRYKER_MUTATE, include the real-broker suite with STRYKER_WITH_INFRA=1.

Details — including the pre-PR ritual and agent instructions — in GUIDELINES_NEST_KAFKA.md.

Initial release scope (0.x)

The initial 0.x release covers:

1. The module — KafkaModule.forRoot() / forRootAsync() / forFeature(), the Kafka driver, and the shared producer.
2. Producer service — KafkaProducerService (send, sendBatch, transactional) and @InjectKafkaProducer() for the raw Confluent producer.
3. Consumers — @KafkaConsumer + @KafkaHandler with the full Nest enhancer pipeline (guards, interceptors, pipes, filters) and request-scoped DI.
4. Parameter decorators, error mapping, graceful shutdown — @KafkaMessage, @KafkaHeaders, @KafkaCtx, @KafkaBatch; commit-or-retry error mapping (#9679); in-flight draining on shutdown.
5. Batch consume + per-topic concurrency — addresses sequential per-topic processing (#12703) and rebalance-safe offsets (#12355), plus backpressure.
6. Transactional producer helper — transactional(work) with sendOffsets for the consume-process-produce pattern.
7. Testing utilities — KafkaTestModule, InMemoryKafkaBroker, createMockKafkaProducer, and a migration guide from @nestjs/microservices.
8. Documentation site and the sample catalog, plus a real-broker CI integration test running against a single-node KRaft Kafka.

See CHANGELOG.md for the per-release detail.

License

MIT © 2026 Rodrigo Nogueira.

Part of the nest-native family, alongside @nest-native/drizzle and @nest-native/trpc.