[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"question-kafka-kak-vyglyadit-primer-konfiguratsii-kafka-consumer":3},{"id":4,"slug":5,"topicId":6,"topicSlug":7,"topicName":8,"topicEmoji":9,"question":10,"answer":11,"codeLang":12,"codeSrc":12,"important":12,"commonMistakes":12,"modernUsage":12,"difficulty":13,"tags":14,"related":15,"progress":16,"seo":17},805,"kak-vyglyadit-primer-konfiguratsii-kafka-consumer",21,"kafka","Kafka","📨","Как выглядит пример конфигурации Kafka Consumer?","Рассмотрим конфигурацию консюмера с разными гарантиями доставки и через разные фреймворки.\n\n### Нативный Kafka Consumer\n\n\u003Cdetails>\u003Csummary>Пример кода\u003C\u002Fsummary>\n\n```java\nimport org.apache.kafka.clients.consumer.ConsumerConfig;\nimport org.apache.kafka.clients.consumer.KafkaConsumer;\nimport org.apache.kafka.common.serialization.StringDeserializer;\n\nimport java.time.Duration;\nimport java.util.HashMap;\nimport java.util.Map;\nimport java.util.Collections;\n\npublic class KafkaConsumerExample {\n\n    public static void main(String[] args) {\n        String bootstrapServers = \"localhost:9092\";\n        String groupId = \"my-consumer-group\";\n        String topic = \"my-topic\";\n\n        Map\u003CString, Object> consumerConfigs = new HashMap\u003C>();\n        consumerConfigs.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);\n        consumerConfigs.put(ConsumerConfig.GROUP_ID_CONFIG, groupId);\n        consumerConfigs.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);\n        consumerConfigs.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);\n        consumerConfigs.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, \"earliest\");\n\n        KafkaConsumer\u003CString, String> consumer = new KafkaConsumer\u003C>(consumerConfigs);\n        consumer.subscribe(Collections.singletonList(topic));\n\n        try {\n            while (true) {\n                var records = consumer.poll(Duration.ofSeconds(1));\n                records.forEach(record -> System.out.println(\"Received: \" + record.value()));\n            }\n        } finally {\n            consumer.close();\n        }\n    }\n}\n```\n\n\u003C\u002Fdetails>\n\n### Гарантии доставки\n\n| Гарантия | Принцип | Когда коммитить |\n|----------|---------|-----------------|\n| **At least once** | Сообщение обработано минимум один раз (возможны дубли) | После обработки (`commitAsync()`) |\n| **At most once** | Сообщение обработано максимум один раз (возможна потеря) | До обработки или `enable.auto.commit=true` |\n| **Mostly once** | Гибрид — обычно один раз, при сбоях возможны дубли | Ручной commit + дедупликация по messageId |\n\n\u003Cdetails>\u003Csummary>At least once — нативный API\u003C\u002Fsummary>\n\n```java\ntry {\n    while (true) {\n        var records = consumer.poll(Duration.ofSeconds(1));\n        process(records);\n        consumer.commitAsync(); \u002F\u002F Commit после обработки\n    }\n} finally {\n    consumer.close();\n}\n```\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\u003Csummary>At most once — нативный API\u003C\u002Fsummary>\n\n```java\ntry {\n    while (true) {\n        var records = consumer.poll(Duration.ofSeconds(1));\n        consumer.commitAsync(); \u002F\u002F Commit перед обработкой\n        process(records);\n    }\n} finally {\n    consumer.close();\n}\n```\n\n\u003C\u002Fdetails>\n\n### С использованием Spring Kafka\n\n\u003Cdetails>\u003Csummary>Конфигурация и listener\u003C\u002Fsummary>\n\n```java\n@EnableKafka\n@Configuration\npublic class KafkaConsumerConfig {\n\n    @Autowired\n    private KafkaProperties kafkaProperties;\n\n    @Bean\n    public ConsumerFactory\u003CString, String> consumerFactory() {\n        return new DefaultKafkaConsumerFactory\u003C>(consumerConfigs());\n    }\n\n    @Bean\n    public Map\u003CString, Object> consumerConfigs() {\n        Map\u003CString, Object> configs = new HashMap\u003C>();\n        configs.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, kafkaProperties.getServer());\n        configs.put(ConsumerConfig.GROUP_ID_CONFIG, kafkaProperties.getConsumerGroupId());\n        configs.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);\n        configs.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);\n        return configs;\n    }\n\n    @Bean\n    public KafkaListenerContainerFactory\u003CConcurrentMessageListenerContainer\u003CString, String>> kafkaListenerContainerFactory() {\n        ConcurrentMessageListenerContainerFactory\u003CString, String> factory = new ConcurrentMessageListenerContainerFactory\u003C>();\n        factory.setConsumerFactory(consumerFactory());\n        return factory;\n    }\n}\n```\n\n```java\n@Service\npublic class KafkaConsumer {\n\n    @KafkaListener(topics = \"my_topic\", groupId = \"group_id\")\n    public void listen(@Payload String message,\n                       @Header(\"traceId\") String traceId,\n                       @Header(\"correlationId\") String correlationId) {\n        System.out.println(\"Received message: \" + message);\n        System.out.println(\"Trace ID: \" + traceId);\n        System.out.println(\"Correlation ID: \" + correlationId);\n    }\n}\n```\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\u003Csummary>At least once — Spring Kafka\u003C\u002Fsummary>\n\n```yaml\nspring:\n  kafka:\n    consumer:\n      enable-auto-commit: false\n      auto-offset-reset: earliest\n      group-id: my-consumer-group\n      max-poll-records: 500\n    listener:\n      ack-mode: manual\n```\n\n```java\n@EnableKafka\npublic class AtLeastOnceConsumer {\n\n    @KafkaListener(topics = \"my-topic\", groupId = \"my-consumer-group\")\n    public void listen(String message, Acknowledgment acknowledgment) {\n        System.out.println(\"Received message: \" + message);\n        acknowledgment.acknowledge(); \u002F\u002F Ручное подтверждение после обработки\n    }\n}\n```\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\u003Csummary>At most once — Spring Kafka\u003C\u002Fsummary>\n\n```yaml\nspring:\n  kafka:\n    consumer:\n      enable-auto-commit: true\n      group-id: my-consumer-group\n      auto-offset-reset: earliest\n      max-poll-records: 100\n```\n\n```java\npublic class AtMostOnceConsumer {\n\n    @KafkaListener(topics = \"my-topic\", groupId = \"my-consumer-group\")\n    public void listen(String message) {\n        System.out.println(\"Received message: \" + message);\n        \u002F\u002F Смещение автоматически зафиксировано после получения\n    }\n}\n```\n\n\u003C\u002Fdetails>\n\n### С использованием Spring Cloud Stream\n\n\u003Cdetails>\u003Csummary>Конфигурация и consumer\u003C\u002Fsummary>\n\n```yaml\nspring:\n  cloud:\n    stream:\n      bindings:\n        input:\n          destination: my-topic\n          group: my-consumer-group\n          content-type: application\u002Fjson\n      kafka:\n        binder:\n          brokers: localhost:9092\n          auto-create-topics: false\n```\n\n```java\n@Service\n@EnableBinding(KafkaProcessor.class)\npublic class KafkaConsumerService {\n\n    @StreamListener(\"input\")\n    public void handle(@Payload String message) {\n        System.out.println(\"Received message: \" + message);\n    }\n}\n```\n\n```java\npublic interface KafkaProcessor {\n\n    @Input(\"input\")\n    SubscribableChannel input();\n}\n```\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\u003Csummary>Mostly once — Spring Cloud Stream (дедупликация)\u003C\u002Fsummary>\n\n```yaml\nspring:\n  cloud:\n    stream:\n      bindings:\n        input:\n          destination: my-topic\n          group: my-consumer-group\n          content-type: application\u002Fjson\n          consumer:\n            ackMode: manual\n            maxAttempts: 3\n```\n\n```java\n@Component\n@EnableBinding(Sink.class)\npublic class MostlyOnceConsumer {\n\n    private Set\u003CString> processedMessageIds = new HashSet\u003C>();\n\n    @StreamListener(Sink.INPUT)\n    public void handleMessage(Message\u003CString> message, @Header(\"messageId\") String messageId) {\n        if (processedMessageIds.contains(messageId)) {\n            System.out.println(\"Duplicate message: \" + messageId);\n            return;\n        }\n        System.out.println(\"Received message: \" + message.getPayload());\n        processedMessageIds.add(messageId);\n    }\n}\n```\n\n\u003C\u002Fdetails>\n\n> **На собеседовании:** ключевой вопрос — объяснить разницу между at-least-once и at-most-once. Покажите на уровне кода: at-least-once — commit после обработки (при сбое сообщение будет обработано повторно); at-most-once — commit до обработки (при сбое сообщение потеряется).","","middle",[7],[],null,{"title":18,"description":19,"ogTitle":18,"ogDescription":19,"keywords":20,"schemaAnswer":19,"featuredSnippetReady":21},"Как выглядит пример конфигурации Kafka Producer? — Gymterview","Рассмотрим три подхода к созданию продюсера: нативный Kafka API, Spring Kafka и Spring Cloud Stream.",[7,13],true]