Apache Kafka Interview Questions and Answers

Q1) .Mention what is Apache Kafka?
Ans: Apache Kafka is a publish-subscribe messaging system developed by Apache written in Scala. It is a distributed, partitioned and replicated log service.

Q2). Mention what is the traditional method of message transfer?
Ans: The traditional method of message transfer includes two methods

•  Queuing:In a queuing, a pool of consumers may read message from the server and each message goes to one of them
  •  Publish-Subscribe:In this model, messages are broadcasted to all consumers
  Kafka caters single consumer abstraction that generalized both of the above- the consumer group.

Q3). Mention what is the benefits of Apache Kafka over the traditional technique?
Ans: Apache Kafka has following benefits above traditional messaging technique

• Fast:A single Kafka broker can serve thousands of clients by handling megabytes of reads and writes per second
  •  Scalable: Data are partitioned and streamlined over a cluster of machines to enable larger data
  •  Durable: Messages are persistent and is replicated within the cluster to prevent data loss
  •  Distributed by Design: It provides fault tolerance guarantees and durability

Q4). Mention what is the meaning of broker in Kafka?
Ans: In Kafka cluster, broker term is used to refer Server.

Q5). Compare Kafka & Flume
Ans:

Criteria	Kafka	Flume
Data flow	Pull	Push
Hadoop Integration	Loose	Tight
Functionality	Publish-subscribe model messaging system	System for data collection, aggregation & movement

Q6). What role ZooKeeper plays in a cluster of Kafka?
Ans: Kafka is an open source system and also a distributed system is built to use Zookeeper. The basic responsibility of Zookeeper is to build coordination between different nodes in a cluster. Since Zookeeper works as periodically commit offset so that if any node fails, it will be used to recover from previously committed to offset.
The ZooKeeper is also responsible for configuration management, leader detection, detecting if any node leaves or joins the cluster, synchronization, etc.

Q7). What is Kafka?
Ans: Kafka is a message divider project coded in Scala. Kafka is originally developed by LinkedIn and developed as an open sourced in early 2011. The purpose of the project is to achieve the best stand for conducting the real-time statistics nourishment.
Q8).Why do you think the replications are dangerous in Kafka?
Ans: Duplication assures that issued messages which are available are absorbed in the case of any appliance mistake, plan fault or recurrent software promotions.

Q9).What major role a Kafka Producer API plays?
Ans: It is responsible for covering the two producers- kafka.producer.SyncProducer and the kafka.producer.async.AsyncProducer. The main aim is to disclose all the producer performance through a single API to the clients.

Q10). Distinguish between the Kafka and Flume?
Ans: Flume’s major use-case is to gulp down the data into Hadoop. The Flume is incorporated with the Hadoop’s monitoring system, file formats, file system and utilities such as Morphlines. Flume’s design of sinks, sources and channels mean that with the aid of Flume one can shift data among other systems lithely, but the main feature is its Hadoop integration.
The Flume is the best option used when you have non-relational data sources if you have a long file to stream into the Hadoop.Kafka’s major use-case is a distributed publish- subscribe messaging system. Kafka is not developed specifically for Hadoop and using Kafka to read and write data to Hadoop is considerably trickier than it is in Flume.
Kafka can be used when you particularly need a highly reliable and scalable enterprise messaging system to connect many multiple systems like Hadoop.     

Q11). Describe partitioning key?
Ans: Its role is to specify the target divider of the memo, within the producer. Usually, a hash-oriented divider concludes the divider ID according to the given factors. Consumers also use the tailored Partitions.

Q12). Inside the manufacturer, when does the Queue Full Exception emerge?
Ans: Queue Full Exception naturally happens when the manufacturer tries to propel communications at a speed which Broker can’t grip. Consumers need to insert sufficient brokers to collectively grip the amplified load since the Producer doesn’t block.

Q13). Can Kafka be utilized without Zookeeper?
Ans: It is impossible to use Kafka without Zookeeper because it is not feasible to go around Zookeeper and attach in a straight line to the server. If the Zookeeper is down for a number of causes, then we will not be able to serve any customer demand.

Q14). What are consumers or users?
Ans: Kafka provides single consumer abstractions that discover both queuing and publish-subscribe Consumer Group. They tag themselves with a user group and every communication available on a topic is distributed to one user case within every promising user group. User instances are in disconnected process. We can determine the messaging model of the consumer based on the consumer groups.

• If all consumer instances have the same consumer set, then this works like a conventional queue adjusting load over the consumers.
• If all customer instances have dissimilar consumer groups, then this works like a publish-subscribe and all messages are transmitted to all the consumers.

Q15). Describe an Offset?
The messages in the partitions will be given a sequential ID number known as an offset, the offset will be used to identify each message in the partition uniquely. With the aid of Zookeeper Kafka stores the offsets of messages consumed for a specific topic and partition by this consumer group.

>> Read more >>