How to Run Apache Kafka as a Windows Service

Automatically launch Kafka whenever your server starts, without having to log in. Survive reboots, crashes and other failures!

Apache Kafka is a distributed streaming platform supporting the publish-subscribe paradigm. It is written in Scala and Java and leverages Apache ZooKeeper to coordinate distributed consumers.

To install Kafka as a windows service with AlwaysUp:

First, ensure that Kafka and its supporting software (Scala, Java and ZooKeeper) are installed and working properly on your system. If you installed Kafka in C:\Kafka and placed your configuration file in the config sub-folder, then this command should start your server:
C:\Kafka\bin\windows\kafka-server-start.bat
C:\Kafka\config\server.properties

Of course, the location of your files may vary. Please adjust accordingly and make a note of the command line that works. We'll need it later.

This is important: If you can't start Kafka from the batch file, you won't get it to work under AlwaysUp!
Download and install AlwaysUp, if necessary.
Start AlwaysUp.
Select Application > Add to open the Add Application window:
On the General tab:
- In the Application field, enter the full path to the batch file that starts Kafka, kafka-server-start.bat. This should be in the bin\windows folder in your Kafka installation directory.
  
  Since we have installed Kafka in C:\Kafka, we'll enter "C:\Kafka\bin\windows\kafka-server-start.bat". As in Step 1 of this tutorial, your location may be different (please use that one instead).
- In the Arguments field, enter the full path to your Kafka settings file, server.properties. This is likely in the config sub-folder.
- In the Name field, enter a name for this application in AlwaysUp. We have used Kafka but you can specify another name if you like.
Click over to the Logon tab and enter the user name and password of the windows account that you log into to start Kafka. Running in this account will ensure that Kafka will find your Scala and Java installations along with their required environment variables (JAVA_HOME) and other necessary settings.
Next, switch to the Startup tab and check the Ensure that the Windows Networking components have started box. This informs AlwaysUp that Kafka needs the TCP/IP networking stack to start and operate properly.
We're now done configuring Kafka to run as a windows service, so click the Save button. In a couple of seconds, an application called Kafka will show up in the AlwaysUp window. It will be in the Stopped state because it is not yet running.
Before you start your new Kafka windows service, please ensure that ZooKeeper is running. You can start it now -- or better yet, setup ZooKeeper as a windows service too!

Note: If ZooKeeper is not available, Kafka will start but will fail after the "zookeeper.connection.timeout.ms" period (configured in your properties file) has elapsed.
To start Kafka, choose Application > Start "Kafka". The service's state should transition to Running soon:
That's it! Next time your computer boots, Kafka will start up immediately, before anyone logs on.

We encourage you to edit Kafka in AlwaysUp and check out the many other settings that may be appropriate for your environment. For example, capture the output from the console to a file, hide the console, boost priority and much more.

Kafka not working properly as a Windows Service?

Consult the AlwaysUp Troubleshooter - our online tool that can help you resolve the most common problems encountered when running an application as a windows service.
From AlwaysUp, select Application > Report Activity > Today... to bring up a HTML report detailing the interaction between AlwaysUp and Kafka. The AlwaysUp Event Log Messages page explains all the messages.
Browse the AlwaysUp FAQ for answers to commonly asked questions and troubleshooting tips.
Contact us and we will be happy to help!