Simple Configuration Distribution Using Zookeeper

Submitted by chris.crowe on Sat, 03/30/2019 - 18:51
So it's been a while since our last post, but I wanted to keep the blog going.

There has been a lot of movement in the distributed space so I wanted to take the chance to discuss a simple, yet powerful, way that you can leverage Apache Zookeeper in your environment for a distributed configuration store.

Let us start with some simple things

The Environment Itself

I personally prefer to access zookeeper using the kazoo module in Python, so that's what I will utilize for these examples. Zookeeper itself is written in Java, and if you prefer to utilize java feel free. The basic principles we will be covering are not language bound (but as a side note, if you prefer Java over Python, please seek the help of a professional as Java contains all of the syntax and overhead baggage of C++ with none of its performance benefits, but whatever makes you happy I guess. And please keep in mind that I write my articles in raw HTML, so take my advice on the easier way with a grain of salt)

Let's assume that you have a working zookeeper deployment at you disposal with 3 nodes in play (although I would recommend 5 for true resilient deployments in production environments). Let's also assume:
  • You are not utilizing the default pot of 2181 for zookeeper client connections as this can collide with other fabric providing clients (use 2180 instead)
  • Likewise, you have change the internal ports for intranode comms from 2888:3888 to something else reasonable to your environment for the same reason as above (this will not affect our work, but is just a good practice)
  • You do not have administrative access to the hardware on which either zookeeper, python, or your program will run (will impact how we build our application for configuration management. We don't want to be reliant on any pre-installed libraries at the OS level
  • Python is version 2.7.5+

With these caveats in mind, let's discuss how we should lay out our distributed configuration management tool, which I will refer to as dman (DEE-man) moving forward.

Basic Directory Structure

Let's assume that the system administrator on our platform (in this case a VM running CentOS 7) has provisioned a user named dman on the host system.
As a side note: If you want to start with exact setup I will be using, please work through this article How to Configure a 3-Node Zookeeper Cluster Without Root Privileges