TimI often see questions on StackOverflow and other Q&A tech sites about running cron jobs on single servers within clusters of AWS instances. Running the same job on each machine tends to be simple – ensure the appropriate crontab exists on every machine (usually through your deployment process, Docker image, or AMI) and you’re good to go.
What happens when you want a scheduled job to run on just one server? The use cases for these jobs are fairly common: generating reports, sending emails, writing to a distributed filesystem, archiving data outside of your backup process, monthly roll-up of analytics data, health checks, etc.
For an easy way to support these types of scheduled tasks without complicating your stack, check out Cronally.
Unless you’re running a stack that includes some type of scheduling services (i.e. CoreOS, Mesosphere, Kubernetes, etc.) the common method for running these jobs is by using locks. The complexity of such a setup depends on how critical your jobs will be. A single Redis instance can act as a mutex, allowing one machine in the cluster to set a key to a particular value and return an error to all other machines that attempt to set the same key. Adding a TTL to the key ensures the lock will expire after some time.
The problem with such a simple implementation is that one Redis instance becomes a single point of failure. If the instance crashes, terminates, runs out of memory, etc. your jobs will never run.
A more sophisticated implementation is a distributed lock that relies on several Redis instances. The failure of one instance should still allow your jobs to run as expected. Due to the complexity of such a setup, there are several utility clients that help with this: http://redis.io/topics/distlock
The following basic example uses the Redsync Go implementation:
Note that we’re not performing any unlocking in this example and simply waiting for the default Redsync TTL to expire the lock, which is about 8 seconds.
Interested in learning more about Cronally? Check it out!