Taking Azure Redis Cache for a ride – testing a simple Producer/Consumer program

In my “younger” days, I worked in a team that developed/maintained an application written in Ada, which used Oracle 8 as its database. Our performance requirements didn’t allow us to query the DB all the time, so the team had developed an in-memory cache on top of the database. We had a full-blown ORM mapping from the DB to Ada, with in-memory querying mechanisms, cache refreshes, and all the good stuff.

But why am I talking about this now? Because this taught me to really like caching mechanism. And now that Azure Redis cache came out, I had to get my hands dirty. I’ve heard a lot about Redis from many sources, but I never had time to learn about it. Last week I decided to give it a try.

The specific use case I had in mind was using it as a FAST medium of communication between servers, in a publish-subscribe fashion. What a delight it was to find that this use case is supported by Redis! I wanted to know how much time it took for a message to go from a consumer to a producer, so I created a simple ping-pong program, where one server sends ping and the other responds pong. I did this double trip because I don’t trust clock sync algorithms enough, and wanted to do all time measurements in the same machine. Since my code is very simple, just dividing by 2 will give the approximate time it takes for a message to get from one server to the other.

I tried the two different cache tiers provided by Azure: the basic tier with no replication, and the standard tier with no replication. Both have caches that are shared (not dedicated) and dedicated (size above 1GB). For my tests, I tread both the Basic and Standard tiers, with 1GB size so that in both cases it is a dedicated cache.

The test code is fairly simple: two azure worker roles, the Producer creates messages and pushes them to a Redis pub/sub queue, and the Consumer subscribes to the messages, and when it receives one, it pushes it back to another Redis queue to which the Producer is subscribed. The Producer checks that the value received is the same as the value sent, and counts the time it took for the message to go back and forth. I used the StackExchange.Redis nuget to access Redis.

This is the code for the Producer:

using System.Diagnostics;
using System.Net;
using System.Threading;
using Microsoft.WindowsAzure.ServiceRuntime;
using StackExchange.Redis;

namespace Producer
{
    public class WorkerRole : RoleEntryPoint
    {
        Stopwatch watch;
        ISubscriber subscriber;
        int count = 0;
        int ITERATIONS = 1000;

        public override void Run()
        {
            Trace.TraceInformation("Starting Test");
            ConnectionMultiplexer connection = ConnectionMultiplexer.Connect("");
            IDatabase cache = connection.GetDatabase();
            subscriber = connection.GetSubscriber();
            watch = new Stopwatch();
            subscriber.Subscribe("pong", (channel, message) => getPong(message));
            watch.Start();
            subscriber.Publish("ping", count);

            while (true)
            {
                Thread.Sleep(1000);
            }
        }

        private void getPong(RedisValue message)
        {
            watch.Stop();
            if (!count.ToString().Equals(message))
            {
                Trace.TraceInformation("Got wrong message. Sent:" + count + ", Received:" + message);
            }
            count++;
            if (count == ITERATIONS)
            {
                Trace.TraceInformation("Finished Test. Running " + count + " iterations. Total time:" + watch.ElapsedMilliseconds + ". Average: " + watch.ElapsedMilliseconds / count);
                finished = true;
                return;
            }
            watch.Start();
            subscriber.Publish("ping",count);
        }
    }
}

And the code of the Consumer (which is pretty simple) is this:

using System.Diagnostics;
using System.Threading;
using Microsoft.WindowsAzure.ServiceRuntime;
using StackExchange.Redis;namespace Consumer
{
    public class WorkerRole : RoleEntryPoint
    {
        public override void Run()
        {
            Trace.TraceInformation("Consumer started");
            Subscribe(ConnectionMultiplexer.Connect(""));
            while (true)
            {
                Thread.Sleep(1000);
            }
        }        public void Subscribe(ConnectionMultiplexer connection)
        {
            IDatabase cache = connection.GetDatabase();
            ISubscriber subscriber = connection.GetSubscriber();
            subscriber.Subscribe("ping", (channel, message) => { subscriber.Publish("pong", message); });
        }
    }
}

The result of the tests in both cases were the same. After running 1000 iterations, both caches gave an average of 15ms latency from the full round trip, and assuming that there was no delay inside the consumer (or that it is negligible), this means that a message takes around 7.5ms to get to its destination. That is pretty damn fast!

I hope to have some more time to learn about Redis and it’s uses, and to have a real project where I can test its capabilities in a real-world environment. But for starters, it is very, very promising.

The Elevator Test, the Theory of Motivation, DRY enough, and are Humans “Resources?” – Vainolo’s Weekly Readings

Can Your Team Pass The Elevator Test?
“Software developers think their job is writing code. But it’s not. Their job is to solve the customer’s problem”. Are you simply writing core or are you solving YOUR customer’s problem?
http://blog.codinghorror.com/can-your-team-pass-the-elevator-test/

95% of Managers Follow an Outdated Theory of Motivation
“a recent study by psychologist Susan David … asked people what made them so engaged and excited about their work. 96% of the employees didn’t mention pay at all… In describing their motivations at work, highly engaged employees “highlighted feeling autonomous and empowered, and a sense of belonging on their teams.”
Food for thought.
http://blog.idonethis.com/management-maslows-hierarchy-needs/

When is your code DRY enough?
Is having duplicate code always bad? Not really – “DRY is not about identical chunks of bits or characters in different places. DRY aims at not repeating domain concepts”
http://blog.8thcolor.com/en/2014/04/when-is-your-code-dry-enough/

People Are Not Resources
“People aren’t resources! People finish work. If you don’t want us to finish projects, let’s decide which projects not to do. Then we can re-allocate people, if we want. But we don’t start with people. That’s crazy.”
http://www.jrothman.com/blog/mpd/2014/08/people-are-not-resources.html

How Microsoft became agile, being a doctor, defining quality, and writing a blog – Vainolo’s weekly reading

How Microsoft dragged its development practices into the 21st century
A story of how Microsoft moved from the good-old waterfall software development methodology to more agile, fast and customer friendly methodologies.
http://arstechnica.com/information-technology/2014/08/how-microsoft-dragged-its-development-practices-into-the-21st-century/

At the office, are you a doctor or a patient?
When you go to your boss, are you talking to him as a doctor or as a patient? Are you asking for solutions or having a conversation on how to solve the problem?
http://www.plexxi.com/2014/07/office-doctor-patient/#sthash.TNCHUaUc.dpbs

Agile and the Definition of Quality
“Quality can be elusive if you don’t know how to find it, or you don’t know where to look.  Worse, even when you know where to look, you need to know how to manage the diversity of conflicting views.”
http://blogs.msdn.com/b/jmeier/archive/2014/07/17/agile-and-the-definition-of-quality.aspx

Why software engineers should maintain a blog
How many times have you found in a blog just the right solution to the problem you are trying to solve? It’s time to give back, even if just a little.
http://chase-seibert.github.io/blog/2014/08/01/why-blogging.html