Consul from Hashicorp is a tool used in distributed architectures to allow service discovery, health checking and kv storage for configuration. This article details how to use Consul for storing the configuration in ASP.Net Core by implementing a ConfigurationProvider.

Why use a tool to store the configuration ?

Usually, the configuration in .Net apps is stored in configuration files such as App.config, Web.config or appsettings.json. Starting with ASP.Net Core, a new and extensible configuration framework appeared, it allows to store the configuration outside of the config files and retrieving them from the command line, the environment variables, etc.
The issue with configuration files is that they can be difficult to manage. In fact, we usually end with a base configuration file and transformations files to override for each environment. They’re delivered at the same time than the binaries and therefore, changing a configuration value means redeploying configuration and binaries. Not very convenient.
Using a separate tool to centralize allows us two thing :

• Having the same configuration across all the machines (no machine out of sync)
• Being able to change a value without redeploying anything (useful for feature toggling)

Introducing Consul

The purpose of this article is not to talk about Consul but instead to focus on using it with ASP.Net Core.
However, it can be useful to remind few things. Consul has a Key/Value store available, it’s organized hierarchically and folders can be created to map the different application, environments etc. Here’s an example of a hierarchy that is going to be used along this article. Each end node can contain a JSON value.

/
|-- App1
| |-- Dev
| | |-- ConnectionStrings
| | \-- Settings
| |-- Staging
| | |-- ConnectionStrings
| | \-- Settings
| \-- Prod
|   |-- ConnectionStrings
|   \-- Settings
\-- App2
  |-- Dev
  | |-- ConnectionStrings
  | \-- Settings
  |-- Staging
  | |-- ConnectionStrings
  | \-- Settings
  \-- Prod
    |-- ConnectionStrings
    \-- Settings

Querying is easy as it is a REST API, the keys are in the query. For example the query for getting the settings of App1 in the Dev environment looks like this : GET http://<host>:8500/v1/kv/App1/Dev/Settings
The response looks like this :

HTTP/1.1 200 OK
Content-Type: application/json
X-Consul-Index: 1071
X-Consul-Knownleader: true
X-Consul-Lastcontact: 0

[
    {
        "LockIndex": 0,
        "Key": "App1/Dev/Settings",
        "Flags": 0,
        "Value": "ewogIkludCI6NDIsCiAiT2JqZWN0IjogewogICJTdHJpbmciOiAidG90byIsCiAgIkJsYSI6IG51bGwsCiAgIk9iamVjdCI6IHsKICAgIkRhdGUiOiAiMjAxOC0wMi0yM1QxNjoyMTowMFoiCiAgfQogfQp9Cgo=",
        "CreateIndex": 501,
        "ModifyIndex": 1071
    }
]

It’s also possible to query any node in a recursive manner, GET http://<host>:8500/v1/kv/App1/Dev?recurse gives :

HTTP/1.1 200 OK
Content-Type: application/json
X-Consul-Index: 1071
X-Consul-Knownleader: true
X-Consul-Lastcontact: 0

[
    {
        "LockIndex": 0,
        "Key": "App1/Dev/",
        "Flags": 0,
        "Value": null,
        "CreateIndex": 75,
        "ModifyIndex": 75
    },
    {
        "LockIndex": 0,
        "Key": "App1/Dev/ConnectionStrings",
        "Flags": 0,
        "Value": "ewoiRGF0YWJhc2UiOiAiU2VydmVyPXRjcDpkYmRldi5kYXRhYmFzZS53aW5kb3dzLm5ldDtEYXRhYmFzZT1teURhdGFCYXNlO1VzZXIgSUQ9W0xvZ2luRm9yRGJdQFtzZXJ2ZXJOYW1lXTtQYXNzd29yZD1teVBhc3N3b3JkO1RydXN0ZWRfQ29ubmVjdGlvbj1GYWxzZTtFbmNyeXB0PVRydWU7IiwKIlN0b3JhZ2UiOiJEZWZhdWx0RW5kcG9pbnRzUHJvdG9jb2w9aHR0cHM7QWNjb3VudE5hbWU9ZGV2YWNjb3VudDtBY2NvdW50S2V5PW15S2V5OyIKfQ==",
        "CreateIndex": 155,
        "ModifyIndex": 155
    },
    {
        "LockIndex": 0,
        "Key": "App1/Dev/Settings",
        "Flags": 0,
        "Value": "ewogIkludCI6NDIsCiAiT2JqZWN0IjogewogICJTdHJpbmciOiAidG90byIsCiAgIkJsYSI6IG51bGwsCiAgIk9iamVjdCI6IHsKICAgIkRhdGUiOiAiMjAxOC0wMi0yM1QxNjoyMTowMFoiCiAgfQogfQp9Cgo=",
        "CreateIndex": 501,
        "ModifyIndex": 1071
    }
]

We can see multiple things with these responses, first we can see that each key has its value encoded in Base64 to avoid mixing the JSON of the answer with the JSON of the value, then we notice the properties “Index” either in the JSON and in the HTTP headers. Those properties are a kind of timestamp, they allow to know if/when a value was created or updated. They will allow us to know if we need to reload the configuration.

ASP.Net Core configuration system

The configuration infrastructure relies on several things in the Microsoft.Extensions.Configuration.Abstractions NuGet package. First, the IConfigurationProvider is the interface to implement for supplying configuration values, then IConfigurationSource has for purpose giving an instance of the implemented configuration provider.
You can observe several implementations on the ASP.Net GitHub.
Hopefully, instead of directly implementing the IConfigurationProvider, it’s possible to inherit a class named ConfigurationProvider in the Microsoft.Extensions.Configuration package which takes care of the boilerplate code (such as the reload token implementation).
This class contains two important things :

/* Excerpt from the implementation */
public abstract class ConfigurationProvider : IConfigurationProvider
{
    protected IDictionary<string, string> Data { get; set; }
    public virtual void Load()
    {
    }
}

Data is the dictionary containing all the keys and values, Load is the method used at the beginning of the application, as its name indicates, it loads configuration from somewhere (a config file, or our consul instance) and hydrates the dictionary.

Loading consul configuration in ASP.Net Core

The first implementation that we can make, is going to use a HttpClient to fetch the configuration in consul. Then as the configuration is hierarchical (it’s a tree), we will need to flatten it, in order to put it in the dictionary. Easy no ?

First thing, implementing the Load method. It doesn’t do much as we need an asynchronous one, this one will just block the asynchronous call (although it is not the best to block, it is inspired by the ASP.Net core implementation).

public override void Load() => LoadAsync().ConfigureAwait(false).GetAwaiter().GetResult();

Then, we are going to query consul to get the configuration values, in a recursive way (see above). It uses some objects defined in the class such as _consulUrls which is an array of urls to consul instances (for fail-over), _path is the prefix of the keys (such as App1/Dev). Once we get the json, we iterate on each key/value pair, decoding the Base64 string and then flattening all the keys and the JSON objects.

private async Task<IDictionary<string, string>> ExecuteQueryAsync()
{
    int consulUrlIndex = 0;
    while (true)
    {
        try
        {
            using (var httpClient = new HttpClient(new HttpClientHandler { AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip }, true))
            using (var request = new HttpRequestMessage(HttpMethod.Get, new Uri(_consulUrls[consulUrlIndex], "?recurse=true")))
            using (var response = await httpClient.SendAsync(request))
            {
                response.EnsureSuccessStatusCode();
                var tokens = JToken.Parse(await response.Content.ReadAsStringAsync());
                return tokens
                    .Select(k => KeyValuePair.Create
                    (
                        k.Value<string>("Key").Substring(_path.Length + 1),
                        k.Value<string>("Value") != null ? JToken.Parse(Encoding.UTF8.GetString(Convert.FromBase64String(k.Value<string>("Value")))) : null
                    ))
                    .Where(v => !string.IsNullOrWhiteSpace(v.Key))
                    .SelectMany(Flatten)
                    .ToDictionary(v => ConfigurationPath.Combine(v.Key.Split('/')), v => v.Value, StringComparer.OrdinalIgnoreCase);
            }
        }
        catch
        {
            consulUrlIndex++;
            if (consulUrlIndex >= _consulUrls.Count)
                throw;
        }
    }
}

The method that flattens the keys and values is a simple Depth First Search on the tree.

private static IEnumerable<KeyValuePair<string, string>> Flatten(KeyValuePair<string, JToken> tuple)
{
    if (!(tuple.Value is JObject value))
        yield break;

    foreach (var property in value)
    {
        var propertyKey = $"{tuple.Key}/{property.Key}";
        switch (property.Value.Type)
        {
            case JTokenType.Object:
                foreach (var item in Flatten(KeyValuePair.Create(propertyKey, property.Value)))
                    yield return item;
                break;
            case JTokenType.Array:
                break;
            default:
                yield return KeyValuePair.Create(propertyKey, property.Value.Value<string>());
                break;
        }
    }
}

The whole class with its constructor and its fields looks this:

public class SimpleConsulConfigurationProvider : ConfigurationProvider
{
    private readonly string _path;
    private readonly IReadOnlyList<Uri> _consulUrls;

    public SimpleConsulConfigurationProvider(IEnumerable<Uri> consulUrls, string path)
    {
        _path = path;
        _consulUrls = consulUrls.Select(u => new Uri(u, $"v1/kv/{path}")).ToList();

        if (_consulUrls.Count <= 0)
        {
            throw new ArgumentOutOfRangeException(nameof(consulUrls));
        }
    }

    public override void Load() => LoadAsync().ConfigureAwait(false).GetAwaiter().GetResult();

    private async Task LoadAsync()
    {
        Data = await ExecuteQueryAsync();
    }

    private async Task<IDictionary<string, string>> ExecuteQueryAsync()
    {
        int consulUrlIndex = 0;
        while (true)
        {
            try
            {
                var requestUri = "?recurse=true";
                using (var httpClient = new HttpClient(new HttpClientHandler { AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip }, true))
                using (var request = new HttpRequestMessage(HttpMethod.Get, new Uri(_consulUrls[consulUrlIndex], requestUri)))
                using (var response = await httpClient.SendAsync(request))
                {
                    response.EnsureSuccessStatusCode();
                    var tokens = JToken.Parse(await response.Content.ReadAsStringAsync());
                    return tokens
                        .Select(k => KeyValuePair.Create
                        (
                            k.Value<string>("Key").Substring(_path.Length + 1),
                            k.Value<string>("Value") != null ? JToken.Parse(Encoding.UTF8.GetString(Convert.FromBase64String(k.Value<string>("Value")))) : null
                        ))
                        .Where(v => !string.IsNullOrWhiteSpace(v.Key))
                        .SelectMany(Flatten)
                        .ToDictionary(v => ConfigurationPath.Combine(v.Key.Split('/')), v => v.Value, StringComparer.OrdinalIgnoreCase);
                }
            }
            catch
            {
                consulUrlIndex = consulUrlIndex + 1;
                if (consulUrlIndex >= _consulUrls.Count)
                    throw;
            }
        }
    }

    private static IEnumerable<KeyValuePair<string, string>> Flatten(KeyValuePair<string, JToken> tuple)
    {
        if (!(tuple.Value is JObject value))
            yield break;

        foreach (var property in value)
        {
            var propertyKey = $"{tuple.Key}/{property.Key}";
            switch (property.Value.Type)
            {
                case JTokenType.Object:
                    foreach (var item in Flatten(KeyValuePair.Create(propertyKey, property.Value)))
                        yield return item;
                    break;
                case JTokenType.Array:
                    break;
                default:
                    yield return KeyValuePair.Create(propertyKey, property.Value.Value<string>());
                    break;
            }
        }
    }
}

Dynamic configuration reloading

We can go further by using the change notification of consul. It works by just adding a parameter (the value of the last index configuration), the HTTP request is now blocking till the next configuration change (or the timeout the HttpClient).
Compared to the previous class, we just have to add a method ListenToConfigurationChanges to listen in background to the blocking HTTP endpoint of consul and refactor a little.

public class ConsulConfigurationProvider : ConfigurationProvider
{
    private const string ConsulIndexHeader = "X-Consul-Index";

    private readonly string _path;
    private readonly HttpClient _httpClient;
    private readonly IReadOnlyList<Uri> _consulUrls;
    private readonly Task _configurationListeningTask;
    private int _consulUrlIndex;
    private int _failureCount;
    private int _consulConfigurationIndex;

    public ConsulConfigurationProvider(IEnumerable<Uri> consulUrls, string path)
    {
        _path = path;
        _consulUrls = consulUrls.Select(u => new Uri(u, $"v1/kv/{path}")).ToList();

        if (_consulUrls.Count <= 0)
        {
            throw new ArgumentOutOfRangeException(nameof(consulUrls));
        }

        _httpClient = new HttpClient(new HttpClientHandler { AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip }, true);
        _configurationListeningTask = new Task(ListenToConfigurationChanges);
    }

    public override void Load() => LoadAsync().ConfigureAwait(false).GetAwaiter().GetResult();

    private async Task LoadAsync()
    {
        Data = await ExecuteQueryAsync();

        if (_configurationListeningTask.Status == TaskStatus.Created)
            _configurationListeningTask.Start();
    }

    private async void ListenToConfigurationChanges()
    {
        while (true)
        {
            try
            {
                if (_failureCount > _consulUrls.Count)
                {
                    _failureCount = 0;
                    await Task.Delay(TimeSpan.FromMinutes(1));
                }

                Data = await ExecuteQueryAsync(true);
                OnReload();
                _failureCount = 0;
            }
            catch (TaskCanceledException)
            {
                _failureCount = 0;
            }
            catch
            {
                _consulUrlIndex = (_consulUrlIndex + 1) % _consulUrls.Count;
                _failureCount++;
            }
        }
    }

    private async Task<IDictionary<string, string>> ExecuteQueryAsync(bool isBlocking = false)
    {
        var requestUri = isBlocking ? $"?recurse=true&index={_consulConfigurationIndex}" : "?recurse=true";
        using (var request = new HttpRequestMessage(HttpMethod.Get, new Uri(_consulUrls[_consulUrlIndex], requestUri)))
        using (var response = await _httpClient.SendAsync(request))
        {
            response.EnsureSuccessStatusCode();
            if (response.Headers.Contains(ConsulIndexHeader))
            {
                var indexValue = response.Headers.GetValues(ConsulIndexHeader).FirstOrDefault();
                int.TryParse(indexValue, out _consulConfigurationIndex);
            }

            var tokens = JToken.Parse(await response.Content.ReadAsStringAsync());
            return tokens
                .Select(k => KeyValuePair.Create
                    (
                        k.Value<string>("Key").Substring(_path.Length + 1),
                        k.Value<string>("Value") != null ? JToken.Parse(Encoding.UTF8.GetString(Convert.FromBase64String(k.Value<string>("Value")))) : null
                    ))
                .Where(v => !string.IsNullOrWhiteSpace(v.Key))
                .SelectMany(Flatten)
                .ToDictionary(v => ConfigurationPath.Combine(v.Key.Split('/')), v => v.Value, StringComparer.OrdinalIgnoreCase);
        }
    }

    private static IEnumerable<KeyValuePair<string, string>> Flatten(KeyValuePair<string, JToken> tuple)
    {
        if (!(tuple.Value is JObject value))
            yield break;

        foreach (var property in value)
        {
            var propertyKey = $"{tuple.Key}/{property.Key}";
            switch (property.Value.Type)
            {
                case JTokenType.Object:
                    foreach (var item in Flatten(KeyValuePair.Create(propertyKey, property.Value)))
                        yield return item;
                    break;
                case JTokenType.Array:
                    break;
                default:
                    yield return KeyValuePair.Create(propertyKey, property.Value.Value<string>());
                    break;
            }
        }
    }
}

Plug everything together

We now have a ConfigurationProvider, let’s have a ConfigurationSource to create our provider.

public class ConsulConfigurationSource : IConfigurationSource
{
    public IEnumerable<Uri> ConsulUrls { get; }
    public string Path { get; }

    public ConsulConfigurationSource(IEnumerable<Uri> consulUrls, string path)
    {
        ConsulUrls = consulUrls;
        Path = path;
    }

    public IConfigurationProvider Build(IConfigurationBuilder builder)
    {
        return new ConsulConfigurationProvider(ConsulUrls, Path);
    }
}

And some extension methods to use easily our source :

public static class ConsulConfigurationExtensions
{
    public static IConfigurationBuilder AddConsul(this IConfigurationBuilder configurationBuilder, IEnumerable<Uri> consulUrls, string consulPath)
    {
        return configurationBuilder.Add(new ConsulConfigurationSource(consulUrls, consulPath));
    }

    public static IConfigurationBuilder AddConsul(this IConfigurationBuilder configurationBuilder, IEnumerable<string> consulUrls, string consulPath)
    {
        return configurationBuilder.AddConsul(consulUrls.Select(u => new Uri(u)), consulPath);
    }
}

We can now declare Consul in our Program.cs the consul source using other sources (such as environment variables or command line arguments) to provide the urls.

public static IWebHost BuildWebHost(string[] args) =>
    WebHost.CreateDefaultBuilder(args)
    .ConfigureAppConfiguration(cb =>
        {
            var configuration = cb.Build();
            cb.AddConsul(new[] { configuration.GetValue<Uri>("CONSUL_URL") }, configuration.GetValue<string>("CONSUL_PATH"));
        })
        .UseStartup<Startup>()
        .Build();

Now, it’s possible to use the standard configuration patterns of ASP.Net Core such as Options.

public void ConfigureServices(IServiceCollection services)
{
    services.AddMvc();
    services.AddOptions();
    services.Configure<AppSettingsOptions>(Configuration.GetSection("Settings"));
    services.Configure<AccountingFeaturesOptions>(Configuration.GetSection("FeatureFlags"));
    services.Configure<CartFeaturesOptions>(Configuration.GetSection("FeatureFlags"));
    services.Configure<CatalogFeaturesOptions>(Configuration.GetSection("FeatureFlags"));
}

To use them in our code, be careful of how you use options, as for options that can be reloaded dynamically, using IOptions<T> you would get a the initial value. Instead, ASP.Net Core requires to use IOptionsSnapshot<T>.
This scenario is really awesome for feature toggling as you can enable and disable new features just by changing the toggle value in consul and, without delivering anything, customers can use those new features. In a same manner, if a feature is bugged, you can disable it, without rolling back or hot fixing.

public class CartController : Controller
{
    [HttpPost]
    public IActionResult AddProduct([FromServices]IOptionsSnapshot<CartFeaturesOptions> options, [FromBody] Product product)
    {
        var cart = _cartService.GetCart(this.User);
        cart.Add(product);
        if (options.Value.UseCartAdvisorFeature)
        {
            ViewBag.CartAdvice = _cartAdvisor.GiveAdvice(cart);
        }
        return View(cart);
    }
}

Conclusion

Those few lines of code allowed us to add the support for consul configuration in our ASP.Net Core application. In fact, any application (even classic .Net app that use Microsoft.Extensions.Configuration packages) can benefit of this. Very cool in a DevOps environment, you can centralize all your configurations in one place and use hot reloading to have feature toggling live.