Enable support for JSON comments

[goldsam]

It is natural and common to use comments in configuration to document reasoning, motivations, and other contextual information. The need and usefulness of configuration comments has a very long history. Many *unix apps have supported # prefixed comments since the beginning of time 🦖.

Pretty please add support JSON comments.

[amerjusupovic]

Thanks for the suggestion! Just to clarify, are you asking about adding comments to JSON key-values in your App Configuration store? This would require some changes outside of just the .NET provider so it might take more time for me to understand how this would be done.

[jimmyca15]

Hello Sam G (@goldsam), I believe the ask here is to handle comments inside of JSON formatted settings with content type application/json. Our implementation follows the JSON spec and comments are not part of the JSON spec. There are problems drifting from the spec that make it difficult to take that step. I am wondering the specific use case here and if perhaps adding tags on the settings in your app configuration could be of use here?

[goldsam]

I gave the context already - configuration often (read always) needs context as comments. Here's a table of existing support for comment in configuration:

Format	Extensions	Comment Syntax
INI	.ini, .cfg	; or #
YAML	.yaml, .yml	#
TOML	.toml	#
JSONC / JSON5	.json	// and /* */
HCL	.hcl, .tf	#, //, or /* */
Dockerfile	Dockerfile	#
Bash script	.sh, .bashrc, etc.	#
Python	.py (also used for config like settings.py)	#
CUE	.cue	//, /* */
Nginx config	.conf	#
Apache config	.conf	#
Makefile	Makefile	#
Git config	.gitconfig	#
Gradle (Groovy)	build.gradle	//, /* */
Gradle (Kotlin)	build.gradle.kts	//, /* */
Systemd unit files	.service, etc.	#

Need further example of real-world support?

Tool / Context	Notes
Visual Studio Code	settings.json, launch.json, etc. use JSONC
TypeScript config	tsconfig.json is parsed with comments allowed
ESLint / Prettier	Can use .jsonc with the right parser
Azure Pipelines Templates	Sometimes use comment-tolerant JSON tooling

I'm guessing you already use vscode. System.Text.Json already support comments - it a trivial change to add support to this library. There is no mime type for JSONC or JSON5 - I welcome you or Microsoft to register one with IANA. In the meantime, I would like pragmatic solution.

[jimmyca15]

Appreciate the feedback on this Sam G (@goldsam). I understand what you mean about System.Text.Json already supporting comments, Typically, changes such as supporting comments in JSON would be something that would not be implemented only in .NET. The typical experience with App Configuration is consistent support across all languages, Azure Portal and CLI. This is the experience with JSON support today. I hope that adds a bit where this is coming from.

On the topic of System.Text.Json supporting JSON comments, and if you're specific scenario is only for .NET without any concern for other languages, Portal support, or Azure CLI support for JSON with comments then you can use the configuration providers Map capabilities.

builder.Configuration.AddAzureAppConfiguration(options =>
{
    options.Connect(new Uri("<your-app-configuration-endpoint>"), new DefaultAzureCredential())
           .Map(setting =>
           {
               if (setting.ContentType == "application/jsonc")
               {
                   // Reserialize without comments
                   setting.Value = JsonSerializer.Serialize(
                       JsonSerializer.Deserialize<JsonElement>(
                           setting.Value,
                           new JsonSerializerOptions
                           {
                               ReadCommentHandling = JsonCommentHandling.Skip,
                               AllowTrailingCommas = true // Optional: handle trailing commas
                           }));

                   // Update content type to application/json for Azure App Configuration provider to later process
                   setting.ContentType = "application/json";
               }

               return ValueTask.FromResult(setting);
           });
});

This approach does have drawbacks

• performing a round-trip serialization. If the number of JSONC settings are few I expect it should be inconsequential.
• Settings with this custom content type won't get expected application/json support in other languages, Azure Portal, Azure CLI.

Please let me know if I misunderstood and/or if you are asking for universal support for JSON with comments across App Configuration.

[goldsam]

With all due respect, configuration files have a rich history of supporting comments since they appeared in computing history - a time well before either of us were born (nice headshot 😉). It's not a nice-to-have - it's a solid requirement of any sys admin. It's kind of absurd that this history and common feature/requirement - core to your domain - was not respected by a service called app configuration.

Let's take a step back though an examine how incoherent the situation really is:

• The core app configuration service supports a content type field which can be virtually anything. The choice was arbitrary to strictly validate JSON content in the portal, but not any other content types whatsoever (ignoring the invented JSON representation for key vault secret references which I don't believe is registered with IANA - so the standards compliance argument is a red herring).
• A library directly making HTTP calls can encode data however it wants to - JSON not required.
• It follows from the last point that it's really up to the language/client to determine how configuration data is serialized/encoded/interpreted.
• Microsoft for obvious reasons favors C#, their own language for which their own library supports JSON with comments.
• Monaco editor, also built by Microsoft and used in their portal for app configuration service, supports JSON with comments.
• All of Microsoft's other products, including vscode for which monaco was created, support JSON with comments in configuration.
• The choice of this library to only support JSON without comments is arbitrary and disregards app service's own design choice of user-specified content types (see Add yaml support #381 which I have also been participating in).

Summary:

• This library (and I guess also the CLI/portal) are already not respecting the content-type feature of the app configuration service coherently and should be fixed.
• It is completely arbitrary that the azure portal strictly validates JSON not supporting comments.
• It's really the client's issue to encode/parse/validate at the end of the day.

Yes - fix the portal, lib, and CLI. Not having comment support is simply unacceptable. Make an option to enforce strict JSON validation in the portal. Or change validation errors to be warnings and still allow saving the settings in the portal. Add a banner with a warning.

I've worked closely with a lot of people in engineering and computing for a long, long time. I imagine how my sys admin friends would react to the statement "You cannot comment or document your configuration". I've worked in many situations where mission critical configuration was supported by reams of inline documentation/comments. This was true in defense as well as in every private company I've worked.

Just promise me if/when YAML support is added that it supports comments. At least that hasn't been screwed up yet in the CLI and portal.

[carlin-q-scott]

This is what Douglas Crockford (inventor of JSON) said about removing comments from JSON.

I removed comments from JSON because I saw people were using them to hold parsing directives, a practice which would have destroyed interoperability. I know that the lack of comments makes some people sad, but it shouldn't.

Suppose you are using JSON to keep configuration files, which you would like to annotate. Go ahead and insert all the comments you like. Then pipe it through JSMin before handing it to your JSON parser.

https://hashnode.com/post/why-did-douglas-crockford-remove-comments-from-json-cir97ves31fxu2i536bp7u37f

So, the creator of the language and spec for it, says it can contain comments if used as a configuration format. And that it is fairly easy to enable support for comments.

I understand your concern that adding comment support in this SDK, may imply to users of other SDKs that they are also supported. But that support should also be implemented in those SDKs, and that works needs to be tracked in the appropriate issue trackers for those SDKS. This issue is for this SDK. Users of this SDK will benefit from the change, and we can deal with complications of interoperability between SDKs if we want to use comments in our configuration.

[jimmyca15]

Sam G (@goldsam) Carlin Scott (@carlin-q-scott)

You are right. We will make some changes as requested here. And we will do it incrementally.

Firstly, we will update this library to handle JSON values with comments, whereas currently it would throw.

Secondly, we will update all other App Configuration Provider libraries to be able to handle comments as well. Most likely this is in tandem with the first item.

Lastly, we will update the App Configuration Portal editor, as well as CLI, to no longer restrict the usage of comments in JSON payloads.

Does this sound like a proper solution? Let me know if I missed anything in your scope.

Thanks for sticking with this and apologies for not realizing it sooner.

[RichardChen820]

Maryanne Gichohi (@MaryanneNjeri) TS import library also needs to support it

[goldsam]

Jimmy Campbell (@jimmyca15) Thank you for your reconsideration of the matter. Greatly appreciated ☺️

[zhiyuanliang-ms]

Hey, Sam G (@goldsam) Carlin Scott (@carlin-q-scott) 8.4.0 has been released. For more information, you can check the feature status overview.

Thank you for the support ;)