Hello,

I have a problem I'm facing and I think many of us have faced it.
I'm wondering what the right approach is or what to do in this situation.

In the beginning there was a config file. This file was plain old dumb data and lived life as a yaml, json, toml or other.

Some human would author it, then it'd get fed into then the main program would read it.

Now, some humans want more power like if statements and loops.

How to satisfy them?

At many places I've worked I've seen this turn into a ‘ProgrammableConfLanguage’ that can be a pain to maintain with a syntax no one really likes.

So, what is a good approach?

Maybe just skipping the ‘programmable conf’ and jumping straight to a language like lua, or python, with a “baking step” that would run the “.lua” and output the resulting dict or definition into a plain old data “.toml”?

After all, the the “programmable conf” either needs an embedded vm in the binary that parses it, or it needs to be cooked beforehand. So why not just have a programming language that goes through a “compile/run” to generate the dumb data?

What do you think is a good way to address this pressure of DumbDataConf → ProgrammableConf, aka devs wanting more power.

(fun thing to remember: both emacs and vim's conf are programming languages)

I like the config files that just use some standard script/interpreted language.

Say nearly all the config for say Ruby on rails which are just Ruby or Factorio using Lua scripts to define all its object prototypes.

Rails.application.configure do
  # Settings specified here will take precedence over those in config/application.rb.

  # In the development environment your application's code is reloaded on
  # every request. This slows down response time but is perfect for development
  # since you don't have to restart the web server when you make code changes.
  config.cache_classes = false

  # Do not eager load code on boot.
  config.eager_load = false

  # Show full error reports.
  config.consider_all_requests_local = true

  # Enable/disable caching. By default caching is disabled.
  # Run rails dev:cache to toggle caching.
  if Rails.root.join('tmp', 'caching-dev.txt').exist?
    config.action_controller.perform_caching = true
    config.action_controller.enable_fragment_cache_logging = true

    config.cache_store = :memory_store
    config.public_file_server.headers = {
      'Cache-Control' => "public, max-age=#{2.days.to_i}"
    }
  else
    config.action_controller.perform_caching = false

    config.cache_store = :null_store
  end
  ...

data:extend(
{
    type = "logistic-container",
    name = "logistic-chest-storage",
    icon = "__base__/graphics/icons/logistic-chest-storage.png",
    icon_size = 64, icon_mipmaps = 4,
    flags = {"placeable-player", "player-creation"},
    minable = {mining_time = 0.1, result = "logistic-chest-storage"},
    max_health = 350,
    ...
})
...

local infinity_chest = util.table.deepcopy(data.raw["logistic-container"]["logistic-chest-storage"])
infinity_chest.type = "infinity-container"
infinity_chest.name = "infinity-chest"
infinity_chest.icon = "__base__/graphics/icons/infinity-chest.png"
...
data:extend({infinity_chest})

It is up to you to decide exactly how the interface between the config and code works. For example you might create bindings between native objects and the script ones, or maybe you do just take a bunch of hash/array/etc, objects and use those as if you would JSON etc.

The former might be useful if you decide to use scripts for a lot of other things, as you might want to access these objects again later at runtime in scripts (missions, AI, etc.).

Doing it in the main game executable is extremely nice for modders, so that might be a consideration. It also just makes changes in general easier as people can just change something then restart the level or such without a special tool step.

Do you need a configuration file at all, or can you hardcode the configuration in the executable (and recompile when the configuration changes)? When hardcoding, you have the full power of your chosen programming language available for configuration.

Does the configuration need to be directly human-editable at all, or can you hide this beneath a separate configuration program? The smarter the configuration program, the dumber the configuration file is allowed to be.

If you can't write a sufficiently powerful configuration program yourself, can you encourage your users to write their own custom scripts for generating configuration files?

If none of these work for you, using an off-the-shelf scripting language is pretty much the least bad option left over.

Plotnus said:
Now, some humans want more power like if statements and loops

Shouldn't the right question be here “What does those humans want to do that they need loops and conditionals for”? Either you have a configuration file like JSON, INI and other or you have batch scripts but what are you trying to achieve and whyt is it used for?

I was at a similar point of ‘enhancing’ my configuration files for our build system that uses different profiles based on a target architecture, a configuration and a platform setting which are loosly coupled into a profile. For this purpose, I decided to have a text-templating language instead that looks some kind similar to C++ preprocessor code with full conditional and include support on top of any possible file format. We choose JSON as our base file format so our Alchemy library produces JSON out of those .alc files.

I honestly never ever needed something more complex for ‘configuring’ a program