I have been using salt for quite a while (oldest commit in my personal salt states repo is 2010), but have usually taken the old approach to distributing modules. By setting the file_roots you can automatically sync all your modules with the saltutil.sync_all command.

file_roots:
  base:
    - /srv/salt
    - /srv/salt-kafka
    - /srv/salt-forgejo

With our file_roots defined, our plugin repo looks like this

tree /srv/salt-kafka
├── LICENSE
├── README.md
└── _engines
    └── kafka_consumer.py

This is useful, but for maintainability, I finally sat down to look at how to package things up more properly.

From the Salt Extensions concepts page, you can see that extensions have been supported for a long time (since 2016), with more user friendly extensions supported since 2020. We therefore want to update our repo layout to look like this.

tree /srv/salt-kafka
├── LICENSE
├── README.md
├── pyproject.toml
└── src
    └── salt_kafka
        ├── __init__py
        └── engines
            ├── __init__.py
            └── kafka_consumer.py

Our minimum pyproject.toml will look like this.

# All the normal project bits and dependnecies
[project]
name = "salt-kafka"
version = "0.1.0"
dependencies = [
    "salt>=3006",
    "confluent-kafka>=2.4.0",
]

# Then we register our entrypoint.
# The name doesn't matter but the path does
[project.entry-points."salt.loader"]
"salt_kafka" = "salt_kafka"

We can then install our Salt extension using the salt-pip wrapper that comes with the onedir distribution.

salt-pip install --editable /srv/salt-kafka

Previously I would run saltutil.sync_all to sync all modules from the salt-master but now with extension modules I will need to install them on each machine. I will likely need to do some kind of state like the following.

Editable Modules:
  pip.installed:
    - editable:
        - /srv/salt-kafka
    - watch_in:
        - service: salt-master
    - pkgs: []

Though for now with the watch_in directive, it restarts the server each time, so I will likely need to figure something else out.

Aside

I am not quite ready to give up Salt for something like Kubernetes yet, though Broadcoms rough support of the Salt project lately does have me a little worried. I did worry in the past that the all inclusive nature of so many modules in the Salt tree would cause a maintenance burden and it looks like that happened. Hopefully when they finish the split, it will breath some new life into the Salt project.