Template was specified incorrectly

After reorganizing my salt configuration, I received the following error:

[ERROR   ] Template was specified incorrectly: False

Enabling some debugging on the command gave me a slight pointer why this occurred:

[DEBUG   ] Could not find file from saltenv 'testing', u'salt://top.sls'
[DEBUG   ] No contents loaded for env: testing
[DEBUG   ] compile template: False
[ERROR   ] Template was specified incorrectly: False

I was using a single top file as recommended by Salt, but apparently it was still looking for top files in the other environments.

Yet, if I split the top files across the environments, I got the following warning:

[WARNING ] Top file merge strategy set to 'merge' and multiple top files found. Top file merging order is undefined; for better results use 'same' option

So what's all this about?

When using a single top file is preferred

If you want to stick with a single top file, then the first error is (or at least, in my case) caused by my environments not having a fall-back definition.

My /etc/salt/master configuration file had the following file_roots setting:

file_roots:
  base:
    - /srv/salt/base
  testing:
    - /srv/salt/testing

The problem is that Salt expects ''a'' top file through the environment. What I had to do was to set the fallback directory to the base directory again, like so:

file_roots:
  base:
    - /srv/salt/base
  testing:
    - /srv/salt/testing
    - /srv/salt/base

With this set, the error disappeared and both salt and myself were happy again.

When multiple top files are preferred

If you really want to use multiple top files (which is also a use case in my configuration), then first we need to make sure that the top files of all environments correctly isolate the minion matches. If two environments would match the same minion, then this approach becomes more troublesome.

On the one hand, we can just let saltstack merge the top files (default behavior) but the order of the merging is undefined (and no, you can't set it using env_order) which might result in salt states being executed in an unexpected order. If the definitions are done to such an extend that this is not a problem, then you can just ignore the warning. See also bug 29104 about the warning itself.

But better would be to have the top files of the environment(s) isolated so that each environment top file completely manages the entire environment. When that is the case, then we tell salt that only the top file of the affected environment should be used. This is done using the following setting in /etc/salt/master:

top_file_merging_strategy: same

If this is used, then the env_order setting is used to define in which order the environments are processed.

Oh and if you're using salt-ssh, then be sure to set the environment of the minion in the roster file, as there is no running minion on the target system that informs salt about the environment to use otherwise:

# In /etc/salt/roster
testserver:
  host: testserver.example.com
  minion_opts:
    environment: testing
more ...

Using salt-ssh with agent forwarding

Part of a system's security is to reduce the attack surface. Following this principle, I want to see if I can switch from using regular salt minions for a saltstack managed system set towards salt-ssh. This would allow to do some system management over SSH instead of ZeroMQ.

I'm not confident yet that this is a solid approach to take (as performance is also important, which is greatly reduced with salt-ssh), and the security exposure of the salt minions over ZeroMQ is also not that insecure (especially not when a local firewall ensures that only connections from the salt master are allowed). But playing doesn't hurt.

more ...

Trying out imapsync

Recently, I had to migrate mail boxes for a couple of users from one mail provider to another. Both mail providers used IMAP, so I looked into IMAP related synchronization methods. I quickly found the imapsync application, also supported through Gentoo's repository.

more ...

New cvechecker release

A short while ago I got the notification that pulling new CVE information was no longer possible. The reason was that the NVD site did not support uncompressed downloads anymore. The fix for cvechecker was simple, and it also gave me a reason to push out a new release (after two years) which also includes various updates by Christopher Warner.

So cvechecker 3.6 is now available for general consumption.

more ...

Switching focus at work

Since 2010, I was at work responsible for the infrastructure architecture of a couple of technological domains, namely databases and scheduling/workload automation. It brought me in contact with many vendors, many technologies and most importantly, many teams within the organization. The focus domain was challenging, as I had to deal with the strategy on how the organization, which is a financial institution, will deal with databases and scheduling in the long term.

more ...

Getting su to work in init scripts

While developing an init script which has to switch user, I got a couple of errors from SELinux and the system itself:

~# rc-service hadoop-namenode format
Authenticating root.
 * Formatting HDFS ...
su: Authentication service cannot retrieve authentication info
(Ignored)
more ...


Using multiple OpenSSH daemons

I administer a couple of systems which provide interactive access by end users, and for this interactive access I position OpenSSH. However, I also use this for administrative access to the system, and I tend to have harder security requirements for OpenSSH than most users do.

For instance, on one system, end users with a userid + password use the sFTP server for publishing static websites. Other access is prohibited, so I really like this OpenSSH configuration to use chrooted users, internal sftp support, whereas a different OpenSSH is used for administrative access (which is only accessible by myself and some trusted parties).

more ...

Maintaining packages and backporting

A few days ago I committed a small update to policycoreutils, a SELinux related package that provides most of the management utilities for SELinux systems. The fix was to get two patches (which are committed upstream) into the existing release so that our users can benefit from the fixed issues without having to wait for a new release.

more ...

Doing away with interfaces

CIL is SELinux' Common Intermediate Language, which brings on a whole new set of possibilities with policy development. I hardly know CIL but am (slowly) learning. Of course, the best way to learn is to try and do lots of things with it, but real-life work and time-to-market for now forces me to stick with the M4-based refpolicy one.

Still, I do try out some things here and there, and one of the things I wanted to look into was how CIL policies would deal with interfaces.

more ...