Lightweight AWX production without overbuilding

AWX becomes useful when it replaces Ansible runs launched from administration workstations, without a central log, without clear credential control, and without visibility on who executed what. The use case here is deliberately simple. A team operates around forty Linux servers across preproduction and production. Playbooks already exist in Git, but everyone runs them in their own way. The goal is not to build a complete automation platform. It is to centralize a few recurring operations with enough control to avoid turning AWX into a dangerous remote console.

The initial scope fits into three job families. Read jobs check server state. Maintenance jobs perform bounded actions, such as restarting an approved service or installing a validated package. Change jobs apply an already tested Ansible role to a limited inventory. Anything that deletes data, modifies networking, reboots machines in bulk, or changes secrets remains out of scope at the beginning.

Build the Ansible repository before the interface

The first mistake is to start creating AWX templates before cleaning up the Ansible repository. AWX should execute versioned code, not become the place where operations are improvised. A simple layout is enough as long as it separates playbooks, roles, and inventories.

ansible-operations/
playbooks/
  check-linux-baseline.yml
  check-disk-usage.yml
  restart-approved-service.yml
  patch-selected-package.yml
roles/
  linux_baseline/
  ssh_hardening/
  package_management/
inventories/
  preprod/hosts.yml
  production/hosts.yml
collections/requirements.yml
README.md

In AWX, the project points to this repository. Production should follow a stable branch. Tests can use another branch or another AWX project. This separation prevents a production job from accidentally running a playbook that is still being modified.

The repository README should document the playbooks that are actually exposed in AWX. It does not need to be long. It should state the job purpose, authorized groups, expected variables, and possible effects. Without that, AWX templates become buttons whose behavior depends on team memory.

Create inventories that prevent obvious mistakes

A single inventory named linux is convenient at the beginning and dangerous later. Environments and roles should be separated. A standard maintenance job should not be able to target a bastion or a database by accident. Group names must make the scope obvious.

all:
children:
  production_web:
    hosts:
      web01.example.local:
      web02.example.local:
  production_tools:
    hosts:
      tools01.example.local:
  restricted:
    hosts:
      bastion01.example.local:

In AWX, job templates should point to the right inventory. A preproduction template should not accept a production inventory at launch time. A template that restarts a service should target a precise group, not every host.

Separate credentials by risk level

A lightweight AWX installation becomes fragile if every job uses the same SSH account with full sudo rights. At minimum, three levels are useful. A read-only account for checks. A maintenance account for routine operations. A more privileged account for a few validated changes.

Read-only credential
Check jobs
No sudo

Maintenance credential
Validated packages
Approved service restarts
Limited sudo

Restricted admin credential
More sensitive roles
Limited inventories
Monitored usage

The restriction must also exist on Linux. AWX stores the credential, but the real guardrail is the remote account capability. Giving NOPASSWD: ALL to the automation account simply moves the risk into a nicer interface.

# Principle example, to adapt
%ansible-maintenance ALL=(root) NOPASSWD: /bin/systemctl restart nginx
%ansible-maintenance ALL=(root) NOPASSWD: /usr/bin/apt-get update
%ansible-maintenance ALL=(root) NOPASSWD: /usr/bin/apt-get install *

# Avoid for a lightweight start
%ansible-maintenance ALL=(ALL) NOPASSWD: ALL

Turn job templates into bounded actions

An AWX template should not be a generic launcher. The more free variables it accepts, the more it behaves like a remote console. For a service restart, it is safer to provide a list of approved services than to leave an open text field.

---
- name: Restart an approved service
hosts: "{{ target_group }}"
become: true
vars:
  allowed_services:
    - nginx
    - telegraf
    - chronyd
tasks:
  - name: Reject unauthorized service
    ansible.builtin.fail:
      msg: "Service not allowed by this job"
    when: service_name not in allowed_services

  - name: Restart approved service
    ansible.builtin.service:
      name: "{{ service_name }}"
      state: restarted

  - name: Check service after restart
    ansible.builtin.service_facts:

  - name: Show service state
    ansible.builtin.debug:
      msg: "{{ service_name }} state: {{ ansible_facts.services[service_name + '.service'].state | default('unknown') }}"

This kind of job is more useful than a basic restart playbook. It refuses unapproved services, executes the action, and prints a check. The AWX report becomes operational evidence instead of only saying that the playbook ended.

Add verification after every change

Any job that modifies something should prove the result. Installing a package without checking the version, applying configuration without validating the service, or restarting without checking state leaves uncertainty. AWX should produce a trace that operations can reuse.

---
- name: Install and verify a selected package
hosts: "{{ target_group }}"
become: true
tasks:
  - name: Install package
    ansible.builtin.package:
      name: "{{ package_name }}"
      state: present

  - name: Collect package facts
    ansible.builtin.package_facts:
      manager: auto

  - name: Display installed version
    ansible.builtin.debug:
      msg: "{{ package_name }} version: {{ ansible_facts.packages[package_name][0].version }}"
    when: package_name in ansible_facts.packages

In AWX, these outputs matter. They allow a change to be reviewed from the interface, show which hosts were affected, and prove that the post-action check actually ran.

Back up what is not in Git

Playbooks are in Git, but AWX also contains objects that do not rebuild themselves: projects, UI-created inventories, credentials, templates, schedules, history, and settings. A database failure without backup forces the team to rebuild the AWX organization by hand.

A lightweight production instance should therefore have a minimal restore test. The goal is not a complicated procedure. It is to prove that an AWX instance can be restored, a Git project can sync, a credential works, and a read job can run.

AWX restore test
Restore the instance or database in an isolated environment
Verify web access
Sync the main Git project
Run check-linux-baseline on a test target
Validate credentials, inventories, and templates
Record recovery time

Conclusion

A successful lightweight AWX production setup is not measured by the number of enabled features. It is measured by the quality of exposed actions. A small set of bounded, versioned, verified jobs with appropriate credentials is better than a broad interface that can run everything everywhere.

The right starting point is concrete: three clean inventories, three credential levels, five useful templates, a tested backup, and AWX reports that prove the result of each action. With that base, AWX becomes a controlled operations tool, not another automation layer that is hard to govern.