Fix Error: cannot re-exec process to join the existing user namespace in Ansible Automation Platform 2

I have a lab environment running Ansible Automation Platform 2.3 (AAP) that I use for my customer demos.

At the end of 2022 I stopped the VM for a couple of days.

Today I start my AAP VM as usual. Everything seemed to work except when I try to run a simple job template.

I get an error “cannot re-exec process to join the existing user namespace” in the job output.

I try to sync the project and get the same error. I sync other projects and they all fail.

A quick Google search takes me to a Podman issue

Podman indeed is the foundation of AAP under the hood, with the concept called Execution Environments (really, those are just containers).

Podman runs as the unprivileged user awx.

I could verify the behavior from the command line:

[root@aap23 etc]# su - awx -c "podman ps -a"
Error: cannot re-exec process to join the existing user namespace

The link above mentions a file that fixes the problem when deleted.

I find a folder /tmp/podman-run-988/libpod/tmp but no

It turns out the temporary folder is overriden by AAP.

You can find the actual temp folder with this:

[root@aap23 etc] su awx -
[awx@aap23 ~]$ podman --log-level debug ps
INFO[0000] podman filtering at log level debug
DEBU[0000] Called ps.PersistentPreRunE(podman --log-level debug ps)
DEBU[0000] Merged system config "/usr/share/containers/containers.conf"
DEBU[0000] Merged system config "/var/lib/awx/.config/containers/containers.conf"
DEBU[0000] Using conmon: "/usr/bin/conmon"
DEBU[0000] Initializing boltdb state at /var/lib/awx/.local/share/containers/storage/libpod/bolt_state.db
DEBU[0000] Overriding run root "/tmp/podman-run-988/containers" with "/tmp/ansible.d1xwha4l/containers" from database
DEBU[0000] Overriding tmp dir "/tmp/podman-run-988/libpod/tmp" with "/tmp/ansible.d1xwha4l/libpod/tmp" from database
DEBU[0000] systemd-logind: Unknown object '/'.

Simple delete the file:

cd /tmp/ansible.d1xwha4l/libpod/tmp

I can now sync my projects again with no restart needed.

Thanks for reading this post!

Did you find an issue in this article?

- click on the following Github link
- log into Github with your account
- click on the line number containing the error
- click on the "..." button
- choose "Reference in new issue"
- add a title and your comment
- click "Submit new issue"

Your feedback is much appreciated! πŸ€œπŸΌπŸ€›πŸΌ

You can also drop me a line below!