Debugging and Logging

1. View Container Logs

podman logs mystay

⚠️ Only stdout and stderr of the initial command are sent to docker logs.

It is possible to pipe to the stdout of the original process, but this is more for demonstration purposes.

docker exec mystay sh -c 'echo "DEMO LOG ENTRY" >> /proc/1/fd/1'

ℹ️ If you were to run a container with --rm, the log would be removed as soon as the container exits along with the container itself.

Piping stdout and stderr

For batch jobs on the farm, stdout and strerr for a job are picked up by slurm and transmitted to /farm_out. If you run a container without -it, the stdout and stdin will be sent to the calling process. On the batch farm nodes, this means the stdout and stderr will be transmitted to /farm_out.

docker run --rm docker.io/library/ubuntu bash -c \
'for i in {1..5}; do echo "stdout: $i"; \
echo "stderr: $i" >&2; sleep 1; done'

2. Stop and Inspect

podman ps
podman stop mystay
podman ps
podman container ls -a

ℹ️ For long runing containers, it is ideal to add a process that can monitor SIGTERM so that it can exit gracefully instead of being killed by the container engine.

3. Look at Image and Container Layers

podman image inspect ubuntu:24.04
podman container inspect mystay

4. Show Image History

podman history ubuntu:24.04

5. View Process Tree

podman top <container-name>

Useful for checking if your service is still running.

6. Inspect Internals

podman inspect <container-name>

Shows full JSON definition including mounts, environment, command, and process state.

---