Skip to content

Commit

Permalink
Merge pull request #1 from ddimatos/ddimatos/ansible-documentation
Browse files Browse the repository at this point in the history
Ddimatos/ansible documentation
  • Loading branch information
ketankelkar authored Feb 6, 2025
2 parents c9547c9 + cb904d0 commit c52c652
Show file tree
Hide file tree
Showing 3 changed files with 61 additions and 60 deletions.
4 changes: 2 additions & 2 deletions docs/docsite/rst/os_guide/index.rst
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
.. _os_guide_index:

###########################################
#######################################
Using Ansible on Windows, BSD, and z/OS
###########################################
#######################################

.. note::

Expand Down
101 changes: 51 additions & 50 deletions docs/docsite/rst/os_guide/intro_zos.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,54 +5,55 @@ Managing z/OS hosts with Ansible
================================


Ansible can connect to `IBM UNIX System Services <https://www.ibm.com/docs/en/zos/3.1.0?topic=descriptions-zos-unix-system-services>`_ to bring your Ansible Automation strategy to IBM z/OS.
Ansible can connect to `IBM UNIX System Services <https://www.ibm.com/docs/en/zos/latest?topic=descriptions-zos-unix-system-services>`_ to bring your Ansible Automation strategy to IBM z/OS.
This enables development and operations automation on IBM Z through a seamless,
unified workflow orchestration with configuration management, provisioning, and application deployment with
the easy-to-use Ansible platform.


Ansible and UNIX System Services
---------------------------------
UNIX System Services can support the required dependencies for an Ansible managed node including running python and
Ansible and z/OS UNIX System Services
-------------------------------------
UNIX System Services can support the required dependencies for an Ansible managed node including running Python and
spawning interactive shell processes through SSH connections.
Ansible can target UNIX System Services nodes to modify files, directories, etc. through built-in Ansible community modules.
Further, anything that one can do by typing command(s) into the UNIX System Services shell can be captured
and automated in an Ansible playbook.

To learn more about z/OS managed nodes,
To learn more about z/OS managed nodes,
see `Red Hat Certified Content for IBM Z <https://ibm.github.io/z_ansible_collections_doc/>`_.


The z/OS Landscape
-------------------
While most systems process files in two modes - binary or UTF-8 encoded text,
IBM Z including UNIX System Services features an additional third flavor - text encoded in EBCDIC.
------------------
While most systems process files in two modes - binary or UTF-8 encoded text,
IBM Z including UNIX System Services features an additional third mode - text encoded in EBCDIC.
Ansible has provisions to handle binary data and UTF-8 encoded textual data, but not EBCDIC encoded data.
This is not necessarily a limitation, it simply requires additional tasks that convert files to/from their original encodings.
It is up to the Ansible user managing z/OS nodes to understand the nature of the files in their automation.

The type (binary or text) and encoding of files can be stored in file "tags".
File tags is a z/OS UNIX System Services concept (part of enhanced ASCII) which was established to distinguish binary
File tags is a z/OS UNIX System Services concept (part of Enhanced ASCII) which was established to distinguish binary
files from UTF-8 encoded text files and EBCDIC-encoded text files.

Default behavior for an un-tagged file or stream is determined by the program, for example,
Default behavior for an un-tagged file or stream is determined by the program, for example,
`IBM Open Enterprise SDK for Python <https://www.ibm.com/products/open-enterprise-python-zos>`__ defaults to the UTF-8 encoding.

Ansible modules will not read or honor any file tags. It is up to the user to determine the nature of remote data and tag it appropriately.
Data sent to remote z/OS nodes is by default encoded in UTF-8 and is not tagged.
This is achieveable with an additional task using the ``builtin.command`` module and applying any necessary encoding conversion.
Ansible modules will not read or recognize file tags. It is up to the user to determine the nature of remote data and tag it appropriately.
Data sent to remote z/OS nodes is by default, encoded in UTF-8 and is not tagged.
Tagging a file is achievable with an additional task using the ``ansible.builtin.command`` module.

.. code-block:: yaml
- name: tag my_file.txt as IBM-1047 EBCDIC.
ansible.builtin.command: chtag -tc ibm1047 my_file.txt
The z/OS UNIX remote shell defaults to an EBCDIC encoding for un-tagged data streams.
The `z/OS shell <https://www.ibm.com/docs/en/zos/latest?topic=shells-introduction-zos>`_ available on
z/OS UNIX System services defaults to an EBCDIC encoding for un-tagged data streams.
This mismatch in data encodings can be resolved with the ``PYTHONSTDINENCODING`` environment variable,
which tags the pipe used by python with the specified encoding.
File and pipe tags can be used for automatic conversion between ASCII and EBCDIC.
But only by programs on z/OS which are aware of tags and honor them.
which tags the pipe used by Python with the specified encoding.
File and pipe tags can be used with automatic conversion between ASCII and EBCDIC, but only programs on
z/OS which are aware of tags can use them.


Using Ansible Community Modules with z/OS
Expand All @@ -65,22 +66,22 @@ On z/OS, since text data (file or stream) is sometimes encoded in EBCDIC and som
Here are some notes / pro-tips when using the community modules with z/OS. This is by no means a comprehensive list.

* ansible.builtin.command / ansible.builtin.shell
The command and shell modules are excellent for automating tasks for which command line solutions already exist.
The thing to keep in mind when using these modules is depending on the system configuration, the Z shell (``/bin/sh``) may return output in EBCDIC.
The LE environment variable configurations will correctly convert streams if they are tagged and make output look sensible on the Ansible side.
The command and shell modules are excellent for automating tasks for which command line solutions already exist.
The thing to keep in mind when using these modules is that depending on the system configuration, the z/OS shell (``/bin/sh``) may return output in EBCDIC.
The LE environment variable configurations will correctly convert streams if they are tagged and return readable output on the Ansible side.
However, some command line programs may return output in UTF-8 and not tag the pipe.
In this case, the autoconversion may incorrectly assume output is in EBCDIC and attempt to convert it and wind up with yet more unreadable data.
If the source encoding can be determined, since the ``ansible.builtin.command`` module allows chaining together commands through pipes,
try piping the output through a call to ``iconv``. You may need to play around with the 'to' and 'from' encodings to determine the correct set.
In this case, the autoconversion may incorrectly assume output is in EBCDIC and attempt to convert it and yield unreadable data.
If the source encoding is known, you can use the ``ansible.builtin.command`` module's capability to chain commands together through pipes,
and pipe the output to ``iconv``. In this example, you may need to select other encodings for the 'to' and 'from' that represent your file encodings.

.. code-block:: yaml
ansible.builtin.command: "some_pgm | iconv -f ibm-1047 -t iso8859-1"
* ansible.builtin.raw
The raw module, by design, ignores all remote environment settings. However, UNIX System Services managed nodes require some base configurations.
One trick to use this module with UNIX System Services is to pass in the bare minimal environment variables as a chain of export statements before the desired command.
The raw module, by design, ignores all remote environment settings. However, z/OS UNIX System Services managed nodes require some base configurations.
To use this module with UNIX System Services, configure the minimum environment variables as a chain of export statements before the desired command.

.. code-block:: yaml
Expand All @@ -99,19 +100,19 @@ Here are some notes / pro-tips when using the community modules with z/OS. This
* ansible.builtin.copy / ansible.builtin.fetch
The built in community modules will NOT automatically tag files, nor will existing file tags be honored nor preserved.
You can treat files as binaries when running copy/fetch operations, there is no issue in terms of data integrity,
just remember to restore the correct tag and encoding once the file is returned to z/OS, as that data will not be stored for you.
but remember to restore the file tag and encoding once the file is returned to z/OS, as tags are not preserved.

* ansible.builtin.blockinfile / ansible.builtin.lineinfile
These modules process all data in UTF-8, so be sure to convert files to UTF-8 beforehand and re-tag the resulting files after.
These modules process all data in UTF-8, you must convert files to UTF-8 beforehand and re-tag the resulting files after.

* ansible.builtin.script
The built in script module copies a local file over to a remote target and attempts to run it.
The issue that UNIX System Services targets run into is that the file does not get tagged as UTF-8 text.
When the underlying shell attempts to read the untagged script file, it will assume the default,
The issue that z/OS UNIX System Services targets run into is that the file does not get tagged as UTF-8 text.
When the underlying z/OS shell attempts to read the untagged script file, it will assume the default,
that the file is encoded in EBCDIC, and the file will not be read correctly and the script will not run.
One work-around is to manually copy local files over (``ansible.builtin.copy`` ) and convert or tag files (with the ``ansible.builtin.command`` module).
With this work-around, some of the niceties of the script module are lost, such as automatically cleaning up the script file once it's run,
but it is trivial to recreate those steps as separate playbook tasks.
One work-around is to copy local files to the managed node (``ansible.builtin.copy`` ) and convert or tag files (with the ``ansible.builtin.command`` module).
With this work-around, some of the conveniences of the script module are lost, such as automatically cleaning up the script file once it's run,
but it is trivial to perform those steps as additional playbook tasks.

.. code-block:: yaml
Expand All @@ -126,15 +127,15 @@ Here are some notes / pro-tips when using the community modules with z/OS. This
- name: Run script.
ansible.builtin.command: "/u/ibmuser/scripts/sample.sh"
Another somewhat convoluted work-around is to store local script files in EBCDIC.
They may be unreadable on the controller, but they will copy over to UNIX System Services targets,
be read in correctly in EBCDIC, and the script will run. This approach takes advantage of the built-in conveniences of the script module,
but storing unreadable files locally makes maintaining those script files difficult.
Another work-around is to store local script files in EBCDIC.
They may be unreadable on the controller, but they will copy correctly to z/OS UNIX System Services targets in EBCDIC,
and the script will run. This approach takes advantage of the built-in conveniences of the script module,
but managing unreadable EBCDIC files locally makes maintaining those script files difficult.

Configure the Remote Environment
-----------------------------------
--------------------------------

Certain Language Environment (LE) configurations enable automatic encoding conversion and automatic file tagging functionality required by python on z/OS systems.
Certain Language Environment (LE) configurations enable automatic encoding conversion and automatic file tagging functionality required by Python on z/OS systems (IBM Open Enterprise SDK for Python).

Include the following configurations when setting the remote environment for any z/OS managed nodes. (group_vars, host_vars, playbook, or task):

Expand All @@ -148,7 +149,7 @@ Include the following configurations when setting the remote environment for any
_TAG_REDIR_OUT: "txt"
Note, the remote environment can be set any of these levels:
Note, the remote environment can be set in any of these options:

* inventory - inventory.yml, group_vars/all.yml, or host_vars/all.yml
* playbook - ``environment`` variable at top of playbook.
Expand All @@ -157,14 +158,14 @@ Note, the remote environment can be set any of these levels:
For more details, see :ref:`playbooks_environment`.

Configure the Remote Python Interpreter
----------------------------------------
---------------------------------------

Ansible requires a python interpreter to run most modules on the remote host, and it checks for python at the default path ``/usr/bin/python``.
Ansible requires a Python interpreter to run most modules on the remote host, and it checks for python at the 'default' path ``/usr/bin/python``.

On z/OS, the python3 interpreter (from `IBM Open Enterprise SDK for Python <https://www.ibm.com/products/open-enterprise-python-zos>`_)
is often installed to a different path, typically something like: ``/usr/lpp/cyp/v3r12/pyz``.

This path to the python interpreter can be configured with the Ansible inventory variable ``ansible_python_interpreter``.
The path to the Python interpreter can be configured with the Ansible inventory variable ``ansible_python_interpreter``.
For example:

.. code-block:: ini
Expand Down Expand Up @@ -192,22 +193,22 @@ Enable Ansible Pipelining
Enable :ref:`ANSIBLE_PIPELINING` in the ansible.cfg file.

When Ansible pipelining is enabled, Ansible passes any module code to the remote target node
through python's stdin pipe and runs it in all in a single call rather than copying data to temporary files first and then reading from those files.
through Python's stdin pipe and runs it in all in a single call rather than copying data to temporary files first and then reading from those files.
For more details on pipelining, see: :ref:`flow_pipelining`.

Enabling this behavior is encouraged because python will tag its pipes with the proper encoding, so there is less chance of encountering encoding errors.
Further, using python stdin pipes is more performant than file I/O.
Enabling this behavior is encouraged because Python will tag its pipes with the proper encoding, so there is less chance of encountering encoding errors.
Further, using Python stdin pipes is more performant than file I/O.


Include the following in the environment for any tasks performed on z/OS target nodes.
The value should be the encoding used by the z/OS UNIX shell on the remote target.
Include the following in the environment for any tasks performed on z/OS managed nodes.
The value should be the encoding used by the z/OS UNIX System Services managed node.

.. code-block:: yaml
PYTHONSTDINENCODING: "cp1047"
When Ansible pipelining is enabled but the ``PYTHONSTDINENCODING`` property is not correctly set, the following error may result.
Note, the ``'\x81'`` below may vary based on the target user and host:
Note, the hex ``'\x81'`` below may vary depending source causing the error:

.. code-block::
Expand All @@ -231,9 +232,9 @@ Using z/OS as a Control Node
----------------------------

The z/OS operating system currently cannot be configured to run as an Ansible control node.
Despite being POSIX-compliant, the UNIX System Services interface also cannot be configured to run as an Ansible control node.
Despite being POSIX-compliant, z/OS UNIX System Services interface also cannot be configured to run as an Ansible control node.

There are some options available on the IBM Z platform to set up as a control node:
There are options available on the IBM Z platform to use it as a control node:

* IBM z/OS Container Extensions (zCX)
* Red Hat OpenShift on IBM zSystems and LinuxONE
Expand Down
16 changes: 8 additions & 8 deletions docs/docsite/rst/reference_appendices/faq.rst
Original file line number Diff line number Diff line change
Expand Up @@ -328,9 +328,9 @@ Running on z/OS
.. error::
/usr/bin/python: FSUM7351 not found

Ansible requires a python interpreter to execute modules on the remote host, and checks for it at the default path ``/usr/bin/python``.
Ansible requires a Python interpreter to execute modules on the remote host, and checks for it at the 'default' path ``/usr/bin/python``.

| On z/OS, the Python 3 interpreter (from `IBM Open Enterprise SDK for Python <https://www.ibm.com/products/open-enterprise-python-zos>`_) is often installed to a different path, typically something like:
| On z/OS, the Python 3 interpreter (from `IBM Open Enterprise SDK for Python <https://www.ibm.com/products/open-enterprise-python-zos>`_) is often installed to a different path, typically something like:
| ``<path-to-python>/usr/lpp/cyp/v3r12/pyz``.
The path to the python interpreter can be configured with the Ansible inventory variable ``ansible_python_interpreter``.
Expand All @@ -347,21 +347,21 @@ Running on z/OS
.. error::
SyntaxError: Non-UTF-8 code starting with '\\x81' in file <stdin> on line 1, but no encoding declared; see https://peps.python.org/pep-0263/ for details

Note, the ``'\x81'`` below may vary based on the target user and host:
Note, the hex ``'\x81'`` below may vary depending source causing the error:

When Ansible pipelining is enabled, Ansible passes all module code to the remote target through python's stdin pipe and runs it all in a single call.
When Ansible pipelining is enabled, Ansible passes all module code to the remote target through Python's stdin pipe and runs it all in a single call.
For more details on pipelining, see: :ref:`flow_pipelining`.

Include the following in the remote environment configuration for any tasks performed on z/OS target nodes.
The value should be the local encoding used by the z/OS UNIX Systems Services shell of the remote target.
Include the following in the environment for any tasks performed on z/OS managed nodes.
The value should be the encoding used by the z/OS UNIX System Services managed node.

.. code-block:: yaml
PYTHONSTDINENCODING: "cp1047"
* Certain language environment (LE) configurations enable auto conversion and file tagging functionality required by python on z/OS systems.
* Certain language environment (LE) configurations enable auto conversion and file tagging functionality required by Python on z/OS systems (IBM Open Enterprise SDK for Python).

Include the following configurations when setting the remote environment for any z/OS managed nodes. (group_vars, host_vars, playbook, or task):

Expand All @@ -375,7 +375,7 @@ Running on z/OS
_TAG_REDIR_OUT: "txt"
Note, the remote environment can be set any of these levels: inventory (inventory.yml, group_vars, or host_vars), play, block, or task with the ``environment`` key word.
Note, the remote environment can be set in any of these options: inventory (inventory.yml, group_vars, or host_vars), play, block, or task with the ``environment`` key word.

.. seealso:: :ref:`working_with_zos`

Expand Down

0 comments on commit c52c652

Please sign in to comment.