Ansible and Db2 for z/OS – Fun with Collections

Introduction

In the last blog on Ansible and z/OS we talked a bit about how we went about extending Ansible to provide some additional capabilities around Db2 for z/OS DDF Aliases.

As promised at the end of that blog, this one will cover how to go about creating a collection of modules, and how to publish this on Ansible Galaxy to make it available to everyone else!

 

Why Create a Collection?

When you have a number of modules that deliver capability around a particular idea – we’re looking at Db2 for z/OS Production Access Path Modelling – then this is a way to package these and make them available as one deployable element. Making it available through Ansible Galaxy is a route to publishing to the wider internet connected community.

 

What’s Involved?

Publishing a collection to Ansible Galaxy involves a couple of setup steps, which we’ll cover below in more detail.

 

Ansible Galaxy Namespace

Before you can publish, you will need a namespace – e.g. ours is “triton”. It is not hard to setup, but make sure that your new namespace is available and get it registered before pushing the development boat out.

The authentication in Ansible Galaxy uses GitHub credentials, so you’ll need a presence there as well, and a GitHub organisation to “own” your namespace.

Namespaces are requested by raising an Issue in the Ansible Galaxy project on GitHub. You can find the form here:

To fill in the details:

• Title – leave this one as “Namespace: FIXME”
• Description:
  • “Namespace:” – this should be “Namespace: <your-namespace-here>”
    • E.g. “Namespace: triton”
  • “Description” – a one-line description for the namespace
    • E.g. “Triton Consulting”
  • “GitHub Org Link” – the web URL to the GitHub organisation that will own the namespace
    • E.g. https://github.com/tritonconsulting
  • “Admins” – a list of GitHub users to be registered as administrators of the namespace.
    • E.g. db2dinosaur (james.gill@triton.co.uk)

Note the warning that the “Admins” need to have logged into Ansible Galaxy to ensure that they have a user presence before submitting the form.

 

Ansible Galaxy API Token

Once you have created the namespace, you will need an API token, which is used to drive publishing your assembled collection into your namespace. To create this:

• Log in to https://galaxy.ansible.com as one of your namespace admins
• In the left-hand navigation panel, click on Collections, then API Token

• Pay attention to the warning – there is only one API token active and available to your collections for your logged in user.
• Click “Load Token” and make a note of the API token generated as you will not be able to find or display this again in Ansible Galaxy.

We’ll refer to this value in the rest of the doc (well, the bit on publishing) as api-token.

 

Creating the Collection

When you installed Ansible, you got the ansible-galaxy command line tool at the same time. We’ve used this previously to install collections from Ansible Galaxy, and in the following sections we’ll see how you can use it to create a template collection, build a deliverable collection package and then publish this to Ansible Galaxy.

There is some good documentation on doing all of this on the Ansible documentation site:

• Developing collections:
  https://docs.ansible.com/ansible/2.9/dev_guide/developing_collections.html
• More – including useful commands:
  https://docs.ansible.com/ansible/latest/dev_guide/developing_collections_creating.html

The following sections cover how we created the triton.db2_zos collection.

 

Creating the Collection Template

The collection template is a framework to add your content into to create the collection. This has a predefined directory structure and a number of supporting files which are used to configure the collection and present information about its contents.

The directory structure will be created under the current directory – to create our db2_zos collection in our triton namespace, we used:

ansible-galaxy collection init triton.db2_zos

james@fx507:~/ansible/working$ ansible-galaxy collection init triton.db2_zos
- Collection triton.db2_zos was created successfully

This creates the template in:

./triton/db2_zos

The resulting contents looks like this:

Note the namespace (triton) and collection names (db2_zos) in the path that is created.

As well as the directory structure, there are some important files built into the template:

• README.md – this markdown README is the basis of the documentation that will be shown on the Ansible Galaxy site about your collection. E.g. https://galaxy.ansible.com/ui/repo/published/triton/db2_zos/docs/
• galaxy.yml – this is the configuration file that Ansible Galaxy will read to process the delivered package (see below). It will be pre-tailored with some information (the namespace and collection name), but will need updating. More on this below.
• meta/runtime.yml – there are quite a few controls for plugin aging and deprecation, but the important aspect that must be set in here is the first (commented) stanza which sets the minimum version of Ansible required to support this collection.
• plugins/README.md – this markdown README provides documentation references for the different plugin types. Our focus in this blog – and collection – is module plugins, but the full list of types and their use can be found (at the time of writing) here:
  https://docs.ansible.com/ansible-core/2.15/plugins/plugins.html

 

Adding the Modules

Before we assemble our collection, it’s probably worth mentioning again (sorry) the need for the documentation to format correctly in your modules. We discussed this in the last blog post, but the thing to note here is that the publishing process will fail if the ansible-doc tool cannot correctly format the documentation and examples embedded in the modules.

We found the messages to be somewhat cryptic and ended stripping each of the documentation elements out into their own, otherwise empty module to get them right before re-inserting them in the correct module. There are probably better ways of doing this!

Having checked that all of your modules clear the ansible-doc test, you’ll want to create the modules subdirectory underneath plugins add the modules to it:

 

There are a few in here, and they’re covered in more detail below.

The next element to manage is the README.md in the collection root. This markdown file forms the basis of the documentation that will be presented on the Ansible Galaxy website for your collection. There are no specific rules about what should be presented here, but we’d suggest some solid use instructions and examples of how to work with the delivered modules and what sort of responses and output might be returned.

 

Updating the Template

The two parts of the template that have to be updated before build and delivery are:

1. meta/runtime.yml
   As mentioned above, there are several stanzas that can be used to control versioning and external references, but the one that you must update is the “requires_ansible” stanza. This sets the minimum level of ansible that the collection will work with. You must supply a value for this, and it shouldn’t be any lower or higher than one that you have successfully tested with 😊
2. galaxy.yml
   This file sets the collection configuration in Ansible Galaxy and includes the following settings that should be established:
   1. 1. namespace – the namespace that you’ve registered in Ansible Galaxy for your collections to land in. In our example, this is “triton”
      2. name – the name of this collection. E.g. db2_zos
      3. version – the version of this collection delivery. E.g. 0.0.1. Note that this must change on each publish – no sneaky bug fixing and reusing the same version number!
      4. readme – where the documentation readme markdown file is, relative to the collection root. E.g. README.md (the default)
      5. authors – a list of the authors of the collection
      6. description – a short (one liner) description of the collection
      7. license – a list of licenses that apply to the content in the collection, or you could use (h), below instead
      8. license_file – path to your license document. Can’t be used with (g)
      9. tags – a list of tags that Ansible Galaxy will associate with your collection, to help with search / indexing

Now that we’ve got the modules in place and the template updated, we can build the collection.

 

Build

The collection build creates a gzipped tar file in the current directory – use the ansible-galaxy collection build command:

ansible-galaxy collection build <path-to-galaxy.yml>

i.e. the path that holds the galaxy.yml file.

e.g.

james@fx507:~/ansible/working$ ansible-galaxy collection build ./triton/db2_zos
Created collection for triton.db2_zos at /home/james/ansible/working/triton-db2_zos-0.0.1.tar.gz

 

Publish

To publish the collection, we again use the ansible-galaxy command, with the api-token that we created on the web portal:

ansible-galaxy collection publish –token api-token <collection.tar.gz>

e.g.

ansible-galaxy collection publish –token api-token triton.db2_zos-0.0.1.tar.gz

 

Cool – What’s In triton.db2_zos?

The original module (db2_ddf_alias) has got some company now, mostly focused on gathering the data for the Db2 Production Access Path Modelling, as mentioned in the introduction. We make use of some environment variables to provide the correct Db2 load libraries, RUNLIB (home of DSNTEP2) and the DSNTEP2 plan name (examples in the collection doc – link below). You can download the collection tar.gz from Ansible Galaxy, open it up and have a look at how we did all of this.

For the most part, we’re just using standard interfaces and reusing sample tools (DSNTEP2 SQL processor and the SYSPROC.ADMIN_INFO_SYSPARM stored procedure) that we think that will be available on most Db2 for z/OS customer sites. The module that retrieves the current ZPARMs (db2_get_zparms) uses a piece of Rexx that we push to the host and run. This is because there is no simple way to call a stored procedure from the supplied tools unless you use the USS based Java Db2 CLP, and a lot of customers do not install this.

Modules in the collection at the time of writing are:

 

Usage notes, example plays and documentation for all of these are available on the Ansible Galaxy documentation page for the collection, here:

https://galaxy.ansible.com/ui/repo/published/triton/db2_zos/docs/

This will be maintained as we add more modules to the collection over time, so is probably a good place to look before nudging us for new things 😊

 

Conclusions

This blog is really a very early look over my shoulder at some things that we thought might be interesting to try. It’s more of a “what we did” rather than the definitive guide on how to create a Db2 for z/OS Ansible collection. Having said that, it does work!

There is definitely more to come on this – especially with respect to embedding service routines in the collection – e.g. an SQL handler – that can be consumed by multiple modules in the collection, and experimenting with dependencies, especially with other collections.

There are many things that we like about Ansible, and the ability to extend the capabilities of the tool through modules and collections is right at the top. Because the environment (Python) is open by nature – you can read the source – we all share our working, how we did things and how we made it work. This has resulted in the Ansible collections contributors becoming a huge community and one of the largest open source projects ever. Come and join us!