Introduction
Hello folks! Today, I’m writing this post to talk about a solution we found to the following problem: how do you export compliance data from a Rudder installation?
The use case was: how do you export Rudder compliance metrics that are currently accessible in the Web UI in a machine-usable format so you can work on them and use them in, for example, a spreadsheet? or, put more crudely:
“I need a compliance output from your software, in a format I can work with”
Well, okay, but Rudder only seems to provide a graphical display and (with a plugin) a PDF summary… what can I do ?
“Tell me what you want, what you really really want!”
The request seemed quite simple at the beginning, but of course, it needed to be analyzed and prioritized:
- What do we need exactly ?
- What do we have to achieve it ?
- How do we implement it ?
The first part is a business need: Rudder compliance data is crucial for enterprise users that need KPI’s on their infrastructure health. Problem is, that exposing this kind of KPI’s can be done in multiple ways: Rudder by default exposes them in the Web UI as components like the main dashboard and context-specific representations (Rules view, Nodes view …).
We need a way to obtain “raw” metrics in a format that can be used in a wide range of applications.
Fortunately, Rudder’s REST API has methods for this!
API to the rescue!
Rudder’s API’s role is to expose all the data it has (compliance, rules, directives, settings…) using a standard REST interface, that enables a user or developer to query various things without having to scrape the Web UI.
Every exchange between the API client and Rudder is done using the JSON representation format, like this:
We created a nice command-line tool called “rudder-compliance” that enables you to easily:
- Get compliance metrics from the CLI
- Export those metrics in a CSV file for further handling
Cool, how can I use it?
Well, first you need:
- A Python interpreter with the “requests” module (available in all good package managers or just run “pip install requests”)
- An operational Rudder installation
Then, getting the tool is a simple as:
wget https://raw.githubusercontent.com/Normation/rudder-tools/master/scripts/rudder-compliance/rudder-compliance
You then need to edit the script rudder-compliance to set the Rudder URL and API password in the “Configuration” area.
Then, just run the tool without arguments to get basic usage explanations:
Then just type what you need:
- -d to get global Directive compliance, or -d <DIRECTIVE UUID> to get the compliance of a specific Directive
- -r to get global Rule compliance, or -r <RULE UUID> to get the compliance of a specific Rule
- In both cases, adding -c <CSV FILE> outputs a CSV file to the given location with all the information requested
It’s as simple as that !
To infinity and beyond!
Just like every tool, rudder-compliance isn’t perfect: it has been made with a specific use case in mind (compliance CSV export), and could probably be improved:
- More compliance details could be obtained / calculated (node compliance?)
- More complex search queries could be handled
- Configuration could be done using CLI arguments / a config file instead of hardcoding them
But it is a good example of what the Rudder API can provide, and actually isn’t the only cool utility that you could use. We’ve got more of them, for example:
- rudder-nodes-list: to export all nodes from a Rudder installation, and their OS / Rudder version. Can be easily expanded to get an arbitrary program version (to track OpenSSL upgrades maybe?)
- rudder-set-node-props: to tag nodes with custom key=values
Just take a look at the rudder-tools repository to discover all of the utilities!
And do not hesitate to create your own tools and send us a Pull Request, we would be more than happy to make them part of the family 🙂
Cheers!
