Rudder 4.1 has been released on March 30th 2017 and brings a lot of much anticipated convenience improvements to make the best of Rudder 4.0 big new features. This article describes the most significant ones. Feel free to read the 4.1 changelog to see the complete list.
Organize Rules and Directives by setting key=value tags (using UI or REST API)
When the scope of your Rudder installation get bigger, you have many Rules and Directives defined and it can be complicated to find which one apply the correct configuration. To improve that situation, we added in 4.1 the possibility to tags Directives and Rules. A tag is a key/value pair ( ie, “environment”/”production” ) that can set without any restriction. You can put as many tags as you want, you can have multiple tags with the same key, so you can adapt the tags to fulfill your needs. We added a section in the edit form of Directive and Rules so you can modify it using the UI. You can also automate the configuration and build a script to set tags in batch using the REST API.
So ok now you have tagged all your Directives and Rules, how to find back those you tagged with “client/reallyBigClientWithHugeNeeds” ? Don’t worry we added tools to find them. We added filters on Directives and Rules, just enter the tag key and value, click on the + and voila. You can search as many tags as you want.
If you want to search a specific key without any value, just keep the value field empty (vice versa if you only know the value of the tag but not its key. Note that those fields (adding tag and filter) have autocompletion enabled, just start typing and the fields suggests how to end with an existing value.
Also you can find you tags using the quicksearch bar using the filter “in:tags” so you can access them from anywhere.
With these tags, we hope to give you a built in way to manage and a better control over you Directives and Rules, What will happen next? Maybe we will extend that system to other elements (node / groups)? Allow to use them in configuration ? Base an authorization system on them ? Time will tell! Please tell us what you think about it, and what you would expect!
Add or remove Node properties directly in the web interface
Since Rudder 3.1.15, we have the possibility to use Node properties, to group nodes based on defined properties, and to add data in the generated policies.
These properties could be set only via the Rudder API, and were displayed in the Web Interface
In Rudder 4.1, we can also add or remove Node properties directly in the Web interface, in the “Properties” tab of the Node detail.
There, you can create properties, where values can be either plain string (“prod”) or JSON data for more complex structure.
Share files between Nodes (via new generic methods)
Rudder 4.1 introduced a way to share files from one node to another. It allows a node to send a file to its relay server, which will make it available for another target node, that has to to specifically download it.
This file sharing method is secured by:
- The control of uploaded file signature by the server, to check it matches the source node’s private key.
- The same mechanism as standard file copy in Rudder to download the shared file from the server.
It also includes a TTL mechanism that allows sharing a file for a limited amount of time.
To use this feature, two generic methods are available in the technique editor:
- sharedfile_from_node: To download a file shared from another node.
- sharedfile_to_node: To make a file available to another node.
Example:
We have a node A, with uuid 2bf1afdc-6725-4d3d-96b8-9128d09d353c which wants to share the file /srv/db/application.properties with node B with uuid 73570beb-2d4a-43d2-8ffc-f84a6817849c.
We want this file to stay available for one year for node B on its policy server.
The node B wants to download it into /opt/application/etc/application.properties.
They have to agree (i.e. it has to be defined in the policies of both nodes) on the id of the file, that will be used during the exchange, here it will be application.properties.
To share the file, node A will usesharedfile_to_node with:
sharedfile_to_node(“73570beb-2d4a-43d2-8ffc-f84a6817849c”, “application.properties”, “/srv/db/application.properties”, “356 days”)
To download the file, node B will use sharedfile_from_node with:
sharedfile_from_node(“2bf1afdc-6725-4d3d-96b8-9128d09d353c”, “application.properties”, “/opt/application/etc/application.properties”)
Trigger agent runs on Nodes via relay servers
Rudder 4.0 introduced an API to trigger agent run on nodes from the server, but it was limited to nodes directly connected to the server, and it could not cross Rudder relays.
In 4.1, we improved this API, and it is now able to cross relays, and can hence cover all your Rudder nodes easily. It is based on a new relay-level API that will enable many new exciting things in the future (and already also permitted the file sharing between nodes for example).
This extended API keeps the exact same interface, it is possible to trigger a run on all nodes, or only on specific one (and in this case, to get the full agent output), but it will now use the internal relay API directly. This internal API is only accessible to the policy server of a relay, or to a local caller (which is the case when using Rudder applyPolicy API).
We also improved the modularity of Rudder, and Rudder root servers now include a standard relay (the same as a standalone relay) to distribute policies.
Many Techniques renamed to be more explicit and now displayed in alphabetical order to make finding them easier
In Rudder 4.1, we renamed almost all provided techniques, with two main goals:
Have a consistent naming scheme
Have easily understandable technique names
We hence decided to write naming conventions for techniques, and to apply them to all existing ones. There also now sorted in alphabetical order to make the list more readable.
New technique names will appear in the interface as soon as you have updated your techniques after upgrading to 4.1, even if you use older versions of the technique (it is the name of the last version available that appear).
Customize Rudder behavior with the new hook system based on server-side actions
Rudder 4.1 introduces the possibility to execute files (hooks), typically scripts, when some predefined event occurs on Rudder. Based on the return code of the execution, the process will continue (on success or warn only), or stop (on error). For now, the supported server side events are related to policy generation and the are already used in Rudder to check the correctness of the generated policies or notify the policy server when a generation succeeds. They will be extended to support node acceptation or deletion events, and latter to be triggerable based on (non-)compliance levels for a node, a group or a rule.
Of course, as usual for that kind of feature, you are able to sort hooks for a given event and only enable the ones you need. You can find the full feature documentation at http://www.rudder-project.org/doc-4.1/_server_event_hooks.html#_server_event_hooks.
Improved user experience, overall improvement of web performance (web resources are cached), many minor UI tweaks (page titles, forms, buttons, filters, …)
Improved user experience, overall improvement of web performance (web resources are cached), many minor UI tweaks (page titles, forms, buttons, filters, …)
To make Web Interface more responsive, especially with large installation (more than 2000 nodes), we implemented caching of web resources and generalized compression of all data over the network. This makes page loading and ajax requests much faster, and also permits as a side effect to avoid visual glitches on Rudder upgrades due to improper caching by the browser of CSS and Javascript
We also worked on improving the design of Web Interface, by making more obvious important information, and hiding non-relevant data. Most notable change is on the Directive page, which benefited from several tweaks, and mainly the Directive tree with warnings about newer Technique version, deprecated versions and Techniques.
Rules page have been updated to give more visibility, save space and offer better experience. It now has an always displayed “New category” button, as well as smaller “Refresh” and “New Rule” colored button.
Group tree has been modified also to remove the “Root of the group and group categories” which doesn’t provide any useful information.
Switched to a new graph rendering library, fixing all performance issues with Firefox
When we introduced dashboard and recent changes graphs in 3.0, we decided to use c3.js (http://c3js.org/) to display graphs. That library uses svg, and firefox has really bad support for svg. This combined with our usage (small graphs in a table) ensues a dramatic loss of performance ( each graph takes hundreds of milliseconds to render, blocking the browser, with high cpu usage). For a few graphs it was not visible but when the number grow high it would block completely the usage of the browser. In 4.1, it’s over! We changed to chart.js (http://www.chartjs.org/) which used canvas, which is gives a huge performance boost on Firefox. Chrome users will not see any change in performance. We also fixed some bugs that were hard to fix in the old library and were really easy to fix in these new library
By changing of library, we fixed a technical debt we had for a long time, with a really devastating impact on you, users of Rudder. We welcome any feedback on the new graph, if you see some performance issues, a change of behavior or simply find it better 😉
New plugin packaging system for easier plugin management
Starting with Rudder 4.1, we introduce a new archive format for our plugin, the “.rpkg” Rudder package format.
That packaging, combine with a new command line tool named “rudder-pkg”, allows to better manage plugin cycle of life. Of course, installation of plugin and their initialization scripts is now a matter of one command, but the tool also handle plugin update and deletion. In the long term, it will be able to check constraints on Rudder API version for compatibility, or dependencies and conflicts between plugins. And it is the first step toward a Rudder store for Techniques and Plugins.
Check it out!
Rudder 4.1 is available now. Here are a few easy ways to try it out:
- Online demo: You can see a demo version of Rudder on http://demo.rudder-project.org/
- Vagrant: Just want to test Rudder without worrying about a full installation, but with real virtual machines so that you can actually configure and test things, unlike the demo? We provide a Vagrant set-up. See Rudder Vagrant for getting started with Rudder on Vagrant.
- Linux server packages: The easiest way to test and use Rudder for prototyping or production is to install it from the provided Linux packages. For the server, the main current .rpm-based (RedHat, CentOS, SLES) and .deb-based (Debian, Ubuntu) distributions are supported. See https://www.rudder-project.org/site/get-rudder/downloads/ for details about repository and available versions.
