Managing configuration files is one of the, if not the, most crucial part of IT infrastructure management and the most efficient way to achieve it is by sharing resources (files, templates) to all your nodes. But, ensuring that the correct files are deployed on hundreds, thousands of machines is a complex and time consuming task (some would say a nightmare), which means that you need tools that provide a reliable, secure and performant way to provide a total control of this process. And this is where Rudder comes into play.
Rudder provides a central server to share your resources through policies you defined in our web UI and ensure that they were correctly distributed to all your nodes with our compliance system. In Rudder latest releases, file distribution has been significantly improved: this article will give you an overview of the current options you have to distribute files with it.
Introduction to file sharing
Since Rudder 6.1, there are 3 options to share files to your nodes:
- Downloading from Rudder server shared folder: Rudder server acts as file repository that nodes can fetch
- Linked ressources with configuration policies: define your configuration and share files packaged with it in one go
- Share files between nodes: Send files from a Node to other Nodes of your IT
We’ll cover more in detail, in the next section, some use cases related to each of the following options.
Downloading from Rudder server shared folder
- Either going through the built in technique in the directive section which is called ‘File Download (Rudder server)’.
- This technique includes few parameters to fill and requires that resources that need to be shared to your nodes to be added to the configuration-repository
- During the agent execution, it ensures that all files are updated within the nodes.
- Usually this method is dedicated to large files transfer.
- You can also create your own policies including file sharing using the technique editor and the following methods: File copy from rudder shared folder File from remote source
‘File Download (Rudder server)’ technique
As we mentioned above, you can simply use the first option via ‘File Download (Rudder server)’ technique.
Once the configuration is OK and the corresponding policy is deployed on agent, during agent execution the whole directory ‘my-app/conf-files’ will be downloaded by the nodes as part of the files transferring process. Then it will be maintained and synchronized whenever there will be changes in its content on the shared-files directory on the server. Of course, if files were changed on the node, they will also be replaced by a fresh copy from the server.
Also you can optionally add a command that will be executed following any changes in the resources content.
Using methods from the technique editor
When building your own technique, you may want to share files from Rudder server to your nodes. The main interest here is that you have more flexibility than the built-in technique and you can choose what to do with the file downloaded from Rudder server, like applying a templating engine on the file to it final path.
There are two methods that can be used:
- File copy from Rudder shared folder
- File from remote source
File copy from Rudder shared folder
File from remote source
Linked ressources with configuration policies
Introduced in 6.1, You can embed files directly within your technique when editing them in the technique editor. This provides a strong link between the resources and the technique and has some advantages over previous download methods.
- Resources are historised with your technique and are committed within Rudder git repository allowing you to track changes over time, checking for changes in both logic and files together.
Resources are downloaded when Rudder updates its policies ensuring that the agent will run with a version of the resource consistent with the policy, whereas previous download methods check and download files at agent execution could fail and lead to inconsistencies between a policy and its resources. - Resources folder path can be accessed directly using the ${ressources_dir} variable within the technique in the technique editor.
For example, let’s consider that you have an ansible playbook that you want to launch to configure some network appliance you can only access through ssh but ensuring that you get ansible feedback and keep a compliance trace in Rudder
File sharing between nodes
The final option covers a different need than the previous methods: sharing files between nodes. This allows to sync state between nodes by setting a flag file to share from one node, or maybe share a configuration file or a build artifact that has finished to build from a node to another.
Conclusion
So this concludes our tour of the different options of sharing files with Rudder, which offers various styles so you can adapt to any of your use cases. We went from highly customizable techniques (with technique editor), to ready to use but more limited built in directives through specific use cases sharing files between nodes. Mixed with other features, compliance, templating and node properties, Rudder is a great tool to ensure that your configuration files are correctly shared with personalized values from your nodes. Don’t hesitate to comment here or contact us about a use case, we may find out together how to achieve it with Rudder, or we can look on how to add this within Rudder in a new Version
