Backups automation for networks using Gitlab & Ansible

Backups automation for network equipement using Gitlab & Ansible we’ll discover what it is. We will see also how to automate backups of various equipment like Unix machines, switches, routers… using open-source solutions. In fact, we base our solution on Ansible for automation and Gitlab for UI and versioning.

The goal is to apply the principles of Git code management for backup management in order to have a platform rich in functionality, flexibility and without breaking the bank.

Git is a decentralized version management software. Git aims to allow several people to participate in the development of the same project.

GitLab.com offers a SaaS platform based on Git with a web interface above. That said, it’s also possible to install your own open-source Gitlab Community Edition platform..

Ansible is a configuration management tool that automates tasks with automation scripts.

Backups automation

Prerequisites

Before starting, you must have :

1. A machine to install Gitlab. In this case, CentOS 8
2. A Linux machine for Ansible server. Here, in fact, another CentOS 8
3. Target machines (Linux, switches, routers, etc.) to retrieve their configurations. Windows machines are also supported.

You can use other Git managers like Gogs.

Another interesting point to note; backups should ideally not be located in the same site as the target equipment. This is very important to have a more solid Disaster Recovery Plan.

A simpler option to avoid managing a Gitlab instance while having an excellent availability rate may simply be to take a paid account at gitlab.com or github.com and create a private repository.

Network equipment backups automation – Installation & Configuration

Gitlab

So let’s start by installing our own Gitlab repository manager.

Installation of Gitlab CE :

There are two versions of Gitlab, but we are going to focus on the Community Edition. (Mainly because it’s free ).

The installation steps differ depending on the Linux distribution used on our instance. Take the case of a CentOS 8 instance, we will need to type the following commands:

a. SSH installation and configuration

# installation of necessary packages 
gitlab:~$ sudo dnf install -y curl policycoreutils openssh-server  

# activation and launch of the ssh service 
gitlab:~$ sudo systemctl enable sshd 
gitlab:~$ sudo systemctl start sshd

b. Firewall configuration

# Check if the firewall status, to know if you need to execute the commands for the UI (http / https): sudo systemctl status firewalld ansible:~$ sudo firewall-cmd --permanent --add-service=http 
ansible:~$ sudo firewall-cmd --permanent --add-service=https  

# Loading the new firewall config 
ansible:~$ sudo systemctl reload firewalld

To enable email notifications, we must configure an email client, Postfix in our case:

gitlab:~$ sudo dnf install postfix 
gitlab:~$ sudo systemctl enable postfix 
gitlab:~$ sudo systemctl start postfix

The following command will add the Gitlab repository, useful for repatriating the RPM installation. (For paranoids, you can run the curl without a pipe, view the contents of the script and run it after ensuring no malicious code is there )

gitlab:~$ curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.rpm.sh | sudo bash

You just have to execute

gitlab:~$ sudo EXTERNAL_URL="https://gitlab.example.com" dnf install -y gitlab-ee

As a last step, you may want to integrate Gitlab with your LDAP for user management.

Gitlab Configuration

To be able to automatically authenticate from your ansible server to your Gitlab server, one quick way is to import the SSH key from your ansible server to the Gitlab server.

On your client, type :

gitlab:~$ cat ~/.ssh/id_rsa.pub

Copy the output and point to Gitlab UI> http://myURL/profile/keys [Settings> SSH Keys] Paste the content like this:

Then click on Add Key

You will now be able to synchronize content without problems.

Now, we can create our first project:

Fields to fill :

• name of the project
• Description
• Type (public / private)
• README initialization choice

Now that we’ve created our project, we need to import it to our Ansible server.

To do this, just copy the following link:

Finally, paste the command starting with git clone :

ansible:~$ git clone ssh://git@myURL/tools/network-backups.git

This command will clone your Gitlab project in a folder in the path where you executed your command /myPath/network-backups.

Ansible

Ansible installation

In your Linux server dedicated to Ansible, type the following command to install Ansible.

ansible:~$ sudo yum install ansible

Inside your Ansible server :

1. ansible:~$ cd /myPath/network-backups
2. Create two new directories ansible/ and backup/ ansible:~$ mkdir ansible/ backup/
3. Inside ansible/ create two files :

• ansible:~$ touch hosts
• ansible:~$ touch backup.yml

If you want to backup a Linux machine, just follow the following steps. For each Linux machine to be backed up, execute the following code:

ansible:~$ ssh-copy-id -i ~/.ssh/id_rsa.pub root@ipdemamachine

Make sure you can connect to SSH automatically without entering a password :

ansible:~$ ssh root@ipdemamachine

Ansible configuration

Linux server case

     1-Put yourself on your Ansible server.

     2-Complete the hosts file with :

[linux_servers] 
LIN-1 ansible_host=myip1 ansible_connection=ssh ansible_user=root 
LIN-2 ansible_host=myip2 ansible_connection=ssh ansible_user=root

PS: Replace myip1 and myip2 with the corresponding IP addresses of the target Linux machines.

     3-fil the file backup.yml with:

-hosts: linux_servers   
 vars:
   root_dir:"myDir"

tasks:
 -name: Specifying a path directly      
  fetch:
  src: /etc/hostname        
  dest:"{{myDir}}network-backups/linux/{{inventory_hostname}}- hostname.txt"
  flat: yes  
     
-name: get date      
  command: date      
  register: mydate  
  
-hosts: localhost   
  vars:root_dir:"myDir"

tasks:
-name: Store date as fact       
  set_fact:myCommitDate:"{{ ansible_date_time.date }}"
-name: sync git       
  shell:"cd {{root_dir}}network-backups && git add . && git commit -m {{myCommitDate}} && git push origin master"

We should have a similar result :

PLAY [linux_servers] **************************************************

TASK [Gathering Facts] ************************************************
ok: [LIN-1] 
ok: [LIN-2]  

TASK [Specifying a path directly] ************************************* 
changed: [LIN-1] 
changed: [LIN-2]  

TASK [get date] *******************************************************
changed: [LIN-2] 
changed: [LIN-1]  

PLAY [localhost] ******************************************************  

TASK [Gathering Facts] ************************************************
ok: [localhost]  

TASK [Store date as fact] ********************************************* 
ok: [localhost]  

TASK [sync git] ******************************************************* changed: [localhost]  

PLAY RECAP ************************************************************
LIN-1                      : ok=3    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0    

LIN-2                      : ok=3    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0    

localhost                  : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

Network equipment case (Cisco Router)

Now if we want to save the config of a network device (Let’s take a Cisco router as an example), all we need to do is :

     1-complete the hosts file with :

[cisco] 
ROU-2691-2 ansible_host=myip ansible_user=admin ansible_password=cisco ansible_network_os=ios ansible_ssh_extra_args=-caes256-cbc

     2-fill the backup.yml file with :

---
-name: BACKUP ROUTER CONFIGURATIONS   
  hosts: cisco   
  connection: network_cli   
  become_method: enable   
  gather_facts: no   
  vars:root_dir: myDir    
  
tasks:
-name: BACKUP THE CONFIG     
  ios_command:commands: show run     
  register: config_output    
-copy:content:"{{ config_output.stdout[0] }}"
  dest:"{{root_dir}}network-backups/cisco/{{ inventory_hostname }}_running.txt"
-hosts: localhost    

tasks:
-name: Store date as fact       
  set_fact:myCommitDate:"{{ ansible_date_time.date }}"
-name: sync git       
  shell:"cd {{root_dir}}/network-backups && git add . && git commit -m {{myCommitDate}} && git push origin master"

Take care to replace myDir with the path where your project is located.

     3-We execute the following command to retrieve the configuration and synchronize everything with Gitlab :

ansible:~$ ansible-playbook ./maconfig.yml -i hosts -b

PLAY [BACKUP ROUTER CONFIGURATIONS] ***********************************
 
TASK [BACKUP THE CONFIG] **********************************************
ok: [ROU-2691-2]  

TASK [copy] ***********************************************************
ok: [ROU-2691-2]  

TASK [sync git] *******************************************************
changed: [localhost]  

PLAY RECAP ************************************************************
ROU-2691-2                 : ok=2    changed=0    unreachable=0    failed=0    

localhost                  : ok=3    changed=1    unreachable=0    failed=0

Now let’s go to the Gitlab UI level:

On the project page that we created we find our directory (either Linux/ or cisco/) :

Change directory to Linux/, we should find our backup (In this case, the hostname of our machine).

To take advantage of Gitlab’s versioning capabilities, click on the button history :

The same is done to see the differences between two backups of the same network equipment.

We choose the folder cisco > name of the file then we click on history.

Gitlab offers a chronological view of our project’s modifications.

Indeed, using Git Blame feature, we can get more details about our file versioning to know exactly at what time each change was made.

To see this, let’s choose any backup file. Then we click on Blame.

We will get a similar view :

Daily execution :

To ensure that our script will be executed daily, all we need to do is create a cronjob on the Ansible machine :

ansible:~$ crontab -e

0 1 * * * /usr/bin/ansible-playbook /myPath/backup.yml

Depending on the version of Ansible, the path of ansible-playbook can vary between /usr/local/bin/ansible-playbook or /usr/bin/ansible-playbook
myPath is to be replaced with the path where your playbook is located.
The latter will be executed every day at 01 a.m.

 

Zen Networks is a leading provider of advanced IT solutions, specializing in log monitoring, automation and DevOps.

Our expert team offers a wide range of services, including IT monitoring, cloud services, agile solutions, and automation. We are also well-versed in the installation and implementation of Elasticsearch, Logstask, and Kibana on Docker.
By leveraging our extensive domain expertise and innovative technologies, we empower our clients to optimize their IT infrastructure and achieve operational excellence. Our solutions are designed to help organizations of all sizes to streamline their processes, improve efficiency, and reduce costs.
We invite you to take advantage of our complimentary consultation and quote service to learn more about how we can help your organization to achieve its IT goals. Contact us now to schedule your consultation and discover the benefits of working with Zen Networks.