Grav, Synchronize content with git

The advantage of a markdown file-based CMS is also the ability to version its content easily. Gratefully, grav has a plugin to synchronize its content with a git repository and therefore GitLab.

Historically, CMS first used databases (SQL then noSQL for some). This is very convenient for developers because these databases are designed to easily and efficiently find content that matches search criteria.

But for webmasters, it is less convenient. To access the content, we must systematically go through a tool (the CMS or the specific client of the base). It adds steps to saving and it is frankly painful if you want to modify an article without going through the CMS.

Then came the "file-based" CMS which, as the name suggests, forget about databases and store everything in files. For developers, this is less convenient because they have to manage searches (and any indexes) themselves and add a caching system to speed up response to queries.

But for users, it is much more convenient as you can access the content with a simple file browser and edit the articles is done with any text editor. Backup by copying a directory, restoration by copying it back...

Russellbande @ pixabay
Russellbande @ pixabay

With a system so easy to copy and duplicate, inevitably we thought about versioning all of these files. We could have used CVS and SVN if they hadn't become obsolete in the meantime. We could also have used mercurial, but fashion wanted us to use git (github probably had something to do with it).

Like all respectable file-based CMSs, grav therefore has a specific plugin to take care of versioning and synchronization with your favorite repositories.

After the classic installation of the plugin, we will configure its connection to the repository, the content to synchronize and finally some security details.

Plugin installation

The installation of the plugin could not be more classic.

On command line

Although this plugin is most useful when your cms has a graphical administration interface, you can install it from the command line like any other.

./bin/gpm install git-sync

Note that if all your files are owned by another user (i.e. www-data), you must use sudo to launch the manager with the identity of that user. The command is then slightly modified as follows:

sudo -u www-data ./bin/gpm install git-sync

Via the administration interface

If you prefer the administration interface, you can go through the “Plugins” menu on the right, then the “+ Add” button at the top, enter the name of the plugin in the input field then click on the “+ Install” button ".

Chose the plugin to install
Chose the plugin to install

The interface will then ask you to confirm, click on "✔ Continue".

Confirm installation
Confirm installation

Once the installation steps are successful, the interface will automatically redirect you to the plugin's configuration page and launch the configuration wizard.

Launching the Wizzard
Launching the Wizzard

Plugin configuration

As long as you have not finished configuring the plugin, just visit its configuration page and grav will automatically launch the wizard for you. If for some reason it doesn't, you can relaunch it via the "Wizard" button at the bottom of the page (in the "Actions" section).

Action button in the footer
Action button in the footer

The configuration of the plugin involves, in parallel, certain configurations of your repository. So, in the following, we will switch between applications regularly.

Connection to the repository

As the plugin will need to authenticate to the repository, you will need to provide it with a valid username and password.

Gitlab / LDAP side

You could give it your own identifiers but I do not find it very secure because in the event of a compromise of your CMS, your own identifiers will be accessible. So I prefer to create a specific user.

Since our GitLab is synchronized with our Active Directory, we just had to create a restricted account.

Grav side

Once you have an account for your repository, you can do the first step of the setup. We chose GitLab, enter the account name and its password for the connection and we click on “Next ❯” at the bottom of the page.

Connexion to the repository
Connexion to the repository

Repository settings

Unsurprisingly, for the plugin to work, it also needs a repository with, and this is important, at least one commit on a branch. As you are going to configure the plugin to follow a branch, your repository must have one with a commit.

Gitlab side

For those who use GitLab and don't want to bother, when creating the project, simply check the box provided to create a README.

README creation with gitlab
README creation with gitlab

Also, keep in mind that for this to work, the plugin user must be able to push their commits to the branch. It is therefore necessary to give him the rights on the project (maintainer on the protected branches, develop otherwise). On GitLab, once on your project page, go to the "Members" menu, then complete the user invitation form and click "Invite".

Allowing grav’s user
Allowing grav’s user

Grav side

Once the repository has been created and the user has the correct rights, you can do step 2. This screen asks you for the repository's parameters: its address (in https) and the branch to follow.

Setup of the repository
Setup of the repository

Fortunately, this screen has a button to check the settings. Abuse it, it allows you to debug as you adjust. Once it works, click on “Next ❯”.

WebHook

So far, the plugin is configured to push changes made with grav to the repository. If you want changes made elsewhere and also pushed to the repository to be automatically transferred to your grav, you must configure a webhook.

You can think of webhooks as addresses that your repository (GitLab and cie.) call when certain events take place to notify sites of the event and allow them to react to it.

Grav side

This time, we start with grav. To increase security, I prefer to check the box "Use Webhook Secret" which deploys a new field. Do not change the values and keep this screen open, we will need it for the configuration on the GitLab side.

Webhook informations
Webhook informations

Gitlab side

On the GitLab side, we have several things to do.

First, if you are using your own servers with local IPs, GitLab will first refuse to connect to them. This is a security measure, to prevent an external user of your GitLab from being able to use your gitlab to access your internal network.

To disable it, go to the administration area of your GitLab, then in the settings, find the "Outbound requests" section and check the "Allow requests to the local network from web hooks and services" box.

Screenshot © gitlab.com
Screenshot © gitlab.com

Disabling the restriction does not necessarily put your network at risk but its partitioning will depends a little more on your firewalls. If there are any restrictions, then they are the ones you should turn to.

Second setting for TLS connections. For security, I'm assuming you've configured HTTPS with a TLS certificate signed by your root. In order for GitLab to recognize your server and agree to send its requests to your server, it must therefore trust your root.

To do this, you need to deploy your root certificate (let's call it root.arsouyes.org) by copying it to the /etc/gitlab/trusted-certs/directory.

sudo cp root.arsouyes.org.crt /etc/gitlab/trusted-certs/

We can now configure the webhook. To do this, via the GitLab graphical interface, go to the "Settings / Webhooks" menu. Here, enter grav's URL by adding the webhook address (in URL) it gave you and then the secret token (in Secret Token). Leave “Push Event” checked.

Webhook setup
Webhook setup

Go down the page, check that SSL verification is activated and click on "Add webhook".

Adding webhook
Adding webhook

No need to run a webhook test now, until you finish the setup with the wizard, grav will respond with errors.

Back to grav

Go back to grav and click on “Next ❯”.

What to sync

Last step, choose what you want to sync. No best choice here, it depends on what you want to do:

Directories to sync
Directories to sync

Make your choices and click on “Next ❯”.

The plugin will launch a first synchronization.

Practical details

At this point, your plugin is functional but there are a few handy details to know before you get started for good.

Pay attention to details, DariuszSankowski @ pixabay
Pay attention to details, DariuszSankowski @ pixabay

Automatic merges

When working with git, it sometimes happens that we suddenly move a branch from one commit to another without worrying about the new commit being a descendant of the old one (i.e. during rebases). In these cases, we must force the git push for the repository to accept to move the branch.

It is generally considered bad practice when the branch is pushed to a central repository because it imposes some precautions with other contributors. But as with all bad habits, we sometimes do it anyway.

For once, the git-sync plugin is not going to help you at all because when it receives the new version, it will have to cope with it itself, and since it does not really know what to think about this rather brutal branch move , it will merge the remote branch and its own then push its result to the central repository.

With luck, everything works out (because the two branches are the same) but I do not guarantee the result you will get.

If you moved your branch to make the commit tree look nice (that's often why we actually rebase), git-sync will not only undo all your great cleanup work, but will make it worse since it will recover the old commits and add a new merge...

With this plugin, avoid rebases and other force commits to the repository.

Ignore files

Second thing, if you need to version the configuration (in general), you can do it by entering the config directory in the plugin configuration. The thing is, you might not want the configuration of some particular plugins to be versioned.

In particular the configuration of git-sync itself. On the one hand, because it contains the login details (and even if the password is protected, that's not a great habit to take). On the other, because it contains the name of the branch it follows (and if you use several branches, with several grav, the merges will be painful since the file of a branch can be overwritten with the data of a other and your instance change follow another branch).

Fortunately (# 197), the developers have added an option to tell the plugin to ignore certain files and directories in particular . If you want the plugin to ignore syncing its own configuration, just add /config/plugins/git-sync.yaml to the "Git Ignore" option.

Ignore its own setup file
Ignore its own setup file

Once this option is added, the plugin synchronizes it and will then ignore any changes to this file. If they are done in grav, they will not be sent to the repository. If they are made in the repository, they will be ignored by grav. You can therefore merge your branches quietly, the configuration of git-sync will not be impacted.

And after

Now the content of your grav application (the pages and if you have chosen it, the themes, plugins and configuration) are synchronized with your GitLab repository. Whether the changes take place on one side or the other, the plugin will keep everyone in sync.

To go even further, you can connect all these applications to an LDAP directory (GitLab and grav) then set the git-sync plugin to consider the logged in user as author of the commit. In the plugin configuration, modify the configuration of "Commits Author" to take the value "Use Grav User Name".

Setup the commit's author
Setup the commit's author

With this setting, changes made by your users are linked to a single account, the one they have in your directory.