A key thing to remember is that your keys must not have a passphrase set when created, otherwise it will prompt, but you’ll not be able to see it and your pipeline won’t run!<\/p>\n\n\n\n
Now in my particular example i’m going to be running the pipeline tasks under “root”, however the documentation https:\/\/docs.gitlab.com\/ee\/ci\/ssh_keys\/<\/a> shows you how to do this under the “gitlab-runner” user instead and also how to create key in more detail. You should place the key in the default location ~.ssh\/id_rsa<\/strong> and ~.ssh\/id_rsa.pub<\/strong>. You can use other locations, the links above will help you with how to do that, but using different paths is beyond the scope of this document.<\/p>\n\n\n\n Now we need to add the keys into gitlab, to do this firstly log on to gitlab. Click on your icon (profile) and then click, “Preferences”.<\/p>\n\n\n\n Now click “SSH Keys”.<\/p>\n\n\n\n Copy and paste the contents of the ~.ssh\/id_rsa.pub<\/strong> file into the “key” box, and give the key a title. You can leave the expiration date as the default for now, which is 1 year. <\/p>\n\n\n\n NOTE! Ensure you are pasting in the Public SSH key, not the private one. The private one is to be kept….private!<\/em><\/p>\n\n\n\n Once the key has been imported, let’s test you can SSH to gitlab from your machine without needing the key, you can do this with:<\/p>\n\n\n\n Assuming all is well you’ll get a “Welcome to Gitlab!” and your username message.<\/p>\n\n\n\n Now we are ready to push our Apache configuration into our gitlab repository. So:<\/p>\n\n\n\n Run these commands to ensure the git configuration specifies the correct user:<\/p>\n\n\n\n In our case the directory is owned by root, so we need to perform these steps as root to ensure the git configuration can be created correctly. So let’s setup git and create our initial commit.<\/p>\n\n\n\n Now refresh the gitlab repo, you should now see all the files have been uploaded.<\/p>\n\n\n\n Okay, so now let’s recap where we are, we have:<\/p>\n\n\n\n Okay, so what’s next. Well we now have the Apache2 configuration in the repository, but we’ll not want any of this to be manual, what we want is that we can clone this repository to our normal workstation (i.e. not this server), make a change to the Apache2 configuration, commit it, then merge it and for the pipeline to start and trigger the gitlab-runner to automatically pull down the latest version of the configuration and apply it. <\/p>\n\n\n\n So, let’s do that now.<\/p>\n\n\n\n So currently we have no CI\/CD pipeline configuration for that we need to add a .gitlab-ci.yml<\/strong> file into our repository, this file determines the pipeline configuration, and although this is a simple example the pipeline can be very complex with a number of stages, inputs, outputs etc. <\/p>\n\n\n\n For this inital setup its recommended to be performing the creation of the next two files directly in the Apache2 configuration directory \/etc\/apache2<\/strong>, so you can just push it up when we’re all done (and trigger the pipeline to start for the first time).<\/p>\n\n\n\n Create the following file .gitlab-ci.yml<\/strong> at the root of the repository (project) with the following contents:<\/p>\n\n\n\n Let’s just explore what this file does. It defines the pipeline, so our pipeline has two stages: “verify” and “deploy”, then within each stage we have the steps that are going to be taken. <\/p>\n\n\n\n In the “verify” stage it uses a Docker image to verify that the configuration is syntactically correct, if this stage was to fail (a typo in the configuration file for example), then the pipeline will stop and it won’t apply duff configuration to your server.<\/p>\n\n\n\n Assuming all is well then the pipeline moves to the “deploy” stage, this quite simply tells the gitlab-runner to trigger the deploy.sh<\/strong> BASH script file, which you now need to create in the root of the repository with the following contents:<\/p>\n\n\n\n You also need to make this file executable with:<\/p>\n\n\n\n Okay create, so now we have the CI\/CD configuration set within<\/p>\n\n\n\n Let’s commit and push this configuration to repository, which will then trigger the first run of our pipeline!<\/p>\n\n\n\n For these steps, we’ll first commit and push our CI\/CD pipeline files, then swap to look at gitlab to monitor what happens.<\/p>\n\n\n\n You’ll now see the little blue clock symbol, this shows that CI\/CD has found a pipeline and has started to run it. Click on the clock symbol and we’ll inspect what is going on….<\/p>\n\n\n\n So the new files have been commited, you may want to make some random (benign) change to a configuration file, but otherwise you should now see the two stages either running or have run (depending on how quickly it scheduled and ran them).<\/p>\n\n\n\n All appears to be fine, so let’s click on the “verify” stage. As you can see all fine, no errors in the configuration.<\/p>\n\n\n\n So if we then click on the next stage (i.e. the “deploy” stage) we can then see:<\/p>\n\n\n\n And again all is good, we can see the deploy.sh has been run, it has pulled down the latest copy of the repository files from gitlab and then successfully reloaded the Apache2 configuration.<\/p>\n\n\n\n I had a few issues with permissions, the first time I ran the pipeline I got the following error. Stack overflow appeared to help here: https:\/\/stackoverflow.com\/questions\/64329037\/gitlab-shell-script-permission-denied<\/a><\/p>\n\n\n\n To fix you can do this:<\/p>\n\n\n\n The CI\/CD will then re-run and you should be good!<\/p>\n\n\n\n And that is it. We have our test web server host setup with a git repository held locally and a CI\/CD pipeline which can trigger tasks to take place on the host via the gitlab-runner.<\/p>\n\n\n\n If you want to check what is going on you can tail the syslog to watch the gitlab-runner do its thing, this can be seen with:<\/p>\n\n\n\n Image Attribution<\/a><\/p>\n","protected":false},"excerpt":{"rendered":" Assuming you’ve read and followed part 1, now on part 2 we’ll get our SSH keys created, and the gitlab-runner registered with gitlab as a “runner” so we can create our pipeline. Configuring the SSH Keys for Gitlab-Runner To make the local gitlab-runner on your machine have the correct keys, you need to generate them … Read more<\/a><\/p>\n","protected":false},"author":4,"featured_media":3822,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[42,43],"tags":[],"class_list":["post-3809","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops","category-git"],"_links":{"self":[{"href":"https:\/\/geekmungus.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/3809","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/geekmungus.co.uk\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/geekmungus.co.uk\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/geekmungus.co.uk\/index.php?rest_route=\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/geekmungus.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3809"}],"version-history":[{"count":6,"href":"https:\/\/geekmungus.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/3809\/revisions"}],"predecessor-version":[{"id":3825,"href":"https:\/\/geekmungus.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/3809\/revisions\/3825"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/geekmungus.co.uk\/index.php?rest_route=\/wp\/v2\/media\/3822"}],"wp:attachment":[{"href":"https:\/\/geekmungus.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3809"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/geekmungus.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3809"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/geekmungus.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3809"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
But as mentioned in our case we need to run under the “root” user, so we’ll generate a SSH keypair within “root”, so when gitlab-runner executes our pipeline and its tasks it can 1. trigger a pull of the most recent configuration from our project repository but 2. have the correct permissions to be able to execute command to say restart services like Apache2.<\/p>\n\n\n\nsudo -s\n(now root)<\/code><\/pre>\n\n\n\nssh-keygen -t rsa -b 2048<\/code><\/pre>\n\n\n\nssh -T git@gitlab.com\nssh -Tvvv git@gitlab.com<\/code><\/pre>\n\n\n\nSetup Git and Push the Initial Configuration<\/h2>\n\n\n\n
sudo -s\n(now root)<\/code><\/pre>\n\n\n\ngit config --global user.name \"Your Name\"\ngit config --global user.email \"thingy@doberry.com\"<\/code><\/pre>\n\n\n\ncd \/etc\/apache2\nsudo git init --initial-branch=main\nsudo git remote add origin git@gitlab.com:tristan.self\/apache2.git\nsudo git add .\nsudo git commit -m \"Initial commit\"\nsudo git push --set-upstream origin main<\/code><\/pre>\n\n\n\nRecap Where we Are<\/h2>\n\n\n\n
\n
Configuring the Pipeline Files<\/h2>\n\n\n\n
stages:\n - verify\n - deploy\n\nverify-apache2-config:\n stage: verify\n image: ubuntu\/apache2\n script:\n - echo \"Checking config\"\n - apachectl configtest\n\nupdate-apache2-config:\n stage: deploy\n script:\n - .\/deploy.sh<\/code><\/pre>\n\n\n\n#!\/bin\/bash\nset -e\n\n# another quick syntax check, in situ rather than in separate CI runner\n# 'set -e' will catch errors, no need to explicitly check\n\necho \"Configuration Updated!\"\n\nuser=\"root\"\ngit_home=\/etc\/apache2\n\necho \"Updating master repository...\"\nsudo -u $user \/usr\/bin\/git --git-dir=${git_home}\/.git fetch\nsudo -u $user \/usr\/bin\/git --git-dir=${git_home}\/.git --work-tree=${git_home} pull origin main\n\necho \"Reload Apache2 Configuration\"\nsudo systemctl reload apache2<\/code><\/pre>\n\n\n\nchmod +x deploy.sh<\/code><\/pre>\n\n\n\nCommit, Push and Run!<\/h2>\n\n\n\n
sudo -s\n(now root)\n\ncd \/etc\/apache2\nsudo git add .gitlab-vi.yml\nsudo git add deploy.sh\nsudo git commit -m \"Added CI-CD files\"\nsudo git push origin main<\/code><\/pre>\n\n\n\n![\"\"](\"https:\/\/geekmungus.co.uk\/wp-content\/uploads\/2023\/07\/image-4-1024x352.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/geekmungus.co.uk\/wp-content\/uploads\/2023\/07\/image-5.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/geekmungus.co.uk\/wp-content\/uploads\/2023\/07\/image-6-1024x569.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/geekmungus.co.uk\/wp-content\/uploads\/2023\/07\/image-8-1024x695.png\")
<\/figure>\n\n\n\nA Bit of Troubleshooting<\/h2>\n\n\n\n
$ .\/deploy.sh\nbash: line 127: .\/deploy.sh: Permission denied<\/code><\/pre>\n\n\n\nsudo git update-index --chmod=+x deploy.sh\n\nsudo git commit -m \"fix\"\n\nsudo git push origin main<\/code><\/pre>\n\n\n\nConclusion<\/h2>\n\n\n\n
tail -f \/var\/log\/syslog<\/code><\/pre>\n\n\n\nAdditional Random Information<\/h3>\n\n\n\n
sudo chmod g+w -R .git<\/code><\/pre>\n\n\n\n