Munki, Gitlab & Git-LFS

NOTE: This is not a HOW TO, more of a rant, the how to that may or may not come later. This is written from the perspective of someone who manages (for the most part) their infrastructure from end to end, vm config, service management, monitoring and patching. When it comes to scale, and complexity your milage may very.


  • there are some caveats this route
  • its worth it
  • git isn’t a one way street, you can use it as little or as much as you want
  • Gitlab as a tool sets you up for more in the CI/CD realm

The idea

This post is just some gernal process / lessons learned from migrating a Munki repo to a git tracked repository

fwiw, “git” and “git-lfs” don’t need Gitlab, its just my life, and these are my thoughts.


  • vcs
    • a version control system was huge, the benefit to see what I changed when I changed was really rad
  • pipelines
    • though I didn’t have that word for it at the time, some flow of dev/test/deploy of a repo
  • multiple users
    • though this still isn’t prod, I wanted the ability for multiple users to make changes, and said changes be tracked
  • no more smb/sfp
    • all the transport security can be done with pub/priv keys, etc

Why Gitlab?

  • I could keep it locally
    • OR use the cloud, privately too
  • The CE edition is rad (and I will do my best to link exclusively to the Community Edition docs)
    • and free
  • It has CI/CD built in
    • (though I didn’t realize how great this was ’till later)

The Reality

There were some kinks that had to be worked out, the first and maybe most obvious was managing large files with git. Gitlab has LFS support built in, which is rad, but still comes with its own nuances.


Git Large File Storage

Git Large File Storage (LFS) replaces large files such as audio samples, videos, datasets, and graphics with text pointers inside Git, while storing the file contents on a remote server like or GitHub Enterprise.

Heres a handy tutorial: Getting started with Git LFS

Git LFS on Gitlab

Sounds great, and can be enabled in Gitlab easily.

LFS on Gitlab also requires you use HTTPS for auth and transport, rather than ssh. Digging into the Gitlab administration docs, we can see the docs straight list some pretty big limitations:

Support for removing unreferenced LFS objects was added in 8.14 onwards.
LFS authentications via SSH was added with GitLab 8.12
Only compatible with the GitLFS client versions 1.1.0 and up, or 1.0.2.
The storage statistics currently count each LFS object multiple times for every project linking to it

But some stuff they don’t tell you that I found out the hard way-

LFS caches (this, srsly)

What I found was when Gitlab was receiving LFS file sit could cache them in a /cache location, then move them to the configured storage location.

This was noticed when my OS disk on my VM filled. womp. Modifying the gitlab.rb you can change the LFS storage location.

Looking at gitlab.rb.template, we have these LFS options:

### Git LFS
# gitlab_rails['lfs_enabled'] = true
# gitlab_rails['lfs_storage_path'] = "/var/opt/gitlab/gitlab-rails/shared/lfs-objects"
# gitlab_rails['lfs_object_store_enabled'] = false # EE only
# gitlab_rails['lfs_object_store_direct_upload'] = false
# gitlab_rails['lfs_object_store_background_upload'] = true
# gitlab_rails['lfs_object_store_proxy_download'] = false
# gitlab_rails['lfs_object_store_remote_directory'] = "lfs-objects"

Oh nice! a gitlab_rails['lfs_storage_path'] option, sweet, so you can store your repo on /dev/sd(whatever), this is good to know- so say- your os disk doesn’t fill…

And what about your client? Most people have Munki running with autopkg or munki admin on a macOS box. So you have to have Git-LFS on your mac, and little less supported.

LFS on macOS

You can do this via the installer git provides (macOS), which is basically the command line extension and a shell script. Or you could use brew, loads of people have loads of opinions about brew, so don’t use it and just keep that to yourself.

Regardless you will need to initialize it-

# Update global git config
$ git lfs install
# Update system git config
$ git lfs install --system

Which is like HEY GIT we going to use LFS now. But for what? and when?, touché git.

Looking into Configuring Git Large File Storage you can, once you’re in a tracked dir,

$  git lfs track "*.psd"
Adding path *.psd

Which is cool, and it gets added to your .gitattributes file, but most admins know what they’re big files in their munki repo are… so somehting like this in your .gitattributes file may be more applicable:

*.pkg filter=lfs diff=lfs merge=lfs -text
*.mpkg filter=lfs diff=lfs merge=lfs -text
*.dmg filter=lfs diff=lfs merge=lfs -text
It still CACHES

Lets take a look at our mcOS git env…

$ git lfs env

That effectively means your local repo folder can be double its actual size. So plan accordingly.

Once its tracked it is as simple as a git push to get those files and changes to the Gitlab server.


I know it sounds pretty negative up until this point, the benefit though, of the hump of getting it setup is git. And you can “git” as much or as little as you want.

Munki & Git

Theres some great documentation on that here. But specifically check out Munki’s Repo Plugins, specifically the GitFileRepo:

GitFileRepo – a proof-of-concept/demonstration plugin. It inherits the behavior of FileRepo and does git commits for file changes in the repo (which must be already configured/initialized as a git repo)

This is rad because once your repo is tracked it does git commits for file changes in the repo. Rad.

Git Theories

master Branch All Day

Once you have all your stuff in git, you can choose how you’re going to use it- I have seen a lot of benefit of a tracked repo that just simply commits changes to master. Since dev, test, and prod are all contained in “munki logic” commit any changes to master allows you to track any changes made to .pkginfo or manifest files.

Which this works, and is totally legit, and will get you a load of good info from tracked files.

Let’s Face(book) It, we need more

Or maybe you don’t but Facebook’s CPE team has a really rad option…

Check out their CPE resources, specifically the autopkg_tools

This is an AutoPkg wrapper script creates a separate git feature branch and puts up a commit for each item that is imported into a git-managed Munki repo.

They have a Getting Started Guide if this sounds like more of what you’re looking for-

Stuff I didn’t touch on, that I could

  • catalogs, not tracking them, making them, making them with a runner
  • git-fat as an option?

In summary

I personally won’t go back, there was a little tweaking to get it all sorted, but the information and tracking git provides to the munki repo is well worth it.

Coming Soon

  • I am going to try and sanitize some of the helpful CI/CD stuff I got rolling in Gitlab and talk about it here.
  • Munki in the cloud stuff
  • macOS monitoring?
  • maybe a similar rant on osquery and


Macadmin Resources

🎥 Mac Justice – Intro to Gitlab (MacDevOPs, shorter)
🎥 Mac Justice – Intro to Gitlab (PSU Macadmins, longer)

🔗 Advanced Munki Infrastructure: Moving to Cloud Services by Rick Heil
🎥 Advanced Munki Infrastructure: Moving to Cloud Services

General Resources

🔗 Git LFS
🔗 Gitlab CE
🔗 Gitlab and LFS

🎥 Git Large File Storage – How to Work with Big Files
🎥 Git LFS Training – GitHub Universe 2015
🎥 Tracking huge files with Git LFS, GlueCon 2016

MunkiAdmin sync on “Save”

The idea was to use MunkiAdmin‘s script features to automatically rsync changes from a management machine to a machine hosting the repo for clients access. My testng case was syncing from a macOS machine to Ubuntu 16.04. This utilizes rsync with psk’s, great documentation specifically on check out Digital Ocean‘s article.

The Script

The main bread and butter is a simple rsync script:

/usr/local/bin/rsync -vrlt -e "ssh -i /Users/$macUSER/.ssh/" --chmod=$symbolic --chown=&nixUSER:$nixGROUP /macOS/munki_repo/* $nixUSER@$nixHOST:/nix/munki_repo/

So to break it down…

  • v
    • verbose
  • r
    • recursive
  • l
    • symlinks (optional? probably not needed in a munki_repo specifically)
  • t
    • preserve times
  • specify the remote shell
    • ssh
    • -i
      • identity file
    • /Users/$macUSER/.ssh/
      • The key you would like to use (that also exists under authorized keys on the receiving server”
  • specify the modification privileges via symbolic
    • 4744=go+r,u+rwxs
    • I just cheated, here.
  • change the ownership
    • user:group
  • local repo
  • destination admin@host
  • :path/to/repo_destination

Tip: That should do it, you can always use -n or –dry-run to check this sync without actually syncing any data.

  • -n, –dry-run “perform a trial run with no changes made”

MunkiAdmin Integration

I added the command as well as some logging items to a bash script, and saved it as repository-postsave.

MunkiAdmin full documentation on custom scripts is available here, though its pretty cut and dry:

  • scripts should be saved in <repository>/MunkiAdmin/scripts/ or ~/Library/Application Support/MunkiAdmin/scripts/.
  • The presave scripts can abort the save by exiting with anything other than
  • All of the scripts are called with the working directory set to the current repository root.

Further more according to MunkiAdmin documentation, MunkiAdmin looks for executable files (with any extension) with the following names:

  • pkginfo-presave
  • pkginfo-postsave
  • manifest-presave
  • manifest-postsave
  • repository-presave
  • repository-postsave

I chose repository-postsave because a sync would be the last thing we would want to do. I moved my script to <repository>/MunkiAdmin/scripts/, reloaded MunkiAdmin, and then added a pkg to test.

Quick Test

I figured why not test it with a worst possible case..? How about a 10.11.6 upgrade pkg, 6.24 GB? Yeehaw.

So I imported via munkiimport, and then reloaded MunkiAdmin. As the script is tied to “Save” in munki admin, no sync occurs until then…

I hit “Save” and everything died:

Screen Shot 2017-05-02 at 7.45.54 AM.png

But not really, I had a hunch that it was just working hard, and MunkiAdmin was waiting until the script exited, and those suspicions were confirmed:

Screen Shot 2017-05-02 at 7.45.59 AM.png

Once the transfer processes completed, MunkiAdmin was back to normal.

Much success! As a note smaller more regular pkg/infos and catalog files really quickly* (your milage may vary depending on your speeds).


rsync 3.1, Your keen eye may have picked up on /usr/local/bin/rsync vs /usr/bin/rsync, as one may expect on macOS. Unfortunately macOS ships with rsync v2.6.9, which does not support the –chown functionality, so I had to brew err pursue other avenues for rsync to completely work in this capacity…


No manual rsync of your repo anymore! Well… actually its still manual on “Save” but its automatic!

If you use MunkiAdmin this scripting has a lot of potential for different automation tasks, git integrations or whatever you may do to your repos after “saving,” to pkgs or whatever your use case may call for- I really like this integration and I just thought I’d share this bit I found useful.

Munki, Docker and why you’d want to even try. The video!

Here  is a video from PSU MacAdmins where I take a high level look at Munki, Docker and why you’d want to even try to get them to play nice with one another- or what better options for hosting your repo may be…


I love what the folks on the PSU MacAdmins team have been doing for the community, you can read more about them and the PSU MacAdmins conference here.

Munkireport-PHP on Docker

In my previous post I delved into getting a munki repo with ssl client-server cert protecting up and running on a docker host. You can read that here.

So the next step I wanted to get up and running next to it was the great tool Munkireport-PHP.  More than likely if you are looking into docker as an option for a munki repo server you have some experience with munkireport-php as a reporting tool for your fleet. I am not endorsing this as the only way or even a preferred way- I just wanted to see what it looked like to get it up and running on a docker host- and it was super easy (in comparison to getting a munki repo configured and up and running).

Continue reading “Munkireport-PHP on Docker”

Munki-Docker & SSL (proofing a concept)


The ustwo docker munki git repo has been removed, thanks to Clayton Burlison you can see a forked version here


This is a stumbling journey of a layman and how he got a proof of concept munki repo secured with ssl in some way on a docker host. Before this I knew little about docker, less about ssl/certs and had barely touched Ubuntu let alone a server version.


The end game: Have a Munki Repo that requires client side ssl certs for authentication (as a means to protect for un authorized access to a repo) and hosted in a docker environment (for easy maintenance and recreation of servers in multiple locations.

Continue reading “Munki-Docker & SSL (proofing a concept)”