I spent quite a lot of yesterday fixing my automated backup system at work.<\/p>\r\n
Most important bit of this post: In recent versions of macOS, if you want a scheduled Bash shell script to write to your disk, you need to explicitly give Bash \u201cFull Disk Access\u201d permissions.<\/p>\r\n
Side note: In most contexts, using a standard backup system like Time Machine or Backblaze would probably address the problem I was trying to solve here. (It\u2019s even possible that the standard backup system at work would solve my problem, but I set up my system before that existed, so I tend to forget about it.) So the specifics of what I was doing are probably something that few people would need to do. But some of the general ideas might still be useful.<\/p>\r\n
Some time ago, sparked by the frequency of losing all my browser tabs on my work computer, I set up shell scripts to make backup copies of various files, such as Chrome\u2019s sessions and tabs files. (Also my Things database, and various other important administrative files.)<\/p>\r\n
I then set up a crontab to run those scripts regularly.<\/p>\r\n
But at some point in the past few months, the cron jobs stopped working.<\/p>\r\n
I couldn\u2019t find any reason for that. When I ran the scripts manually, they worked fine. I checked the crontab syntax and it seemed to be fine. But the backups weren\u2019t happening.<\/p>\r\n
As part of my investigation, I checked on whether the computer\u2019s idea of what time it was matched mine. But I did that using the UNIX I then spent quite a bit of time trying to figure out why the command line didn\u2019t seem to know my local time zone, even though the GUI did. I thought that maybe cron was going by UTC time instead of local time, and that\u2019s why my tests weren\u2019t working.<\/p>\r\n Eventually, I just gave up on this whole line of inquiry. I never did figure out what was going on with the time zones.<\/p>\r\n While I was investigating various things about cron, I discovered that Apple doesn\u2019t recommend using it to schedule things!<\/p>\r\n It turns out that, as of several years ago, Apple instead recommends using (There\u2019s a third-party GUI tool that does various launchctl things for you, but I figured that was unlikely to be usable on my work computer.)<\/p>\r\n So I started trying to figure out So I created an Automator workflow to call a shell script. And I scheduled running that workflow using Calendar. (After a frustrating side trip when the Calendar pick-a-file GUI froze up for no clear reason and I misinterpreted what was going on.)<\/p>\r\n And then I re-discovered that at work, we can\u2019t run unapproved applications. (I\u2019m oversimplifying here, but close enough to true for this context.) So I couldn\u2019t use Automator for this task.<\/p>\r\n But in many contexts, I think using Automator to call the backup script and scheduling it using Calendar might be the simplest approach to the problem I was trying to solve.<\/p>\r\n (Well, simplest approach other than just using a standard backup system.)<\/p>\r\n So I created a launchctl .plist file and used \u2026and nothing happened.<\/p>\r\n Except that the status-of-last-run in So I spent a very long time trying all sorts of things to get it to work.<\/p>\r\n One tool that I wish I had known about sooner: Another tool that I wish I had tried sooner: the It turns out that if you\u2019re calling a Bash script from (Giving Full Disk Access to a shell is slightly complicated: you need to open \/bin in Finder, then drag the shell icon into the right part of the System Preferences window.)<\/p>\r\n As soon as I gave Full Disk Access to Bash, everything started working.<\/p>\r\n And it didn\u2019t occur to me until I wrote up this post that that might have been the problem with the cron jobs in the first place. Possibly if I had given Full Disk Access to Bash from the start, the cron jobs would have worked fine.<\/p>\r\n Some day, I need to learn how to get scheduled shell scripts to provide me with useful error reporting. But today is not that day.<\/p>\r\n\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":5,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[83,79,68,86,24,133],"tags":[],"class_list":["post-19313","post","type-post","status-publish","format-standard","hentry","category-apple","category-computers","category-programming","category-security","category-software","category-writing-samples"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.kith.org\/jed\/wp-json\/wp\/v2\/posts\/19313","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kith.org\/jed\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kith.org\/jed\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kith.org\/jed\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kith.org\/jed\/wp-json\/wp\/v2\/comments?post=19313"}],"version-history":[{"count":1,"href":"https:\/\/www.kith.org\/jed\/wp-json\/wp\/v2\/posts\/19313\/revisions"}],"predecessor-version":[{"id":19314,"href":"https:\/\/www.kith.org\/jed\/wp-json\/wp\/v2\/posts\/19313\/revisions\/19314"}],"wp:attachment":[{"href":"https:\/\/www.kith.org\/jed\/wp-json\/wp\/v2\/media?parent=19313"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kith.org\/jed\/wp-json\/wp\/v2\/categories?post=19313"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kith.org\/jed\/wp-json\/wp\/v2\/tags?post=19313"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}date<\/code> command, which gave me the time in UTC instead of my local time.<\/p>\r\nmacOS cron is deprecated?<\/h3>\r\n
launchctl<\/code>. (cron is still supported, but not recommended.) You specify a bunch of parameters in a .plist file (including saying what to do and when to do it), and then you load the .plist file into the launch system, and the launch system runs the code at the specified time(s).<\/p>\r\nlaunchctl<\/code>, but then I remembered that Apple now has a GUI for automating stuff.<\/p>\r\nAutomator to the \u2026 oops, never mind<\/h3>\r\n
Back to launchctl<\/h3>\r\n
launchctl load<\/code> to prepare it to run, and I used launchctl list<\/code> to make sure it was loaded, and then I waited for the time that I had specified in the .plist file\u2026<\/p>\r\nlaunchctl list<\/code> changed from 0 to 1, suggesting that some kind of error had happened on the last run, but of course no indication of what kind of error.<\/p>\r\nplutil<\/code>, which (among other things) checks the syntax of .plist files. It turned out that my syntax was fine, but it would have been nice to know that a lot sooner so that I could have skipped spending time manually checking syntax.<\/p>\r\nStandardErrorPath<\/code> parameter in the .plist file, which lets you specify a file for the system to log error messages to. When I finally tried that, the resulting error messages were super helpful, and quickly led me to the solution.<\/p>\r\nSolution: Full Disk Access<\/h3>\r\n
launchctl<\/code>, and that script writes to your disk, you need to explicitly give Bash \u201cFull Disk Access\u201d permissions. (In System Preferences, Security & Privacy preferences, Privacy tab.) Even if you\u2019re calling the script as superuser, it won\u2019t write to disk without Full Disk Access.<\/p>\r\n