Lorem Ipsum

Summary of today’s Apple Books adventure, so I don’t forget what’s happened so far. This is primarily for my own future reference; not expecting anyone to read the whole thing.

There were two problems:

• I have something like 700-800 ebooks. I used to be able to sync them, using iTunes, from my Mac to my iPhone. Sometime in the past week or two, half of them disappeared; only 450 of them were visible in iTunes sync, though all of them were still listed in Apple Books (formerly known as iBooks).
• When I tried to investigate this issue, I discovered that many of my books now have invisible text when viewed in Apple Books on my Mac. When I open a book, my highlights are still visible, and some chapter numbers, but the text doesn’t render.

After about five hours of poking at the first issue today, much of it on the phone with Apple tech support, I learned that the missing books had been moved to iCloud. At some point, my new iPhone had gotten the option enabled that synced non-Apple ebooks via iCloud, and somehow even after I turned that option off, all the non-Apple ebooks were still only in iCloud. Apple Books/Mac still knows about them, but they’re not in the folder where Apple Books files live (~/Library/Containers/com.apple.BKAgentService/Data/Documents/iBooks/Books/). (Side note: That directory is made annoying and complicated by all of the filenames being long strings of numbers instead of being, say, the title of the book.) Instead, they’ve been moved to another folder, one that’s invisible in the Finder even when the show-hidden-files option is turned on: ~/Library/Mobile Documents/iCloud~com~apple~iBooks/Documents. (And yes, those are tildes, not hyphens.)

The support person indicated that that directory is a temporary directory, used for moving stuff to and from iCloud, but I think they were wrong. When I use the command-line open command to open the directory, the Finder thinks that that directory is iCloud Drive/Books. But if I open the iCloud Drive folder in Finder, there’s no Books folder there. The support person said that Apple had probably hidden the Books folder in iCloud Drive to prevent piracy, but I’m dubious about that too.

Anyway. The support person was definitely right about one thing: part of the problem was that my iCloud drive was running out of space while trying to copy all these ebooks to it. I paid the 99¢ to upgrade to 50GB of iCloud storage, and the ebooks commenced further uploading. After a couple of hours of uploading, the process concluded, with 644 ebooks in iCloud, and 220 outside of it.

Clearly, the next step is to get the books out of iCloud and back into local storage. The support person was convinced that that would be feasible after we unstuck the upload process by adding more storage. But we haven’t yet figured out how to move the books back. It seems like turning off iCloud for Books should work, but (a) there are four different places to do that, with different effects (one on the Mac, in System Preferences > iCloud > iCloud Drive > Options > Books.app, and three on the iPhone, one in Preferences under iCloud and two in preferences under Books); and (b) so far, turning it off doesn’t seem to tell the system to put the moved books back into local storage. But I haven’t yet tried turning off all four of the options; I should probably try that next.

The support person indirectly suggested that I could just restore from backup. If I had a working backup from the past couple weeks, that would be the clear best answer. (My backup systems have all failed recently, in various ways.) As it is, I probably have a backup of the com.apple.BKAgentService directory (the local versions of all the books) from sometime in the past month, but I’m hesitant to try to restore from that. But that may end up being by far the simplest/easiest/briefest approach.

At any rate, even if we do manage to resolve issue 1, issue 2 (the blank pages) may still be a problem.

Note to self: Next time you think about getting a new computer, remind yourself that you may lose literally weeks’ worth of time and energy getting everything working again. So if part of the point is to get a fresh start because things are broken on your old computer, you may want to consider not taking that path.