10 October 2017

Adventures in Kernel Panic Land

Jimmy McGill chose "Saul Goodman" as his pseudo for a new venture to sell his now useless TV commercial slots, because "it's all good, man!"
That's exactly what your Mac tells you every day. Everything works fine, and everything will work out fine tomorrow, until it doesn't, but then, it's a Mac, so you're all good.

You never know that you need a backup until you do. That's the rule. I learned it about 10 years ago, the hard way. Massive hard disk failure, no backup at all, no extra software to check/fix/save the disk. Nothing, but a big deadline 2 days away. At the time I was running an Itanium Powerbook. I had bought it second hand a while before and it was a real workhorse. I also had a greenish clamshell iBook that my wife used to work with, before her company lent her a Windows machine.

When the machine's hard disk died on me, I called the project manager. He roughly told me that I still had 2 days and that the schedule would not move an inch. I don't remember what I did to fix the mess, but I did manage to scrap most of the data from the disk, then I must have moved all that to the iBook, resumed work, delivered, celebrated, and decided to create regular backups.

In the 21st century, external drives are cheap. They are cheaper than whatever hourly rate you apply to the time spent on fixing a non-backed-up dead system. They come in all sorts of flavour, shape, colour and capacity. And if your main machine is a laptop and it's a pain to physically connect it to your backup disk every time you put it back on your desk, you even have Wifi backups.

We tried half-backed solutions for a while. I bought terabytes of external drives, ugly as crap, all Firewiry and USB-Cy, but when you want a transparent solution that just works, all this just doesn't scale. Full disclosure: I have no financial interests in Apple. But damn, Time Capsule rocks. So yes, we got ourselves a Time Capsule, just 2 TB. We had 2 iMacs and 2 notebooks at the time, we thought we'd be short with the capacity but the 3TB Capsule seemed too expensive for just one extra terabyte. But even with the "low" capacity, it's been with us since. It does everything over Wifi, no need to connect the machines, just turn them on and Time Machine does its thing, and in no time you have your incremental backup ready for when you need it. You do need to turn the Time Machine on and select Time Capsule as your backup target, but it's a one time setting, and then you can forget about it, pretty much.

When things work fine, the only time you need a thorough backup is when you upgrade your system. I don't know how many more OS upgrades this "early 2011 Macbook Pro" will be allowed (our 24" iMac could not go further than Yosemite) but I was not going to miss that High Sierra upgrade after seeing the APFS talks on internet. The rest of the icing, I was not very interested in, although I love the way Safari does a lot to keep advertisers from wasting our CPU (and brain) cycles.

What had been bothering me for a while was that the machine was getting slower and slower everywhere and after having an issue with how slow BBEdit was, the Barebones support team confirmed that it was most probably a hard disk issue. With APFS coming I was not going to replace my disk with a rotating thing again. I was done with that. I was going to go the SSD way. All flashy, all speedy, all slick electronics.

My son had done it a few months back. We bought him an MBP when he entered university, one of those new but still coming with the antiquated DVD player. In his department, he was the only freshman to use a Mac. When he came back for Christmas last year, I paid for the replacement Samsung SSD and he spent a few hours doing the thing: backup your data and follow the instructions on Youtube. Easy enough. Then he reboot the machine and everything was faster. And it was before High Sierra and APFS.

I was going to go incremental on this. First install High Sierra, then get an SSD, and last reinstall everything. But before all, the backup. Time Machine being automatic, you really don't have to do much usually. But when you want to do it "now", well, there is a menu item for that. It's called "Back Up Now". To make sure you don't forget to save a few last-minute settings, just quit all the applications you run, leaving only Finder on (although if you know how to quit it, you can too), then let Time machine prepare the backup, do the backup and clean up the backup.

Then, you can start the upgrade process itself. Go to apple.com, click on "Upgrade" and everything goes super smoothly. At one point you are presented with a dialog that asks whether you really agree with all the terms, you just click "Agree" unless you're a hard-core free software advocate, but in this case you're not running a Mac and you're not reading this page, and then files get downloaded and installed and you can go watch your favourite show in the meanwhile, or read a book.

That's what I had planned anyway. Either finish season 3 of "Better call Saul" or read "Consider Phlebas", but it did not happen that way. What happened is that after a long while and not much happening, the Mac complained that it could not be upgraded. Just like that, two thirds into the install process, when the machine had overwritten all those important parts of the disk and you know that if you don't find a solution, you're done.

Rebooting won't do it. The old OS has been blown to pieces by the newer files partially written over it and the machine is programmatically speaking a giant spaghetti mess that the CPU has no idea how to deal with. When that happens, you have a Kernel Panic, and when the kernel panics it is of the utmost importance that you don't panic. Whatever you imagine has happened. All the pics of the newborn, all the thousands of whatever currency you spent on music and movies. Everything is gone. Except that, it's on your backup, and the likeliness that your computer and your backup die on you at the exact same time is so low that it's even lower than the likeliness of you being under the atomic fire of a North Korean missile. Although, granted, the day this happens, you'll have things more important to worry about than a broken backup.

So, you see the Kernel Panic thing that actually looks like a personal reboot of The Matrix on your screen and you feel all warm inside because you know it's a Mac. The idea here is that you need a handy way to get information from the Net. I don't have a phone, but I have that extra 24" iMac that's now mostly used as a Minecraft machine by the 2 little ones, but it also runs Safari (mostly used by the same little ones to check new "mods" for said Minecraft) and so I have a powerful entry point into the internets of information.

What you need to know here (and Safari is your friend if you don't) is that your Mac comes with secret programs that you can launch when you need them because they are located somewhere that doesn't get to be erased by a gigantic mess up of overwritten OS files. One such program is "Recovery Mode". You can launch it by hitting Command+R right after you restart your machine. If your system is down, like mine was, and you don't have a boot disk or USB key, then you want to enter Internet Recovery Mode, which requires an internet connection to download to your Mac all the things necessary to fix your issues (you may want to physically connect to your router/Time Capsule with anything that is faster than Wifi).

"Recovery Mode" be it local or internet based can do four things. The first one is get a Time Machine backup and restore the full machine from there. The second one is reinstall the OS from scratch. The third one is launch Safari to give you access to Apple Support for help. And the fourth one is give you access to Disk Utility so that you can diagnose/fix disk problems.

I figured that since I did not have a separate boot system my best bet was a full recovery from Time Machine. But I tried Disk Utility before that, checked the disk and since it told me everything was normal I just went on and decided to test an OS install. Depending on the key you call Recovery with, the version of the OS that's proposed to you is different. Either you have the OS your machine came installed with when you bought it, or you get the latest version that was installed, or you get the latest compatible version.

I tried High Sierra. And it did not work. I got a number of "cannot access the recovery server" or similar messages and gave up. I rebooted, again in Recovery Mode, tried again with the originally bundled version (Lion ?) and install started. And never stopped. It was late in the night when a dialog announced that the install would really not work.

This kind of moments are when you desperately need to cling to your faith in Macs (slightly backed by actually very few horror stories of messed up machines that refused to recover), and try one last thing before you go to bed. Reboot/Recovery Mode/Reinstall, it was staring to become a routine. But for some reason, this time it worked, and in the morning I had a (not so High) Sierra machine up and running, or rather ready to accept my backup data. A few hours and 250 GB later, I was running my machine, with a fresh Sierra install and all my data in place. I was basically at the exact same point I was the day before, but stronger in my faith that Macs just work and when they don't they still do.

The SSD arrived a few days later. That kind of manipulation involves a slightly higher level of wizardry. You need a few tools and an iFix video. Tools mean screws, and you really don't want to lose the screws. So here we go. Old hard disk removed, screws put in separate boxes to not mix them all, new SSD thing installed, screws back in place, power supply back on, reboot and Internet Recovery Mode launch.

Normally, an all brand new disk is not formatted to accept data copying. It is just a blank slate that the OS must put in shape for it to be of any use. But the OS still recognizes that there is a disk willing to be formatted attached to the machine. Normally.

In my case, the SSD thing was nowhere to be seen. The law of probability was plain about that: the thing itself is not at issue. Maybe an internal connection ? Would I have to open the Mac again and check everything ? I'm lazy and I prefer the Aspirin of Computer Fixing: reboot. A reboot/recovery mode later, I had the thing under my eyes. Go figure. Disk Utility was offering me to format the disk to APFS, which would also erase the data I had spent one full day reinstalling. Go for it, I thought, I'd just spend one more day going through the recovery hoops to get my machine running with the new OS.

Here again, things did not happen as planned. It took a small number of attempts to get High Sierra to be installed on the disk, and then I had to reinstall my data again from the Time Capsule backup.

I was lucky. We had a 3 day weekend here and I had decided to remove the curtains in both our room and the kids' and replace everything with wood or cloth window shades. Plus I had a lot of TV shows to watch, and all the books I brought back from France last summer just waiting to be read. Every once in a while I would check the Mac's screen, just to make sure, and this morning, eventually, I was able to contemplate the new setup, running smoothly, with a few new visible features, but mostly the new boot and launch speed (and damn, that SSD boot is faster than anything I've seen in a long while).

I know that putting a new third party disk into a box that's starting to show its age, adding the new version of the OS when it's just fresh out of master copy and using a new file system that was still in beta not so long ago is the exact opposite of hedging one's bets. And after reading a few cautionary articles after the fact I'm realizing that I may have been a little bold, but whatever happens, I have a backup that's regularly and automatically updated and I can just start from scratch any time I need... So I guess, it's all good, man!