Mac Boot Mysteries

This is long, digressive story about diagnosing a hardware problem on a Mac; if you dislike such stories, feel free to leave now.

About a week ago, my wife Candy complains to me that her Mac won't boot up. This is my hand-me-down Mac (we have a new policy in our house: Candy gets my hand-me-down computers, and I get her hand-me-down cameras), which means that it's about 2 years old, but it has a relatively new hard drive that I installed last November. A long time ago, I had set the startup option to always run in Verbose startup mode (on demand by holding COMMAND-V upon startup, or permanent by issuing the following command:

sudo nvram boot-args="-v"

Anyway, I could see from the startup porn that she was having a kernel panic with 2 likely suspects: the fan control daemon and something about Cisco VPN. Now, Candy doesn't have a Cisco VPN, but given that this was my hand-me-down machine, that explains why some of that stuff is there. Candy hadn't installed anything in the last week or so, leading me to think that one of these two was the culprit. She had been complaining that her machine was getting slower and slower, including things like window resizing, which had me puzzled. Perhaps a dying fan was causing the processor to overheat and thus slow down?

I tried safe boot (no joy), and at this point I suspect the fan. I'm certainly not afraid to crack open a Mac (with proper respect), but replacing a fan isn't high on my list of fun things to do, so we made an appointment at the Genius bar. To Candy's credit, she had a SuperDuper! backup that was just a couple of days old, so virtually everything was safe.

We went to the genius bar where the GenX slacker (this is a compliment) booted the Mac from an external drive. I hadn't tried this (even though I have several bootable drives laying around) because I was fixated on the fan problem. After booting it up, his suspicion now lies with the VPN stuff, and I reluctantly concur (especially after he ran some fan diagnostics). Now, though, the question remains: why did this problem suddenly occur? What was his (depressing) advice to fix this problem? Reinstall Leopard and all your applications. What?!? Is this a freakin' Windows machine? I couldn't believe that was real Genius advice. I've never yet had to do a ground-up reinstall of everything, but if that's the only way...hmmmm. He was very knowledgeable, and obviously he doesn't tread in the realm of VPN stuff. He also correctly pointed out that a bad fan shouldn't cause slowness: redrawing windows is mostly handled by the GPU on the Mac. The slowness was as far as I can tell a red herring.

When I got home, the first thing I did was boot Mac OS X from an external drive and get a real SuperDuper! snapshot, getting the real current snapshot. Once I have that, I can play. Candy has already agreed to the pain and degradation of reinstalling everything, but I have to think there's a better way. Then, I had a brain storm: I took the SuperDuper! snapshot I just made and booted the machine from the external drive. Success! That suggests that some part of the internal hard drive that houses the VPN stuff has somehow gotten corrupted, but still allows it to boot using the same image from an external drive. Because I have the SuperDuper! safety net, I decided on an experiment. I reformatted the internal hard drive and ran Drive Genius on it to scan for bad sectors. Nothing of note came from that, but then I overlaid my most recent SuperDuper! snapshot back onto the internal drive.

Success! The internal drive now boots, and everything appears back to normal. I'm guessing that my bad sector theory was correct.

Lessons:

• Don't reinstall everything! My record is still clean on that account: I have never had to do that on a Mac yet (and it was a once-a-year chore on Windows because of bit rot).


• Always have good backups. This would have been a tragedy rather than a comedy if Candy hadn't been using SuperDuper!. It has yet to let me down, and it has saved my bacon on several occasions.


• I immediately latched onto the fan because it seemed to support other observed phenomena. I should have booted it myself from an external drive and run Drive Genius, but I thought I had it figured out.


• Stop and think. It was a good thing that we had dinner plans with a neighbor when we got back from the genius bar. It was over dinner that I had the idea of just overlaying the snapshot again. If I had started on it as soon as we got back, I would have been creating a lot of movement without a lot of forward progress. Sitting and thinking about it opened my mind to alternative options.


• SuperDuper! rocks. I can't imagine life without it.

SmartSleep Preference Pane

A while back, Erik blogged about the Unsafe Sleep setting for Macs, allowing for ultra-quick sleep and wake up. The only two problems: 1) it takes a little terminal magic to set new defaults for your machine and 2) you can't change the battery when the machine is only asleep (thus, "unsafe" sleep).

Now that's changed with the SmartSleep preference pane. It allows you to set all the possible settings plus a new one. From the preference pane itself:

• smart sleep: sleep if battery charge is above the sleep & hibernate level. Hibernates only when battery charge is below 5% or less than 5 minutes.
• sleep: machine will go to sleep only (saves state in RAM only, battery keeps RAM contents)
• sleep & hibernate: machine sleeps and hibernates. (default)
• hibernate only machine will go to hibernate only. (saves state on disk, battery will not be used)

The first one is new: it dynamically allows the machine to do "unsafe sleep" until the battery gets low, then switch to "sleep + hibernate", allowing changing the battery while the machine is asleep. Very nice: best of both worlds.

MenuMeters & Text

Lots of OS X geeks I know use Menu Meters to monitor the goings on for their machines. But here's a nuance that remembered last week, watching one of my colleagues give a presentation. I can't remember exactly where I heard (overheard?) this, but apparently it's quite relatively expensive to paint letters and numbers on the task area of the menu bar as opposed to graphics. Not sure why, and it may just be a rumor, but in any case, I tend to keep the graphical version of the CPU meters up, not the numeric one, just in case.

Open in Terminal (4 Ways)

One of the cool little helpers I talk about in The Productive Programmer is "Command Prompt Here" or "Bash Here": little context menu items for Windows that allow you to select a folder in Explorer, right click, and open a command prompt (or bash shell, if you are using Cygwin) at that location. Someone recently asked me if you can do this on the Mac, and it turns out you can, in a surprisingly large number of ways.

The first way leverages Automator to create a little script that makes an Automator plug-in for Finder. This trick appears here. The cool thing about this tip is the stuff it teaches about building Finder extension via Automator, which is a pretty cool subject unto itself. However, the bad thing about the final solution proffered by this tip is the location where the context menu appears: 2 levels down in the right-click menu, under Automator. Too much clicking.

Not being satisfied with the above, I found an alternative, which creates a toolbar icon at the top of Finder that opens the current selected folder (or the folder of the currently selected file) when you click on it. This trick appears here. The only downside of this version is the number of windows it spawns: one per request, instead of opening new tabs (it was created before Terminal supported tabs in Leopard). So, not to be outdone, the author describes the steps to upgrade it to tabbed terminal in Leopard here. One gotcha exists in his instructions: he says to replace the original script with the one found on this page, but I assumed you could do that with a text editor. It turns out that you need to use the script editor that launches when you double-click the main.scpt file. Otherwise, it works like a charm.

The reason I haven't pursued this before on the Mac? I generally use PathFinder, which includes not only a context menu for "Open in Terminal", it also has an embedded Terminal, which opens in the current folder. Highly recommended, but it you don't want to spend the money, you can get the same effect by harvesting the links in this entry.

The New Backup Strategy

Before Leopard, I had a specific backup strategy for both my volatile content (things like source code, documents, etc) and my entire hard drive, which was a snapshot backup. The volatile documents solution was Subversion, kept on a remote, universally accessible machine. The snapshot backup was handled by SuperDuper, which backs up your entire drive in a bootable state, essentially creating a complete snapshot of your hard drive. A couple of problems reared up because of this setup.

1. I had to handle the "package" files (the ones created by Apple iWork, like Keynote and Numbers) files specially, because Subversion didn't like the way the applications managed the contents. Basically, I had to zip/unzip them for version control. This wasn't terrible (I automated the process to a large degree), but still a little annoying.
   
2. My Subversion repository was huge, because I had my entire Documents folder in it. However, most of the documents were there just so that I could have a backup, not because I wanted to version them. The only files I really versioned where the source files and related content.
   

The advent of Leopard and Time Machine changed my strategy. First, I separated out the Documents stuff for which I really only wanted backups and let Time Machine handle them. I put all my versionable files (like source files) in a new, much smaller Subversion repository. And, even though I have Time Machine, I still use SuperDuper to create backups. The reason I use both:

• I want the hourly, behind the scenes backup provided by Time Machine.
  
• I want to be able to browse backwards in time to look at previous versions of those files, and the Time Machine UI is gorgeous for that.
  
• Time Machine alone isn't sufficient. To restore from Time Machine, you have to boot your machine from a startup disk, then restore the backup. Yuck! I still like SuperDuper's snapshot approach, which I've proven to myself works flawlessly (see Don't Crack Open Your Mac for the story).
  
• SuperDuper and Time Machine can share the same drive, so I have a single 500 GB drive that has all my backups on it.
  
• It's now easier to replicate the source code in more places (IOW, more machines) because it's much smaller.
  
• You can tell Time Machine to only backup certain directories (or, more specifically, exclude directories you don't want backed up). Because I only use Time Machine for my Documents folder, it takes only a little space.
  
• Because SuperDuper creates a bootable drive image, and my external drive is FireWire, I can boot another MacBook Pro with the external drive. Yes, it's slow, but if the worst happened while on the road, and I've got to present at a conference, as long as I can borrow/steal another machine, I can boot into my machine from backup and do my presentation.
  

I've been using this approach for a while, and it works nicely. I leave the external hard drive hooked up all the time, and start a snapshot backup at bedtime every night. SuperDuper has a nice option that will sleep the computer when it's finished it's work. So far, this it working out really well. I haven't had to restore the whole drive from SuperDuper yet on this machine (but I know that works -- I've done it on other machines), and I have used Time Machine to grab a file that accidentally got deleted.

Keyboard Zoom

This is a re-blog from Jack Dempsey, setting up a keyboard shortcut to Zoom.

This works well because almost all Mac applications include the Zoom menu item on the Window menu.

Thanks, Jack.

Quick Blank

This was left by Emilio as a comment to the Quick Sleep entry from before, but I'm afraid not enough people read the comments so I'm promoting it to full-blown entry.

The keychord CTRL-SHIFT-EJECT instantly blanks the screen, without invoking the screen saver. It's just instant blank. Note that it will not lock the screen if you have passwords turned on for the screen saver, it just makes the screen blank. Still, pretty cool.