Batch converting HTML files to PDF using OS X and 'convert' utility

At work we are converting content to a new website. As a part of that conversion, some older content will be archived on the new site in the form of PDF documents.

I needed something to convert HTML documents to PDF. We own Adobe's software and there is an option to convert entire web URLs to a single PDF, but that's not what we needed.
I could convert single URLs using the same Adobe software, but that wasn't an optimum solution either. My boss, through a Google search had found a utility on the Mac called 'convert' which does this.

I automated this through a Bash script call, but we still had problems because the pages were truncated. I went to the directory where the "convert" application was, and found it links to 'cupsfilter' in /usr/bin.

By figuring out what cupsfilter does, I was able to determine the parameters necessary to make the PDFs landscape, and use the page size of A4, which was enough to have it work properly.

The great thing about this is if we would have used the "Save As" feature to save each page to a PDF it would have taken hundreds of hours. Using a Bash shell it took an hour to convert three directories of HTML files with about 150+ files per directory. Even though I used 'convert' I suspect you could do the same thing by using the cupsfilter directly on any UNIX variant.

The key parameter was "landscape" but when using "convert" it wasn't obvious how to specify the parameters correctly. Through cupsfilter man pages I found out what I needed; in cupsfilter it's with a "-o" option, but in "convert" it's using -a. For media format options were "Letter" "Legal" and "A4" but A4 worked best. Letter was a little too small and ended up truncating some of our documents.

Here's my Bash Shell Command that walked through the current
directory finding HTML files with the extension HTM, and for
the output file name used SED (Stream EDitor) to convert the
HTM in the filename to the output file type of PDF.

for name in `ls *.htm` ; do /System/Library/Printers/Libraries/convert -f $name -o `echo $name | sed s/htm/pdf/` -a landscape -a scaling=75 -a media=A4; done

I have to point out that if I was still using a PC I could have probably done this with CygWin but the articles we found on Google indicate the people that did this used convert and I don't know I would have figured out to use cupsfilter instead which is what Mac OS X linked to.

Firefox like Shortcuts in Safari

I've been a Firefox user for years. It was leaps above the other solutions available when I first discovered it, but as of late, the general slowness and high crash frequency have had me wondering once again what else is out there.

I like Safari for the most part, but really hate the default shortcut combinations--opt+cmd-f does not speak to me. Thankfully, I finally stumbled across a wonderful post that details how you can setup Firefox like shortcuts in Safari. It does require a small external app, but I haven't noticed it yet, and the speed of the shortcuts feels as if they're native Safari commands.

Take a look at Roberto's post here if you're interested. Enjoy!

Encrypted mail storage without FileVault

If you want to or, like in my case, have to make sure that email is stored encrypted on your laptop the easiest option is to turn on FileVault. This has two disadvantages, though: you lose much of Time Machine's transparency and access to all files in your home directory is slowed down; nothing you'd note when just looking at documents but a slight annoyance when building code. So, what's the alternative?

Apple's Mail.app stores its local data in ~/Library/Mail so all we really need to achieve is encryption of this folder. There are a three simple steps to follow that do exactly that. If you are unfamiliar with the command line this might not be for you, though.

First, you need to create an encrypted disk image. The easiest way to do this is with the DiskUtility application. Make sure that you choose an encrypted read/write disk image with enough room for all your mail. I chose 4 gig as this has a fair bit of room left for me but will allow me to make easy backups onto DVDs. Note that it's not advisable to create a sparse image because these have a tendency to become corrupted on hard resets. Regarding the name for the image, if in doubt use "Mail" as the name. The following screen shot summarises my settings:


Next, mount the image by double clicking it and use Finder to copy the contents of your ~/Library/Mail folder to the disk image. You might want to quit Mail and iCal before doing this.

Then drop into a command line and replace the Mail folder with the disk image: (The trailing dot on the third command is important.)

cd ~/Library
rm -Rf Mail
ln -s /Volumes/Mail .

And that's it! Now all your local email is stored in an encrypted disk image.

When you restart your computer or log out and back in, the Mail volume gets unmounted. If you forget to remount it no harm is done, Mail.app will just crash. So, just mount the image and restart Mail.

There are different opinions on whether it is safe to store the password for the encrypted image in the keychain. If in doubt, don't. This means you will get prompted for the password whenever you mount the image but if you mostly keep logged in and don't restart your computer often this isn't much of a problem.

Update: If you follow the steps above full-text search will not work because that uses Spotlight, which isn't enabled on disk images by default. The following command fixes this problem:

sudo mdutil -i on /Volumes/Mail

It is probably safer to quite Mail before running this command, and it can take a few minutes before the initial index is built; the command returns immediately but you should see activity for a while by a background process.

Building a Hackintosh

Just a quick note to say that I've successfully built a Hackintosh. The full report can be found here.

QuickLook for Markdown Documents

I'm a huge fan of the Markdown simplified markup language and try to use it for most of my prose. There are lots of tools out in the world to transform Markdown into other formats: Markdown itself does HTML, but you can also get a tool like Pandoc with a huge number of output formats for Markdown.

However, one piece I was missing is the ability to "Quick Look" Markdown documents within Finder. Be default, it shows the generic "I'm a file" icon when you do Quick Look, and I would like either just the text or the rendered results.

Found it! QLMarkdown is a utility that you add to your ~/Library/QuickLook folder and gives you the rendered rich output from your markdown source document (and it works with either the ".md" or ".markdown" extensions). One more piece of the puzzle solved.

Tidying up Finder's "Open With" menu

Many applications can open common file types such as HTML or JPG, and they all, conveniently, get an entry in Finder's "Open With" contextual menu. Under many circumstances this is even true for applications that are installed in VMware Fusion or Parallel Desktop. Unfortunately, when you delete these applications, or you decide that you don't want applications in virtual machines to open files, the context menu items remain. In my case some context menus became cluttered to the point of being unusable.

This is such an obvious problem that I expected to find an answer using Google straight away. Maybe my search skills failed me but it took me a good while to find the right answer for Leopard, which is why I want to share it here.

There is this article on Lifehacker. However, the article itself has it wrong; the real solution is in a response by "mtts" halfway down the page, and even then the command doesn't work as described on Leopard. What did the trick for me, on Leopard, was the following command:

/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/Support/lsregister -kill -r -domain local -domain system -domain user

There is no manpage for lsregister but invoking it without any arguments will display some help.

Mac Boot Mysteries

This is long, digressive story about diagnosing a hardware problem on a Mac; if you dislike such stories, feel free to leave now.

About a week ago, my wife Candy complains to me that her Mac won't boot up. This is my hand-me-down Mac (we have a new policy in our house: Candy gets my hand-me-down computers, and I get her hand-me-down cameras), which means that it's about 2 years old, but it has a relatively new hard drive that I installed last November. A long time ago, I had set the startup option to always run in Verbose startup mode (on demand by holding COMMAND-V upon startup, or permanent by issuing the following command:

sudo nvram boot-args="-v"

Anyway, I could see from the startup porn that she was having a kernel panic with 2 likely suspects: the fan control daemon and something about Cisco VPN. Now, Candy doesn't have a Cisco VPN, but given that this was my hand-me-down machine, that explains why some of that stuff is there. Candy hadn't installed anything in the last week or so, leading me to think that one of these two was the culprit. She had been complaining that her machine was getting slower and slower, including things like window resizing, which had me puzzled. Perhaps a dying fan was causing the processor to overheat and thus slow down?

I tried safe boot (no joy), and at this point I suspect the fan. I'm certainly not afraid to crack open a Mac (with proper respect), but replacing a fan isn't high on my list of fun things to do, so we made an appointment at the Genius bar. To Candy's credit, she had a SuperDuper! backup that was just a couple of days old, so virtually everything was safe.

We went to the genius bar where the GenX slacker (this is a compliment) booted the Mac from an external drive. I hadn't tried this (even though I have several bootable drives laying around) because I was fixated on the fan problem. After booting it up, his suspicion now lies with the VPN stuff, and I reluctantly concur (especially after he ran some fan diagnostics). Now, though, the question remains: why did this problem suddenly occur? What was his (depressing) advice to fix this problem? Reinstall Leopard and all your applications. What?!? Is this a freakin' Windows machine? I couldn't believe that was real Genius advice. I've never yet had to do a ground-up reinstall of everything, but if that's the only way...hmmmm. He was very knowledgeable, and obviously he doesn't tread in the realm of VPN stuff. He also correctly pointed out that a bad fan shouldn't cause slowness: redrawing windows is mostly handled by the GPU on the Mac. The slowness was as far as I can tell a red herring.

When I got home, the first thing I did was boot Mac OS X from an external drive and get a real SuperDuper! snapshot, getting the real current snapshot. Once I have that, I can play. Candy has already agreed to the pain and degradation of reinstalling everything, but I have to think there's a better way. Then, I had a brain storm: I took the SuperDuper! snapshot I just made and booted the machine from the external drive. Success! That suggests that some part of the internal hard drive that houses the VPN stuff has somehow gotten corrupted, but still allows it to boot using the same image from an external drive. Because I have the SuperDuper! safety net, I decided on an experiment. I reformatted the internal hard drive and ran Drive Genius on it to scan for bad sectors. Nothing of note came from that, but then I overlaid my most recent SuperDuper! snapshot back onto the internal drive.

Success! The internal drive now boots, and everything appears back to normal. I'm guessing that my bad sector theory was correct.

Lessons:

• Don't reinstall everything! My record is still clean on that account: I have never had to do that on a Mac yet (and it was a once-a-year chore on Windows because of bit rot).


• Always have good backups. This would have been a tragedy rather than a comedy if Candy hadn't been using SuperDuper!. It has yet to let me down, and it has saved my bacon on several occasions.


• I immediately latched onto the fan because it seemed to support other observed phenomena. I should have booted it myself from an external drive and run Drive Genius, but I thought I had it figured out.


• Stop and think. It was a good thing that we had dinner plans with a neighbor when we got back from the genius bar. It was over dinner that I had the idea of just overlaying the snapshot again. If I had started on it as soon as we got back, I would have been creating a lot of movement without a lot of forward progress. Sitting and thinking about it opened my mind to alternative options.


• SuperDuper! rocks. I can't imagine life without it.

SmartSleep Preference Pane

A while back, Erik blogged about the Unsafe Sleep setting for Macs, allowing for ultra-quick sleep and wake up. The only two problems: 1) it takes a little terminal magic to set new defaults for your machine and 2) you can't change the battery when the machine is only asleep (thus, "unsafe" sleep).

Now that's changed with the SmartSleep preference pane. It allows you to set all the possible settings plus a new one. From the preference pane itself:

• smart sleep: sleep if battery charge is above the sleep & hibernate level. Hibernates only when battery charge is below 5% or less than 5 minutes.
• sleep: machine will go to sleep only (saves state in RAM only, battery keeps RAM contents)
• sleep & hibernate: machine sleeps and hibernates. (default)
• hibernate only machine will go to hibernate only. (saves state on disk, battery will not be used)

The first one is new: it dynamically allows the machine to do "unsafe sleep" until the battery gets low, then switch to "sleep + hibernate", allowing changing the battery while the machine is asleep. Very nice: best of both worlds.