Bug or Feature?

This entry doesn’t have anything to do with books or writing. I just finished a draft, and I’m so burned out I can’t see straight, so I’m going to write about something that is not writing, and is it just me or is there some kind of recursive weirdness there?

Anyway. If you’re not curious about computers, file systems, music, or the single-minded persistence of software people, feel free to skip this one.

So about a month ago, I started noticing weird things about iTunes.

(I should clarify, of course, that these weird things were not typical iTunes weirdness. It’s trendy to hate on iTunes, and while I try not to be a reflexive trend-follower, it’s somewhat justified. It’s often frustrating and inconsistent, and seems optimized for actions that nobody I know actually wants to perform.

And at this point, I sort of love it, the way you love the ugly feral cat that slinks up to your back door and disdainfully accepts a dish of cheap wet cat food in exchange for keeping the rats out of your garage. We need each other, I think, every time I open iTunes. I will keep using you despite your hideous contextual menus, and you don’t have to tell anyone I own two versions of the soundtrack to Cats.)

The advent of digital downloads completely changed how I listened to music. Years ago, I would listen to entire albums at home. For commuting, I would make mix tapes, constrained by the 45-minutes-a-side restriction of cassettes. It was a source of some pleasure to me, going through my collection and choosing the songs I most loved at the time, thinking about pace and flow, making the hard choices about what wouldn’t fit. (Yes, there were 120-minute and longer tapes; but a) the tape tended to be thinner, and thus more fragile; and b) cognitively, 60 minute chunks seemed too big, probably because so many albums I’d grown up with clocked in at 35-45 minutes.) But with digital downloads – and then the iPod on top of it – time restrictions, as well as the need to make choices, became artificial, and I stopped making mix tapes. And yeah, I’m kind of sorry about that.

But now, when I’m home, I tend to either toss on a trance compilation, or just shuffle my entire library. (This isn’t always wise. There is some really questionable stuff in my library.) Sometimes I’ll put together a quick playlist (like “Stuff I Bought In The Last Six Weeks And Totally Forgot About Because When It’s Digital I Lose Track Instantly”), and shuffle that.

And, iTunes being iTunes, that’s where the problem started to show up.

Some of you iTunes users may be familiar with this little icon:

For those of you who’ve not experienced this particular bit of iTunesness, this icon means that the underlying file associated with the song you’re trying to play is missing. I started noticing this showing up as I was building playlists.

Geek note #1: This suggests that iTunes does a cursory existence check when building playlists that it doesn’t do – at least not right away – when you open up the app.

“But Liz!” I hear you cry. “Weren’t you alarmed? I mean, wasn’t Apple just in the news over missing music?”

Alarmed? No. Not because I’m particularly trusting of iTunes, but because in the past four years I’ve had three disk failures that have destroyed all local copies of my music library, and I’ve been able to recover nearly all of it. (The only things I’ve lost are one episode of my long-defunct podcast, and a music video that isn’t available anymore.) (“Down In It” by Nine Inch Nails, in case you’re curious.) And almost everything in my library is either ripped from CDs sitting in my attic, purchased from iTunes, or purchased from Amazon and matched.

This is probably also the place to mention that I keep my library on an external drive. After the failure of a number of inexpensive external hard drives, we invested in a RAID accessed via our in-house wireless network. That’s where I parked my library.

Geek note #2: Yes, fellow geeks. The NAS is relevant.

Now, most days when iTunes is being weird, I just ignore it. After all, when I’m shuffling thousands of songs, iTunes getting persnickety about a handful of them isn’t a big deal. But some days – usually when I have a deadline, or am in the middle of a chapter that is being a huge pain in the ass – I go back to my geek roots and decide I must figure out what’s going on.

Fixing the problem was easy: remove the download (via the right-click context menu), and redownload it from the cloud. But the same songs kept turning up missing, so I went to the next step: what do they have in common?

As it turns out, they were all albums purchased from Amazon and matched/uploaded.

Geek note #3: Yes, this is also relevant, but not for the obvious reasons.

I took a look at the files in question. They seemed entirely fine, although the file names were a little cluttered (track/disk numbers prefacing song names, which, yeah, is probably configurable, but I’m not going to sort through 18,000 songs and tweak the offending ones if I don’t have to). As far as I could tell, the metadata wasn’t corrupted.

But the same files kept disappearing.

I deleted them and redownloaded them from Amazon, reimported them, let them get rematched. In a day or two, the same problem would reappear.

Here it’s relevant to repeat the number I threw out above: 18,000 songs. This means that the problem didn’t necessarily take two days to reappear. It’s very possible the problem reappeared almost immediately, and I just didn’t notice because I hadn’t made my way back to those songs yet.

And I got evidence of this likelihood when I had a song disappear right in the middle of iTunes playing it.

So here, because I’ve been nattering for a thousand words already, I’ll drop the clue that led to the answer:

Every problematic Amazon-purchased song was from a compilation.

There are two bits of metadata you can attach to compilations in iTunes: you can tag it a compilation (which will cause iTunes to file it under Compilations in the file system), or you can add an Album Artist. The Amazon albums had an Album Artist, but were not tagged as compilations. The Album Artist in this case was either “Various Artists” or “Various artists.”

Geek note #4: You’ve all figured it out already, haven’t you? I hadn’t. My only excuse is that I’ve been out of the business for nearly a year.

At this point, what had become clear was this: iTunes was chatting with the underlying file system, and getting conflicting responses. And something about those colliding Album Artists was likely the reason.

ALERT: GEEK SPEAK.

Mac OS X is a variant of UNIX. Unlike early versions of Windows (and, for all I know, current versions of Windows), UNIX has always (at least as long as I’ve played with it) had a concept of case sensitivity; that is, whether or not the directories SYSTEM and System are treated as identical or not. Related is the concept of case preservation: whether the creation of the directory System is permitted if the directory SYSTEM already exists. They are similar concepts, but not quite the same. Most OSes are either case-sensitive and case-preserving, or case-insensitive and non-case-preserving.

Mac OS X, for what I’m sure seemed like super-good reasons, is case-insensitive and case-preserving.

What this means is that you can create the directory SyStEm, and when you list your files, it’s listed properly as SyStEm. But if you want to cd to it, you can enter cd SYSTEM, and you get stuffed into the right place.

What this should mean is that if you have a directory called Various Artists, you shouldn’t be able to create another directory called Various artists (which you can do with a case-sensitive and case-preserving OS).

Which means OMG LIZ YOUR MAC IS HAUNTED well, no.

Remember that RAID? The massive external drive we got because so many of our cheap drives keeled over with no notice?

That RAID is running a case-sensitive and case-preserving version of UNIX. And was perfectly happy to create both Various Artists and Various artists when iTunes asked it to.

Oops.

Geek note #5: FFS, Liz, you are slow on the uptake.

I had to use the RAID’s web interface to move the files out of Various artists and into Various Artists, which seemed to make iTunes happy. And since then? No troubles. (And some of the weird BBOD issues I was hitting now and then have also disappeared.)

IN CONCLUSION (no, really):

What’s fascinating to me about this is not the bug itself (which, okay, isn’t really a bug; but: really, Mac OS X? Really??), but what it reveals about iTunes. Given conflicting data from the underlying file system, it kept trying. Sometimes it would even succeed; as often as not, if I tried to play a file marked missing, the icon would disappear and it would play just fine. Despite the OS sending a now-you-see-it, now-you-don’t message, iTunes gamely gave it another shot. So despite its patchwork nature, it really does try to be flexible, and give your data every chance to come through.

I suppose I should say the same for the underlying OS, which really did seem to do its best to deal with a concept it was, by design, not supposed to understand. I would have expected it to simply never see one version of the directory – but it did, sometimes. Why, I don’t know. (Often it would work right after a reboot. Hm…)

Software is difficult, and it’s not just writing the stuff. It’s the schedules and the staffing and the executives that keep asking why does it take so long? and then get confused when customers get cranky about quality. I’m well aware that I’ve got an odd configuration: an iTunes library on a UNIX NAS with different casing settings than Mac OS X itself. I sincerely doubt it was on anybody’s test matrix, and if it was, it probably made perfect business sense to drop it in favor of a tighter schedule. I’m not even going to report it as a bug, because arguably it’s not: I should have realized what could happen and set the configuration of the RAID accordingly.

But it does remind me of one of the things I loved about working in software: hitting a weird, inconsistent problem, getting to the bottom of it, and figuring out how to deal with it. There’s real satisfaction in getting to the why of something that’s just plain strange.

So I guess my real conclusion is:

TL;DR: iTunes got weird, and I figured it out, and it was fun.