The announcement of Apple’s new iPad brought along a number of updates to many existing Apple iOS apps, among them iBooks. One particular new feature in iBooks immediately caught my attention:
Use your finger as a highlighter when swiping over text
I was particularly curious about this new feature because i’ve always felt that the poor text selection mechanism on iOS is a major hurdle that prevents me from getting any meaningful, serious work done on these devices. Maybe i’m an oddball (though i doubt this, having observed how some of my colleagues work), but text selection and extensive copy-pasting between apps is a common part of many of my workflows, whether it’s preparing a powerpoint slide deck, programming, doing administrative stuff, writing reports and papers or even something as mundane as writing an email. The promise of a new text highlighting feature in iBooks got me excited that Apple was exploring new means of text selection on their touchscreen devices. (Let me note that i’m aware that text highlighting is something different from text selection on a functional level, but they are pretty similar as far as interaction style is concerned.)
So i downloaded the new iBooks version and set out to give this new highlighting feature a try, immediately running into a problem: i couldn’t figure out how the text highlighting was supposed to work from Apple’s one-line description in the patch notes. I swiped across sentences and paragraphs, left to right, only to find myself leafing through pages. I tried swiping top to bottom, thinking that maybe Apple was trying to avoid gesture overload by distinguishing swipe directions, but with no result. After a minute or two i gave up and turned to Google in hope of finding a description of how this new text highlighting feature was supposed to work. So much for the intuitiveness of gestural interfaces…
When i couldn’t find any helpful tips or instructions i returned to iBooks, determined to figure it out myself. After a few more minutes of experimentation i finally discovered how text highlighting in iBooks works: Before you can highlight text with a finger swipe, you need to initiate a very short long press on the text. If you immediately begin with your swipe-gesture nothing happens, but if you first rest your finger on the screen for a very short while (i haven’t been able to time this, but it’s probably much less than half-a-second) and then swipe over the screen, the highlighting feature works.
The text highlighting works pretty great once you’ve figured it out, but it introduces another interesting new problem (and inconsistency between iBooks and pretty much every other app on iOS): Normally on iOS, a long press on a body of text (where you touch the screen with your finger and leave it there motionless for a short period of time) opens a magnifying glass to start a text selection operation. During the long press, while leaving your finger on the screen, you can drag your finger around to move the magnifying glass to change your text selection, a very useful feature in case you positioned your finger ever-so-slightly incorrectly, which, let’s be honest, can easily happen given the size of a typical finger compared to the size of a single word on a tiny iPhone screen.
The magnifying glass is gone in iBooks 2.1
Of course now that iBooks highjacks the long press to initiate text highlighting, this couldn’t possibly work anymore. Instead, text selection in iBooks 2.1 works like this: You start a long press on the screen and iBooks waits for you to move your finger (in case you want to highlight text) all the while providing no visual feedback whatsoever. If you don’t move your finger but lift it off the screen in place, the text selection operation is triggered, with the word that your finger rested on selected. Gone is the magnifying glass for text selection. There are three major drawbacks with this method compared to how text selection used to work in iBooks and continues to work in other iOS apps that immediately come to mind:
- Because the magnifying glass is gone, you don’t get any immediate feedback while touching the screen that you are about to perform a text selection operation triggered by the long press. You also don’t get any feedback for your text highlighting operation until you move your finger. I guess that’s why it took me so long to figure text highlighting out…
- You can no longer change your initial text selection in case the position of your finger was a little off. Instead you have to manually adjust the selection using the two small handle-bars to the left and right of the selection marker (a particularly onerous and annoying operation in my opinion), or you keep long press-stabbing at the screen until you hit the right word.
- Because people have a reasonable expectation of how text selection is supposed to work in iOS from prior experience with pretty much any other app on iOS, it becomes all the more difficult to figure out why nothing is happening when you long press on the text body in iBooks.
I think it would have been preferable to keep this text highlighting feature as part of the action menu that pops up after a long press (of which it’s a part of anyway) instead of introducing this confusing new shortcut.
Admittedly, text selection is a pretty arcane affair on iOS anyway, marred by inconsistencies across different apps and obscure shortcuts. Did you know that you can select a word in Notes by double tapping it? Of course that won’t work in Safari, because a double tap will trigger the context-aware zoom…
More importantly though, this tiny issue got me thinking that we’re now at a point where we’re running into the expressive boundaries of multitouch gestural interaction, inevitably leading to small inconsistencies like this one while we’re trying to get the most out of a limited gesture vocabulary. Gestural interaction increasingly epitomizes a style of interaction devoid of consistency, affordances and intuitiveness. Look no further than Lukas Mathis excellent critique of iPhoto on iOS to see that this is no isolated case.
When i think back how bright the future of gestural interaction looked back in 2007 when the first iPhone launched, i can’t help but wonder at which point we lost the magic. Maybe we’re just trying too hard, trying to do too much with a too limited vocabulary. And maybe buttons and menus weren’t such a bad idea in the first place.
See also: Two Kinds of Gestures by Phillip Bowden.