The field of design, like most early- or non-sciences, struggles with abstracting or generalizing its principles for many reasons —some addressable, some not— and as such many of its practitioners have only thousands of “rules of thumb” with scant systematizing or organizing structures behind them.

So there is no fixed answer; but here are some considerations:

Text advantages:

  • Text wraps, scales, and has been “responsive” for many years in most UIs. Since filenames and font-sizes vary, to say nothing of displays and different character sets, OSes, apps, and the web all handle variations in text’s dimensions very well; the conventions are established. Thus: text is more variable than a visual element, in most cases, and can display well in more varied conditions.
  • Text is universally comprehensible, if it’s read; while many users seemingly skip text anywhere they see it —or rather, they don’t see it— it is still less ambiguous than many of iOS 7’s iconographic glyphs. Visual metaphors are much likelier to be misunderstood than language, since they are usually representations of language, words abstracted into constrained illustrations
  • Text is cheap. You can easily iterate on text, push changes to text, A/B test copy or textual calls-to-action, without bugging your designer (should your designer still be sequestered in the world of deliverable PSDs) or your UI engineers. While edits may sometimes have consequences one must address, they’re nearly always lesser than the comparable consequences of changes to a very visual design.
  • Related to its cheapness is its ubiquity: text works on any device, wether interaction is accomplished via touch or mouse-click or speech or shake. As devices and device-types proliferate, text seems likely to be the easiest means to achieving cross-platform coherence. See: One Interface to Rule Them All: iOS 7 & Future Apple Products.
  • Thanks to what experts call “the world wide web,” users are somewhat accustomed to clicking or tapping on text to interact with software.

Text disadvantages:

  • Text is commonly the content. When your Facebook, Twitter, Quora, WhatsApp, Baidu, Mail, or Messages apps have all-text UIs, the UI text competes with the text you’re actually there to read. Why not break up the visual plane with some easily-spotted and used visual elements?
  • No one reads text; no one reads instructions, captions, tooltips, tabs, titles, headers, footers, or paragraphs. Users click around, make their cow-paths, and stick to them; thus any gains in “explicitness” with text amount to very little.
  • While hypertext links on the web are widely understood, it isn’t clear that this convention is portable. Users may contextually recognize that underlined words (that trigger UI changes on hover) are clickable on the web without then thinking that all words in all UIs are clickable. Indeed: they’re not! Despite Apple’s efforts in iOS 7, it is clear that without affordances of some sort, there is too little distinction between “text” and “UI text” in many apps.

Visual element advantages:

  • UIs made of visual elements or visualizations of metaphors can be really intuitive. When one-year-old babies attempt to “Slide to Unlock” their parents’ iPads, it isn’t because it says “Slide to Unlock.” Pre-verbal and a ways away from understanding what “lock” means in this sense, they understand the motion of the elements and the responsiveness of the screen, and that’s enough.
  • There seems to be a correlation between use-expertise and preference for information density and textual UIs; while a novice is happy with the tradeoffs involved in Cover Flow, experienced users don’t benefit as much from it. They understand regular data views (like iTunes) and favor precision and access. Thus: visual elements are most useful in new use-cases or with newer users.
  • Visual elements can be beautiful, exciting, fun, funny; they have aesthetic qualities which exceed those of text (which is nevertheless capable of carrying aesthetic data), and can be enticing or appealing to users simply for those qualities. Aesthetics are important, especially experientially. When Steve Jobs bragged about OS X’s (now deprecated) aesthetic by saying “we made the buttons on the screen look so good you’ll want to lick them,” he wasn’t being superficial; making UIs approachable by developing visual elements is smart.
  • It is increasingly easy to code visual elements, giving them many of the advantages of text (scalability, ease of iteration, etc.); most of the disadvantages listed below can be mitigated or eliminated by using coded visual elements rather than rasterized image assets or the like.

Visual element disadvantages:

  • Adding visual elements, supporting interactive physics, or even just generating illustrative assets for a UI is costly relative to text, both initially and in terms of continued development / maintenance. This cost comes in several forms, but most important to good development practices is the cost to speed of iteration. Changes to text typically take the time to type them; new PSDs or renders or drawings can take much, much longer, especially when every visual item needs to be rendered for many devices, display densities, etc.
  • Visual elements are less universal and less variable. If someone with assistive features in their OS activated —say, larger fonts or higher contrasts or a “zoomed in” view— comes to your product, how will it fare? Text is far likelier than a graphic or interaction or icon to scale appropriately, play well in a different configuration, handle the many environments in which it might be displayed. In multi-device or multi-platform applications, the importance of this cannot be overstated.
  • Visual elements seem to age more quickly. To again bring iOS into the discussion: Apple’s transition from iOS 6 to iOS 7 was taken by many to indicate a maturing, increasingly savvy user-base, one bothered by (or at least indifferent to) skeuomorphic elements like the Corinthian leather of Find My Friends and so on. No one plausibly argued that the skeuomorphism made iOS difficult to use; they simply stopped liking it. Thus: increasing the amount of aesthetic design (in the form of visual elements) also increases the degree to which one must chase subjective tastes. While “timeless” designs —long-lived ones, that is— are possible, visual elements can age. Not all visual elements age, of course; but many do.

This is not comprehensive, of course; there are lots of other potential factors to consider, such as internationalization issues (with both the textual and the visual), device capabilities (screen, processing, graphics, and network considerations), ecosystem concerns (as with iOS), searchability (the semantic value of text is worth considering), and more. Thanks to what can be accomplished in CSS and HTML, visual elements needn’t cost bandwidth or come from Photoshop; at the same time, an ever-tech-savvier world population using the web means that visual elements may be decreasingly necessary in some applications. Newer applications and devices will always benefit from making their features and UIs analogously comprehensible, however, and for lots of uses visual UIs are simply better. It seems likely to remain a case-by-case or even element-by-element question.

Mills Baker