touchscreens and the approaching tipping point for musical instrumentation

Touchscreens have been around for quite awhile, but they're starting to get seriously good, offering decent response times from devices scarcely bulkier or heavier than an ordinary LCD screen. Pressure sensitive screens are just around the corner.

So what do touchscreens have to do with music? They make instantly reconfigurable, virtual control panels possible. I'll revisit that point and connect the dots later.

Take the iPhone for example. The iPhone was far from being the first device to use a touchscreen, but it may be the first to make really good use of one, combining vivid graphics with multitouch technology and gesture detection. While it uses a menu system much like phones have had for years, navigation through those menus is practically effortless, because current options are presented simultaneously, and choosing one is as simple as putting your finger on it. Each menu uses almost the entire screen to handle a limited set of options, maintaining a moderate density of information and presenting a moderate selection of controls. Each of those controls has a very specific purpose and action, unlike the overloaded buttons on an ordinary cell phone, which change what they do according to context.

So a single touchscreen can offer many different control panels, switching between them as needed, and the controls on those panels - buttons, dials, sliders, steppers, etc. - can have both recognizable identity and clearly visible state.

Now let's take a minute to think about musical instruments as control systems. Physical instruments have a static set of controls, which can typically be applied in various combinations - multiple notes played together on a piano, multiple valves closed/opened on a clarinet, multiple, independently fretted strings plucked or strummed singly or together on a guitar - but even those combinations constitute a fairly limited set of options. Woodwinds produce harmonics of a single fundamental; change instruments to change fundamentals. Brass instruments allow you to change the fundamental by changing the length of the resonant cavity, using slides or valved extensions, but their voices also change, and doing that on the fly is a somewhat awkward way to generate more options in any case. Guitar frets are typically arranged to provide a twelve-tone, equal tempered scale. They could have more frets, or could be strung and tuned to offer six steps between each pair of successive twelve-tone notes, but the latter would mean sacrificing range and all but the simplest chording, and the frets already get crowded as you reach for higher notes.

Electronic keyboards are considerably more flexible, with programmable processors that are more than adequate to generate any sound you might want an instrument to produce, but the options they make available are still mapped to twelve keys per octave, and any use which doesn't at least approximate that arrangement is unlikely to be intuitive. Other instrument-driven synthesizers are similarly constrained by the control device.

That's where touchscreens come in. A touchscreen can show you just the options you need for a particular purpose, arranged in a manner that makes good sense or is at least easily recognizable. It could, for instance, present you with just enough controls to produce the notes that constitute a particular scale. That much isn't new. What is new is that a system using a touchscreen interface could instantly transpose that scale, up or down, or just as quickly switch to a different one, with visual cues so your brain is able to follow the change. It could also offer two or three scales side-by-side, with cues to show which notes are common among them, and allow you to rearrange them by simply dragging them around with your fingertips, elaborate them, perhaps making a copy first, or make them shrink and run to a corner to keep them close and ready for use. When playing, note patterns could also be visual pattens, meaningful bursts of light, and both scale manipulations and played notes could be incorporated into loops and scores, which could also be displayed. Creating, altering, and combining such loops during performance should make for some interesting high-wire acts.

As a scale-generator, such a system could also be used to unify the sound of other, more conventional instruments, bringing them into tune with each other and with scales they weren't designed to produce.

There's some advantage in all of this, even if the only notes you're interested in using are those found on the piano or guitar, but if you're fond of combining pure intervals, you'll quickly discover that the variations are endless, far beyond the capabilities of any static interface. Touchscreens will make this vast set of possibilities the musician's playground, really for the first time.