Sunday, November 11, 2007

touchscreens and the approaching tipping point for musical instrumentation

Touchscreens have been around for quite awhile, but they're starting to get seriously good, offering decent response times from devices scarcely bulkier or heavier than an ordinary LCD screen. Pressure sensitive screens are just around the corner.

So what do touchscreens have to do with music? They make instantly reconfigurable, virtual control panels possible. I'll revisit that point and connect the dots later.

Take the iPhone for example. The iPhone was far from being the first device to use a touchscreen, but it may be the first to make really good use of one, combining vivid graphics with multitouch technology and gesture detection. While it uses a menu system much like phones have had for years, navigation through those menus is practically effortless, because current options are presented simultaneously, and choosing one is as simple as putting your finger on it. Each menu uses almost the entire screen to handle a limited set of options, maintaining a moderate density of information and presenting a moderate selection of controls. Each of those controls has a very specific purpose and action, unlike the overloaded buttons on an ordinary cell phone, which change what they do according to context.

So a single touchscreen can offer many different control panels, switching between them as needed, and the controls on those panels - buttons, dials, sliders, steppers, etc. - can have both recognizable identity and clearly visible state.

Now let's take a minute to think about musical instruments as control systems. Physical instruments have a static set of controls, which can typically be applied in various combinations - multiple notes played together on a piano, multiple valves closed/opened on a clarinet, multiple, independently fretted strings plucked or strummed singly or together on a guitar - but even those combinations constitute a fairly limited set of options. Woodwinds produce harmonics of a single fundamental; change instruments to change fundamentals. Brass instruments allow you to change the fundamental by changing the length of the resonant cavity, using slides or valved extensions, but their voices also change, and doing that on the fly is a somewhat awkward way to generate more options in any case. Guitar frets are typically arranged to provide a twelve-tone, equal tempered scale. They could have more frets, or could be strung and tuned to offer six steps between each pair of successive twelve-tone notes, but the latter would mean sacrificing range and all but the simplest chording, and the frets already get crowded as you reach for higher notes.

Electronic keyboards are considerably more flexible, with programmable processors that are more than adequate to generate any sound you might want an instrument to produce, but the options they make available are still mapped to twelve keys per octave, and any use which doesn't at least approximate that arrangement is unlikely to be intuitive. Other instrument-driven synthesizers are similarly constrained by the control device.

That's where touchscreens come in. A touchscreen can show you just the options you need for a particular purpose, arranged in a manner that makes good sense or is at least easily recognizable. It could, for instance, present you with just enough controls to produce the notes that constitute a particular scale. That much isn't new. What is new is that a system using a touchscreen interface could instantly transpose that scale, up or down, or just as quickly switch to a different one, with visual cues so your brain is able to follow the change. It could also offer two or three scales side-by-side, with cues to show which notes are common among them, and allow you to rearrange them by simply dragging them around with your fingertips, elaborate them, perhaps making a copy first, or make them shrink and run to a corner to keep them close and ready for use. When playing, note patterns could also be visual pattens, meaningful bursts of light, and both scale manipulations and played notes could be incorporated into loops and scores, which could also be displayed. Creating, altering, and combining such loops during performance should make for some interesting high-wire acts.

As a scale-generator, such a system could also be used to unify the sound of other, more conventional instruments, bringing them into tune with each other and with scales they weren't designed to produce.

There's some advantage in all of this, even if the only notes you're interested in using are those found on the piano or guitar, but if you're fond of combining pure intervals, you'll quickly discover that the variations are endless, far beyond the capabilities of any static interface. Touchscreens will make this vast set of possibilities the musician's playground, really for the first time.

Sunday, September 02, 2007


Tenori-on is a new type of electronic musical instrument, developed by Toshio Iwai in collaboration with Yamaha. It's interface is a grid of 16 X 16 LED-illuminated button switches, in a frame that contains a few additional controls.

This device goes on sale in the U.K. soon. There will be a lauch event this Tuesday, September 4th.

Click here to go to Yamaha's Tenori-on product page.

Sunday, June 03, 2007

of primes, powers, ratios, and representations

So what does all this have to do with music?

As was known to the ancient Greeks, there is a tangible relationship between a tone and other tones the frequencies of which are small integer multiples of the first. (The Greeks thought of it in terms of dividing a string into lengths which are integer fractions (1/2, 1/3, 1/4, ...) of the length of the string.)

There are also tangible relationships between the lower members of a harmonic series, for example between the 4th and 5th harmonics. These relationships begin to break down above the 7th harmonic (some would say above the 11th), except as their numerical expression is reducible to a ratio composed of low powers of small primes, in which case even a tiny interval, such as that between the 55th (5 x 11) and 56th (7 x 2^3) harmonics, can be musically significant. (Because there are no common prime factors between 55 and 56, the interval 56:55 is already expressed in least terms and is irreducible.)

Such ratios can be represented by a short list of small integers which are the powers of the first five primes (2, 3, 5, 7, and 11), the values of which may either be positive, if the prime and its positive exponent belong in the numerator, or negative, if the prime and its positive exponent belong in the denominator. Using this approach, the number 2 (2/1) would be {1, 0, 0, 0, 0} and the fraction 1/3 would be {0, -1, 0, 0, 0}. This might seem overly complex, except that it is a comprehensive representation which can easily accomodate any interval that might be considered musically significant by virtue of possessing a degree of consonance.

Moreover, the set of values which might reasonably be used in such a list of prime powers is quite small. The power of 11 would never be anything other than -1, 0, or 1. For the sake of flexibility, one might wish for the range of the power of 7 to be a little wider, say from -2 to 2. For 5 we might wish a choice of powers from -3 to 3, and for the prime 3 possibly even a range from -5 to 5. The lowest prime, 2, is a special case; how many octaves would you like to be able to span? If you want to be able to accomodate fundamentals as low as 1 Hz and at the same time accomodate tones at the very upper limit of human hearing, and to be able to reference either from the point of view of the other, you'll need a minimum range of -14 to 14. Let's call it -16 to 16 for good measure.

That's 33 x 11 x 7 x 5 x 3 or 38115 possible combinations of values, a number smaller than the range of an unsigned 16-bit integer, not that I'd really suggest using 16-bit values and a translation table just to save space, not with memory and storage as abundant and cheap as they've become, but it wouldn't be unreasonable to use an array of five single-byte values (signed char in C).

Note that what's being described by such lists of prime powers are intervals in an abstract sense, the value of the ratio between two tones, which can be applied to any tone where the resulting tone isn't so far outside human hearing as to be irrelevant.

Most compositions won't come close to making use of the full range of the set of prime powers {2^(-16..16), 3^(-5..5), 5^(-3..3), 7^(-2..2), 11^(-1..1)}, but what they may do is to make use of several harmonic series, the fundamentals of which are related by similar ratios. In such cases, it's always possible to reduce the system to a single harmonic series with a lower fundamental, but doing so is very likely to be less meaningful than using several harmonic series and low-numbered harmonics.

As a matter of pragmatism, in the programmatic context it would be preferable to define a collection of harmonic series by ratios relating all of their fundamentals to a single base tone, rather than to generate a string of ratio-related fundamentals as each section of a composition gives way to the next. That progression can easily be represented by reference.

But a harmonic series, even a set of them, is not a scale. To produce a scale it's necessary to specify which members of those series one intends to include, and again powers of primes can be useful, in this case only positive powers. Integers greater than one are either prime themselves or a product of primes, and harmonic numbers are simply integers. You can use limits on prime powers to constrain how many times each prime may appear as a factor, and thereby constrain the set of harmonic series members in use. An example would be {3, 2, 1, 0, 0}, which would generate a scale composed of members 1, 2, 3, 4, 5, 6, 8, 9, 10, 12, 15, 18, 20, 24, 30, 36, 40, 45, 60, 72, 90, 120, 180, and 360. This approach may prove adequate in some circumstances and not in others. It would be wise to maintain at least the ability to tweak the results manually.

So, to recap, ratios described in terms of powers of the first few primes relate the fundamentals of harmonic series to a base tone, as well as relating the members of those series to each other, and lists of limits to prime powers contrain the set of series members in use. With these tools one can define scales.