COMO SE ESTABLECE UNA CONEXION

using the CT device 4.5.3.1 Introduction In Section 1.5.1.1, we discussed the

importance of a fundamental question: What forms of instrumental control are possible

to implement? Here, a relevant sub-question is: What human control actions are possible? In the context of unidirectional fingertip movement orthogonal to a surface (see Section 1.2.5), the human actions that are possible often will be ones that produce

downward or upward fingertip movement. However, human actions suppressing

downward or upward fingertip movement exist, too. Therefore, here, our immediate

research question will be: Can computed touch be used to enable instrumental control

that is based on finger rigidity?

And then, also: Can such control occur simultaneously with control based on down-

and upward fingertip movement? This is relevant, as a positive answer here could point to ways of obtaining more control possibilities per finger, for simultaneous change to the sound-generating process. A positive answer also may serve to clarify and

demonstrate the difference between what we here refer to as “movement-producing” and “movement-suppressing” actions.

To answer the two-part research question, prototype 2 of the CT device (see Section 2.2.4) was used to test an approach based on first generating a wave pattern in fingerpad-orthogonal force output, to then multiplex a rigidity degree-of-freedom (DOF) on the orthogonal fingertip movement input [De Jong 2008]. Here, a form of computed sound – again parametrized waveform synthesis (see Section 1.4.2.2) – was implemented so as to also demonstrate actual control of a sound-generating process.

4.5.3.2 The sound-generating process As before, computed sound was based on a

constantc_{sampling rate}(Hz), with time t ∈ ℕ (see Section 4.2.2.2). Over time, the

amplitude series of audio output was characterized by: o_audio[t] = f_{audio band noise}(t , s_{center frequency}[t], s_bandwidth[t])

Here, f_{audio band noise}computed a noise signal, characterizable as a sum of sine waves

diminishing in amplitude below and above a given center frequency. This s_{center frequency}[t]was variable across a [100, 24000] Hz range. How quickly sine wave amplitudes would diminish over frequency was determined by the bandwidth

parameter s_bandwidth[t], variable across a [20, 320] unitless range. Variation across both

ranges seemed independently perceivable in the resulting heard sound.

4.5.3.3 Computed touch Touch output via the prototype 2 CT device was controlled

by a sine wave amplitude series:

s_{to touch output}[t] = sin(t /c_{sampling rate}×2π ×c_{force sine frequency})

Here, the frequency used was fixed atc_{force sine frequency}= 10 Hz. The output transducer

then mapped the s_{to touch output}[t]amplitude range to the full magnetic field strength range

available (see Sections 2.2.3.3, 2.2.4.5, and 2.2.4.6). Effectively, this mapped to the full range from minimum (downward) to maximum (upward) force currently possible for the given keystone distance. However, in a final adjustment, the digital amplitude

range of s_{to touch output}[t]was reduced by 10%, to avoid collapses in magnetic field

strength output (see Section 2.2.4.8).

4.5.3.4 Instrumental control of musical sound The prototype 2 CT device provided a

linear distance inputi_{z distance}[t](see Sections 2.2.4.2 to 2.2.4.4). From it, a “nearness”

signal was computed, using a moving average: snearness[t] =

1 n

∑

i=n−1

iz distance[t−i]

Here,t ≥ n−1 andn = c_{sampling rate}/c_{force sine frequency}, so as to average out variation

Then, the absolute deviation of distance inputi_{z distance}from thes_nearnessaverage was used, also averaged over the most recent output wave cycle, to compute a “rigidity” signal that tracked how much the fingertip attached to the keystone dampened vibration amplitude:

s_rigidity[t] = f_{distance compensation}(s_nearness[t], 1 n

∑

j=n−1

∣

∣

This withnas before, and nowt ≥ 2n−2 . Here, the f_{distance compensation}function

compensated for the amplitude of the output force wave itself already varying over fingertip distance. This compensation was based on measurements, made both while keeping a finger maximally rigid and while keeping it maximally loose, of the

uncompensated rigidity signal across the s_nearnessrange.

The remaining causal relationships which then enabled instrumental control can be roughly characterized by:

s_{center frequency}[t] ≈ f_{map to Hz}(s_nearness[t])

s_bandwidth[t] ≈ f_{map to bandwidth range}(s_rigidity[t])

Here, f_{map to Hz}mapped the s_nearnessrange to [100, 24000] Hz, and f_{map to bandwidth range}

mapped the s_rigidityrange to [20, 320] (see Section 4.5.3.2).

4.5.3.5 Evaluation When executed, the algorithm described above in Sections

4.5.3.2 to 4.5.3.4 produced a continuous sensation of the fingertip being shaken up and down, rapidly and regularly. During this, performing slow up- or downward fingertip movements (see Figure 4.6, left side) would lower or raise the pitch of a continuously heard sound similar to white noise. Simultaneously, performing muscle contractions tensing the finger (see Figure 4.6, right side) would change the timbre of the noise [De Jong 2008].

Figure 4.6 To the left: fingertip up/down movement, varying s_nearness. To the right:

fingertip tensing, varying s_rigidity.

This can be seen in Figure 4.7, where I/O of the algorithm is shown during a slow downward, then upward fingertip movement, during which the fingertip also was

tensed and relaxed twice. Correspondingly, two s_rigiditypeaks can be seen below a

audio output. The algorithm was publicly demonstrated at the 2008 international conference on New Interfaces for Musical Expression in Genova, Italy.

Figure 4.7 Simultaneous I/O of the algorithm, recorded during 6 seconds of

instrumental control. Top: i_{z distance}is shown here as “proximity”: using a linear but normalized and inverted scale, where a larger value means a smaller z distance.

Upper middle: the derived s_nearnessparameter, also shown using a normalized and inverted scale. Lower middle: the derived s_rigidityparameter, shown using a normalized scale. Bottom: sonogram of the audio output, shown using a linear frequency scale.

The above demonstrates a positive answer to the research question posed in Section 4.5.3.1: Computed touch can be used to enable instrumental control that is based on finger rigidity; and this control may occur simultaneously with control based on down- and upward fingertip movement.

Finally, the above also shows – based on both the movement over time of the fingertip, and on the described resulting perception – how the algorithm has yielded a

new type of fingertip control action: fingertip tensing during force wave output. This directly enables the construction of new causal relationships between human actions and changes in heard musical sound. Thereby, it demonstrates computational liberation.

5. Excursion I: One-press