Sector financiero

Reduplication is a linguistic process of a continuous sign language in which gesture movement is repeated over and over to inflect a basic lexicon. Repetition patterns are the auxiliary components of a sign that complement its meaning. As they constitute a repetition of a complete sign so they can be considered as the boundary points of a sign. Repetition detection also provides a complete linguistic reference of a lexicon which is highly required for better recognition.

Sign repetitions are short termed and they are hardly detected by the pause detector or the directional variation features of the DAD signature. Instead these special types of inter-sample variations are modelled by the repetition features. These features are formed by multiple recurrence of a length of a sign in its delayed window. In a normal prosody, the repetitions happen so fast that the segment of a lexical component does not vary significantly in a short span of its history. Hence the DAD vector inside the repeating segment contains a number of zero values (showing number of matches with its previous values). The number of repetitions and the length of repeating portions are the DAD’s boundary features.

To explain the working of DAD’s repetition detection, a sinusoidal signal is used because it comprises of repeating segments having their maximum similarity with other segments delayed by its time period. Although we have real sign data which contains

0 100 200 300 400 10 20 30 Si gn pa ra m et er 0 100 200 300 400 20 40 60 The ta 200 300 400 Frame number

School of Engineering and Advanced Technology 135 some local repetitions but the amplitudes of the repetitions are so small for our video resolution that they appear as pause. Shown in Figure 72 is a simulated signal which starts as a constant and contains two sinusoids of time periods 10 and 20. Both the sinusoids are separated with a pause period. The DAD transformation of the time varying signal clearly visualizes all the pause features including the small local pauses at the extrema of the sinusoids. The signature also shows the directional features due to significant directional variations found at the extrema of the signal. Repetitions are the black horizontal lines showing the similarities of a segment to the patterns lying at delays equal to its time period.

Figure 72: DAD's repetition features

For example, at the delay of 20 a repetition feature (black horizontal line) in the DAD signature points to the fact that every value of the first sinusoidal signal repeats with a delay of its time period. Because we are observing the repetition patterns in a small delay of 30, only the first recurrence is visible. On the other hand, the DAD’s features of

the high frequency sinusoid have multiple replicas visible within its delay window. For instance, two reoccurrence of the same point P are shown by two red arrows and their

Frame number Frame number

P

r _Pause

School of Engineering and Advanced Technology 136 corresponding similarity scores in the signature. Because both the repetitions are short termed and occur within the last 30 samples (delay window) of the signal, they are modelled as the repetition features of a signal.

Assuming the minimum length of a segment (Kr) and the number of allowed

repetitions (N), DAD features can identify the right candidate for the sign segmentation.

5.6.1 Feature extraction

Suppose a segment of length Kr starts at n=p to n=p+ Kr and has N short term

quasi-repetitions. As shown in Figure 73, DAD signature of the repeating segment transforms these variations into N patterns of maximum similarity. Each DAD vector inside the repetitive pattern has N local minima which show the minimum differences of a sample with its corresponding delayed version. Local minima acquired of all the DAD vectors return many candidate repetitions which require further filtering on the basis of repetition length and the total count of a repetition.

Figure 73: Length of DAD repetition feature by computing the horizontal projection (sum of Kr adjacent local minima starting from first one)

Length of a repeated pattern is the total numbers of connected local minima in the adjacent the DAD vectors. A simple way to extract the length feature is to take the horizontal projection (equation 5.16) from each candidate (p) in a DAD vector to p+Kr

number of adjacent vectors.

Detected candidate (local minimum) P1 P2 Other value Hp(P1)>> δ Hp(P2)< δ Kr PN _H p(PN)< δ p

School of Engineering and Advanced Technology 137

ܪ௣ሺ݊ሻ ൌ ෍ ܦܣܦሺ݅ǡ ܲ௡ሻ

௄_ೝ

௜ୀଵ

5.16

where, horizontal project (Hp) is the sum of Kr adjacent values in the DAD signature

starting from a candidate repetition PNof a signal sample X[n]. If the projection results

are above the threshold (δ), the candidate is rejected. Otherwise more points are added up in the segment until it crosses the threshold.

As discussed in the algorithm, the sum of Kr adjacent values is calculated on each

candidate point in the DAD vector. For the first candidate P1, horizontal projection results

in a large number because of the accumulation of the dissimilar values (red dots). On the other hand, for P2, a connected line containing at least Kr similar values can be created.

Sum of all the points in the detected repeated segment is minimum because they are the maximum similarity scores (minimum DAD value) with their corresponding samples/segments.

Although the repetitions pertaining to this research are short-termed i.e. they occur in short interval of time, yet their trajectory can undergo small variations due to any local movement. In this situation, the existing algorithm fails because the repetition feature appears as a curved line (as shown in Figure 74).

Figure 74: Feature distortion due to an inexact repetition. Black dots (repeated points) are slightly delayed in the signal which causes a bend in the repetition feature failing the horizontal

projection approach

This means that the feature extraction should cope with the distortions due to an inexact match of a repetition. The modified solution is discussed in the following section.

Detected candidate (local minimum) P1 Other value Hp(P1)>> δ Kr

School of Engineering and Advanced Technology 138

5.6.2 Repetition feature by shortest path

The length of the repetition pattern can be calculated by searching a shortest path from the candidate boundary point that traverses Kr nodes in a way that the total path

penalty is minimum. Dynamic programming is successfully used for such kind of optimizations. According to the shortest path algorithm, at every candidate point in the DAD, the next node is selected based on its maximum resemblance with the first one. Path penalty is accumulated along the traversal of the optimum route and if it exceeds a certain threshold, that candidate is rejected.

Inside a DAD signature, an optimum path can interlink any segment of the high similarity score. For example, a feature path can be detected as a vertical line instead of a horizontal curve which is completely misleading in terms of finding the repetition patterns. To limit the dynamic programming to the detection of an optimum horizontal path, the search is conducted in a window which is setup at every candidate point. This window helps in confining the extent of an optimization. This window also allows the optimizer to choose a short path allowing constrained deviation from the initial point. Figure 75 shows the modified searching strategy in which the next candidate is selected only within a defined region. This ensures the detection of a repetition feature through a maximally flat path.

Figure 75: Modified algorithm searches for the adjacent local minima using shortest path