Situación problemática

For CBNDVC detection for novelty re-ranking the goal is to discover as many novel clusters as possible, not just cluster of single seed video and their near duplicates, but all possible novel videos and their near duplicates using given Relevant concept set. The problem of CBNDVC datasets is that we do not know the possible number of novelty clusters in advance. The system is incremental, and new videos can be added and deleted. Also, we need a characteristic description for an identified group to infer the concept represented by the cluster. COBWEB [43] is one such conceptual clustering algorithm that matches most of the requirements for the proposed CBNDVC detection. Unlike conventional clustering that identifies groups of similar objects, conceptual clus- tering finds characteristic descriptions for each group, where each group represents a novel category or class. Its quality is not solely a function of the individual objects. Rather, it incorporates factors such as generality and simplicity of the derived concept description [51].

We made important modifications to the existing COBWEB conceptual clustering algo- rithm such that the data as shown in Figure 5.4 can be processed to completely satisfy the proposed method’s requirements. The concept of cluster representative did not exist in the COBWEB algo- rithm. We enable COBWEB to incorporate the notion of cluster representative, which is to identify the video representing the concept of the cluster in the best possible way compared to other videos residing in the same cluster.

Algorithm for Conceptual Clustering of Multivariate Time-series:Here, we introduce the conceptual clustering algorithm COBWEB along with an example (dataset in Figure 5.4) and describe the required changes made to the algorithm. Whereas some iterative distance-based clus- tering algorithms, such as K-Means, go over the whole dataset until convergence occurs, COBWEB works incrementally, updating the clusters video by video. COBWEB creates hierarchical cluster- ing in the form of a classification tree. The leaves of the tree represent every individual concept, the root node represents the whole dataset and the branches represent the hierarchical clusters within the dataset. The total number of clusters can be as many as the total number of video objects in the given dataset, if all the videos are having significantly different concepts within them and thus in turn none of them is CBNDVC video.

COBWEB starts with a tree consisting of just the root node (seed video). From there, instances are added one by one, with the tree being updated accordingly at each stage. When a video instance is added, there are four possible actions: (1) Classifying the video object into an existing class (2) Creating a new class (3) Combining two classes into a single class (merging) and

Seed Cluster H H H H H H H H H H H H H H H H H H L L L M H L L H H L L L M H L L H H Novelty1 Cluster Root

Figure 5.6: Conceptual cluster generated for example dataset in Figure 5.4. One CBNDVC cluster of seed video is discovered as Seed Cluster and another Novelty1 cluster which is semantically different from seed video. Here H = High, M = Medium and L = Low represents attribute values.

(4) Dividing a class into several classes (splitting). The Algorithm 1 will choose the action with the biggest Category Utility (CU), defined by the following function:

∑n k=1P (Ck)[ ∑ i ∑ jP (Ai= Vij|Ck)2− ∑ i ∑ jP (Ai = Vij)2] n (5.3.2)

Where Vij is a potential value of attribute Aiin our example data. We have Vij ={High, Medium,

Low} and Ai ={Outdoor, Person, Room, Running, Singing, Talking, Standing, Stadium, Dancing

}, i = {1,2,. . .,9} semantic concepts from the Relevant Concept Set. And q is the number of nodes,

concepts or categories forming a partition{C1, C2, , Cq} at a given level of the tree. For our example

data in Figure 5.6 we see q = 2 at level 1 Seed Cluster and Novelty1 cluster. Category Utility is the increased amount of the expected number of attribute values that can be correctly estimated from a partition. This expected number is P (Ck)[

∑

i

∑

jP (Ai = Vij|Ck)2 and the expected number

of correct estimates without such knowledge is the term∑_i∑_jP (Ai = Vij)2. Category Utility

rewards intra-class similarity and inter-class dissimilarity where:

• Intra-class similarity is the probability P(Ai = Vij | Ck). The larger this value is, the greater

the proportion of class members that share this attribute-value pair will be. Hence the class members are more predictable. For our example in Figure 5.6, Seed Cluster is expected to have P(Ai= High| Seed Cluster) = 1 for i = {1,2,. . .,9} because all the videos are CBNDVC

to seed video and thus have high match for all the attributes.

• Inter-class dissimilarity is the probability P(Ck| Ai= Vij). The larger this value is, the fewer

pair belongs to a certain class. Thus, we will have P(Seed Cluster| Ai = High) = 1 for i =

{1,2,. . .,9}.

We modified the COBWEB data structure for storing the video objects such that we can update the cluster representative whenever new video is added or old video is deleted. When the Algorithm 1 chooses one of the four actions for video object clustering, we update the cluster representative if required. We consider the length of video as one important feature to consider it as a Conceptual Cluster Representative. We find length as an important feature because we are looking for conceptually near duplicate videos. Thus longer videos with similar concept distribution of concepts from the Relevant Concept Set are assumed to have some other concepts contained in them. Even if these extra concepts in longer videos may be useful or not useful, we can consider longer videos as better representatives for the conceptual cluster. Each of the conceptual clusters have videos kept in descending order of their lengths, thus the top video is with the highest length and considered as cluster representative. Of course, these cluster representatives do not guarantee best audio and video quality but definitely possess a better representation of a cluster’s concepts within them.