Tendencia ​Fast fashion

To compare the QE and QEE methods, we tested them under different param- eter settings. First we examined standard QE under different parameters. We show results for QE using different numbers of feedback documents (R) with different numbers of expansion terms (k). The results are shown in Figure3.2. We make the following observations:

Figure 3.2: Results for QE for the WikipediaMM test collection using a fixed number of feedback terms.

Figure 3.3: Results for QE for WikipediaMM collection using a fixed number of feedback documents.

1. The best result is obtained when setting R=5 and k=5.

2. When fixing the number of expansion terms, the MAP value decreases when more feedback documents are added.

These findings suggest that bringing more feedback documents into the RF process does not help to improve retrieval effectiveness in this case. Our results indicate when using QE on short length documents retrieval tasks, more feedback documents do not add more useful information into the RF process. Thus it does not help selection of good expansion terms for QE. These findings are different from the previous conclusions of experiments on the TREC collections [Buckley et al.,1994a;Robertson et al.,1994]. In the earlier TREC experiments [Buckley et al., 1994a], the more feedback terms added from relevant documents, the better the recall-precision, up to a steady-state value. For the Okapi feedback method [Robertson et al., 1994], the previous results show that adding a reasonable large number of feedback terms will benefit to the results (such as term number as 60).

We show results using various numbers of expansion terms when the number of feedback documents is fixed in Figure 3.3. From Figure 3.3, we can see that the results indicate that more expansion terms do not change the retrieval effectiveness very much. When we use only 5 good feedback docu- ments, adding too many expansion terms also hurts the final results (k >50). Using 100 feedback documents and 5 expansion terms gives the worst re- sult for QE method in our experiments. Also when using many nonrelevant documents for QE (R = 100), more expansion terms could help to improve the final retrieval effectiveness. This evidence indicates that QE only needs

good expansion terms in the good feedback documents for this short-length documents retrieval task. The results indicate that the target corpus provides limited useful feedback documents and expansion terms for short-length doc- uments retrieval task.

We hypethesis that external resources may help to relieve data sparse prob- lem in IR tasks better than the search target collection. We test the results of the QEE method under various parameter settings. First we examine the ef- fect of different numbers of feedback documents when using a fixed number of expansion terms. The results of this experiment are shown in Figure 3.4. Comparing Figure 3.4 with Figure 3.2, the observable difference is that the MAP scores in Figure 3.4 do not decrease with the addition of more feed- back documents from the external resources when using a fixed number of expansion terms, as was the case in Figure3.2. The difference of QE and QEE is illusrated in Figure 3.5 using the fixed number of feedback terms set at 5. This observation can be explained as the external resource containing more useful information relevant to the query than the target corpus. Thus adding more feedback documents does not add significantly more noisy documents into the process of relevance feedback. However, the best result of the QEE method does not outperform the best result of the QE method (the best results of different methods are shown in Talbe 3.3).

We also show results for different numbers of expansion terms when using a fixed number of feedback documents in Figure 3.6. The results show that adding too many expansion terms from external resources harms the retrieval. This indicates that the expansion terms should be limited to a reasonable number when using external resources.

Figure 3.4: Results for QEE for the WikipediaMM test collection using fixed number of feedback terms.

Figure 3.5: Comparision of QE and QEE using fixed number of feedback terms 5.

Figure 3.6: Results for QEE for WikipediaMM collection using fixed number of feedback document.

Although QEE does not produce better results than QE, it is interesting to test the impact of using queries expanded from external resources to ex- pand from the target corpus again for retrieval (QEE+QE). Also runs of using queries expanded from target corpus to expand from external resources for retrieval (QE+QEE) can be tested. For our experiment, we selected the best run obtained using the QEE method where R = 100 and K = 5. This selec- tion ensures that we have the best queries from QEE method. After QEE, the new queries are sent to the target corpus for QE. The results using different parameter settings are shown in Figure 3.7 and Figure3.8.

The QEE+QE method produces similar curves to the QE method with better results. Figure 3.7 shows that more feedback documents for QE hurts the final retrieval effectiveness when using a fixed number of expansion terms. Figure3.8shows that adding more expansion terms does not change the final

retrieval effectiveness when using a fixed number of feedback documents. These are similiar conclusions to those we found in the QE experiments. These results also indicate that QEE provides useful feedback information to the original queries since QEE+QE gives better results than QE.

Figure 3.7: Results for QEE+QE for the WikipediaMM collection. Furthermore, we test the results of QE+QEE. For QE, we select the param- eter which produced the best result among our QE Runs, where R = 5 and k = 5. After QE from the target corpus, the expanded queries were applied to the Wikipedia collection for QEE. The results of QE+QEE Runs are shown in Table 3.3. The best result of QE+QEE outperforms the best result of QE method, but it does not outperform the best result of the QEE+QE method.

To compare the different query expansion methods, we show detailed re- sults in Table3.3. For each method, the best result after parameter tuning was selected. In Table3.3, Okapi is the baseline run using only the Okapi retrieval

Figure 3.8: Results for QEE+QE for the WikipediaMM collection. model without QE process from any collection. For the results, our analysis is based on MAP values with NDCG, R-Prec, P@10 also included in Table3.3. The results show that:

• QE is an effective method compared to run without QE.

• QEE achieves comparable results to QE, and the difference between the QEE and QE is not significant by the MAP values.

• QE+QEE does not achieve signifcantly better result compared to the QE method.

• QEE+QE outperforms the Okapi, QE, QEE for four different evaluation metrics including MAP, NDCG, R-Prec and P@10. The result of QEE+

Figure 3.9: Results for QE+QEE for the WikipediaMM collection. values.

Table 3.3: Results of different query expansion methods. ’+’ means the im- provements over the baseline are statistically significant for the MAP scores.

Runs MAP NDCG R-Prec P@10

Okapi 0.2338 0.4931 0.2805 0.3453

QE 0.2588+ +10.69% 0.5014 0.3035 0.3720

QEE 0.2551+ +9.11% 0.5253 0.3011 0.3427 QEE + QE 0.2678+ +14.54% 0.5268 0.3071 0.3720

QE + QEE 0.2609+ +11.59% 0.5255 0.2969 0.3693

In the next section, we propose a Definition-based Relevance Feedback (DRF) method using external resources for the text-based image retrieval task. In this method, we hypothesis that the definition documents (the documents from external resources whose title contains the query terms) are good feed- back documents to provide feedback information for a user query, and that

these defintion documents can help to focus on the useful feedback docu- ments.