Propuestas de proyectos de desarrollo e investigación que utilizan el Simulink y

While results from this study were promising, this research has thrown up many questions in need of further investigation. The areas of further research can be categorized into three categories. These areas are described in the paragraphs that follow.

Prototype. While overlook is a working system, it is a proof of concept system that can be expanded upon. The main area of improvement is in the included heuristics. Various errors in visualizations can be expressed as constraints or heuristics. For example, the product of the cardinality of the datasets fields, when mapped to therowandcolumnchannel can be constrained, limiting the number of elements of trellis plots.

When these rules interact with the data used for the visualization, it is idiomatic to implement them in python. When this is not required, we propose to follow the approach of Moritz et al., who showed that logical rules can be expressively expressed in ASP programs, and provide a database of design guidelines expressed in ASP programs. ASP programs can be parsed and included in the Z3 model.

In evaluation, overlook recommended a set of bar, horizontal bar, or line charts. This allowed for comparison to the baseline system. The implementation can be expanded to support additional visualization types, or to recommend multiple visualization types in one query. Visualizations of different types are disjunct; thus, the number of states in the Z3 model scales linearly with the number of visualization types requested.

A natural progression from this is to remove the restriction that fields are only used for a single visual variable and allow the assignment of a field from the dataset to multiple visual variables. When this is supported, a regularization penalty could be added as an optimization goal to ensure efficient allocation of visual variables.

With regard to the information available to the VizRec algorithm, overlook does not query data sources during recommendation. That implies that (selection specific) summary statistics of datasets are not available. Future research might explore methods for incorporating the shape of the data during recommendation, for example, by sampling collection statistics.

Evaluation. This study has used an evaluation approach positioned between the evaluation of a

visualization system and IR evaluation. While this is valuable, when evaluating detailed features of a system, or when a large number of responses is needed, further research might choose another approach. Two approaches are common.

When investigating how users experience a system, other studies often use mixed methods designs with multiple treatments. This approach can evaluate multiple interventions and allows for a balanced design. It is also common to design a task that is similar to IIR, where users need to select or save results out of all results. This data can then be used as training data. A disadvantage of this approach is that it is not feasible to evaluate on multiple datasets since the number of respondents needed scales with the number of datasets.

Another approach is to use crowdsourcing2 to gather assessments. Crowdsourcing is especially useful when a high number of assessments and or assessors is needed. For example, Kim and Jeffrey Heer use crowdsourcing to measure the error rate of assessors when interpreting visual encodings, for various primitive tasks, and use that to rank encodings by visual effectiveness [KH18]. Ç. Demiralp, Bernstein, and Heer evaluate the usage of different judgment types to create an effectiveness ranking for different shapes for shape marks using crowdsourcing.

Ranking. After the SMT problem is solved, overlook ranks the candidate visualizations by the sum of

their heuristic scores. A system that presents results to a user, ordered by a utility function, is using a ranking function [Bur+05]. When labeled examples are available, machine learning can be used to learn a ranking function. The term Learning to Rank (LTR) refers to this application of machine learning.

LTR has been applied in VizRec [Luo+18; Mor+19]. Using this technique, visualization assessments from this study, visualization ratings from other studies (as used by Moritz et al. [Mor+19]), or user

behavior3 can be used to improve the ranking of visualizations and make the system increasingly more effective after initial deployment. For overlook in specific, LTR can be used to learn weights for heuristic scores or to re-rank visualizations based on how visualizations use fields (e.g., number of values on the x-axis).

Another ranking technique to be considered is using heuristic scores to include diversity in the ranked results. For example, a heuristic can express the difference in encoding between a clustered bar chart and a stacked bar chart.

Finally, further research might investigate the usage of machine learning to learn the primitive tasks a visualization is suitable for. This classifier can then be used as either decision support, where a system suggests what tasks a visualization is suitable for; or as a ranking problem, where a system returns a list of visualizations ranked for the task(s) given.