Universidad de Huelva.
7.2. Ciclo del carbono.
In a future study, there are a number of changes that should be made to the experimental design to more accurately test whether how the amount of data ink on a graph affects graph perception and learning.
First, one of the issues that subjects raised in their comments is that the graphs used in the study were not necessarily representative of graphs seen online. In a future study, it would be useful to have graphs that look like they could have appeared online at USA Today or CNN's website. Or, as an alternative, the study could include both academic and popular media graphs.
An additional problem that subjects pointed out with the study was that the bars of the graphs were not always organized in ascending or descending order. Subjects noted that having the bars out of order in some cases made the graphs harder to read. In a future study, all of the graphs should either have bars increasing or decreasing order or out-of- order. Although the order of the bars does not always reflect the way the data appears, this factor can be controlled when selecting data sets to use in the experiment.
A third improvement that could be made is that the graphs should have varying levels of difficulty and there should be at least two graphs of each difficulty level and type. For example, there could be eighteen graphs total with six graphs for each level of difficulty and six graphs for each data-ink ratio. Related to the level of difficulty, it would also be helpful to have some open-ended questions that required subjects to look more closely at the graphs and that did not have a specific “correct answer.”
In addition, one area for improvement in the study relates to the type of graph tested. It would be interesting to test whether subjects have different data-ink level preferences depending on the type of graph used. A number of subjects pointed out that bar graphs are “very simple to read.” In a future study, it would be interesting to also how graph perception changes with varying data-ink levels in test scatter plots, linear regressions and graphs with non-linear relationships.
In addition to varying the type of graph used, in a future study it would be useful to vary the Y-axis intervals for the grid lines. In this study, all of the grid lines were at rounded values such as 200. However, in a future study, it would be useful to test whether having grid lines becomes more important to subjects when they are asked to find Y-axis values such as 213.
A final area for improvement relates to accounting for scrolling during the experiment. One problem with this study was that it was difficult to isolate eye-movement patterns while subjects looked at each graph because subjects were always scrolling down to see the questions and then up again to look at the graph. To accurately measure eye-
movement, future studies should have subjects answer the questions orally or answer questions on a subsequent screen.
6
CONCLUSION
The intent of this study was to gain a preliminary understanding of whether Tufte's notion of graph quality would hold up in a lab setting. For the purpose of the study, quality was defined by response accuracy, attention focus, and enjoyment level.
My initial hypothesis was that subjects would prefer graphs with a moderate data-ink ratio and that they would process information more accurately on graphs with a moderate data-ink ratio than a low or high data-ink ratio. The data from this preliminary study suggests that the first part of this hypothesis may have validity. Subjects indicated that they preferred the graphs with a medium and high data-ink ratio to the graph with the lowest data-ink ratio. Although preference between high data-ink and medium data-ink is unclear, there does appear to be a preference for both options over graphs with low data- ink.
Unfortunately, the results do not provide insight into the second part of the hypothesis. Due to the fact that only one question was answered incorrectly throughout the study, we cannot make any conclusions about the relationship between the type of graph and the accuracy of responses.
Through this preliminary study, a number of additional questions emerged concerning how to measure graph quality and the data-ink ratio. Previous studies testing the data-ink ratio have focused on comparing data-ink levels and question response accuracy (Kelly, 1989; Gillian, 1994). However, the eye-tracking presents the opportunity for finding new ways to measure how people interpret graph quality visually.
One potential future research topic is to further study how people define graph quality and to compare patterns in eye-movement for graphs that are perceived as high-quality and graphs that are perceived as low-quality. In the subject responses, a number of subjects focused on the content of the graph itself and not its design when asked what they liked and disliked about the graphs. In a future study, it would be useful to explore how much perceived graph quality is connected to the content of a graph and how much it is connected to the graph's visual design.
Additionally, it would be useful to study whether people with different educational backgrounds and working memory capacities define graph “quality” in distinct ways (Shah & Carpenter, 1995). In this homogeneous group of well-educated subjects, common indicators of quality included the easiness of interpreting the data, and the clarity of the data labels. However further testing should be done to determine whether people from less-educated backgrounds identify the same items as graph quality indicators.
A final useful topic of study would be to gain a better understanding of the importance of the data-ink ratio for non-designers. When the data-ink ratio is compared to other graph design features such as graph color and graph title location, which feature do subjects rate as being the most important? Although the design community has identified the data-ink ratio as a key indicator of graph quality, further research should be done to determine whether non-designers agree with this design standard.
References
Card, S. K. The psychology of human-computer interaction / Stuart K. Card, Thomas P. Moran, Allen Newell: Hillsdale, N.J. : L. Erlbaum Associates, 1983.
Carpenter, P. A., & Shah, P. (1998). A model of the perceptual and conceptual processes in graph comprehension. Journal of Experimental Psychology-Applied, 4(2), 75- 100.
Cleveland, W. S., & McGill, R. (1985). Graphical Perception and Graphical Methods for Analyzing Scientific-Data. Science, 229(4716), 828-833.
Disabato , N. J. (2006). The effects of Ajax web technologies on user expectations: a workflow approach. A Master’s Paper for the M.S. in I.S degree. April, 2006. 51 pages
Gillan, D. J., & Richman, E. H. (1994). Minimalism and the Syntax of Graphs. Human Factors, 36(4), 619-644.
Hess, E.H. The Tell-Tale Eye: How Your Eyes Reveal Hidden Thoughts and Emotions. Van Nostrand Reinhold Co, Chicago, 1975.
Kelly, J. D. (1989). The Data-Ink Ratio and Accuracy of Newspaper Graphs. Journalism Quarterly, 66(3), 632-639.
Kosslyn, S. M. (1989). Understanding Charts and Graphs. Applied Cognitive Psychology, 3(3), 185-226.
Larkin, J. H., & Simon, H. A. (1987). Why a Diagram Is (Sometimes) Worth 10000 Words. Cognitive Science, 11(1), 65-99.
Levy, E., J. Zacks, et al. (1996). Gratuitous graphics? Putting preferences in perspective.
Proceedings of the SIGCHI conference on Human factors in computing systems:
common ground. Vancouver, British Columbia, Canada, ACM Press.
Lohse, G. L. (1997). The role of working memory on graphical information processing.
Behaviour & Information Technology, 16(6), 297-308.
Lohse, J. (1991). A cognitive model for the perception and understanding of graphs.
Proceedings of the SIGCHI conference on Human factors in computing systems:
Reaching through technology. New Orleans, Louisiana, United States, ACM Press.
Lord, C. G., Ross, L., & Lepper, M. R. (1979). Biased assimilation and attitude
polarization: The effects of prior theories on subsequently considered evidence.
Journal of Personality and Social Psychology. 37, 2098-2109.
Munro, G. D., & Ditto, P. H. (1997). Biased assimilation, attitude polarization, and affect in reactions to stereotype-relevant scientific information. Personality and Social Psychology Bulletin, 23(6), 636-653.
Jakob Nielsen (1997). How Users Read on the Web. Retrieved December 1, 2006 from Alertbox.com: http://www.useit.com/alertbox/9710a.html
Nunes, T. Street mathematics and school mathematics / Terezhina Nunes, Analucia Dias Schliemann, David William Carraher: Cambridge [England] ; New York, NY, USA : Cambridge University Press, 1993.
Pinker, S. (1990). A Theory of Graph Comprehension. In R. Freedle (Ed.), Artificial intelligence and the future of testing: Hillsdale, N.J. : L. Erlbaum Associates, c1990.
Shah, P., & Carpenter, P. A. (1995). Conceptual Limitations in Comprehending Line Graphs. Journal of Experimental Psychology-General, 124(1), 43-61.
Shah, P., & Hoeffner, J. (2002). Review of graph comprehension research: Implications for instruction. Educational Psychology Review, 14(1), 47-69.
Simkin, D., & Hastie, R. (1987). An Information-Processing Analysis of Graph Perception. Journal of the American Statistical Association, 82(398), 454-465. Teghtsoonian, M. (1965). The Judgment of Size. American Journal of Psychology, 78(3),
10.
Tufte, E. R. The visual display of quantitative information / Edward R. Tufte (2nd ed. ed.): Cheshire, Conn. : Graphics Press, c2001.
Wainer, H. (1984). How to Display Data Badly. American Statistician, 38(2), 137-147. Wainer, H., & Thissen, D. (1981). Graphical Data-Analysis. Annual Review of
Psychology, 32, 191-241.
Wickens, C. D., & Carswell, C. M. (1995). The proximity compatibility principle: its psychological foundation and relevance to display design. v37(n3), p473(422).
Appendix A University of North Carolina-Chapel Hill
Consent to Participate in a Research Study Adult Participants
Social Behavioral Form
________________________________________________________________________ IRB Study #_____________________
Consent Form Version Date: November 14, 2006 Title of Study: Graphs via Ink
Principal Investigator: Julia Kulla-Mader
UNC-Chapel Hill Department: School of Information and Library Science UNC-Chapel Hill Phone number: (919) 962-3701
Email Address: [email protected]
Funding Source: School of Information and Library Science
Study Contact telephone number: (919) 620-1442 Study Contact email: [email protected]
_________________________________________________________________ What are some general things you should know about research studies?
You are being asked to take part in a research study. To join the study is voluntary. You may refuse to join, or you may withdraw your consent to be in the study, for any reason, without penalty.
Research studies are designed to obtain new knowledge. This new information may help people in the future. You may not receive any direct benefit from being in the research study. There also may be risks to being in research studies.
Details about this study are discussed below. It is important that you understand this information so that you can make an informed choice about being in this research study. You will be given a copy of this consent form. You should ask the researchers named above, or staff members who may assist them, any questions you have about this study at any time.
What is the purpose of this study?
The purpose of this research study is to learn about how people interact with online graphs.
You should not be in this study if you have extensive experience with graphs, are a Ph.D. Student, or have a science or social science background.
How many people will take part in this study?
If you decide to be in this study, you will be one of approximately twelve people participating.
How long will your part in this study last?
The study will take between forty five minutes and one hour. No follow-up contact will be required.
What will happen if you take part in the study?
If you choose to participate in the study, you will first be fitted with an eye-tracking system headset. You will then complete a questionnaire detailing your experiences interacting with graphs online. Next, you will view three separate graphs and answer questions about each. Finally, you will complete a questionnaire comparing the graphs. Your use of the graphs during the study will be logged with a statistics and key logging software package.
What are the possible benefits from being in this study?
Research is designed to benefit society by gaining new knowledge.
What are the possible risks or discomforts involved from being in this study? There may be uncommon or previously unknown risks. You should report any problems to the researcher.
Participants are not at risk from the eye-tracking equipment. The eye-tracking equipment uses a camera positioned more than one foot from the subject to track eye movements and operates at a level at least an order of magnitude lower than what is considered to be safe for reflected light. The safe chronic ocular exposure estimates for near infrared exposure is 10 mW/sq.cm (David Sliney of US Army Environmental Hygiene Agency and Myron Worbarsht of Duke University Medical Center) and our ASL device uses between 0.1 and 0.3 mW/sq. cm on average. Note that near infrared is technically referred to as IR-A, light between the 760 and 1400 nanometer wavelength. The ASL system operates at the 850 nanometer wavelength. The head tracker receiver is
positioned 12-24 inches from the transmitter that produces a magnetic field between one- eighth and one-quarter (depending on head position) of the normal earth field of 0.6 gauss (manufacturer specifications). This configuration of eye and head tracker supplied by ASL Laboratories has found wide applications in government, university, and
corporate laboratories.
How will your privacy be protected?
Participants will not be identified in any report or publication about this study. Although every effort will be made to keep research records private, there may be times when
federal or state law requires the disclosure of such records, including personal
information. This is very unlikely, but if disclosure is ever required, UNC-Chapel Hill will take steps allowable by law to protect the privacy of personal information. In some cases, your information in this research study could be reviewed by representatives of the University, research sponsors, or government agencies for purposes such as quality control or safety.
Study participants will be assigned ID numbers. The names of the participants will not be recorded. The study records will be stored in a read/write/execute protected folder on the School of Information and Library Science server.
What will happen if you are injured by this research?
All research involves a chance that something bad might happen to you. This may include the risk of personal injury. In spite of all safety measures, you might develop a reaction or injury from being in this study. If such problems occur, the researchers will help you get medical care, but any costs for the medical care will be billed to you and/or your insurance company. The University of North Carolina at Chapel Hill has not set aside funds to pay you for any such reactions or injuries, or for the related medical care. However, by signing this form, you do not give up any of your legal rights.
Will you receive anything for being in this study?
Participants will receive a $5 gift certificate to the UNC bookstore. Participants who withdraw from the study before completing it will not be eligible for the gift certificate.
Will it cost you anything to be in this study? There will be no costs for being in the study What if you are a UNC student?
You may choose not to be in the study or to stop being in the study before it is over at any time. This will not affect your class standing or grades at UNC-Chapel Hill. You will not be offered or receive any special consideration if you take part in this research. What if you are a UNC employee?
Taking part in this research is not a part of your University duties, and refusing will not affect your job. You will not be offered or receive any special job-related consideration if you take part in this research.
What if you have questions about this study?
You have the right to ask, and have answered, any questions you may have about this research. If you have questions, or concerns, you should contact the researchers listed on the first page of this form.
What if you have questions about your rights as a research participant?
All research on human volunteers is reviewed by a committee that works to protect your rights and welfare. If you have questions or concerns about your rights as a research
subject you may contact, anonymously if you wish, the Institutional Review Board at 919-966-3113 or by email to [email protected].
- - - Participant’s Agreement:
I have read the information provided above. I have asked all the questions I have at this time. I voluntarily agree to participate in this research study.
_________________________________________ _________________ Signature of Research Participant Date
_________________________________________ Printed Name of Research Participant
_________________________________________ _________________ Signature of Person Obtaining Consent Date
_________________________________________ Printed Name of Person Obtaining Consent
Appendix B Pre-Study Questionnaire
Please answer the following demographic questions. How old are you?
18-25 26-34 35-54 55-64
65 and over
What is your gender? female male
What is the highest level of education you have completed? Less than High School
High School / GED Some College
2-year College Degree 4-year College Degree Master's Degree Doctoral Degree
Professional Degree (JD, MD)
In what discipline did you receive your undergraduate degree?
What graduate degree will you receive when you graduate? MSLS
MSIS PhD
How often do you go online?
Never
Every other Month Once a month
Once every 2 weeks Once a week
A Few Times a week Daily
How often do you look at graphs online? Never
Every other Month Once a month
Once every 2 weeks Once a week
A Few Times a week Daily
On a scale from 1 to 5 (5 being most expert) please rate
1 2 3 4 5 Your expertise
with computers Your expertise with graphs
Appendix C Graphs and Associated Questions
Graph 1
Graph 1 Questions
1. The highest number of people said that, yesterday, they spent: z Less than 15 minutes online
z Half hour or more but less than 1 hours online z About an hour online
z 2 hours or more but less than 3 hours online z 4 hours or more online
2. Approximately how many people said that they spent a half hour or more but less than 1 hour online yesterday:
z 50 z 100 z 150 z 200 z 250 z 300
3. True or False: More people spend 4 hours or more online than spent less than 15 minutes online
z True
Graph 1: Low Data-Ink Ratio
Graph 2
Graph 2 Questions
1. The highest number of people said that they use the Internet from work :