User Study Details
In a user experiment, we firstly investigated the readability of IPTPs compared to node-link diagrams both without color gradient and interactive features. The user study followed a within-subjects design. We chose node-link diagrams as source of comparison because those diagrams are the most widely used, well established, de-facto standard for hierarchy visualization. Each test of the visualizations included three dataset sizes and three tasks. Questions were designed so that subjects had to answer in forced-choice fashion.
A stochastic algorithm generated all datsets synthetically. The dataset construction was parameterized by the size of the hierarchy in terms of the number of vertices and the maximal depth. One constraint was that the presentation space was identical for both techniques. The experiment included three tasks:
- Find the least common ancestor of two leaf vertices.
- Check the existence of an identical subhierarchy elsewhere in the plot.
- Estimate which of two subhierarchies is the larger one.
All tasks are important when exploring attributes that are attached to all hierarchy levels since patterns in the set of attributes may be caused by corresponding hierarchy levels that may show similar hierarchical patterns. We chose the tasks as a result of a pilot study.
We chose a within-subjects study design with 30 participants. They had to answer questions that were recorded by an operator. Subjects could additionally mark preferences and provide comments by filling in questionnaires.
Environment Conditions and Technical Setup
The user experiment was conducted in a laboratory that was insulated from outside distractions. All visualizations were presented on a 24 inch Dell 2408 wfp ultrasharp TFT screen at a resolution of 1920 x 1080 pixels with 32bit color depth. To avoid wrong results the subjects' responses were recorded by an operator pushing two specially marked keys on a PC keyboard. We assume that the delay associated with the operator for every task execution and recording loop was approximately the same amount of time (fault tolerance < 100ms).
Thirty (23 male, 7 female) subjects were recruited. Gender was not considered a confounding factor for this study. Twenty-seven participants were undergraduate students of our university and three were graduate students. Twenty-three subjects were computer scientists and seven were engineers. The average age was 27 years (minimum 22, maximum 53). Subjects were paid 10 Euros for participating in the user experiment. Twelve stated that they were familiar with visualization techniques or had attended a lecture with this topic. Eighteen stated that they were not familiar with visualization techniques. All participants had normal or corrected-to-normal color vision, which we confirmed by an Ishihara test and a Snellen chart to estimate visual acuity.
First, subjects had to fill out a short questionnaire about age, field of study, and prior knowledge in visualization techniques. Then, they read a two-page instruction manual on IPTPs and node-link diagrams. After the participants were given time to read this tutorial, we did a practice run-through of the user tasks. The time duration of the complete training was 10 minutes. During this practice test, subjects could ask questions about the visualization technique and clarify potential problems or misinterpretations. We also used the practice test to confirm that the subjects understtod both IPTPs and node-link diagrams.
Then, we continued with the main evaluation that took between 15 to 20 minutes depending on the fitness of the subject. There was a "Give Up" option, but it was not used by the subjects. Tasks, tree sizes, and visualization types were randomized and balanced to compensate for learning effects. Each participant had to perform Task 1, Task 2, and Task 3 for each tree size and visualization type. One task consisted of seven trials per tree size. The time limit for every trial was 20 seconds.
In task 1, the child nodes were marked by red colored circles in case of node-link diagrams, and by red colored triangles in case of IPTPs. The area of circles and triangles is equal sized. Two possible ancestors were colored in green and blue. In task 2, one subhierarchy was marked with a red starting node. In task 3, the starting nodes of two subhierarchies were marked green and blue. Subjects had to respond with blue or green (task 1 and task 3) and Yes or No (task 2). Completion times and correctness of answers were recorded for the seven trials. This procedure resulted in a total of 126 measurements for each participant since we showed the combination of two visualization techniques, three tasks, three dataset sizes, and seven trials.
After the main evaluation, subjects were given a second questionnaire in which they marked their preferences in using one of the two visualization techniques. Finally, participants were given the opportunity to provide open, unconstraint comments.
Study Results (Completion Times for Node-Link Diagrams (NLDs) and Indented Pixel Tree Plots (IPTPs)
|Task 1||Task 2||Task 3|
|NLDs||1995 (545)||5328 (1634)||3576 (1215)|
|IPTPs||2016 (934)||5090 (1293)||4514 (1473)|
|NLDs||2110 (541)||5930 (1723)||5107 (2183)|
|IPTPs||2059 (987)||6156 (2018)||4500 (1166)|
|NLDs||2388 (562)||8685 (3001)||3325 (926)|
|IPTPs||2386 (1315)||8603 (2181)||3976 (1257)|