Menu hierarchies are useful in organizing information and reducing the
number of alternatives that must be considered at any one time. This advantage
is achieved with some cost. As seen in the last chapter, a major cost is the
added time to make menu selections at each level. However, a much more
significant cost is often accrued by menu depth. Items are buried under
successive levels of menus and the user may have to search up and down the tree
to locate the desired item. If menu frames give sufficient information
regarding the location of the target at each level and users make no mistakes,
there would be no problem. Unfortunately, this is not always the case.
The problem is that when items are clustered together in the menu hierarchy, category names must be assigned to those sets of items. In many cases the category names are not adequate descriptors of the subset. Categories such as "Options," "Tools," "Windows," "Functions," fail to guide inexperienced users. Menu frames provide insufficient information when:
(1) items within a cluster are not be adequately characterized by the category name;
(2) the user infers that additional items are in a category; or
(3) category names overlap.
Menu items are rarely worded such that users know for certain what selections are required to get to a particular target frame. Items are often vaguely or ambiguously worded so that at first users can at best make only educated guesses about what leads to what. With sufficient use of the system, users learn the idiosyncrasies of the menu and improve their performance. But until that point, users must search as best as they can. This problem is particularly troublesome in vast videotext systems in which one may develop familiarity with only parts of the tree. In such systems the probability of making one or more errors per search is quite high (Frankhuizen & Vrins, 1980). Whalen and Mason (1981) investigated the problems caused by three types of defects in menu systems: miscategorization of items, ambiguous category labels, and synonymous labels. The most serious defect was found to be miscategorization. Miscategorized items were rarely found by users. While ambiguous and synonymous labels reduce certainty in location, miscategorization produces total uncertainty. If an item is miscategorized, looking for it is like looking for a needle in a haystack.
This chapter investigates the problem of search in hierarchical menus. When searching for an item, users typically follow the most probable path down the hierarchy until either (a) they get to the bottom and the item is either found or not found or (b) they come to a menu frame that offers little or no possibility of leading to the desired item. The questions are "How do users redirect search when they come to a dead end, what menu structures facilitate search, and how do users differ in strategies and abilities in menu search?"
While performance on well-practiced menus is largely a function of response times along a relatively direct path, performance on unfamiliar and ill-defined menus is largely a function of the number of frames visited during the search. Search behavior may be characterized by the number of off-path choices, the amount of backtracking required, and the frequency of restarting the search. Depending on the menu structure, users may adopt different strategies for backtracking and restarting. Depth and breadth play an important part in search performance in difficult menu hierarchies.
9.1 Menu Focusing Through Structure
From the consumer's perspective there are different advantages to searching for an item at a department store or a shopping bazaar. Aside from selection of goods, the department store provides a focusing structure to direct the consumer to the areas of the store. The bazaar bewilders the naive shopper with a large number of specialty shops along crowded narrow streets. However, once in the right shop, the variety may provide an easy choice of a needed item. Certainly, the relative efficiency of shopping one or the other will depend on knowing what you are looking for. Moreover, the shopper's search strategy will differ greatly between a department store and a shopping bazaar. Similarly, the efficiency of searching a menu structure may depend on how well the target item is specified and the user's search strategy may vary with the type of menu structure.
It is clear that the depth vs. breadth trade-off has a strong effect on performance. However, it is also likely that other characteristics of the menu structure influence performance when depth is constant. Most real-world menu systems vary in breadth at different levels of the tree. The natural organization of the hierarchy may require groupings that make it top heavy, bottom heavy, or bulging in the middle. In some cases the designer may have sufficient leeway to organize the items in several equally meaningful ways. The question is "What sort of structure best facilitates search?" It may be best to have a broad range of alternatives at the top of the menu and focus in on a small number of choices at the bottom. Or it may be best to have a narrow range at the top and a broad selection at the bottom. McEwan (1981) observed that most navigational errors are made at or near the top of the menu hierarchy. This implies that the greatest effort in menu design should be directed at the top level menu.
9.1.1 Varying Menu Breadth. Can search in difficult hierarchical menus be facilitated by varying the breadth of the menu at particular points in the tree? Norman and Chin (1988) investigated menu structures having different shape while holding depth constant. Five structures were used (Figure 9.1).
The constant menu (4x4x4x4) served as a baseline of comparison since it represents the uniform symmetric menu. It has the advantage of providing the user with a fixed number of items at each level. The user knows that there will be four items at each level. Choice uncertainty and decision difficulty are evenly distributed across all levels of the tree.
The decreasing menu (8x8x2x2) gives a larger number of items at the beginning and narrows the range at the end. Users are initially faced with a broad range of categories leading to a small number of finer distinctions further on. This structure may have the advantage of helping to resolve the general location at the beginning of the search, but it provides little flexibility in search at the bottom.
The increasing menu (2x2x8x8) gives a small number of choices at the beginning and increases breadth at the end. Users are faced with a small number of general categories leading to a larger number of finer distinctions further on. The increasing menu may have an advantage if the top level choices are difficult. Furthermore, it provides search latitude at the bottom of the menu.
The convex menu (2x8x8x2) gives a narrow range at the top, increases in the middle, and then narrows at the bottom. Search latitude is provided at the intermediate levels of the menu. If users redirect search at intermediate levels this menu has an advantage. On the other hand, if users tend to get lost in the middle of the tree, this is a distinct disadvantage.
Finally, the concave menu (8x2x2x8) gives a broad range of choices at the beginning, narrows in the middle, and then broadens at the end. The concave menu has the advantage of search latitude at the beginning and end of a path. If users redirect search locally or at the top of the menu, the concave menu has a distinct advantage.
The 5 menu structures covered 256 gift items from a popular merchandising catalog. Items were first clustered into groups of 2 or 4. These clusters were successively grouped into clusters of 2, 4, or 8 and so on to generate the 5 menu structures shown in Figure 9.1. Category labels were generated to describe the items in the cluster. Figure 9.2 gives examples of the menu frames through one path of each of the 5 structures. Pilot studies were conducted to ensure that clusterings and semantic labels were equally meaningful for target items across the 5 structures.
Subjects searched the menu for 8 "explicit" targets in which the actual gift name was presented and for 8 "scenario" targets in which a scenario was given and subjects were asked to find the most appropriate gift for the situation. Figure 9.3 shows typical situations used in the scenario condition. Subjects continued to search until the designated target was located. If subjects selected a gift that was not designated as the target, they were told to continue searching. In the case of scenario targets, they were told that the item they selected was not in stock and that they had to search for another item that would be appropriate.
Subjects had two menu commands to traverse back up the tree: a "previous" command to move up one level at a time, and a "top" command to return to the beginning of the menu hierarchy.
Overall, scenario targets required twice as much time as explicit targets in all tree structures except the concave tree (8x2x2x8) where search times were nearly equal (Figure 9.4). For explicit targets, search times were relatively constant across menu structures and did not vary significantly. However, for scenario targets, search time was fastest for the concave menu (8x2x2x8). Search time was nearly twice as long for the convex menu (2x8x8x2). Although the increasing menu (2x2x8x8) resulted in faster times than the decreasing menu (8x8x2x2), the difference was not significant. The constant menu (4x4x4x4) appears to fall in the middle with the concave and increasing menus being faster and the convex and decreasing menus being slower.
The minimum number of frames traversed would be 4 given no wrong selections. Overall, subjects tended to visit an average of 8 additional frames per search (Figure 9.4). The pattern of results is highly similar to that for search time. Overall, scenario targets resulted in more frames traversed except for the concave menu (8x2x2x8). Groups did not differ significantly for explicit targets. However, for scenario targets there was a significant effect. The concave menu resulted in the fewest frames traversed and the convex in the largest number. The increasing menu resulted in significantly fewer frames traversed than the decreasing menu, and again the constant menu fell in the middle.
The superiority of the concave menu indicates that breadth is advantageous at the top and bottom of the menu. The fact that increasing menu was superior to the decreasing menu further indicates that breadth at the bottom is of prime importance. It may be that a wide number of choices at the top helps the user to formulate an idea for generating an item that meets the conditions of the scenario and at the bottom the specific listing of items may help to suggest a solution to the scenario. Furthermore, since users generally get lost in the middle of the tree rather than at the beginning or end, the concave menu reduces the probability of a wrong selection in the middle of the tree.
If users have an explicit target and merely need to locate it in the tree, the shape of the tree does not seem to matter as long as depth is constant. It is only when users are searching for a solution or a likely candidate to meet a set of conditions that breadth at the top and bottom makes a difference. Since this is the case with many database retrieval problems, menus for such systems should opt for a concave structure.
9.1.2 Decision Uncertainty Evidence for Breadth. Menu search is a problem because target items are in a sense hidden under successive layers of vague clues. Consequently, the user has to spend a lot of time peeking under the shells to find the pea. Large memory demands are placed on the user to remember where things are buried in the menu structure. Breadth, however, allows the system to lay the cards on the table, so to speak.
There is a fundamental difference between choices among subcategories in upper level menu frames and choices among items in terminal menu frames (Norman & Chin, 1988). The upper level menus involve uncertainty; whereas, the terminal menu frames provide choice certainty. Menu structures with greater breadth at the bottom reduce the total uncertainty of the menu. In all levels of the menu tree except the bottom, there is a degree of uncertainty as to whether a choice will lead to the target item. As selections are made by the user, the system reveals the next level of items.
Cognitive control in menu search involves the transmission of information from the user to the system and from the system to the user. In hierarchical menu structures user choice information is conveyed to the system each time the user makes a selection. Menu information is conveyed to the user each time the system displays a new set of options. User choice information resolves uncertainty as to what the user wants to do while menu information resolves uncertainty on the part of the user as to what an item means and where it leads.
Choice uncertainty depends on the perspective. From the system's perspective, uncertainty as to what the user will select is constant across all structures containing the same number of items. However, from the user's perspective uncertainty as to what leads to what is reduced by semantic labeling of items. If the labels are perfect (or the user had perfect memory of the menu structure), uncertainty would be zero. At the other extreme, if the category labels conveyed no information, then there would be maximal uncertainty at all but the terminal level. Of course, actual uncertainty falls somewhere between the two extremes. An analysis of choice uncertainty in the menu structures used by Norman and Chin (1988) using information theory (see Chapter 3) is shown in Table 9.1. It is interesting to note that choice uncertainty from the user perspective accurately predicts search performance (Figure 9.4). The increasing and concave menus were superior to the constant menu which was superior to the decreasing and the convex menus.
Table 9.1
Choice Uncertainty of 5 Menu Structures (from Norman & Chin, 1988)
This analysis reinforces the rule that breadth is most advantageous at the
terminal level of the menu tree where the specific names of the items are
listed rather than category labels. The guideline for designers should be to
maximize the number of alternatives at the bottom.
9.1.3 Number of Discrete Menu Frames. Menu structure not only affects depth and breadth but also the total number of menu frames in the system. Table 9.2 lists the menu structures used by various investigators. For example, in Miller's (1981) study the number of frames required varied from 1 to 63. In the study by Norman and Chin (1988), one menu required only 39 frames; whereas, another required 201. If the menu system is stored as discrete frames, a broader menu requires less storage space, particularly when each frame is stored with a title and other overhead and housekeeping information.
Table 9.2
Number of Frames Required for Different Menu Structures
From the user's perspective, each frame represents a bundle of information.
Frames need to be recognized and scanned. Familiarity with frames, acquired by
repeated exposure, aid users in recognizing and selecting items. With a large
number of menu frames it is difficult for the user to become familiar with all
of them. Part of the advantage of broad menus may be that there are only a
small number of menu frames for the user to learn.
The results of the four studies listed in Table 9.2 indicate a strong relationship between number of frames and performance time. Although number of frames is confounded with menu depth in all but the Norman and Chin (1988) and Kiger (1984) studies, the results suggest that number of menu frames serves as a good predictor of performance. Designers should strive to reduce the overall number of frames required in the system. A small number of highly familiar, albeit broad, menu frames may prove to be superior to a large number of infrequently encountered menus.
9.2 Patterns of Search
What do users do when after a series of selections, they come to the terminal level of the menu hierarchy and do not find the target? They may give up, believing that the desired information or function is not a part of the database or the system. Whalen and Latrémouille (1981) note that searches may be prematurely terminated in poorly designed menu trees if users are not sure that the item exists. On the other hand, if there is reason to believe that the item is somewhere in the tree, users may continue the search in other places.
Search may be redirected in several ways. The user may back up the menu tree one level at a time to a point where another branch downward is selected. On the other hand, the user may simply start over again at the top of the tree. The structure of the tree determines to some extent the most efficient way in which it may be searched. Some parts or levels of the tree may provide better points from which to redirect search. These may be points that offer users the highest probability of finding a successful path or they may be points that are cognitive landmarks.
9.2.1 Reposition to Breadth. One strategy for redirecting search is to move to a level of the menu tree that affords the greatest breadth of choice. The study by Norman and Chin (1988) varied the breadth of the menu tree at different levels. The number of "previous" and "top" commands issued by subjects searching for explicit or scenario targets were recorded. The means are shown in Figure 9.5 for each of the five menu structures searched.
It can be seen that the type of repositioning varied greatly among the types of menu structures. The most "previous" commands were issued in the convex menu (2x8x8x2) for both explicit and scenario targets. It would appear that subjects gravitated toward the greatest concentration of breadth in the tree. Interestingly, the fewest "previous" commands were issued in the concave menu (8x2x2x8) for scenario targets. Once down at the bottom of the tree, moving to a previous frame provided only a binary choice except at the top of the menu.
The pattern of "top" commands also suggests that subjects repositioned their search at the greatest breadth of the tree. For explicit targets, the greatest number of "top" commands were issued in the concave tree (8x2x2x8). For scenario targets, the least number were issued in the increasing tree (2x2x8x8) where the top of the menu provided the least breadth.
It would appear that when a particular path has failed, users attempt to reposition the search at a level that affords the greatest opportunity to locate a more likely path. The greater the number of items the greater the probability of making a better selection. Consequently, users gravitate toward greater breadth whether at the top of the tree or further down and use the menu commands to reposition search to that point. These results, and those in the previous chapter, suggest that menus should be designed with maximum breadth. However, when menu systems do vary in breadth, they might be designed so that if the user needs to backtrack, one command would automatically reposition search to a higher level giving the greatest breadth of choice. Thus, if a user were at the terminal level of an 8x8x2x2, the command would reposition search to the second level (equivalent to two "previous" commands).
9.2.2 Reposition to High Probability Paths. In general, users backtrack to the point where they suspect that they took a wrong turn. Most likely this is a point at which there was uncertainty in choice. Thus, if a user were fairly sure about a choice, that frame would not be a good point to reposition search. But if the choice was a toss-up, then that frame would be a good candidate for repositioning.
Rather than using actual menus in which the perceived confidence of choice would vary from user to user, Norman and Butler (1989) generated menus which listed the initial probabilities that targets would be found by selecting each item. Probability distributions were varied to favor different levels. Subjects were asked to search for a file in a hierarchical filing system. A mental model of hierarchical filing was given. Files were described as being located in folders in drawers of file cabinets. Subjects were told that they could only go by the probabilities for determining where the file might be. Searches began by selecting the cabinet, then a drawer, and finally a folder. Once the folder was selected, the subject was told if it contained the desired file or not. If it did not, the subject was to continue to search. At any level the subject could decide to switch cabinets, doors within the same cabinet, or files within the same door (Figure 9.6).
Patterns of search indicated that subjects tended to perform local searches at the bottom of the hierarchy. Specifically, once within a drawer, they would check all of its folders in descending order of their probabilities. When folders were exhausted, they tended to reposition the search at either the top level (cabinet) or middle level (drawer). The level selected depended on the location of the next highest probability. For example, subjects would move to set {.6, .3, .1} and select the .3 alternative before moving to set {.6, .2, .2} and selecting either .2 alternative. Thus, subjects followed a conditional probability model in which they selected paths based on updated probabilities.
An additional finding was that although searches tended to be systematic at first, subjects tended to repeat paths and forget where they had looked before. When targets were at low probability branches, searches took much longer than a systematic algorithm either using the given probabilities or random probabilities. For example, in the case of the 3x3x3 menu the expected number of frames visited would be 13.5 (27/2). The average number of frames visited by subjects was much greater.
Obviously, designers seek to avoid uncertainty in real menus. Unfortunately, few menus are perfect and many are fraught with uncertainty at numerous points. Since users seek to reposition search at points of uncertainty, it may be useful to provide a "bread crumb" capability to the menu. When users come to a point in the menu where uncertainty is high, they may issue a "bread crumb" command to mark that frame. If they further traverse the menu and do not find the target, they can activate the "bread crumb" command to bring them back to the frame in question.
In cases where search is particularly difficult, the user may be aided by efficient search algorithms supplied by the computer based on subjective probabilities supplied by the user. Rather than making an overt selection, the user would input a probability distribution for the alternatives in each frame traversed. The menu system would then make selections automatically based on these probabilities, updating them when paths are exhausted, or allowing the user to update them based on new information. Whenever new frames are encountered, the system would solicit subjective probabilities from the user. Although time consuming, the advantage of systematic search might outweigh the burden of inputing probabilities on the part of the user.
9.2.3 Reposition to Cognitive Landmarks. When menus are organized according to a semantic network, there are generally pivotal landmarks at which fundamental turns are taken. When we give directions to geographical locations, we generally use landmarks. If one gets lost, one can return to the landmark in order to redirect search. Major intersections, highways, and shopping centers serve this purpose. Similarly, in menu selection systems, users who have a sufficient mental model of the system use its cognitive landmarks to find their way around. When they get lost or fail to find a target item, they tend to return to focal points of the system. Landmarks tend to be frequently accessed nodes in the system. Furthermore they tend to be transition points from one type of environment or state to another. For example, if the menu of the animal kingdom shown in Figure 8.1 were continued down to the species, one might want to reposition to another genus, order, phylum, or kingdom. It would be less likely that one would want to reposition to suborder, subclass, or subkingdom. To facilitate this, the system could include the alternatives "Return to Genus," "Return to Order," "Return to Phylum," "Return to Kingdom."
The top or root node of a hierarchical menu system is by its nature a landmark. Similarly, the "Home Card" in stackware applications is a landmark. Consequently, when users get lost in the system, they are more likely to reposition to the root node or home card. Unfortunately, this can be very unproductive when a local search is required. If one were in a department store looking for an electronic blender and came to a table of electric can openers, he or she would not reposition the search by going out of the store and starting over again. Instead one would back up to the level of electric kitchen appliances and scan the selections from there. Unfortunately, too often users reposition in computer menu systems by returning to the top for lack of better cognitive landmarks (Norman & Chin, 1988; Norman, 1988).
Many complex office automation systems, electronic mail systems and the like contain a number of focal points toward which users gravitate. A few highly familiar, distinctive screens serve as landmarks to start and end local search processes. Systems which present more global views of the environment make use of such landmarks. For example, in the Rooms environment such landmarks are displayed graphically. The more salient such landmarks can be made the better they will serve as focal points in search processes. Repositioning to such points rather than all the way back to a root node should greatly improve search performance.
9.3 Individual Differences in Search Behavior
Tremendous variability exists in performance, style of search, cognitive ability, knowledge, and prior experience among users. This is particularly true in the area of search behavior. The average time to find items in hierarchical data bases by different users can vary by an order of magnitude. Researchers are interested in both characterizing this variability (e.g., in what ways and to what extent do users vary?) and determining its sources (e.g., what person variables affect search performance?).
9.3.1. Characterizing Differences. Individual differences can be conceptually divided into inherent differences in perceptual ability, memory capacity, and cognitive processing and acquired differences in specific knowledge of the subject matter domain and general knowledge of problem solving strategies. In terms of characterizing individual differences among computer users, inherent differences affect users' ability to scan choices, perceive patterns, remember items and paths, and plan paths. Acquired differences affect performance in that users learn information and strategies over experience that facilitates performance. Research on the development of expertise helps to characterize the differences between novices and experts.
Forward Search versus Backward Search. Larkin (1980) found that novices and experts search for solutions to geometry proofs and physics problems in quite different ways. Experts tend to reason from the givens to the goal in a forward search. Novices on the other hand, reason from the goal to the givens in a backward search. Larkin speculates that the direction of search is the result of the subjects record of past experience. In the past, particular configurations of information resulted in patterns of successful inference. These patterns may have originally been discovered in backward search, but once discovered, subjects begin to transform them to forward production rules.
Although Larkin's studies were on geometry and physics problems, the idea clearly generalizes to search in difficult hierarchical menu structures. In general, hierarchical menu structures promote forward search; however, when novice users are more familiar with the target item than they are with the forward path, backward search may be used. If novices are searching for an explicit target item, they may initially try to think backwards up the tree, "What do I select to get X? If Y leads to X, what do I select to get Y?" In general, this line of reasoning is time consuming and fraught with problems. However, with experience, the user may acquire forward production rules that start at the top of menu hierarchy. Search, and the reasoning processes that support it, are then performed in the same direction as the intended hierarchical structure of the menu.
Recognition of Recurring Patterns. Studies show that experts tend to perceive recurring patterns much more than novices. Experts are more likely to see clusters of items, to chunk things into larger perceptual units, and to pick up on the overall organization of things. Experience results in a large repertoire of patterns that can be applied to current problems.
Simon and Gilmartin (1973) estimate that chess masters have acquired something on the order of 50,000 different chess patterns. These patterns are quickly recognized during play, and this accounts for the superior memory performance of experts over novices when asked to reconstruct board positions. Similarly for computer programming, experts show better memory. It is argued that the development of programming expertise depends on the acquisition of a large number of patterns or templates (Soloway, 1980). Programmers associate these templates with the goal of the program. Given a goal, they can generate the appropriate templates; and given a novel program, they infer its function from the templates that it contains.
In menu search, expert users, no doubt, perceive recurring patterns of menu structure, common menus, types of menu screens, and paths through menu structures. Experts are able to recognize the intent of menus, identify potential paths for search, and search chunks of menu structures rather than individual items. Extensive familiarity with computer menus allows experts to direct search more efficiently by eliminating large sections of the menu hierarchy and inferring possible locations from past experience. Well-designed interfaces promote rapid expertise by enhancing the commonality of menu patterns and screens across the whole interface.
Planning. People differ greatly in the degree to which they plan before they act. Planners work out solution and contingency plans in great detail before they attempt implementation. Others jump immediately into action letting trial and error guide them. In a way, menu selection appeals to the trial and error method. However, trial and error as a method of search becomes impractical (a) when the menu system is very extensive and (b) when computer response time is very slow. In these cases, prior planning is imperative. By formulating a plan, the user accomplishes much of the search mentally rather then overtly. The plan may eliminate unfruitful branches, determine the most direct path to the target, and incorporate backup search procedures if the item is not found on the first try.
Planning ability depends greatly on the user's knowledge of the system and on the user's cognitive ability to transform, manipulate, and process the mental elements that represent the system. Planning requires a certain degree of expertise in the system structure and the tendency to think out a course of action in depth rather than pursue a breadth of immediately available alternatives.
9.3.2. Predictors of Search Performance. Direction of search, recognition of patterns, and degree of planning behavior help to characterize differences between expert and novice users in terms of the acquired knowledge. The next question is whether there are inherent differences among users that predict search performance in complex hierarchical menus.
Vicente, Hayes and Williges (1987) conducted an extensive study on individual differences in the ability to find information in a hierarchical arrangement of files. A large number of candidate factors were investigated which are listed in Table 9.3. The six spatial abilities were assessed using the Kit of Factor-Referenced Cognitive Tests (Ekstrom, French, and Harmon, 1976). Reading rate, vocabulary, and comprehension were assessed using the Nelson-Denny Reading Test (1973). The demographic variables were assessed by simply asking the subjects. Abstractness was assessed by the Abstract Orientation Scale (O'Connor, 1972) and field dependency was assessed using the Embedded Figures Test (1977). Anxiety was assessed using the State-Trait Anxiety Inventory (1983). Finally, information processing rate was assessed by giving a choice reaction time test among 2, 4, or 8 alternatives and calculating the slope of the Hick's Law function for one, two, and three bits of uncertainty (Wickens, 1984).
Table 9.3
Candidate Person Variables Investigated by Vicente, Hayes and Williges (1987).
Six person variables correlated significantly with the performance variable of time, total number of commands and/or number of different commands (see Table 9.4). The variable of time seemed to be the most diagnostic performance variable. A regression analysis indicated that the best prediction equation for time included only the variables of vocabulary and spatial visualization. Other variables did not contribute substantially once these were taken into account. For example, computer experience had a correlation of -.34 (p = .06) with time; however, when spatial visualization was partialled out, computer experience had a negligible correlation with time. Either those with computer experience gained a higher spatial visualization or those with higher spatial visualization gained more computer experience. All that can be concluded is that there appears to be a relationship between the two variables and that spatial visualization was a better predictor.
Table 9.4
Correlations between Person Variables and Performance Variables Investigated by Vicente, Hayes and Williges (1987).
* p < .05
** p < .01
*** p < .001
The difference in performance between subjects with low and high spatial visualization ability was quite dramatic. Spatial visualization accounted for approximately 25% of all individual differences. Furthermore the magnitude of the difference is shown in Figure 9.7. A median split of the subjects into low and high groups of equal size reveals that low visualization subjects took nearly twice as long as high visualization subjects.
Vicente et al. attempted to isolate the components of the search process that seemed to account for the difference in times to locate information. They found that of 12 menu commands available, subjects with low spatial ability tended to use three of them significantly more than subjects of high spatial ability. One command was a procedure to move up one level of the hierarchy (ZOOM OUT). Subjects with low spatial ability tended to go to incorrect files and then had to back up the hierarchy. In addition, these subjects used two commands for searching through the files (SCROLL UP and SCROLL DOWN) significantly more often than subjects with high spatial ability. Vicente et al. conclude that subjects with low spatial ability tended to get lost more often both searching for the right file and searching for the right information in a file. Subjects with low spatial ability may have had a harder time visualizing paths between files and visualizing the location of information within files. Consequently, rather than manipulating the structures mentally in order to plan and test paths, they did so directly resulting in longer search times and a greater number of commands issued.
9.6. Summary
With the increasing complexity of menu selection in information retrieval systems and in command and control systems, it can no longer be assumed that menu selection will guide the user along an error free path. Consequently, when the first try doesn't work, users must rely on strategies for redirecting search. The concern of the designer should be to identify ways that the system can help to redirect and facilitate that search.
A major determinant of effective search is breadth of selection at the bottom of the menu hierarchy. The resulting guideline is that menu structures should be designed so that they are bottom heavy. Once the user has traversed down to the bottom of the menu, a wide range of alternatives should be selectable. Users tend to redirect search to a level that provides the greatest breadth of choice. Since search is generally redirected after a failure at the bottom of the tree, it makes sense to provide breadth at the same level.
A secondary effect is that breadth at the top of the menu facilitates search. The opening menu that provides a wide range alternatives is superior to a limited, focused menu. Since search is often redirected to the top of the menu, it makes sense to provide a number of alternative search routes at this point. On the other hand, menu breadth in middle levels of the structure seems to hinder search. It is here that search may need to be focused so as to reduce the probability path errors.
Menu structure determines the number of discrete frames in the system. It is suggested that menus that minimize the total number of menu frames may facilitate search in that the user has fewer frames to search and may have a greater likelihood of remembering the critical menu frames that lead to the target item.
Users vary greatly in their ability to locate information in hierarchical menu structures. As users gain experience with a system their search behavior improves. Expertise is characterized by (a) a forward search from the given situation to the target item rather than a backward search from the target back up the hierarchy, (b) the ability to recognize recurring patterns in menu structures, and (c) the ability to plan out a search path. Finally, the best predictors of search performance are (a) spatial visualization ability and (b) vocabulary and comprehension abilities.
The problem is that when items are clustered together in the menu hierarchy, category names must be assigned to those sets of items. In many cases the category names are not adequate descriptors of the subset. Categories such as "Options," "Tools," "Windows," "Functions," fail to guide inexperienced users. Menu frames provide insufficient information when:
(1) items within a cluster are not be adequately characterized by the category name;
(2) the user infers that additional items are in a category; or
(3) category names overlap.
Menu items are rarely worded such that users know for certain what selections are required to get to a particular target frame. Items are often vaguely or ambiguously worded so that at first users can at best make only educated guesses about what leads to what. With sufficient use of the system, users learn the idiosyncrasies of the menu and improve their performance. But until that point, users must search as best as they can. This problem is particularly troublesome in vast videotext systems in which one may develop familiarity with only parts of the tree. In such systems the probability of making one or more errors per search is quite high (Frankhuizen & Vrins, 1980). Whalen and Mason (1981) investigated the problems caused by three types of defects in menu systems: miscategorization of items, ambiguous category labels, and synonymous labels. The most serious defect was found to be miscategorization. Miscategorized items were rarely found by users. While ambiguous and synonymous labels reduce certainty in location, miscategorization produces total uncertainty. If an item is miscategorized, looking for it is like looking for a needle in a haystack.
This chapter investigates the problem of search in hierarchical menus. When searching for an item, users typically follow the most probable path down the hierarchy until either (a) they get to the bottom and the item is either found or not found or (b) they come to a menu frame that offers little or no possibility of leading to the desired item. The questions are "How do users redirect search when they come to a dead end, what menu structures facilitate search, and how do users differ in strategies and abilities in menu search?"
While performance on well-practiced menus is largely a function of response times along a relatively direct path, performance on unfamiliar and ill-defined menus is largely a function of the number of frames visited during the search. Search behavior may be characterized by the number of off-path choices, the amount of backtracking required, and the frequency of restarting the search. Depending on the menu structure, users may adopt different strategies for backtracking and restarting. Depth and breadth play an important part in search performance in difficult menu hierarchies.
9.1 Menu Focusing Through Structure
From the consumer's perspective there are different advantages to searching for an item at a department store or a shopping bazaar. Aside from selection of goods, the department store provides a focusing structure to direct the consumer to the areas of the store. The bazaar bewilders the naive shopper with a large number of specialty shops along crowded narrow streets. However, once in the right shop, the variety may provide an easy choice of a needed item. Certainly, the relative efficiency of shopping one or the other will depend on knowing what you are looking for. Moreover, the shopper's search strategy will differ greatly between a department store and a shopping bazaar. Similarly, the efficiency of searching a menu structure may depend on how well the target item is specified and the user's search strategy may vary with the type of menu structure.
It is clear that the depth vs. breadth trade-off has a strong effect on performance. However, it is also likely that other characteristics of the menu structure influence performance when depth is constant. Most real-world menu systems vary in breadth at different levels of the tree. The natural organization of the hierarchy may require groupings that make it top heavy, bottom heavy, or bulging in the middle. In some cases the designer may have sufficient leeway to organize the items in several equally meaningful ways. The question is "What sort of structure best facilitates search?" It may be best to have a broad range of alternatives at the top of the menu and focus in on a small number of choices at the bottom. Or it may be best to have a narrow range at the top and a broad selection at the bottom. McEwan (1981) observed that most navigational errors are made at or near the top of the menu hierarchy. This implies that the greatest effort in menu design should be directed at the top level menu.
9.1.1 Varying Menu Breadth. Can search in difficult hierarchical menus be facilitated by varying the breadth of the menu at particular points in the tree? Norman and Chin (1988) investigated menu structures having different shape while holding depth constant. Five structures were used (Figure 9.1).
The constant menu (4x4x4x4) served as a baseline of comparison since it represents the uniform symmetric menu. It has the advantage of providing the user with a fixed number of items at each level. The user knows that there will be four items at each level. Choice uncertainty and decision difficulty are evenly distributed across all levels of the tree.
The decreasing menu (8x8x2x2) gives a larger number of items at the beginning and narrows the range at the end. Users are initially faced with a broad range of categories leading to a small number of finer distinctions further on. This structure may have the advantage of helping to resolve the general location at the beginning of the search, but it provides little flexibility in search at the bottom.
The increasing menu (2x2x8x8) gives a small number of choices at the beginning and increases breadth at the end. Users are faced with a small number of general categories leading to a larger number of finer distinctions further on. The increasing menu may have an advantage if the top level choices are difficult. Furthermore, it provides search latitude at the bottom of the menu.
The convex menu (2x8x8x2) gives a narrow range at the top, increases in the middle, and then narrows at the bottom. Search latitude is provided at the intermediate levels of the menu. If users redirect search at intermediate levels this menu has an advantage. On the other hand, if users tend to get lost in the middle of the tree, this is a distinct disadvantage.
Finally, the concave menu (8x2x2x8) gives a broad range of choices at the beginning, narrows in the middle, and then broadens at the end. The concave menu has the advantage of search latitude at the beginning and end of a path. If users redirect search locally or at the top of the menu, the concave menu has a distinct advantage.
The 5 menu structures covered 256 gift items from a popular merchandising catalog. Items were first clustered into groups of 2 or 4. These clusters were successively grouped into clusters of 2, 4, or 8 and so on to generate the 5 menu structures shown in Figure 9.1. Category labels were generated to describe the items in the cluster. Figure 9.2 gives examples of the menu frames through one path of each of the 5 structures. Pilot studies were conducted to ensure that clusterings and semantic labels were equally meaningful for target items across the 5 structures.
Subjects searched the menu for 8 "explicit" targets in which the actual gift name was presented and for 8 "scenario" targets in which a scenario was given and subjects were asked to find the most appropriate gift for the situation. Figure 9.3 shows typical situations used in the scenario condition. Subjects continued to search until the designated target was located. If subjects selected a gift that was not designated as the target, they were told to continue searching. In the case of scenario targets, they were told that the item they selected was not in stock and that they had to search for another item that would be appropriate.
Subjects had two menu commands to traverse back up the tree: a "previous" command to move up one level at a time, and a "top" command to return to the beginning of the menu hierarchy.
Overall, scenario targets required twice as much time as explicit targets in all tree structures except the concave tree (8x2x2x8) where search times were nearly equal (Figure 9.4). For explicit targets, search times were relatively constant across menu structures and did not vary significantly. However, for scenario targets, search time was fastest for the concave menu (8x2x2x8). Search time was nearly twice as long for the convex menu (2x8x8x2). Although the increasing menu (2x2x8x8) resulted in faster times than the decreasing menu (8x8x2x2), the difference was not significant. The constant menu (4x4x4x4) appears to fall in the middle with the concave and increasing menus being faster and the convex and decreasing menus being slower.
The minimum number of frames traversed would be 4 given no wrong selections. Overall, subjects tended to visit an average of 8 additional frames per search (Figure 9.4). The pattern of results is highly similar to that for search time. Overall, scenario targets resulted in more frames traversed except for the concave menu (8x2x2x8). Groups did not differ significantly for explicit targets. However, for scenario targets there was a significant effect. The concave menu resulted in the fewest frames traversed and the convex in the largest number. The increasing menu resulted in significantly fewer frames traversed than the decreasing menu, and again the constant menu fell in the middle.
The superiority of the concave menu indicates that breadth is advantageous at the top and bottom of the menu. The fact that increasing menu was superior to the decreasing menu further indicates that breadth at the bottom is of prime importance. It may be that a wide number of choices at the top helps the user to formulate an idea for generating an item that meets the conditions of the scenario and at the bottom the specific listing of items may help to suggest a solution to the scenario. Furthermore, since users generally get lost in the middle of the tree rather than at the beginning or end, the concave menu reduces the probability of a wrong selection in the middle of the tree.
If users have an explicit target and merely need to locate it in the tree, the shape of the tree does not seem to matter as long as depth is constant. It is only when users are searching for a solution or a likely candidate to meet a set of conditions that breadth at the top and bottom makes a difference. Since this is the case with many database retrieval problems, menus for such systems should opt for a concave structure.
9.1.2 Decision Uncertainty Evidence for Breadth. Menu search is a problem because target items are in a sense hidden under successive layers of vague clues. Consequently, the user has to spend a lot of time peeking under the shells to find the pea. Large memory demands are placed on the user to remember where things are buried in the menu structure. Breadth, however, allows the system to lay the cards on the table, so to speak.
There is a fundamental difference between choices among subcategories in upper level menu frames and choices among items in terminal menu frames (Norman & Chin, 1988). The upper level menus involve uncertainty; whereas, the terminal menu frames provide choice certainty. Menu structures with greater breadth at the bottom reduce the total uncertainty of the menu. In all levels of the menu tree except the bottom, there is a degree of uncertainty as to whether a choice will lead to the target item. As selections are made by the user, the system reveals the next level of items.
Cognitive control in menu search involves the transmission of information from the user to the system and from the system to the user. In hierarchical menu structures user choice information is conveyed to the system each time the user makes a selection. Menu information is conveyed to the user each time the system displays a new set of options. User choice information resolves uncertainty as to what the user wants to do while menu information resolves uncertainty on the part of the user as to what an item means and where it leads.
Choice uncertainty depends on the perspective. From the system's perspective, uncertainty as to what the user will select is constant across all structures containing the same number of items. However, from the user's perspective uncertainty as to what leads to what is reduced by semantic labeling of items. If the labels are perfect (or the user had perfect memory of the menu structure), uncertainty would be zero. At the other extreme, if the category labels conveyed no information, then there would be maximal uncertainty at all but the terminal level. Of course, actual uncertainty falls somewhere between the two extremes. An analysis of choice uncertainty in the menu structures used by Norman and Chin (1988) using information theory (see Chapter 3) is shown in Table 9.1. It is interesting to note that choice uncertainty from the user perspective accurately predicts search performance (Figure 9.4). The increasing and concave menus were superior to the constant menu which was superior to the decreasing and the convex menus.
Table 9.1
Choice Uncertainty of 5 Menu Structures (from Norman & Chin, 1988)
| Menu Structure | System Perspective | User Perspective |
|---|---|---|
| Constant | 2+2+2+2 = 8 | 2+2+2+0 = 6 |
| Decreasing | 3+3+1+1 = 8 | 3+3+1+0 = 7 |
| Increasing | 1+1+3+3 = 8 | 1+1+3+0 = 5 |
| Convex | 1+3+3+1 = 8 | 1+3+3+0 = 7 |
| Concave | 3+1+1+3 = 8 | 3+1+1+0 = 5 |
9.1.3 Number of Discrete Menu Frames. Menu structure not only affects depth and breadth but also the total number of menu frames in the system. Table 9.2 lists the menu structures used by various investigators. For example, in Miller's (1981) study the number of frames required varied from 1 to 63. In the study by Norman and Chin (1988), one menu required only 39 frames; whereas, another required 201. If the menu system is stored as discrete frames, a broader menu requires less storage space, particularly when each frame is stored with a title and other overhead and housekeeping information.
Table 9.2
Number of Frames Required for Different Menu Structures
| Frames at Level | |||||||
|---|---|---|---|---|---|---|---|
| Menu Structure | 1 | 2 | 3 | 4 | 5 | 6 | Total |
| Miller (1981); Parkinson et al (1983); Kiger (1984) | |||||||
| 26 | 1 | 2 | 4 | 8 | 16 | 32 | 63 |
| 43 | 1 | 4 | 16 | - | - | - | 21 |
| 82 | 1 | 8 | - | - | - | - | 9 |
| Parkinson et al (1983) | |||||||
| 641 | 1 | - | - | - | - | - | 1 |
| Kiger 1984) | |||||||
| 4x16 | 1 | 4 | - | - | - | - | 5 |
| 16x4 | 1 | 16 | - | - | - | - | 17 |
| Norman & Chin (1988) | |||||||
| 4x4x4x4 | 1 | 4 | 16 | 64 | - | - | 85 |
| 8x8x2x2 | 1 | 8 | 64 | 128 | - | - | 201 |
| 2x2x8x8 | 1 | 2 | 4 | 32 | - | - | 39 |
| 2x8x8x2 | 1 | 2 | 16 | 128 | - | - | 147 |
| 8x2x2x8 | 1 | 8 | 16 | 32 | - | - | 57 |
The results of the four studies listed in Table 9.2 indicate a strong relationship between number of frames and performance time. Although number of frames is confounded with menu depth in all but the Norman and Chin (1988) and Kiger (1984) studies, the results suggest that number of menu frames serves as a good predictor of performance. Designers should strive to reduce the overall number of frames required in the system. A small number of highly familiar, albeit broad, menu frames may prove to be superior to a large number of infrequently encountered menus.
9.2 Patterns of Search
What do users do when after a series of selections, they come to the terminal level of the menu hierarchy and do not find the target? They may give up, believing that the desired information or function is not a part of the database or the system. Whalen and Latrémouille (1981) note that searches may be prematurely terminated in poorly designed menu trees if users are not sure that the item exists. On the other hand, if there is reason to believe that the item is somewhere in the tree, users may continue the search in other places.
Search may be redirected in several ways. The user may back up the menu tree one level at a time to a point where another branch downward is selected. On the other hand, the user may simply start over again at the top of the tree. The structure of the tree determines to some extent the most efficient way in which it may be searched. Some parts or levels of the tree may provide better points from which to redirect search. These may be points that offer users the highest probability of finding a successful path or they may be points that are cognitive landmarks.
9.2.1 Reposition to Breadth. One strategy for redirecting search is to move to a level of the menu tree that affords the greatest breadth of choice. The study by Norman and Chin (1988) varied the breadth of the menu tree at different levels. The number of "previous" and "top" commands issued by subjects searching for explicit or scenario targets were recorded. The means are shown in Figure 9.5 for each of the five menu structures searched.
It can be seen that the type of repositioning varied greatly among the types of menu structures. The most "previous" commands were issued in the convex menu (2x8x8x2) for both explicit and scenario targets. It would appear that subjects gravitated toward the greatest concentration of breadth in the tree. Interestingly, the fewest "previous" commands were issued in the concave menu (8x2x2x8) for scenario targets. Once down at the bottom of the tree, moving to a previous frame provided only a binary choice except at the top of the menu.
The pattern of "top" commands also suggests that subjects repositioned their search at the greatest breadth of the tree. For explicit targets, the greatest number of "top" commands were issued in the concave tree (8x2x2x8). For scenario targets, the least number were issued in the increasing tree (2x2x8x8) where the top of the menu provided the least breadth.
It would appear that when a particular path has failed, users attempt to reposition the search at a level that affords the greatest opportunity to locate a more likely path. The greater the number of items the greater the probability of making a better selection. Consequently, users gravitate toward greater breadth whether at the top of the tree or further down and use the menu commands to reposition search to that point. These results, and those in the previous chapter, suggest that menus should be designed with maximum breadth. However, when menu systems do vary in breadth, they might be designed so that if the user needs to backtrack, one command would automatically reposition search to a higher level giving the greatest breadth of choice. Thus, if a user were at the terminal level of an 8x8x2x2, the command would reposition search to the second level (equivalent to two "previous" commands).
9.2.2 Reposition to High Probability Paths. In general, users backtrack to the point where they suspect that they took a wrong turn. Most likely this is a point at which there was uncertainty in choice. Thus, if a user were fairly sure about a choice, that frame would not be a good point to reposition search. But if the choice was a toss-up, then that frame would be a good candidate for repositioning.
Rather than using actual menus in which the perceived confidence of choice would vary from user to user, Norman and Butler (1989) generated menus which listed the initial probabilities that targets would be found by selecting each item. Probability distributions were varied to favor different levels. Subjects were asked to search for a file in a hierarchical filing system. A mental model of hierarchical filing was given. Files were described as being located in folders in drawers of file cabinets. Subjects were told that they could only go by the probabilities for determining where the file might be. Searches began by selecting the cabinet, then a drawer, and finally a folder. Once the folder was selected, the subject was told if it contained the desired file or not. If it did not, the subject was to continue to search. At any level the subject could decide to switch cabinets, doors within the same cabinet, or files within the same door (Figure 9.6).
Patterns of search indicated that subjects tended to perform local searches at the bottom of the hierarchy. Specifically, once within a drawer, they would check all of its folders in descending order of their probabilities. When folders were exhausted, they tended to reposition the search at either the top level (cabinet) or middle level (drawer). The level selected depended on the location of the next highest probability. For example, subjects would move to set {.6, .3, .1} and select the .3 alternative before moving to set {.6, .2, .2} and selecting either .2 alternative. Thus, subjects followed a conditional probability model in which they selected paths based on updated probabilities.
An additional finding was that although searches tended to be systematic at first, subjects tended to repeat paths and forget where they had looked before. When targets were at low probability branches, searches took much longer than a systematic algorithm either using the given probabilities or random probabilities. For example, in the case of the 3x3x3 menu the expected number of frames visited would be 13.5 (27/2). The average number of frames visited by subjects was much greater.
Obviously, designers seek to avoid uncertainty in real menus. Unfortunately, few menus are perfect and many are fraught with uncertainty at numerous points. Since users seek to reposition search at points of uncertainty, it may be useful to provide a "bread crumb" capability to the menu. When users come to a point in the menu where uncertainty is high, they may issue a "bread crumb" command to mark that frame. If they further traverse the menu and do not find the target, they can activate the "bread crumb" command to bring them back to the frame in question.
In cases where search is particularly difficult, the user may be aided by efficient search algorithms supplied by the computer based on subjective probabilities supplied by the user. Rather than making an overt selection, the user would input a probability distribution for the alternatives in each frame traversed. The menu system would then make selections automatically based on these probabilities, updating them when paths are exhausted, or allowing the user to update them based on new information. Whenever new frames are encountered, the system would solicit subjective probabilities from the user. Although time consuming, the advantage of systematic search might outweigh the burden of inputing probabilities on the part of the user.
9.2.3 Reposition to Cognitive Landmarks. When menus are organized according to a semantic network, there are generally pivotal landmarks at which fundamental turns are taken. When we give directions to geographical locations, we generally use landmarks. If one gets lost, one can return to the landmark in order to redirect search. Major intersections, highways, and shopping centers serve this purpose. Similarly, in menu selection systems, users who have a sufficient mental model of the system use its cognitive landmarks to find their way around. When they get lost or fail to find a target item, they tend to return to focal points of the system. Landmarks tend to be frequently accessed nodes in the system. Furthermore they tend to be transition points from one type of environment or state to another. For example, if the menu of the animal kingdom shown in Figure 8.1 were continued down to the species, one might want to reposition to another genus, order, phylum, or kingdom. It would be less likely that one would want to reposition to suborder, subclass, or subkingdom. To facilitate this, the system could include the alternatives "Return to Genus," "Return to Order," "Return to Phylum," "Return to Kingdom."
The top or root node of a hierarchical menu system is by its nature a landmark. Similarly, the "Home Card" in stackware applications is a landmark. Consequently, when users get lost in the system, they are more likely to reposition to the root node or home card. Unfortunately, this can be very unproductive when a local search is required. If one were in a department store looking for an electronic blender and came to a table of electric can openers, he or she would not reposition the search by going out of the store and starting over again. Instead one would back up to the level of electric kitchen appliances and scan the selections from there. Unfortunately, too often users reposition in computer menu systems by returning to the top for lack of better cognitive landmarks (Norman & Chin, 1988; Norman, 1988).
Many complex office automation systems, electronic mail systems and the like contain a number of focal points toward which users gravitate. A few highly familiar, distinctive screens serve as landmarks to start and end local search processes. Systems which present more global views of the environment make use of such landmarks. For example, in the Rooms environment such landmarks are displayed graphically. The more salient such landmarks can be made the better they will serve as focal points in search processes. Repositioning to such points rather than all the way back to a root node should greatly improve search performance.
9.3 Individual Differences in Search Behavior
Tremendous variability exists in performance, style of search, cognitive ability, knowledge, and prior experience among users. This is particularly true in the area of search behavior. The average time to find items in hierarchical data bases by different users can vary by an order of magnitude. Researchers are interested in both characterizing this variability (e.g., in what ways and to what extent do users vary?) and determining its sources (e.g., what person variables affect search performance?).
9.3.1. Characterizing Differences. Individual differences can be conceptually divided into inherent differences in perceptual ability, memory capacity, and cognitive processing and acquired differences in specific knowledge of the subject matter domain and general knowledge of problem solving strategies. In terms of characterizing individual differences among computer users, inherent differences affect users' ability to scan choices, perceive patterns, remember items and paths, and plan paths. Acquired differences affect performance in that users learn information and strategies over experience that facilitates performance. Research on the development of expertise helps to characterize the differences between novices and experts.
Forward Search versus Backward Search. Larkin (1980) found that novices and experts search for solutions to geometry proofs and physics problems in quite different ways. Experts tend to reason from the givens to the goal in a forward search. Novices on the other hand, reason from the goal to the givens in a backward search. Larkin speculates that the direction of search is the result of the subjects record of past experience. In the past, particular configurations of information resulted in patterns of successful inference. These patterns may have originally been discovered in backward search, but once discovered, subjects begin to transform them to forward production rules.
Although Larkin's studies were on geometry and physics problems, the idea clearly generalizes to search in difficult hierarchical menu structures. In general, hierarchical menu structures promote forward search; however, when novice users are more familiar with the target item than they are with the forward path, backward search may be used. If novices are searching for an explicit target item, they may initially try to think backwards up the tree, "What do I select to get X? If Y leads to X, what do I select to get Y?" In general, this line of reasoning is time consuming and fraught with problems. However, with experience, the user may acquire forward production rules that start at the top of menu hierarchy. Search, and the reasoning processes that support it, are then performed in the same direction as the intended hierarchical structure of the menu.
Recognition of Recurring Patterns. Studies show that experts tend to perceive recurring patterns much more than novices. Experts are more likely to see clusters of items, to chunk things into larger perceptual units, and to pick up on the overall organization of things. Experience results in a large repertoire of patterns that can be applied to current problems.
Simon and Gilmartin (1973) estimate that chess masters have acquired something on the order of 50,000 different chess patterns. These patterns are quickly recognized during play, and this accounts for the superior memory performance of experts over novices when asked to reconstruct board positions. Similarly for computer programming, experts show better memory. It is argued that the development of programming expertise depends on the acquisition of a large number of patterns or templates (Soloway, 1980). Programmers associate these templates with the goal of the program. Given a goal, they can generate the appropriate templates; and given a novel program, they infer its function from the templates that it contains.
In menu search, expert users, no doubt, perceive recurring patterns of menu structure, common menus, types of menu screens, and paths through menu structures. Experts are able to recognize the intent of menus, identify potential paths for search, and search chunks of menu structures rather than individual items. Extensive familiarity with computer menus allows experts to direct search more efficiently by eliminating large sections of the menu hierarchy and inferring possible locations from past experience. Well-designed interfaces promote rapid expertise by enhancing the commonality of menu patterns and screens across the whole interface.
Planning. People differ greatly in the degree to which they plan before they act. Planners work out solution and contingency plans in great detail before they attempt implementation. Others jump immediately into action letting trial and error guide them. In a way, menu selection appeals to the trial and error method. However, trial and error as a method of search becomes impractical (a) when the menu system is very extensive and (b) when computer response time is very slow. In these cases, prior planning is imperative. By formulating a plan, the user accomplishes much of the search mentally rather then overtly. The plan may eliminate unfruitful branches, determine the most direct path to the target, and incorporate backup search procedures if the item is not found on the first try.
Planning ability depends greatly on the user's knowledge of the system and on the user's cognitive ability to transform, manipulate, and process the mental elements that represent the system. Planning requires a certain degree of expertise in the system structure and the tendency to think out a course of action in depth rather than pursue a breadth of immediately available alternatives.
9.3.2. Predictors of Search Performance. Direction of search, recognition of patterns, and degree of planning behavior help to characterize differences between expert and novice users in terms of the acquired knowledge. The next question is whether there are inherent differences among users that predict search performance in complex hierarchical menus.
Vicente, Hayes and Williges (1987) conducted an extensive study on individual differences in the ability to find information in a hierarchical arrangement of files. A large number of candidate factors were investigated which are listed in Table 9.3. The six spatial abilities were assessed using the Kit of Factor-Referenced Cognitive Tests (Ekstrom, French, and Harmon, 1976). Reading rate, vocabulary, and comprehension were assessed using the Nelson-Denny Reading Test (1973). The demographic variables were assessed by simply asking the subjects. Abstractness was assessed by the Abstract Orientation Scale (O'Connor, 1972) and field dependency was assessed using the Embedded Figures Test (1977). Anxiety was assessed using the State-Trait Anxiety Inventory (1983). Finally, information processing rate was assessed by giving a choice reaction time test among 2, 4, or 8 alternatives and calculating the slope of the Hick's Law function for one, two, and three bits of uncertainty (Wickens, 1984).
Table 9.3
Candidate Person Variables Investigated by Vicente, Hayes and Williges (1987).
- Spatial
- Flexibility of closure
- Perceptual speed
- Spatial orientation
- Spatial scanning
- Spatial visualization
- Visual memory
- Verbal
- Vocabulary
- Reading rate
- Comprehension
- Demographic
- Sex
- Computer experience
- Computer courses
- Cognitive Style
- Abstractness
- Field dependency
- Other
- Anxiety
- Information-procession rate
Six person variables correlated significantly with the performance variable of time, total number of commands and/or number of different commands (see Table 9.4). The variable of time seemed to be the most diagnostic performance variable. A regression analysis indicated that the best prediction equation for time included only the variables of vocabulary and spatial visualization. Other variables did not contribute substantially once these were taken into account. For example, computer experience had a correlation of -.34 (p = .06) with time; however, when spatial visualization was partialled out, computer experience had a negligible correlation with time. Either those with computer experience gained a higher spatial visualization or those with higher spatial visualization gained more computer experience. All that can be concluded is that there appears to be a relationship between the two variables and that spatial visualization was a better predictor.
Table 9.4
Correlations between Person Variables and Performance Variables Investigated by Vicente, Hayes and Williges (1987).
| Time | Number of Total Commands | Number of Different Commands | |
|---|---|---|---|
| Vocabulary | -.41* | -.42* | -.34 |
| Comprehension | -.37* | -.35 | -.26 |
| Spatial scanning (1) | -.38* | -.34 | -.33 |
| Flexibility of closure | -.41* | -.30 | -.25 |
| Spatial visualization (1) | -.47** | -.42* | -.44* |
| Spatial visualization (2) | -.57*** | -.46* | -.46* |
** p < .01
*** p < .001
The difference in performance between subjects with low and high spatial visualization ability was quite dramatic. Spatial visualization accounted for approximately 25% of all individual differences. Furthermore the magnitude of the difference is shown in Figure 9.7. A median split of the subjects into low and high groups of equal size reveals that low visualization subjects took nearly twice as long as high visualization subjects.
Vicente et al. attempted to isolate the components of the search process that seemed to account for the difference in times to locate information. They found that of 12 menu commands available, subjects with low spatial ability tended to use three of them significantly more than subjects of high spatial ability. One command was a procedure to move up one level of the hierarchy (ZOOM OUT). Subjects with low spatial ability tended to go to incorrect files and then had to back up the hierarchy. In addition, these subjects used two commands for searching through the files (SCROLL UP and SCROLL DOWN) significantly more often than subjects with high spatial ability. Vicente et al. conclude that subjects with low spatial ability tended to get lost more often both searching for the right file and searching for the right information in a file. Subjects with low spatial ability may have had a harder time visualizing paths between files and visualizing the location of information within files. Consequently, rather than manipulating the structures mentally in order to plan and test paths, they did so directly resulting in longer search times and a greater number of commands issued.
9.6. Summary
With the increasing complexity of menu selection in information retrieval systems and in command and control systems, it can no longer be assumed that menu selection will guide the user along an error free path. Consequently, when the first try doesn't work, users must rely on strategies for redirecting search. The concern of the designer should be to identify ways that the system can help to redirect and facilitate that search.
A major determinant of effective search is breadth of selection at the bottom of the menu hierarchy. The resulting guideline is that menu structures should be designed so that they are bottom heavy. Once the user has traversed down to the bottom of the menu, a wide range of alternatives should be selectable. Users tend to redirect search to a level that provides the greatest breadth of choice. Since search is generally redirected after a failure at the bottom of the tree, it makes sense to provide breadth at the same level.
A secondary effect is that breadth at the top of the menu facilitates search. The opening menu that provides a wide range alternatives is superior to a limited, focused menu. Since search is often redirected to the top of the menu, it makes sense to provide a number of alternative search routes at this point. On the other hand, menu breadth in middle levels of the structure seems to hinder search. It is here that search may need to be focused so as to reduce the probability path errors.
Menu structure determines the number of discrete frames in the system. It is suggested that menus that minimize the total number of menu frames may facilitate search in that the user has fewer frames to search and may have a greater likelihood of remembering the critical menu frames that lead to the target item.
Users vary greatly in their ability to locate information in hierarchical menu structures. As users gain experience with a system their search behavior improves. Expertise is characterized by (a) a forward search from the given situation to the target item rather than a backward search from the target back up the hierarchy, (b) the ability to recognize recurring patterns in menu structures, and (c) the ability to plan out a search path. Finally, the best predictors of search performance are (a) spatial visualization ability and (b) vocabulary and comprehension abilities.
No comments:
Post a Comment