article overview contact/home

This evaluation uses established criteria to identify usability hazards and prioritize solutions. Minor changes can then be implemented before conducting a thorough user test. (Part 1 of 2.)

Objective evaluation of likely usability hazards – preliminaries for user testing

by Jasper Sprengers

Abstract

This article is an attempt to consolidate and apply the recommendations from several excellent resources for good usability design of websites into a workable test. It will then provide a formal yardstick to evaluate the relative expense and benefits of rectifying flaws and implementing improvements. It is written both as a research effort and as a collection of useful advice and warnings that we hope will be of use to everyone involved in copywriting, graphic design and technical implementation of websites.

• Part 1
  • Objective usability criteria versus tastes in design
  • Categories for evaluation outlined
  • Evaluation into hazards, benefits and expenses
• Part 2
  • Language criteria
  • Layout and graphics criteria
  • Information architecture criteria
  • User interface criteria
  • General criteria for design practice
• Reporting the results
• Conclusion and recommendations

Part I: objective usability criteria versus tastes in design

We define usability here as the decisive quality criterion for a website. It involves the overall success of the user's experience and therefore involves more than speed and ease of navigating. We have categorized the recommendations from these sources into a checklist of objective criteria that have proven to help or hinder usability in sites that vary both in size and objectives. Despite the inevitable shortcoming that it overlooks – or rather postpones – the experience of the individual user, a single tester can at least apply the criteria objectively and unbiased.

Evaluation criteria for website usability seldom distinguish explicitly between value judgments and objective criteria. Perhaps researchers prefer to deny that objective criteria have any relevance. Usability is after all a user's experience; not the verdict of an expert. No objective method can tell us whether a website is really usable unless we have observed ordinary people using it. Many sites, however, suffer from frequent flaws that are well documented. These are easy to identify, often simple to remedy, and can be carried out by a single expert.

Figure 1: Blue underlined text is recognised by users as a hyperlink.

The distinction we employ is not only pragmatic. Design decisions of an objective and subjective nature often compete over a single feature. This is especially true when a trade-off between aesthetic effects and ease of use is involved. Let us take blue underlined link colours as an example. Graphic designers can argue about the aesthetics, but because they are an established practice we all recognize them at once as links. The positive effect on usability is hard to dispute and something which many major sites have weighed up in their redesign, including Kodak.

While this test cannot identify individual user problems specific to a single site, it can make a subsequent task-based user test more meaningful and reliable by first solving objective usability hazards. The user will eventually not only have a more favourable impression of that site, but we can be confident that common flaws do not obscure any site-specific problems, which are always harder to locate and often harder to solve.

Truly objective criteria are a matter of measurable degree. When the mere presence or absence of a certain feature is a criterion in itself (for example: 'provide a copyright notice'), we naturally have to disqualify those instances of it that are incomplete or otherwise of poor usefulness, such as 'Bob has the copyright to this page'. This decision needn't be a subjective judgment. If the necessary features of an item cannot be objectively defined – as they most certainly can for a copyright notice – the item has no place in our present evaluation. There is after all no need to carry the argument about subjective versus objective to a philosophical level.

To identify which guidelines are concerned with objective features we have defined the following two characteristics. There is no direct correlation between each characteristic and specific usability hazards: it is only there to help us sift the criteria.

Figure 2: The way hyperlinks are rendered can be counted.

• Guidelines concerning the simple presence or absence of information items, either visible ('date last modified', copyright notice) or as part of the document's meta-data (keywords, document description). Proper coding conventions for HTML, style sheets and scripts belong to the same group: they urge the programmer to include certain attributes or they suggest a preferred alternative to reach the same visual effect.
• Guidelines that concern how visible elements are rendered in countable and measurable terms. Some examples:
  • Size of page elements and their relative or absolute position on the page.
  • Use of fonts: types, colours, size. (E.g. only use underlined fonts for hyperlinks).
  • Length of text units in headings, lines, paragraphs and pages.
  • Assigning hyperlinks to images or text elements and how they are made visible (underlined, coloured, with rollover effect).

Categories for evaluation

We have identified five categories, each representing an area where usability hazards are common. The rationale behind defining these categories is as follows:

• Verbal or visual rendering of information items make up categories 1 and 2
• The logical coherence between information items at all levels of the site's hierarchy (headings, paragraphs, tables, pages) and the ease of navigating through these levels make up categories 3 and 4.
• Working practices to ensure overall quality fall under category 5. These cannot be evaluated from observing the site alone, and belong to a site designer's self-evaluation of her methods and ideas.

1. Language

This refers to the choice of words used to present information. Much good advice on making a text easy to understand and well-structured is too subjective for the scope of this text. However, spelling, sentence length and use/avoidance of certain words ('cool', 'cyber-', etc.) do lend themselves to more objective judgements.

2. Layout & graphics

This concerns how elements are visually rendered on the page, but we identify a particular issue only as a layout problem when it can be remedied by adding/changing visual features such as size of elements, colours and fonts. A long body of text, for example, can pose a layout problem when fonts, bulleted lists, or paragraph breaks could improve it. If it becomes better manageable by breaking it into several linked sections, it is an information architecture issue (see below). Thirdly, it becomes a subjective language problem if we should condense the text itself. In reality it can however be a combination of all three.

3. Information architecture

Good information architecture means a clear, intuitive division of tasks and topics. The site's content and features can be arranged in more than one way, both with regard to the number of branching (sub)sections (width) as in the levels of hierarchy (depth). Careful wording of titles, introductions and summaries are features that help emphasize this structure. Objective criteria for proper structuring are rare, because the best arrangement of information items is highly specific to each site and its purpose.

4. User interface

The user interface of a site determines the ease of navigating through its content. Any feature designed to facilitate the user's quest belongs to this category. Aspects of user interface discussed here are specific to the Web and typically of a technical nature, whereas the arrangement of tasks and topics is a question of information architecture, which extends to printed media as well.

5. General

This category contains warnings and recommendations that apply to the general practice of design and maintenance. Proper use of coding conventions (HTML, CSS, Scripts) are examples of essential requirements. Many criteria can be collected in a checklist of useful 'things to do' and often apply more to the way people work than to the final product.

Examples: 'conduct regular user testing', 'invite comments and reply to them'.

Categorization is based on areas of common usability hazards, not on a list of HTML elements or content items. A single page element can present usability problems in three areas, as explained above with a large body of text. As a further example, consider a navigation bar rendered as a clickable image map:

• The number of links and the rationale behind the division of topics belongs to the site's information architecture.
• The ease of using these links belongs to the site's user interface (e.g. how are they recognizable as links? Does a link open a new window?).
• The choice of words for the link labels is a language feature. Words like 'cool', 'hip' and 'stuff' carry little meaning.

Evaluating the results

For our evaluation of the test results we have marked each usability criterion for three features:

• Whether the criterion that was tested for poses a usability hazard or a benefit. One might argue that these are opposite sides of the same coin: removing a hazard means implementing a benefit, while omitting a beneficial feature poses a hazard. While there is some truth in that, we choose to interpret them as follows:
  • Hazards are features that must be rectified or implemented, because they can do real damage to user satisfaction and indeed the credibility of the entire site. Omitted features can also pose hazards.
  • Benefits are features that enhance usability, but do not strike users as a serious threat to usability when they are not present. To some extent this happens because they have not become standard practice yet. They are similar to Jakob Nielsen's 'good deeds'. While hazards should take priority over benefits in maintenance work, it is good to remember that benefits can become common practice over the course of time, and become a hazard when left out.
• The potential improvement on the user experience. These three categories apply both to hazards and benefits. Harmful features must be rectified and positive features can be implemented if they are feasible.
  1. Minor hazards can go unnoticed and not strike users as either especially poor, or helpful. Minor benefits are not worth the effort of a major redesign, but can be implemented throughout the development of the site.
  2. Medium hazards of this kind can be the cause of lost visitors when they are frequent and persistent. They should get high priority during site maintenance. Medium improvements are especially recommended when they only require minor expenses.
  3. Major hazards are very likely to chase away visitors forever and at the worst harm the organization's entire reputation. They must be solved before the site goes public. Major benefits can put a site ahead of its competition, but may cost accordingly.
• The necessary expense in terms of time, skills and hardware to solve the problems. We distinguish the following three levels:
  1. Minor. Quickly and easily resolved without advanced tools or experience: typically less than 30 minutes per page. Some of this troubleshooting could be automated.
  2. Medium. Requires more skills and experience from various disciplines (programming, graphic design, editing, information architecture). Although possible to solve quickly, tasks cannot be automated.
  3. Major. Issues involve a drastic redesign or addition to the site, requiring considerable time and effort to implement, especially when new hardware is involved. Such decisions should not be undertaken until after a task-based user survey.

Proceed to: part 2, the test categories.

last updated: 5 October 2004 (links)

Copyright 1999 Jasper Sprengers.

article overview contact/home