Designing for User Satisfaction:Web site quality

7.4 Web site quality – the customer perspective

Participative design and a concern for job satisfaction and democracy in the workplace are important aspects of achieving user satisfaction. A more general concern is for the quality of the web site and the user experience, which will affect employees and customers. In understanding the subjective experience of the web site user we will draw on the Web Qual instrument developed by Barnes & Vidgen at the University of Bath.

7.4.1 Web site quality

The WebQual instrument is a way of assessing the quality of web sites – particularly sites with an e-commerce aspect – and was inspired by work in quality function deployment (QFD). QFD is a structured process to identify and carry the voice of the customer through each stage of product and/or service development and implementation. Based upon a distinction of 'what' and 'how', a series of matrices are used to deploy customer-demanded qualities through design requirements, product functions, part characteristics, and manufacturing operations into production requirements. The starting point for QFD is the 'voice of the customer' – an expression of quality requirements from the perspective of, and in the words of, the customer (King, 1989).

One way to get access to the voice of the customer is to hold a quality workshop. In the context of the theatre this could involve getting a group of theatre-goers together and asking them: 'What are the qualities of an excellent theatre booking web site?'. In the workshop the attendees first work individually and write down everything they can think of on post-it notes (one idea per post-it). Some of the items will be qualities, such as 'easy to use' and some will be product functions, such as a 'restaurant booking facility'. At this stage we are interested primarily in qualities rather than characteristics and functions, but everything is collected ready for use in later stages of QFD, or as input to the requirements specification. The individuals now work together in teams of 3 or 4 and look for groupings in their qualities. Working quietly they move the post-it notes around a work surface bringing together qualities that seem to have an affinity. They can either promote one of the existing items in a group to be a header for that category or come up with a new header item that in some way describes the group. This process may seem rather vague and imprecise, but remember that the aim is to understand how the customer conceptualizes web site quality – not how the supplier (IS developer and theatre management) sees it. For further details of running a quality workshop see Bossert (1991).

Quality workshops were held at every stage of the development of the WebQual instrument. The findings were then checked against the literature to see where previous work might inform the development of an instrument for assessing web site quality. In WebQual 4.0 web site users are asked to rate offerings against a range of qualities (table 7.4) using a 7-point scale. The respondents are also asked to rate each of the qualities for importance (again, using a 7-point scale), which helps the web site supplier gain an understanding of which qualities are considered by the user to carry most weight. WebQual consists of three dimensions for assessing e-commerce website quality: usability, information quality, and service interaction quality.

Usability

Usability draws on the field of human computer interaction and is concerned with the pragmatics of how a user perceives and interacts with a web site: is it easy to navigate? Is the design appropriate to the type of site? It is not, in the first instance, concerned with design principles such as the use of frames or the percentage of white space, although these are concerns for the web site designer who is charged with improving usability.

Information quality

Assuming that a web site is usable, then the user's attention can turn to the content of the web site. Questions 9 through 15 address the quality of the information presented on the web site. There is a longstanding body of IS literature examining aspects of information and information quality. Most of this literature predates the explosion in web commerce and builds on the seminal work of Shannon and Weaver from the 1940s on communications. They proposed 'information' as the message in a communication system, from sender (S), via a communication channel, to receiver (R). This can be measured at a number of levels: technical, referring to the accuracy and efficiency of the system producing information; semantic, referring to the success of the system in conveying intended meaning; and, effectiveness, referring to the effect or influence of the information on the receiver. Such a conception is most poignant, particularly to web commerce, where organizations aim to transmit data efficiently and accurately over the Internet, such as product offerings, which convey a desired meaning and achieve a desired effect, such as ticket sales.

Service interaction quality

Information quality is clearly important in conducting e-commerce, but it does not capture the interaction aspects. A service encounter is the period of time during which a consumer directly interacts with a service. This encounter may or may not involve a human interaction element and in the case of Internet transactions will probably not. The SERVQUAL instrument was developed as a means of assessing customer satisfaction with the service level of a firm (Zeithaml et al., 1990). SERVQUAL has been applied in many domains, including appliance repair, banking, dentistry, accounting, and department stores. There is a service element involved in the delivery of any product, whether it is intangible (such as opening a new credit card account), or tangible (such as buying a book). Although important, in many industries cost is not the sole determinant of competitiveness; the customer experience has come to be recognized as a significant basis for differentiating suppliers. SERVQUAL was developed in the context of a physical world of buildings and human service representatives, but many of the concepts underlying the instrument can be adapted for the Internet and e-commerce.

7.4.2 The application of WebQual to online bookstores

To illustrate the WebQual approach we will summarize an application of the instrument to online bookstores (Barnes & Vidgen, 2002). Buying books over the Internet is one of the early applications of B2C e-commerce and has matured to become relatively stable, at least in Internet terms, with 5.4% of total global book sales in 1999. In the fall of 2000 we conducted a survey of the three largest UK online bookshops: Amazon, Bertelsmann Online (BOL) and the Internet Bookshop (IBS).

Amazon.com was launched in July 1995. However, although Amazon.com is accessible from all over the world, in the last few years the company has also established a localized presence in other international markets – including the UK, Germany, France and Japan – to comply with publishers' territorial rights while minimizing shipping costs. Amazon established a UK presence in 1998, Amazon.co.uk, with headquarters in Slough, a town 25 miles west of London.

Owned by the media conglomerate Bertlesmann AG, BOL (Bertelsmann Online) is the newest of the sites examined – launched in 1999. The launch of BOL follows Bertelsmann's acquisition of a 50% share in US-based Barnes and Noble's online book retailing subsidiary BarnesandNoble.com for $200 million in October 1998. Barnes and Noble is one of the World's largest booksellers with around 1000 bookstores.

The Internet Bookshop was established in 1993, making it one of the UK's longest-established Internet bookstores. The owner of the IBS web site, WH Smith, is a traditional UK high-street business, selling newspapers, books, music and stationery. WH Smith have an estimated £330 million (19.4%) of the UK book market and is estimated to be in second place for Internet sales behind Amazon.co.uk, although there are no accurate figures for its web operations.

Data collection

The WebQual approach to data collection is to use an online questionnaire targeted at real users of an e-commerce offering. In the case of Internet bookstores the students and staff of a university make excellent subjects given that both groups buy books as a matter of course and have considerable experience of this area of e-commerce.

The questionnaire was administered online and resulted in 280 usable responses. Respondents were directed to the start page for the survey, where they could read a set of brief instructions and then click the 'start' button. This button opened a second window that was used to collect questionnaire data (see figure 7.6). By clicking on the question number the user could see a dictionary entry for that item – a paragraph of text expanding on the quality and giving examples as appropriate. A random number was generated to pick which of the three sites would be presented to any given respondent. Questionnaires were checked online prior to submission to ensure that valid responses had been entered for all questions (an option 'not applicable' was included to ensure that responses were not forced).

To ensure a deeper level of commitment to the evaluation of the site the respondents were asked to find a book on the site that they would like as a runners-up prize. Respondents were asked to supply the title, price, and delivery cost. These are 'facts' that the respondent is asked to determine, which made it more likely that they would engage in searching and navigating the site. Furthermore, in setting a task we aimed to maintain a more natural and representative flow of interaction between the user and the web site.

The results

Table 7.5 shows the top 6 and the bottom 6 questions in terms of importance to respondents. Those questions considered most important by the respondents are heavily tied to information accuracy, usability and issues of trust. Such questions concern security and reliability regarding completion of transactions, receipt of goods and personal information, accuracy of content, as well as ease of site use and navigation. Conversely, when examining those items considered least important there is a quite different variety of questions. This group revolves around empathy with the user (communication, community and personalization), as well as site design issues (site experience, appropriateness of design and aesthetics). Given the high profile given to Amazon.com's reader reviews of books, it is interesting to note that respondents rate the community aspects as low in importance. Perhaps the basic building blocks of e-commerce – accurate information and safe to complete transactions – are still uppermost in web site users' minds.

Weighted results for each of the sites can be calculated by multiplying the score given to the site for an item by the importance attached to that item by

the individual. However, the individual weighted scores for each question make it difficult to give an overall benchmark for the sites. One way to achieve this is to index the total weighted score for a site against the total possible score (i.e. the total importance multiplied by 7, the maximum rating for a site). This allows a WebQual Index (WQI) to be calculated for the overall site. The results of the WQI analysis are:

Factor analysis is a statistical method for finding 'natural' groupings of questions in the data. Factor analysis showed five constructs to be present in the data. The usability questions contained two constructs, which the authors labeled usability and design, and service interaction, which subdivided into trust and empathy. Information questions grouped together in a single construct. These five dimensions give a statistically verifiable indication of how web site users perceive quality in the online bookshop domain. To summarize the performance of the three cyber bookshops the data was organized around the five questionnaire subcategories. Then, and similarly to the overall WebQual Index, the total score for each category was indexed against the maximum score (based on the importance ratings for questions multiplied by 7). Figure 7.7 is the result – a comparison of the three sites in graphical form showing how each performed in terms of the five constructs of web site quality. Note that the scale has been restricted to values between 0.4 and 1.0 to allow for clearer comparison.

Figure 7.7 demonstrates very clearly that the Amazon UK site stands 'head and shoulders' above the two rivals. The indices for the Amazon subcategories make a clear circle around the other two sites, with Trust rating particularly well. Other areas are less strong, in relative terms, although clearly ahead of the nearest rival. Empathy had the lowest WQI for Amazon, with other categories somewhere in between. The scores of the two other bookshops are very close, with IBS edging only slightly ahead of BOL for three of the subcategories. Information quality presents the largest discrepancy with IBS leading BOL. IBS also score marginally higher for empathy and usability, with the remaining categories containing almost equal scores.

Interpretation of the results

The survey indicates that the differences in usability between the three sites are relatively small, suggesting that once a basic level of usability is achieved then the design of the web site is unlikely to be a differentiating competitive factor. With regard to design, the difference is more marked, indicating that the design

of the Amazon site is preferred, although this may be due in part to respondents being more likely to be familiar with the Amazon site. It is a relatively straightforward task for an organization to benchmark the usability of its web site, for example by holding a usability workshop where users are given tasks to complete and are monitored while doing so.

Respondents rated 'accurate information' as the most important item in the WebQual instrument. This indicates that e-commerce businesses need to pay particular attention to the content of their web sites. Lack of control over content is evident when, for example, organizations do not remove special offers that have expired, or, more seriously, when pricing errors are introduced, such as a dishwasher being offered at £2.99 rather than £299. Managing information quality is likely to be rather more difficult than improving web usability since the content management cycle needs to manage web documents from creation through publishing to archival and, eventually, destruction. Whereas usability can be evaluated quickly, information quality is likely to require an enterprise-wide approach that addresses all the sources of content, encompassing authors, existing systems and databases.

While usability and information quality might be addressed largely through internal changes, interaction quality requires a stronger external perspective. The greatest differentiator of the sites is 'trust', where Amazon is a long way ahead of its competitors. Yet the concept of 'trust' has a degree of ambiguity; a

variety of definitions of trust have been proposed, including those related to benevolence, integrity, competence and predictability (McKnight & Chervany, 2001). Interestingly, in the online bookshop application of WebQual, of the four questions that comprise 'trust', three of the questions were rated as second, third, and fourth most important by respondents (only 'accurate information' rated more highly in terms of customer importance).

It is unlikely that excellent web site design and judicious use of new technology will increase the perception of trust by customers, since trust is affected by external factors, such as the strength of the brand, the customer's previous experiences, and the whole range of communications generated by the brand-owner, the media, and word of mouth. There are web site design implications for 'trust', such as making sure that the privacy policy is visible, and displaying the logo of a third party for accreditation of security mechanisms. However, given Amazon's first-mover advantage and the switching costs incurred by customers in moving to a competitor, BOL and the Internet Bookshop need to go further than web site design considerations and offer something that distinguishes them from Amazon. In the case of the Internet Bookshop this might involve integration of online activity with the physical high-street network of its owner, WH Smith, and for BOL with its partner Barnes and Noble.

Summary

• A successful web-based information system will typically need to meet the requirements of at least two sets of stakeholder: employees and customers.

• Employee job satisfaction can be enhanced through participative design and sociotechnical approaches such as ETHICS and the Scandinavian School.

• Customer satisfaction requires a marketing approach and can be assessed through instruments such as WebQual.

Exercises

1. How might an ETHICS-based development be applied to the creation of a new research student admission process (appendix B)?

2. How would you safeguard against a sociotechnical design process being used in a managerialist way in the single-minded pursuit of profit?

3. Conduct a web site quality workshop for your organization to elicit your customers' quality requirements. How closely do the qualities map to the WebQual instrument?

Search This Blog

information systems course