Methodology: Accessibility data generated with Bobby

The methodology used in this survey is essentially identical to that used in my previous (1999) study. For a more detailed discussion, please consult its methodology section. The following remarks are not meant to be exhaustive; they are meant to highlight and clarify some important aspects:

1. Scope of investigation:

Two sets of web pages were included in this survey: general campus pages and library pages. Each set included the respective home page and the pages linked to it that were within the same domain or directory.

Text-only or non-frame versions were used in this survey only when they were accessible from the top of the home page.

Only the thirteen UW campuses that offer four-year programs are included in this survey. No data pertaining to the UW Centers were collected.

2. Evaluation tool

I used the latest download-version of Bobby (3.1.1), which is an accessibility validator created by the Center for Applied Special Technology. Bobby was developed to assist people in checking the accessibility of their web pages. For each page checked, Bobby provides information pertaining to the type, number, and location of accessibility errors--both minor and major ones. Bobby also issues a summary report for each set of web pages. Web pages that contain any major ("high-priority") error do not receive Bobby's approval.

Bobby checks for compliance with the W3C/WAI Web Content Accessibility Guidelines. The developers of Bobby 3.1.1 emphasize that passing the automated Bobby check is not sufficient proof for a web page's accessibility. To get full "Bobby approval," a manual check involving human judgment is also required.

3. Limitations of this study

In this study, the term "Bobby-approved" is used in a rather lax manner.  It refers to those pages which passed the automated Bobby check.  No systematic manual checks were performed. Similarly, the error data presented on this site refer exclusively to those errors that were detected during the automated Bobby check.

Bobby's automated checks provide only a rough measure of web page accessibility. As indicated above, the automated Bobby checker will miss certain access problems that, at this point, only  humans can detect. Since Bobby checks for compliance with the W3C accessibility standards, it may determine a page to be inaccessible even though its authors have employed creative, nonstandard ways to ensure universal access. (The home page of the UW-La Crosse is a good case in point. At the time the data were collected, Bobby did not approve of its image map and the hot links contained therein, even though the alternative text provided ["Welcome image. Same links as below."] would steer users with text-only browsers toward an equivalent set of accessible links.)

One particular shortcoming of Bobby is its inability to distinguish between degrees of impact between different manifestations of the same error.  For example, a bullet icon without alternative text registers as equal in status (i.e., as having "high priority") to that of an image (also without the ALT tag) that is packed with crucial information. Furthermore, Bobby classifies certain types of accessibility errors as equally high in priority even if the barriers they constitute differ to a significant degree. For example, the lack of alternative text associated with a purely decorative image registers as an error equal in need of correction to the lack of frame labels in multi-frame pages.

Finally, recent modifications in Bobby (in response to changes in the evolving W3C standards) should call for initial caution in comparing last year's with this year's data.  However, since the most frequently occurring errors (i.e., images without alternative text and inaccessible hot-spots in image maps) have been unaffected by the changes in the W3C standards, a comparison of old and new Bobby-generated data is feasible and meaningful.

Back to the top of this page

Back to the page entitled: Accessibility data generated with Bobby

Back to the Contents/Overview page

Researched and created by Axel Schmetzke, Library, University of Wisconsin-Stevens Point.
Last updated 05/10/00 .
Comments are welcome!

Valid HTML 4.0! Bobby 3.0 Approved Valid CSS!