Methodology: Accessibility data generated with Bobby

The methodology used in this survey is essentially identical to that used in my previous studies that focused, more narrowly, on the campuses within the University of Wisconsin System. The following remarks are not meant to be exhaustive; they are meant to highlight and clarify some important aspects. 

1. Scope of investigation:

For each of the 24 campuses, I collected accessibility data on the top-layer web pages for the general campus, the library and the library school (SLIS). The top-layer pages included the homepage and all the pages within the same domain that were directly linked to it.

Text-only or non-frame versions were used in this survey only when they were accessible from the top of the home page. 

The universities that were in included in this study do not constitute a random sample. Since I was particularly curious about the accessibility of SLIS pages, I selected the campuses with the nation's most highly ranked library schools (according to U.S. News). While the validity of U.S. News rankings is highly questionable, one may reasonably assume that these rankings shape the perceptions of prospective library science students and influence their decisions. Many students will enroll in these highly-ranked LIS schools with the expectation that these are the leaders in the field.

2. Evaluation tool

I used the latest download-version of Bobby (3.1.1), which is an accessibility validator created by the Center for Applied Special Technology. Bobby was developed to help people to check the accessibility of their web pages. For each page checked, Bobby provides information pertaining to the type, number, and location of accessibility errors--both minor and major ones. Bobby also issues a summary report for each set of web pages. Web pages that contain any major ("level 1") error do not receive Bobby's approval.

Bobby checks for compliance with the W3C/WAI Web Content Accessibility Guidelines. The developers of Bobby 3.1.1 emphasize that passing the automated Bobby check is not sufficient proof for a web page's accessibility. To get full "Bobby approval," a manual check involving human judgment is also required.

3. Limitations of this study

In this study, the term "Bobby-approved" is used in a rather lax manner.  It refers to those pages which passed the automated Bobby check.  No systematic manual checks were performed. Similarly, the error data presented on this site refer exclusively to those errors that were detected during the automated Bobby check.

Bobby's automated checks provide only a rough measure of web page accessibility. As indicated above, the automated Bobby checker will miss certain access problems that, at this point, only  humans can detect. Since Bobby checks for compliance with the W3C accessibility standards, it may determine a page to be inaccessible even though its authors have employed creative, nonstandard ways to ensure universal access. (The home page of the UW-La Crosse is a good case in point. At the time the data were collected, Bobby did not approve of its image map and the hot links contained therein, even though the alternative text provided ["Welcome image. Same links as below."] would steer users with text-only browsers toward an equivalent set of accessible links.)

One particular shortcoming of Bobby is its inability to distinguish between degrees of impact between different manifestations of the same error.  For example, a bullet icon without alternative text registers as equal in status (i.e., as a "level 1" error) to that of an image (also without the ALT tag) that is packed with crucial information. Furthermore, Bobby classifies certain types of accessibility errors as equally severe even though the barriers they constitute may differ to a significant degree. For example, the lack of alternative text associated with a purely decorative image and the lack of meaningful frame labels in multi-frame pages both register as "level 1" errors. 

Back to the top of this page

Back to the Contents/Overview page

Researched and created by Axel Schmetzke, Library, University of Wisconsin-Stevens Point.
Last updated 07/20/00 .
Comments are welcome!

Valid HTML 4.0! Bobby 3.0 Approved Valid CSS!