Author: Ruzegea, Mboni
Date published: February 1, 2012
Interface for user input and system output is among the components of IR system. Different Online Catalogs (online IR systems) have different user interfaces which would allow user to navigate or search information within and outside their library collections. Due to the increase in web technology, designers in user-interface industry compete in making different designs to allow ease-of-use of these interfaces so that users can have access to information they need. Yet, most of the designs of OPACs' interfaces are not that much effective in helping the users during their search for information. Some interface designs in university libraries' OPACs are less user-friendly and would not allow interactivity with the user during search sessions rendered them less effective, inefficient and bring low satisfaction on users. Libraries' Online Public Access Catalogs (OPCs) are one of the highly visible end user searching tools. Online catalog user studies have revealed, among other findings, that catalog users have the most difficulty with information searching and place the highest priority for improvements on various information search enhancements (Markey, 1983; Hildreth, 1985).
The OPACs allow users to access resources of libraries, publishers, and online vendors (Guha&Saraf, 2005). OPACs can be accessed by from anywhere in the world, even from the palm of their hand. According to Guha&Saraf, this new generation of OPACs also incorporates advanced search features and new designs from other types of IR systems, such as allowing users searching OPAC and online databases via OPAC interface. Most of OPACs interfaces were designed to minimize online connect time and printing options (Brantley et al, 2006). It is therefore expected that, a user-friendly designed interfaces would have for instance, a simplified menu-driven interface utilizing off line storage of search strategy, automatic logon procedures, and software-controlled navigated searching techniques.
Search and retrieval of library materials has become easy due to OPAC. But it has been observed in some instances, that users are not coping with this change. There seems to be two reasons for this. Firstly, some users lack computer knowledge and hence are reluctant to accept the change and secondly, the designs of the interfaces of some systems are not user friendly (Umarani et al, 2008). Umarani and others observed that personal and extended help is possible from library staff to the users to search OPAC effectively within the library. But it becomes difficult to provide such a help to online users. Therefore, it becomes essential to design user friendly OPACs and to test them for usability on a regular basis.
Usability testing is a means for measuring how well people can use some humanmade object (such as a web page, a computer interface, a document, or a device) for its intended purpose. Usability testing tries to find out 'user-friendliness' of the system, which is obviously subjective. Repeated user interviews, surveys, video recording of user sessions, and other techniques can be used for this purpose. Apart from these, technique of task analysis also can be used where in, certain tasks are assigned to the users and observations are made and further analyzed .The academic libraries OPAC users manifest special and unique needs and problems during their searching for information. Few user studies could be located which focused exclusively on how user-interfaces may have different impact to different categories of university library user communities (Umarani et al 2008; Galitz& Wiley, 2002; and Mayhew & Kaufmann, 1999).
This study intends to investigate the usability and effectiveness of user-interface features of Library OPAC among postgraduate students of International Islamic University Malaysia (IIUM), in relation to variables that influence the usability and the extent that the interface features can enhance search and informational retrieval for the users (i.e. effectiveness). This study will also seek to understand users' perception on the effectiveness of IIUM's OPAC's interface design (perception of overall ease - of - use) and whether or not they are satisfied. To this regard, user's background information concerning the exposure and use of OPACs and levels of computer literacy skills are important dimensions to be looked at.
1. To what extent the IIUM library OPAC's user- interface features have influenced the use of the system to meet users information needs?(Relevance)
2. Is library OPAC's user-interface user friendly, (support smooth navigation?; highly supportive, minimally supportive, little supportive)
3. Are there any experienced difficulties in the use of library OPAC's user -interface? (many difficulties, minimum difficulties, few difficulties, no difficulties)
4. To what extent users' characteristics/background have influenced usability of OPAC user-interface?
According to the Web dictionary (Wikipedia), the user interface (or Human Computer Interface (HCI)) is the aggregate of which people - the users - interact with the system - a particular machine, device, computer program or other complex tool. The user interface provides means of:
a) Input, allowing the users to manipulate a system
b) Output, allowing the system to produce the effects of the users' manipulation.
For quite a long time (in 20 years period) many researches about OPAC, as appeared in the Journal of the American Society for Information Science (JASIS) covered issues on state-of-the-art OPAC research (Beaulieu and Borgman, 1996); new design models for on-line catalogs (Hildreth, 1995a); and analytical review of recent OPAC research is provided by Large and Beheshti (1997). These authors focus on the various methodologies employed in OPAC studies, and summarize research-based recommendations under three headings: database record enhancement, search capabilities, and interface design (Hildreth, 2001).
Few studies were conducted to date on the performance of online catalogs with specific focus on effectiveness and usability of user-interface features (Hildreth, 1995b as cited in Hildreth, 2001) Looked at the new graphical user interfaces (GUIs) that are being applied to older, conventional, second generation OPACs, Hildreth warned that "users may be too easily impressed with these systems, systems that deliver the same old level of poor results". In her insightful 1996 article, Borgman, asks, "Why are on-line catalogs still so hard to use?" Perhaps we should be asking now, "Why do easy OPACs still produce such poor results?" (Borgman, 1996 as cited in Hildreth, 2001).
In response to these questions, the current study presumes that most IR systems are still system-centered and not user-centered. This study seek to expose users' views and suggestions as to how they want the system to be(have) in regards to easy-of use when interacting with systems' user-interface features. The system may expose several user interfaces to serve different kinds of users. For example, a computerized library database might provide two user interfaces, one for library patrons (limited set of functions, optimized for ease of use) and the other for library personnel (wide set of functions, optimized for efficiency (Wikipedia). This study focuses on usability as the question of how well users can use that user interface features. Mansor (2007) conducted a study on "Heuristic Evaluation of Interface Usability for a Web-based OPAC" at the IIUM university Library, and the results revealed that there was lack of visibility of interface status in IIUM Web PAC interface. The study further revealed that the most obvious weakness of the interface is the lack of a proper messaging system, to inform users on the system's status during delays, as reported by 60% of respondents.
The design of a user interface affects the amount of effort the user must employ to provide input for the system and to interpret the output of the system, and how much effort it takes to learn how to do this. Usability is the degree to which the design of a particular user interface takes into account the human psychology and physiology of the users, and makes the process of using the system effective, efficient and satisfying (Wikipedia). Schneiderman (1998) as cited by Othman, (2006) proposed criteria that focus on user interfaces. The criteria include (a) consistency in terminology (b) shortcuts for experienced users (c) informative feedback about the search (d) usability to undo or modify action (e) user control in specifying parameters (f) clear error messages and correct errors easily and (g) alternative interface for expert and novice users. Generally, usability is mainly a characteristic of the user interface, but is also associated with the functionalities of the product and the process to design it. It describes how well a product can be used for its intended purpose by its target users with efficiency, effectiveness, and satisfaction, also taking into account the requirements from its context of use (Wikipedia).
Different types of IR systems such as OPACs have different interface designs and different search mechanisms. Research in information retrieval has traditionally concentrated on building representations of content and queries, different IR techniques and indexing methods, however, one problem for IR is support and designing for IR interaction (ERCIM, 1996).
Search and retrieval of library materials have been easy due to OPAC. But it has been observed in some cases that users are not coping with these changes amongst the reasons being the designs of interfaces of some systems are not user friendly (Umarani, Nagarcar&Jagtap, 2003). The same authors (ibid) argue that, although personal and extended help is possible from library staff to the users to search web OPAC effectively within the library, but it becomes difficult to provide such a help to online users. Therefore it is essential to design user friendly Web OPACs and to test them for usability in regular basis.
Saracevic (1996) points out that "IR interaction is a complex process that is very much situation or context dependent: it starts from and relates to users, their tasks or problems, competencies, knowledge states and intents on the one hand, but it also involves characteristics and capabilities of the system, the information resources, and the interface, on the other hand" (p. 5). Interactivity is a fundamental characteristic of searching in digital environments. Users are able to interact with online catalog systems, as well as their collection via multiple avenues. The inherent interactive nature of Web-based IR systems poses a challenge for users (Xie, 2003). According to Xie, there is lack of control in interacting with OPACs through its user interface, although they proved to have ease-of-use of interface design. The existing OPACs do not support both ease-ofuse and user control (Xie& Cool, 2000). Accordingly, the design of any online IR systems such as OPCs need to be clear about user involvement and system role to facilitate user-system interaction (Bates, 1990; White & Ruthven, 2006; Xie, 2003).
The above idea is also supported by Dillon, (2004) who noted that these IR systems (OPACs) were not designed to take into consideration of aspect of the users, the thing which hinders the effectiveness of user-system interactions. According to Shneiderman (1997), search interfaces should provide helpful messages to explain search results and to support progressive refinement. Speaking on the effectiveness of user system, Shneiderman informs that, "if a stop word or misspelling is eliminated from a search input window, or stemmed terms, partial matches, or variant capitalizations are included, users should be made aware of these changes to their query. If the two words in a phrase are not found proximally, then feedback might be given about the occurrence of the words individually. If multiple phrases are being sought, then perhaps documents containing all phrases should be shown first and identified, followed by documents containing subsets, but if no documents are found with all phrases, this would be indicated. A fairly elaborate decision tree (perhaps 50 to 100 branches) of search outcomes and messages might be specified" (Shneiderman, 1997).
The overall research design for this research integrated both qualitative and quantitative research methodologies. The targeted audience was Master students at International Islamic University Malaysia (IIUM), from the Faculty of ICT which has two departments- IT department and Library Science department. The reason behind the selection of the population was for convenience of researcher and ease of access to students as the researcher come from the same faculty. Another reason was to get different view from cross section of postgraduates (Master students), who are the main users of the library, when they were looking for references to their proposals and research report works. Also, bearing the fact that time for study was too short for the researcher to study large sample which could include other faculties at the university, it was convenient for her to stick to one faculty only.
The sampling frames were chosen from the IIUM Masters students from Kulliya (faculty) of ICT and from Library IT departments respectively. IT department constitute a total of 44 students which included 28 male and 16 female. Department of Library science has 23 students which include 9 male and 14 female. In total there are 67 Master students in the faculty of ICT. The proportionate sample of 30 students was drawn out of 67 students. Fifteen students were randomly selected from IT department to include 10 male and 5 female (proportionately). And another fifteen students were randomly selected to include 6 male and 9 female (proportionately).
Quantitative method was used to quantify issues related to respondents' demographic characteristics, awareness, usability, and attitude toward Library OPAC interface features through filling of a set of questionnaire. This method was used to gain general understanding of the extent of effectiveness and efficiency of OPAC interface features to respondents (library users) and how they applied them to accomplish their document/information search from the system.
A 14-item questionnaire was designed, piloted and administered. Each of these questions contained set of alternatives for respondent to choose the desired one. Except for question six where respondents had to circle features mostly familiar to them. The questionnaire was administered to 30 Master students from KICT department from Library and IT departments, randomly selected. Quantitative method for this study also involved the bird's eye view technique, where by separate library environment was selected and respondents given tasks to be performed in their computers while being observed. From the sample of thirty respondents, five (5) respondents were randomly selected and given three types of search tasks to include browsing, basic and advanced searches. Browsing task had 3 questions to be accomplished, basic search task had 5 questions and advance search task had 4 questions. The researcher documented, categorized and evaluated their search tasks.
Qualitative methodology was also applied through face to face interview with randomly selected respondents who performed assigned user tasks and their opinions were coded, recorded and interpreted. The face to face interview helped to enhance our subjective understanding of respondents' interactions with OPAC systems and their levels of achievements in terms of tasks accomplishment in browsing, basic and advance searches. The SPSS software was used to code and analyze data from the survey and interview and its analysis presented in form of tables and histogram to show frequency, percentage and associations.
3.0. FINDINGS AND DISCUSSION
3.1. Overview of Research Findings
The section on research findings and discussion covers seven major areas, including:
a. Demographic characteristics of the respondents: department and gender of respondents
b. The baseline survey of the awareness (knowledge) of OPAC interface features including familiarity with features
c. Usability of the Library OPAC interface features
d. Attitude toward Library OPAC interface features
e. Difficulties/barriers faced by respondents on using the OPAC interface features
f. Implication of the study
3.2. Demographic Characteristics of Respondents
The office records indicate that, the total number of the Masters students in the faculty of ICT is overall of 67 students from IT (44) and Library Science (23) students. (Source: Faculty/Kulliya of ICT General Office, 2008/2009 intake).
The demographic characteristics of the study sample included IIUM Master students from Kulliya of ICT (KICT) which were randomly selected from two faculty departments of Library and Information Science and IT respectively. Fifteen students were selected from each Kulliya department. Total students' sample constituted thirty (30) respondents. The overall response rate of 29 (97%) respondents from the two departments was achieved. The sample respondents' characteristics were as follows:
Master of Library Science= 15 respondents sample out of 23 students in Library department (which included 9 male and 14 female). The sample of 15 respondents constituted 6 male and 9 female (proportionately).
3.3. Awareness (Knowledge) of OPAC Interface Features /Familiarity with Features
Findings of study based on questionnaire distribution to both IT and Library students revealed that awareness about OPAC as enhanced library user-based system and its interface design (features) within IIUM institution is an important first step towards access and eventually increased usage (usability) of the literature materials and other relevant information in library and outside libraries or online databases to aid students in their learning process. To measure respondents' knowledge OPAC interface features the questions on whether they have heard about it and are aware of its basic features were asked. The results of the findings revealed that all students (100%) wer.e aware of the OPAC and its interface features
Discussion on Gender Dimension
The study wanted to find out students level of awareness of interface features presented in their Library OPAC system, and if there was any association or differences based on gender. The overall study findings revealed that there were no significant association between male and female on the issue of awareness of the OPAC interface features at the University Library. Based on the proportionate sample, all respondents (100%), (male and female) reported to be aware with existence of Library OPAC. This response can be attributed to the library user orientation when students join the university. Table 1 below shows the SPSS output results of this response category.
Discussion on Familiarity to the OPAC Interface Features
However, the study wanted to establish the extent of awareness by asking them about familiarity with the range of features provided to them and asked them to indicate how frequent they have been using them by indicating "always", "rarely" or "never" used it. The few, basic interface features were selected to indicate respondents' familiarity to them. The features labeled as "always" indicated the highest level of familiarity; the ones labeled "rarely" indicated least familiar and those labeled - "never" indicated they were unknown to the respondents. The features included:
1 Graphics in the interface (can help user grasp the concept of the system)
2 Dialog boxes (help in recognition of success and recovery from error)
3 Online help (back for users if they get stuck)
4 Tutorials /Wizards (guide user step by step)
5 Use of shortcuts (for rapid access to information)
6 Language suggestions (spelling error check)
7 Help browser/tool tip
The study findings have shown that the frequently- "always" used OPAC interface features are: dialog box (86.7%), followed by use of shortcuts (67.7%) and help browser/toolkit (60%). The finding also revealed the "rare" use of interface features like graphics (56.7%) followed by tutorials/wizard (76.7%), online help (46.7%) and Language suggestion (46.7%). However, significantly, 50% of respondents reported to have "never" used wizard feature during their search. (See Appendix C for tabled results from table 2-8).
These findings suggest that most respondents are well familiar with the use of dialogue box to search for information and are also employing shortcuts features to aid them during their search. The higher percentages of "rare" use of graphics, tutorials/wizards, online help and language suggestion, indicate the unfamiliarity of those features by the respondents. The scope of this study did not allow the researcher to go deep establishing the reasons for each one of the rarely used feature, but one general assumption could be that those rarely used interface feature could be due less exposure of the respondents to information search skills and or computer skills.
3.4. Usability of the Library OPAC interface features
Three questions on the usability of interface features of Library OPAC were asked to respondents on:
1. Whether or not those features helped them and how. This question was provided by set of answers for respondents to choose like (a) query modification/restructuring (b) easy access of materials/timely (c)doing multiple search and (d) aid in visual display
2. Whether or not speed of navigation was fast or slow
3. Whether or not OPAC has Multimedia features (audio, real play etc)
4. Whether or not the OPAC system allow multiple navigation
Help from the interface features
The results of the findings revealed that for the issue of help from the interface features, majority (46.7%) reported to have found interface feature helpful to them in query modification or restructuring. The 30% of respondents reported to have been helped in easy access of materials/timely; in doing multiple search 16.7% and aid in visual display 6.7%. These findings support the previous studies on the various methodologies employed in OPAC studies in importance of good interface design (Hildreth, 2001). The low use of multiple search and visual aid implicate difficulties faced by respondents (users) in applying some of the interface features due to complexity of design or just mere lack of technology know-how on users' part (see output results elaborated on table 9 below).
Majority of respondents (66.7%) agreed that the speed of navigation was fast, and 33% reported to have slow speed when they navigated OPAC pages. Those reported to experience slow speed the reason could be attributed to internet traffic where sometimes due multiple users on the searched pages or sights the delay on page display can obviously be encountered. However this might be seen as rare experience as shown in the response rate (see table 10 below).
Multimedia interface features
The issue of multimedia interface features in OPAC has been the focus of many studies on the interactive OPACs knowledge, concerning how users interact with such system (Kim et al, 1999:89). In this study it was found that there was no any multimedia interface feature in the IIUM Library OPAC system. Majority of respondents (76.7%) reported that Library OPAC has no multimedia interface like video, real players and audio features (see table 11 below). This rendered a weakness to the system as it reduces the level of interaction of users to the system. These results underscore the conclusion reached by other studies that the design of any online IR systems such as OPCs need to be clear about user involvement and system role to facilitate user-system interaction (Bates, 1990; White & Ruthven, 2006 and Xie, 2003). The absence of multimedia interface features in IIUM Library OPAC system help to cripple users' search creativity and inhibit more productive search outputs. OPACs design should take into account also downloading of sound documents to increase users' interactions.
Multiple navigation is ability to search multiple sources in one interface, for example searching library's catalog, an online journals database, and/or a local image database through the library's OPAC. The benefit of multiple navigation is that it allows quick, efficient access to multiple sources of information without the user leaving the OPAC. It leverages the knowledge inside the library such as digital and digitized grey literature. The findings of this study revealed that the IIUM Library OPAC system interface supports multiple navigation, as majority of respondents (80%) reported to have agreed on its functionality (see table 12 below).
Attitude toward Library OPAC search through interface features
The respondents' attitude towards OPAC interface features were gauged by asking their opinions and general feelings towards the use of interface features (as explained earlier this study) in relation to browsing, basic and advance searches, 73% reported the features to be friendly, while 26% said the features were unfriendly (see the table below).
However, a more concrete attitude was indicated through respondents opinions given during face to face interview, on the internet improvement whereby majority of respondents (40%) felt that there should be guidelines in form of tutorials to enable them to understand the functionality of the interface features.
Thirty three percent wanted to OPAC to be improved in terms of support of natural language query. That is users should be able to post their query in query box or through voice query, using natural language string (phraseology). The current textbased search query gives limited search results based indexed terms pre-defined by computer program.
Some respondents showed their concern on improvement on speed of navigation (10%) and other useful features not identified in this study (13%).
User tasks Report
The result of users search related to Browsing, Basic search and Advance search were labeled as "successfully accomplished" and didn't "accomplished" were assigned to respondents' responses which were presented in form of percentages.
The total number of respondents were five who previously participated in survey and interview sessions.
Tasks for Browsing
1. Search the documents having keyword "information" in subject (97.7% successfully accomplished)
2. Search the all documents of some authors: Van Rijsbergen, Michel Gorman(60% successfully accomplished)
3. Find out the documents having word "information retrieval" in title (66.7% successfully accomplished)
Tasks for Basic search
1. Search the book on automation in libraries by Michael D. Cooper (66.7% successfully accomplished)
2. Find out book on terrorism published in the year 2005? (56%successfully accomplished)
3. What is the name of the journal having ISSN "0022-2240"? How one can see display resultsin different format such as printer friendly, MARC format, etc.(76% Didn't accomplish)
4. Find out the document having title fundamentals of computer algorithms? (70%successfully accomplished)
5. Find out the book on artificial intelligence having ISBN 0070522634? (80%successfully accomplished)
Tasks for Advanced search
1. Find out documents on "information management NOT knowledge management" (66.7% successfully accomplished)
2. How many references do you get on information technology in the year 2007- 2008 published in English language? (83% didn't accomplish)
3. How many books published in Malaysia and India on library automation in the year 2005-2008? (76.7% didn't accomplish)
4. Find out the Video's (DVDs) on Women in Islam. (80% didn't accomplish)
Discussion Based on Tasks Results
The finding results, as far as browsing tasks were concerned, showed that majority of respondents successfully accomplished the assigned tasks (see APPENDIX B on browsing tasks output). This implies that respondents ware well conversant with this search criteria.
As far as the basic search was concerned the respondents show that they were capable of performing all the tasks given in this block, except for the task number three where they were required to find the name of the journal having ISSN "0022- 2240". This failure can be due to the fact that many users do not search by using International standard serial number (ISSN) as it was also seen elsewhere in the study findings and other studies that majority of user prefer to search by other search criteria like using keyword, title and author name.
However, the results for advance search showed that respondent did not successfully accomplish the tasks given except for task number one which asked them to look for document which contain "information management NOT knowledge management" (66.7%). This indicates the ease of use of Boolean search criteria. But the failure to accomplish the rest of the tasks can be attributed to their low level of skills in performing advance searches.
3.5. Barriers or difficulties in using the OPAC interface features
The study findings revealed the existence of some barriers on using interface features in the Library OPAC system. The barriers were related in terms of visibility of features (50%), accessibility (16.7%), usability (23%) and navigation (10%). (See table 15 below). This variation of results indicates significant difference in terms of weight of the problems or difficulties faced by the respondents in relation to use. The fact that majority rated visibility as more problematic (50%) is supported by other study findings in the same university by Mansor and Widyawati (2007) which also found lack of visibility of interface status in IIUM Web PAC interface, as reported by sixty (60 percent) of the respondents.
3.6. Implication of the study
The findings of the study support the underlying assumption of this study that the effectiveness and usability of user-interface features in library OPAC would allow easy-of-use and quick retrieval of information and online resources. However, despite previous investigation on usability of IIUM OPAC system (Mansor, 2007), this current study was able to identify interface's areas of user difficulty, particularly in the construction of search strategy using advanced search features.
Generally, the findings of this study revealed the degree of usage of OPAC interface features in the IIUM library but its potential has not fully been utilized. Some issues pertaining to awareness of OPAC interface features and usability such as frequency of use, multiple searches, navigation speed and or duration i.e. time taken to accomplish search task, can render the IIUM Library OPAC less effective. In this study respondents' failure to accomplish the assigned tasks can be paired to the difficulties in usability of interface features. Students' background has no much influence on the findings of this study in terms of the departments they belong and gender. However, their prior knowledge to search strategies need to be re-investigated. Lastly, this study, through its empirical findings has contributed to methodology of usability studies and has increased our knowledge and understanding of the current status of IIUM students in relation to the usability of Library OPAC interface features.
There were some undergoing efforts by IIUM library trying to make a track of its users' effective use of library OPAC. For example the online menu on "The How to Effectively Use the IIUM Library OPAC: Advanced Search" written by Customer Services Division in 2007 (see lib.iiu.edu.my/resources/howto/Advanced_Search.pps), was an initiative towards this end. However, such efforts need to be solidified by devising more effective strategies to help students optimize their searches. Usability study is an essential element of implementing a discovery interface or next generation online catalog. This study recommends for an improvement of IIUM Library OPAC system through collaborations with other IR systems within and outside university. For instance, accommodation of OPAC search skills modules and IR system designs into students' curriculum in their degree programs may help to inform students of ongoing system technological trends with regards to new interface designs which OPACs might have adopted. Students need this knowledge because to them OPAC is an essential gateway to their academic achievements.
This evaluation study of usability of user- interface features for IIUM library OPAC system is a step towards improvement of the overall Library OPAC functioning. The study conclusion is that the interface features of the IIUM Library OPAC were not optimal for serving user needs, with apparent limitations in its design, lack of aid in visual display and lack of multimedia features (audio, real play etc). Yet, an improvement of the interface cannot be the only line of effort in developing a good OPAC system of library. Aspects such as the technological factor, mechanisms for the better use OPAC, or the heightened preparation and awareness of the users (through continuous education) with regards to search techniques, are areas that deserve further attention.
1. Bates, Marcia J. (1990). Where should the person stop and the information search interface start? Information Processing and Management, 26: 575-591.
2. Beaulieu, M., &Borgman, C.L. (1996). A new era for OPAC research: Introduction to the Special Topics Issue on Current Research in Online Public Access Systems. Journal of the American Society for Information Science, 47(7), 491-492.
3. Belkin, N.J., Oddy, R.N., Brooks H.M (1982). "ASK for Information Retrieval: Part 1," Journal of Documentation, Vol.38 pp61-71.
4. Borgman, C.L. (1987) "Individual Differences in the Use of Information Retrieval Systems: Some Issues and some Data" Proceedings of 10th Annual International ACMSIGIL Conference on Research and Development in Information Retrieval. Pp 61-71
5. Borgman, C.L. (1986) "Why are Online Catalogs Hard to Use? Lesson learned from Information Retrieval Studies". Journal of the American Society for Information Science, Vol. 37 pp. 387-400
6. D-Lib Magazine, January 1997. Shneiderman, Ben "Clarifying Search: A User- Interface Framework for Text Searches"
7. Dillon, A. (2004), Designing Usable Electronic Text: Ergonomic Aspects of Human Information Usage, 2nd ed., CRC Press, London.
8. Dix, A.J. et al (1998) Human Computer Interaction. 2nd Edition. Prentice Hall, Europe, London
9. ERCIM News No.27 - October 1996
10. GuhaT. Kand Saraf, V. (2005) OPAC Usability: Assessment through Verbal Protocol.The Electronic Library 23(4): 463-473
11. Hildreth, C.R. (2001). "Accounting for users' inflated assessments of on-line catalogue search performance and usefulness: an experimental study". Information Research, Vol. 6 No. 2, January 2001
12. Hildreth, C. (1995). Online Catalog Design Models: Are We Moving in the Right Direction? Washington, D.C.: Commissioned by the Council on Library Resources.
13. Hildreth, Charles R.(1985). Online Public Access Catalogs. In: Martha E. Williams (Ed.) Annual Review of Information Science and Technology (ARIST), NY: Knowledge Industrial Inc. Vol. 20, pp. 232-285
14. Jansen B.J. & Pooch, U. (2001). A review of web searching studies and a framework for future research, Journal of the American Society for Information Science and Technology, v.52 n.3, p.235-246, Feb. 1, 2001
15. Kim, H. et al, (1999). "Correlations between Users' Characteristics and Preferred Features of Web-based OPAC Evaluation". ETRI Journal. Vol. 21, no4, pp. 83-9. Electronics and Telecommunications Research Institute, Taejon, COREE, REPUBLIQUE DE
16. Large, J.A. &Beheshti, J. (1997). OPACs: A Research Review. Library and Information Science
17. Mansor and Widyawati (2007), "Heuristic evaluation of interface usability for a web-based OPAC". Library Hi Tech, Vol. 25 No 4. pp: 538-549. Emerald Group Publishing Limited
18. Markey, Karen. (1983). "Thus spake the OPAC user." Information Technology and Libraries. 2(4): 381-387
19. Nielson, J. (1993)Usability Engineering, Academic Press, Boston, San Diego, New York etc
20. Othman, Roslina (2006) "Quality Indicators for an Information Retrieval System: Users' Perspective". Faculty of ICT, International Islamic University, Malaysia.
21. Schneiderman, B (1998). Designing the User interface: Strategies for Effective Human Computer Interaction. Addison-Wesley, Longman, Menlo Park, Califonia.
22. Umarani, A., Nagarkar, S., and Jagtap, V. (2003). Usability testing: study of three WEB OPACS from three different countries. Department of Library and Information Science, University of Pune.
23. White R.W & Ruthven,I. (2006). A study of interface support mechanisms for interactive information retrieval. Journal of the American Society for Information Science and Technology archive.Volume 57, Issue 7 (May 2006).
24. Xie, H. (2003). Supporting ease-of-use and user control: Desired features and structure of Web-based online information retrieval systems. Information Processing & Management, 39(6), 899-922.
25. Xie, H. & Cool, C. (2000). Ease-of-use versus user control: An evaluation of Web and non-Web interfaces of online databases. Online Information Review, 24(2): 102-115
(ProQuest: Appendix omitted.)