Duff & Cherry on archival orientation

 

• over the last decade, interest in evaluating archival services and systems has grown; archivists and academic researchers have begun to investigate how different user groups search for and use archival material

 

• impact: the difference something makes, resulting in a "change in state, attitude or behavior of an individual or a group after engagement with the output." impact can be short, medium or long term; indirect or direct; intentional or unintentional; critical or trivial; simple or complex; negative or positive.

 

• this paper reports on a small exploratory study that investigated the impact of archival orientation sessions and was part of the Archival Metrics Project which "seeks to promote a culture of assessment in the archival domain by creating standardized user-based evaluation tools and other performance measures."

 

• 2005: Arc. Mets. Project investigators and advisory board (nine archival partner institutions) met to determine assessment tools needed

 

• Duff/Cherry et al focused on the impact that archival services have on users - and decided to conduct two small exploratory studies - a study of members of a Canadian historical society and the the impact of orientation sessions at Yale.

 

• interested to develop tools to investigate the impact of archival orientation sessions; an essential role of university archivists is to help students develop archival literacy

 

• there is little research that investigate how students learn about archives or the role of the archivist in providing such education

 

• Literature Review (very thorough! nice introduction to it)
  • much has been written about the value of teaching how to use archival sources - public school and university levels.

 

• university archivists use orientation sessions to teach archival literacy

 

• 2002 Yakel found that there has been no empirical work evaluating the outcomes of different types of archival user education. she interviewed some archival users - their archival experience was not memorable

 

• Robert Tissing conducted a survey about archival instruction in 12 institutions, and from the findings developed guidelines for conducting orientation interviews

 

• Jill Katte used Tissing's framework to assess content of online tutorials and produced a model a web based archival user orientation

 

• Susan Allen surveyed instructional programs in special collections at 75 liberal arts colleges - over 80% provided instruction through tours, exhibits and class visits.

 

• Anna Allison did a mail survey - 96% of archives/manuscripts departments at major research insti. provide classroom instruction for undergraduate majors

 

• archives make an impact on individuals, groups, society at large; archival scholars and practitioners have only recently discussed the need to assess these impacts with methods that systematically gather statistics, facts or stories that can articulate impact...

 

• in the UK, the Museums, Libraries and Archives Council have sponsored impact studies

 

• 2002, Wavell, Baxter, Johnson and Williams undertook extensive review of the literature to identify studies of the impact of libraries, archives, museums - the sector does have an impact, but hard evidence from museums and archives is lacking; gap in research impact evaluation/evidence of impact in the archive domain. evidence though that the domain itself is beginning to rectify this. identified a few research studies, one being the longitudinal survey by Public Service Quality Group (PSQG) in Great Britain

 

• the above four people created a web portal to survey to assess archives/library/museum practitioners' awareness and experience with impact evaluation - found 52% of organizations were engaged in assessing social, learning or economic impact  - but evaluation itself was at an early stage, impact evaluation is one step beyond that

 

• 2006 Horton and Spence reviewed research on social and economic impact of archives and noted lack of research on the social, learning and economic impact of archives.  Developed archives-specific impact models

 

• discussion of archival impact in North American literature is scarce, and often focuses on short term outcomes and performance evaluation.

 

• Horton and Spence: social impact - inclusion or overcoming exclusion of individuals or groups in terms of poverty, education, race or disability and may also include issues of health, community safety, employment and education

 

• Wavell et al argue that that literature fails to deliver evidence of a causal relationship between archives and social impact but note the potential for archival impact on social inclusion..
• PSQG study shows a public perception that archives impact social development by preserving culture, strengthening family and community identity, providing learning opportunities, supporting business and administrative activity

 

• studies of economic impact focus on ways archival users contribute to local economy - by supporting public transportation, local shops, restaurants and hotels.

 

• Wavell et al  - learning impact:  individual or community's progression towards and change in their knowledge base through a variety of channels

 

• Horton and Spence argues that the archives sector knows most about learning impact

 

• conclusion of literature review: paucity of research on impact of archives and need to develop methods for gathering data that provide evidence of he difference archives make in individual's lives. need for longitudinal, national studies, using robust tools and indicators that provide evidence of causal links and report on how archives change the way people behave

 

• Developing Models
  • Brophy - Levels of Impact model, rates impact of information services on users in six levels (Horton and Spence adapted it and mapped it to Generic Social Outcomes and Generic Learning Outcomes)

 

• literature suggests that few studies have investigate impact of archives on students' impressions of archival orientation sessions.

 

• Impact of an Orientation Session 
  • authors  surveyed professors and students who attended four orientation sessions at Yale University Library Manuscripts and Archives with the intent to explore methods for assessing impact of archi. instr. sessions. focused on two areas 1. students' level of confidence in finding archival material 2. subsequent use of material

 

• used questionnaires to gather data from professors and students before the orientation session and at the end of the term

 

• (see paper for detailed findings of study)
  • prior to the orientation, over all students had not used many types of primary resources; over half not at all familiar with using archival sources; almost half rated their confidence in finding sources in archives as somewhat to not confident.
  • after the orientation, mean satisfaction with the orientation was at 8 (10 being highest); they included ways to improve the sessions; described what they liked most and least; their confidence in using archival sources increased; professors reported that the orientation had an impact on the students' assignments, but the professors' satisfaction differed in regards to the range of sources used by students

 

• Future Research 
  • measuring impact is very difficult; further research is needed

 

• replicating the study at other university archives may provide data on the impact of different types of orientation sessions - would be helpful to wrok more closely with professors and analyze citation from students' papers to identify archival materials used and in-depth interviews with students