Reference statistics sampling proposal

Timeline -

April 23 and/or 30th - Subgroup meets to prep for meeting with Steve.
May 1 - Decide on weeks to sample. Use this time to work on some of the little things below, like creating a little blurb about ref vs. other questions, set up the instructions on the wiki; Remlee target date for completing email
May 4 - Meeting with Steve and Jess re: reporting ref stats in the new model. Bring any final questions for Steve.
May 18 - Complete first draft of email that will be sent out. Send draft to our little subgroup and Steve for comments.
~May 20 - Finalize wording for email. Finalize public wiki
June 1 - Steve to send email introducing the new policy (include link to wiki).
June 15 - Send reminder of upcoming changes. Keep public wiki updated
July 1 - Reference statistics sampling model lauches! Stop taking daily statistics. Send reminder.

Note: Remlee will be away May 8-21.

To do -

1. Draft message describing new system for Steve to send to all-lib? ps-lib?. - (First draft by ~May 15, Remlee)

• Make it clear that everyone who answers any questions from the public should be involved (or however we want to clarify that).
• Keep it brief, so that people will read it.
  • Instead of recording statistics every day, we will be doing recording them four weeks out of the year
  • The 4 weeks will be (x,x,x,x)
  • Remember to record all transactions, whether they happen at the desk or away.
  • A reminder of the difference between reference and other
  • Talk about potential for "other" surveys by committees and divisional libraries. (RISG to talk about coordination of these efforts at May 20th meeting)

2. Make a web / wiki page describing the new system. - Heather & Bill

Reference Statistics Policy 2010

• Is that title too formal? - I think you could go simpler: Reference Statistics Policy, or something like that, with a date DONE
• Link off of RISG page, where?  - We could link off RISG page, but I think we should somehow highlight it from the staff web pages. Right now, there is this: http://libstaff.mit.edu/statistics/.
• Send link with email to all-lib
  * Include attached .pdf of ref statistics sheet.
• What is going to go here? Same wording as email? - Perhaps use the current page on the staff web as an example?

3. Meet with Steve and Jess to figure out logistics.

Monday, May 4th, 2-3 pm in the Hayden Conference Room - Thank you, Remlee!

For this meeting we will have prepared:

• Definition of reference (I'm pretty sure that I missed something about this that maybe came up at the PSLG meeting? Where does this fit in to reference statistics? Does it have to do with whether a question is reference or other or who should be recording statistics?) - Mostly following the RUSA guidelines but replace the word consultation with transaction.
• A rough timeline of how we plan to roll out the policy (when & how we'll publicize, etc.) Heather started; Bill will take a look at this and see if additions or changes need to be made.
• Some ideas for whether the reference statistics sheet needs to change at all? Heather - Incorporate definition of reference to the sheet?
• Discuss which weeks we will sample? It was suggested that Steve picked these before and would likely be willing to continue to pick them. I like that idea. We could possibly make some recommendations, don't bridge two months, yada, yada.
  Bill is going to look at online random number generators.
• Our audience will be all-lib as there are people outside of ps-lib who handle reference questions. Include a note that says, "think this doesn't apply to you? Think again." Include example ref transactions. Craft 2 different message? Ask Steve. What are the boundaries, what are we trying to capture.
• Talk to Jess about reporting structure. What do people without reference coordinators do? Who do they give their sheets to? Send to Jess directly?
  Print or electronic? Both?
• Who will answer questions about the reference statics process after the launch? (Particularly for people that don't have a reference coordinator.)
• Need to think about who will coordinate "special studies" being taken at other times of year? Wait until after re-org?

3.1 Meeting with Steve.

• Choosing weeks to collect data (Steve) - Steve has traditionally chosen the weeks for sampling. We know it is biased, but we accept this. He volunteers to continue choosing these dates.
• Changes to form (Heather) - take out "sample week" indicator, add distinction between reference and other
• Definition of reference - Modified RUSA definition, incorporates reference work in addition to reference transactions
• Process
  Current:
  Create Excel file (Director's office)
  Reference coordinators collect statistics from the units and turn in to the director's office.
  Jess compiles and adds to spreadsheet on R drive.

Future:
Jess does not need statistics at individual level, only at the unit level. Continue to aggregate them around divisions/departments
As of July 1, only needs 1 summary sheet that has away & at desk needs to be maintained.
Ref/other,
at/away from desk
in person/phone/electronic

What about other groups (non-traditional public service), that don't fall within a divisional library?
Nicole H.
Digprob
Sean Thomas
Director's office
Ask each person to fill out their own form. Aggregate at department level? (Technology Services, ALS, Director's office) Make sure this is clear on the form to identify department. Public service side = work through reference coordinator. Everyone else, send form to Jess.
Steve/director's office sends out sample week reminder. People outside of normal PS, send/scan sheet and email to Jess by x date. Send email before, maybe another after. Jess can create list of who she is getting them from and follow up with regular contributors to the statistics.

Communication:
Steve to inform all at DLG meeting of the change. Remind at beginning of sample period.
Before sending formal note to staff, talk to people about the change.

Target lists:
PS-Lib + targeted list. Look at PS-Lib list and see who is on it. A lot of the targeted people may be on the list already.
Look at staff list by department.

4. Publicize the new model email, personal communication within local units

• Make timelines for reminding people to keep statistics
  • Timeline for FY 2009 and lead in to sampling method
  • Timeline for FY 2010 to send reminders that sample weeks are coming and how to find stats sheets.
• Remind them how to distinguish whether a question is "reference" or "other"
• Send wiki link with correspondence
  

*5 Extra refstats data gather or the "other" surveys

• • Short-term user studies?
  • How to coordinate these studies among committees and divisional libraries.
  • Possibly use this wiki space to link to information about other studies: https://wikis.mit.edu/confluence/x/jgYZAg

6. DONE! Decide which definition of reference to use. - I'm thinking we probably want to know this (or have a good sense of where we are,) by the meeting with Steve, May 4.

Definition of Reference for the purpose of the MIT Libraries

Reference and User Services Association (RUSA)
Approved by RUSA Board of Directors, January 14, 2008
Submitted by subgroup of RSS Executive Committee

Revised per RISG 4-15-09

Reference Transactions are information interactions in which library staff recommend, interpret, evaluate, and/or use information resources to help others to meet particular information needs.

Other transactions include exchanges that provide assistance with locations, schedules, equipment, supplies, or policy statements. Other transactions also include lengthy circulation questions that are not counted through the Aleph system.

Each time a library staff member encounters and helps an individual, regardless of number of questions answered within that transaction, counts as a transaction (a tick mark) for statistics-taking purposes.
Reference Work includes reference transactions and other activities that involve the creation, management, and assessment of information or research resources, tools, and services.

(The following bullets clarify what is meant by terms within the Reference Work definition.)

• Creation and management of information resources includes the development and maintenance of research collections, research guides, catalogs, databases, web sites, search engines, etc., that patrons can use independently, in-house or remotely, to satisfy their information needs.
• Assessment activities include the measurement and evaluation of reference work, resources, and services.

How to quantify reference work for statistical purposes will be determined for each relevant assessment project.

Definition that we are not using:

Association of Research Libraries (ARL)

A reference transaction is an information contact that involves the knowledge, use, recommendations, interpretation, or instruction in the use of one or more information sources by a member of the library staff. The term includes information and referral service. Information sources include (a) printed and nonprinted material; (b) machine-readable databases (including computer-assisted instruction); (c) the library's own catalogs and other holdings records; (d) other libraries and institutions through communication or referral; and (e) persons both inside and outside the library. When a staff member uses information gained from previous use of information sources to answer a question, the transaction is reported as a reference transaction even if the source is not consulted again. If a contact includes both reference and directional services, it should be reported as one reference transaction. Duration should not be an element in determining whether a transaction is a reference transaction. Sampling based on a typical week may be used to extrapolate to a full year for Question 34. Please indicate if the figure is based on sampling. Exclude simple directional questions.

A directional transaction is an information contact that facilitates the logistical use of the library and that does not involve the knowledge, use, recommendations, interpretation, or instruction in the use of any information sources other than those that describe the library, such as schedules, floor plans, and handbooks. See The ARL Statistics Web page for links to documents containing current definitions.

It is listed in the instructions for the ARL statistics survey: ARL Statistics survey which is available at: http://www.arl.org/bm~doc/08instruct.pdf

It is based on the ANSI/NISO Z39.7-2004: Information Services and Use: Metrics & statistics for libraries and information providers — Data Dictionary, 7.3, http://www.niso.org/dictionary/section7/. The one online is listed from 2004.

Other Materials

Proposal sent to PSLG

Reference Statistics Review - This links to citations & notes from the literature review we did as background, notes from earlier meetings, also notes related to different reference statistics-taking options including the READ scale, subscription services (such as AltaRama and DeskTracker) and macros in Excel that we tried out. A tool of interest may be Zoho Creator, which allows users to create on the fly databases with some decent options.