Monday, January 26, 2015

AALL Releases "The Economic Value of Law Libraries" Report-- Long on Rubrics-- Short on ROI

In October 2013 the American Association of Law Libraries issued a Request for Proposal  seeking consultants to deliver a report on the economic value of law libraries. The RFP specifically stated that “When complete, the report should offer law librarians and the institutions and businesses they serve important metrics (italics added) that can help them calculate the return on investment law libraries provide. AALL set the bar high. I and many others in the organization expected a report that delivered actual ROI metrics.  The final Economic Value of Law Libraries report is 42 pages long. It does not attempt to calculate the value of law libraries or law librarians at all.
 AALL leaders previously indicated that one of the key reasons for selecting the HBR consulting firm was that they were well known to the legal community and therefore the metrics delivered in the “value report” would have credibility with law firm partners and  law school deans. Unfortunately this report contains no data which would ever be shared beyond the members of AALL. In fact the report is devoid of “important metrics.”

If the Australians Could do it...?In April 2014 a consortium of Australian  Library organizations including the Australian Law Library Association hired a consultant SGS Economics and Planning which, developed a methodology and then conducted surveys which actually generated a  financial metric on the value. According to their study  Putting A Value on Priceless, the ROI of information resources and services is $5.43 for every dollar invested.

What Kind of Report is This?  I am still puzzled.  It is too simplistic to be called a blueprint. It is really a collection of rubrics and best practices for a “do it yourself” project. The recommendations in this report simply validate the kind of advice that is routinely given in management programs at the AALL annual conference: “Align yourself with your organizations strategy,” ”adjust your message to your audience,” “ determine your stakeholders communication preferences,” “don’t use library jargon when speaking to management.”   We needed a consultant to tell us that? The only thing report does is to validate common sense and good management practices with some stakeholder data.

What this Report is Not. Many of us already collect sophisticated service metrics. What we needed was a tool or a formula  or a methodology for converting that data into a dollar value. The report doesn't deliver anything at that level of sophistication. The report is a theoretical narrative which is short on math and statistical analysis. The report doesn't follow one of it's own quantitative recommendations:

"Go beyond the mere measurement of activities and utilize methods that measure and demonstrate success or impact on organizational services."

This report never moves from measurement to methodology or impact.  
One size does not fit all. The biggest flaw that becomes apparent in reading the report is that the 3 major constituencies in AALL: private firm, academic and Government/court law libraries operate in drastically different environments supporting stakeholders with dramatically different support needs and service expectations. All of the data provided in the report aggregates the responses from all 3 types of organizations. The institutional  differences invalidate the aggregated responses. For example, the aggregated data indicate that “business development” activities are not highly valued. Anyone who works in a law firm will disagree with that assessment. But two thirds of the respondents work in environments (academic and Government/Court) where this type of service has no meaning.  Unfortunately there is no attempt to report data based on organization type.
The report suggests that  since there was no easy way to measure value across all organization types, HBR and the committee threw up their hands and decided not to try. They use these known differences as an  excuse for not delivering a real ROI metric.  I don’t buy that. They could have and should have been able to develop  methodologies and metrics for individual types of organizations. If it could not have been done all at once, AALL should have allowed the Academic, Private Firm and Government and Court Library SIS’s to work with HBR or another consultant to develop a custom study  for each type of organization. Maybe that needs to happen in the future.

These flaws were completely foreseeable  and  I would have expected a prestigious organization like HBR to recommend a survey methodology to  overcome this kind of problem. That issue should have been obvious at the time they responded to the RFP.

Do it Yourself? Really? The bottom line is that AALL and HBR have produced a report that says “we couldn’t figure out how to measure your value – we hope you have better luck on your own.“ Given the  reduced staff and time pressures we all face, this is an absurd recommendation.  Of course we will all continue to try to hone our own metrics but we expected a  report that reached well beyond what we are able to do as individuals. We expected AALL and HBR to do some heavy lifting and instead they have passed the problem back to the members. 

I do not disagree with many of the report's recommendations… I simply disagree that they delivered the kind of report that was needed and expected by many members... especially members who have already undertaken the kind of initiatives and activities recommended in the report.
Who Needs To Read This Report If you have never attended an AALL conference  and if you don’t  read professional literature, don’t follow any  professional blogs and never speak to your colleagues, this report will  be a real shocker. 

Is there is anyone out there who doesn’t currently report on the activities of their organization?  Thirty percent of the responding directors  (in all types of organizations) indicated that they do not provide any reports to management because they are not required to do so. That is a dangerous and naive place to be.  For that 30% this report should be a "wake up call" and the report will provide valuable guidance on beginning the process of thinking about how to communicate with management and what services are worth measuring.
Here are the key findings of the report:

Qualitative Information

1. Employ both formal and informal communications regularly.

2. Provide context with qualitative data.

3. Use testimonials to highlight the impact of delivered services.

4. Tailor the value message to stakeholder preferences.

Quantitative Information and Analysis

1. Go beyond the mere measurement of activities and utilize methods that measure and demonstrate success or impact on organizational services.

2. When reporting metrics related to specific library activities, report them in the context of the larger frame of importance to the organization.

3. Define the measurements within your organization used to demonstrate value to their stakeholders (clients, elected officials, board of trustees) and adjust library metrics accordingly.

4. Identify the external resources used by stakeholders to evaluate organizational success and especially law library success. Adapt internal processes as appropriate.

5. Identify current formal valuation methodologies used within your organization. Evaluate the applicability of that method for use by the law library and adjust the related processes accordingly.


1. Determine your stakeholder’s information delivery preferences, including distribution channels through a collaborative process with the stakeholder.

2. Create a schedule of reporting that fits into the strategic planning and decision making cycle of the organization to ensure that critical library data is delivered in a relevant timeframe.

3. Refrain from using library jargon when sharing qualitative data.

4. Find or make opportunities to present the information in formal meeting settings.

5. Determine the communication styles that match organizational preferences and get comfortable with different communication styles that can be easily adjusted to formal and informal settings as well as stakeholder preferences.

Comprehensive Best Practices

1. Form strong interpersonal relationships with stakeholders.

2. Employ short, frequent communication through existing channels as defined by stakeholder preferences.

3. Create templates that highlight the key considerations for current strategic initiatives in both narrative and numerical formats.

4. Adjust information delivery to meet a mix of demands and expectations.

5. Define foundational library services and emerging service opportunities within your organizations through a collaborative strategic process with stakeholders.

6. Embrace the leadership responsibilities and expectations of the management and director role.

Draw your own conclusion.
I encourage everyone to read the report and draw their own conclusion. I was expecting a lot more from AALL and HBR. If this report meets the needs of a substantial number of members  then it will have been worth the time and cost invested. 

This report  does not break new ground or  "move the ball forward"  for many professionals who have already developed metrics and effective communications.  A grand slam would have been the development of a formula for converting service metrics into dollar values. I would say that this report stops at "first base."


  1. Excellent analysis of the anemic analysis presented to us. Fingers crossed, they'll take heed. I don't want to know how much they paid HBR. It seems that whatever it was, it was too much. Thank you for contributing your time, once again, to the betterment of the profession for all of us.

  2. thanks, Jean. Ascerbic and right to the point. You expressed my thoughts as I read through the report. Disappointing but I am hopeful it gives some impetus to start working on their relationships and reporting opportunities.
    Karl Gruben

  3. Yeah, I don't a certain extent this report echoes the SLA report, that librarians don't put themselves out there enough. Is it so bad to put it on us? It's our profession and we can't hire someone to rip off the blinders that many of our stakeholders have about its value. Those of you in the Big Cities are awesome, but there are quite a few librarians who don't network or follow trends and are just treading water.

    1. I agree with you and with the report that ultimately it is our responsibility to communicate value. My main point is that data speaks loader than anecdotes and the report failed to compute a value metric for various services ( as I mentioned had been done in the Australian survey). The report also failed to provide a methodology or metric to enable members to compute a value metric themselves. AALL paid top dollar for an outside consultant and I would have expected an outside consultant to deliver more value than a volunteer committee have could deliver on their own for free using SurveyMonkey.