Awareness and use of altmetrics among LIS scholars and faculty

Sarah W. Sutton, Ph.D.Rachel Miles, MLS

January 6, 2015ALISE Annual Conference

Abstract

Altmetrics measure the impact of scholarship via mentions in social media and other non-traditional venues. For LIS faculty, altmetrics are also a new area for research. The focus of this presentation is the results of a survey of LIS scholars’ awareness and use of altmetrics.

Outline

• A little bit about altmetrics– Recent literature– LIS faculty awareness

• The study– Who, what, how, when?– Limitations

• Results

What is the level of LIS scholars’ and faculty awareness of altmetrics?

0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

30.00%

35.00%

12.34%

24.68% 24.03%

32.47%

6.49%

1 - never heard of them 2 3 4 5 - I'm an expert

The literature from 2015: Efficacy and use of metrics

• In academic libraries– Suiter and Moulaison (2015) – Booth and Hendrix (2015)

• Information policy– De Groote, Shultz, and Smalheiser (2015)

• Influential altmetrics– Bornmann (2015)– Zahedi, Costas, and Wouters (2015)

• Spurious metrics– Gutierrez , Beall , and Forero 2015– Davis, 2015

The literature from 2015: Explanations and definitions

• Criticism– Gaming– Correlation with citations– Non-academic social media mentions

• Benefits– Non-article research output– Replacing JIF– Measuring attention from the general public(Roemer & Borchardt, 2015)

The literature from 2015

• Related LIS faculty surveys– Peekhaus & Proferes (2015)– Syn and Oh (2015)

• Developing metrics: Relative Citation Ratio – Hutchins, Yuan, Anderson, and Santangelo (2015)

The Survey

• Based on Sutton and Miles. (2015)• Surveygizmo.com • 2,312 invitations sent, 159 responses received• 3 weeks, 1 reminder• 25 – 30 questions• ~ 25 minutes to complete

Limitations & Criticism

• Response rate = 6.88%• No “Goodness of Fit”• Anonymity• Part-time faculty• No research

Variables

• Appointment (full time, part time)– Full time teaching appointment: teaching area – Full time teaching appointment: research area

• Tenure• Years of teaching • Academic status (assistant, associate, full,

emeritus)

Familiarity with altmetrics by appointment type

1 - never heard of

them

2 3 4 5 - I'm an expert

0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

30.00%

35.00%

40.00%

Full time (n=111)Part time (n=43)

Familiarity with altmetrics by tenure track status

1 - neve

r heard

of them 2 3 4

5 - I'm

an exp

ert0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

30.00%

35.00%

40.00%

Tenure track (n=97)Non-tenure track (n=11)

Familiarity with altmetrics by years teaching

< 1 year (n=10)

1 - 5 years (n=51)

6 - 10 years

(n=34)

11 - 20 years

(n=30)

> 20 years (n=28)

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

1 - never heard of them2345 - I'm an expert

Familiarity with altmetrics by years of teaching

1 - never heard of

them

2 3 4 5 - I'm an expert

0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

30.00%

35.00%

40.00%

>=5 years (n=61)<= 6 years (n=92)

Distributions of faculty by years teaching

ALL (n=155) Part-time (n=43) Full-time (n=112)

0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

30.00%

35.00%

40.00%

45.00%

50.00%

Less than one year1 - 5 years6 - 10 years11 - 20 yearsMore than 20 years

Familiarity with multiple metrics

1 - never heard of them 2 3 4 5 - I'm an expert0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

Other metrics mentioned• None, 10• Google Scholar Metrics, 8• Qualitative measures of impact (e.g. who is saying what) , 6• Aggregations (e.g. Altmetric.com, Impact Story,

Academia.edu), 4• SciMago, 3• Research Gate, 2• G-index, 2• Eigenfactor, 2• Books sold/library holdings , 2

Other metrics mentioned once

• Scientometrics• Epistemetrics• Entitymetrics• M-quotient• SNIP (Source Normalized Impact per Paper)

Questions? Comments?

Please contact me:

ssutton3@emporia.edu620-341-5816

References & BibliographyBooth, H., A. & Hendrix, D. (2015). Libraries and institutional data analytics: Challenges and opportunities. The Journal of Academic Librarianship, 41(5), 695–699. http://doi.org/10.1016/j.acalib.2015.08.001 Bornmann, L. (2015a, March 10). How much does the expected number of citations for a publication change if it contains the address of a specific scientific institution? A new approach for the analysis of citation data on the institutional level based on regression models. Retrieved from http://figshare.com/articles/How_much_does_the_expected_number_of_citations_for_a_publication_change_if_it_contains_the_address_of_a_specific_scientific_institution_A_new_approach_for_the_analysis_of_citation_data_on_the_institutional_level_based_on_regression_models/1330139 Bornmann, L. (2015b, March 13). Overlay maps based on Mendeley data: The use of altmetrics for readership networks. Retrieved March 13, 2015, from http://figshare.com/articles/Overlay_map_for_Science_Nature_PNAS/1334179 Bornmann, L. (2015c, March 20). Usefulness of altmetrics for measuring the broader impact of research: A case study using data from PLOS and F1000Prime. Retrieved from http://figshare.com/articles/Usefulness_of_altmetrics_for_measuring_the_broader_impact_of_research_A_case_study_using_data_from_PLOS_and_F1000Prime/1344583 Davis, P. (n.d.). Knockoffs erode trust in metrics market. Retrieved from http://scholarlykitchen.sspnet.org/2015/03/10/knockoffs-erode-trust-in-metrics-market/ De Groote, S. L., Shultz, M., & Smalheiser, N. R. (2015). Examining the impact of the National Institutes of Health public access policy on the citation rates of journal articles. PLoS ONE, 10(10), e0139951. http://doi.org/10.1371/journal.pone.0139951

References & BibliographyDhlman, A. K. (2015). Bibliometrics to altmetrics: Changing trends in assessing research impact. DESIDOC Journal of Library & Information Technology, 35(4), 310–315. Ding, Y., Song, M., Han, J., Yu, Q., Yan, E., Lin, L., & Chambers, T. (2013). Entitymetrics: Measuring the impact of entities. PLoS ONE, 8(8), e71416. http://doi.org/10.1371/journal.pone.0071416 Gutierrez, F. R. S., Beall, J., & Forero, D. A. (2015). Spurious alternative impact factors: The scale of the problem from an academic perspective. BioEssays, n/a–n/a. http://doi.org/10.1002/bies.201500011 Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520(7548), 429–431. http://doi.org/10.1038/520429a Hutchins, B. I., Yuan, X., Anderson, J. M., & Santangelo, G. M. (2015). Relative Citation Ratio (RCR): A new metric that uses citation rates to measure influence at the article level. bioRxiv, 029629. http://doi.org/10.1101/029629 Look out for Bogus Impact Factor Companies. (n.d.). Retrieved from http://scholarlyoa.com/2013/08/06/bogus-impact-factor-companies/ Orduna-Malea, E., Ayllón, J. M., Martín-Martín, A., & López-Cózar, E. D. (2015). Improvements in Google Scholar Citations are for the summer: Creating an institutional affiliation link feature. arXiv:1509.04515 [cs]. Retrieved from http://arxiv.org/abs/1509.04515

References & BibliographyPeekhaus, W., & Proferes, N. (2015). How library and information science faculty perceive and engage with open access. Journal of Information Science, 41(5), 640–661. http://doi.org/10.1177/0165551515587855 Roemer, R. C., & Borchardt, R. (2015). Issues, controversies, and opportunities for altmetrics. Library Technology Reports, 51(5), 20–30. Suiter, A. M., & Moulaison, H. L. (2015). Supporting scholars: An analysis of academic library websites’ documentation on metrics and impact. The Journal of Academic Librarianship, 41(6), 814–820. http://doi.org/10.1016/j.acalib.2015.09.004 Sutton, S. W., & Miles, R. (2015, September). Using alternative metrics for collection development. Presented at the Kansas Library Association / Missouri Library Association Joint Conference, Kansas City, MO. Syn, S. Y., & Oh, S. (2015). Why do social network site users share information on Facebook and Twitter? Journal of Information Science, 41(5), 553–569. http://doi.org/10.1177/0165551515585717 Wasserman, M., Zeng, X. H. T., & Amaral, L. A. N. (2015). Cross-evaluation of metrics to estimate the significance of creative works. Proceedings of the National Academy of Sciences, 112(5), 1281–1286. http://doi.org/10.1073/pnas.1412198112 Zahedi, Z., Costas, R., & Wouters, P. (2014). How well developed are altmetrics? A cross-disciplinary analysis of the presence of “alternative metrics” in scientific publications. Scientometrics, 101(2), 1491–1513. http://doi.org/http://link.springer.com/article/10.1007%2Fs11192-014-1264-0