Normalizing Scientometric Indicators of Individual Publications Using Local Cluster Detection Methods on Citation Networks
One of the major shortcomings of widely used
scientometric indicators is that different disciplines cannot be
compared with each other. The issue of cross-disciplinary
normalization has been long discussed, but even the classification
of publications into scientific domains poses problems. Structural
properties of citation networks offer new possibilities, however, the
large size and constant growth of these networks asks for precaution.
Here we present a new tool that in order to perform cross-field
normalization of scientometric indicators of individual publications
relays on the structural properties of citation networks. Due to the
large size of the networks, a systematic procedure for identifying
scientific domains based on a local community detection algorithm
is proposed. The algorithm is tested with different benchmark
and real-world networks. Then, by the use of this algorithm, the
mechanism of the scientometric indicator normalization process is
shown for a few indicators like the citation number, P-index and
a local version of the PageRank indicator. The fat-tail trend of the
article indicator distribution enables us to successfully perform the
indicator normalization process.
[1] Garfield, E. (1998). The Impact Factor and Using It Correctly. Der
Unfallchirurg, 101(6), 413–414.
[2] Bergstrom, C. T., West, J. D., Wiseman, M. A. (2008). The eigenfactor
metrics. Journal of Neuroscience, 28(45), 11433–11434.
[3] Davis P. M. (2008). Eigenfactor: Does the principle of repeated
improvement result in better estimates than raw citation counts. JASIST,
59(13), 2186–2188.
[4] Bollen, J., Rodriguez, M. A., Van de Sompel, H. (2006). Journal status.
Scientometrics, 69(3), 669–687.
[5] Hirsch, J. E. (2005). An index to quantify an inidividual‘s scientific
research output. PNAS, 102(46), 16569–16572.
[6] Schubert, A., Braun, T. (1996). Cross-field normalization of scientometric
indicators. Scientometrics, 36(3), 311–324.
[7] Radicchi, F., Fortunato, S., Castellano, C. (2008). Universality of citation
distributions: Toward an objective measure of scientific impact. PNAS,
105(45), 17268–17272.
[8] Waltman, L., van Eck, N. J. (2013). Source normalized
indicators of citation impact: an overview of different
approaches and an empirical comparison, Scientometrics, 96, 699,
https://doi.org/10.1007/s11192-012-0913-4.
[9] Bouyssou, D., Marchant, T. (2016). Ranking authors using fractional
counting of citations: An axiomatic approach. Journal of Informetrics,
10(1), 183–199, https://doi.org/10.1016/j.joi.2015.12.006.
[10] Zitt, M., Small, H. (2008) Modifying the journal impact factor by
fractional citation weighting: The audience factor. J. Am. Soc. Inf. Sci.,
59(11), 1856–1860, http://dx.doi.org/10.1002/asi.20880.
[11] Kostoff, R. N. (1997) Citation analysis cross-field normalization: a new
paradigm. Scientometrics., 39(3), 225-230.
[12] Seglen, P. O. (1997). Why the impact factor of journals should not be
used for evaluating research. BMJ, 314(7079), 498–502.
[13] Opthof, T. (1997). Sense and nonsense about the impact factor.
Cardiovascular Research, 33(1), 1–7.
[14] Bornmann, L., Daniel, H.-D. (2008). What do citation counts measure?
A review of studies on citing behavior. Journal of Documentation, 64(1),
45–80.
[15] Web of Science. http://www.webofknowledge.com, Accessed on
07/05/2018. [16] Leydesdorff, L., Wagner, C. S., Bornmann, L. (2017). Betweenness
and diversity in journal citation networks as measures of
interdisciplinarity A tribute to Eugene Garfield. Scientometrics.
https://doi.org/10.1007/s11192-017-2528-2.
[17] Page, L., Brin, S., Motwani, R., Winograd, T. (1999). The PageRank
citation ranking: Bringing order to the Web, Technical Report 1999-66,
Stanford InfoLab, November 1999. http://ilpubs.stanford.edu:8090/422/.
Accesed 21 August 2013.
[18] Papp, I., Ercsey-Ravasz, M., Deritei, D., Sumi, R., J´arai-Szab´o, F.,
Florian, R. V., Cabuz, A. I., L´az´ar, Zs.I. (2013). The P-Index: Hirsch
Index of Individual Publications. Proceedings of ISSI, 2013, 2086–2088.
[19] Waltman, L., van Eck, N. J. (2012), A new methodology for constructing
a publicationlevel classification system of science, J. Am. Soc. Inf. Sci.
Technol., 63, 2378–2392.
[20] Ruiz-Castilloa, J., Waltman, L. (2015), Field-normalized citation impact
indicators using algorithmically constructed classification systems of
science, Journal of Informetrics, 9, 102–117.
[21] ˘ Subelj, L., van Eck, N. J., Waltman, L. (2016), Clustering Scientific
Publications Based on Citation Relations: A Systematic Comparison of
Different Methods. PLOS ONE, 11, e0154404.
[22] van Eck, N. J., Waltman, L. (2017) Citation-based clustering of
publications using CitNetExplorer and VOSviewer. Scientometrics, 111,
1053-1070.
[23] Newman, M. E. J. (2001). The structure of scientific collaboration
networks. Proc. Natl. Acad. Sci. USA, 98(2), 404–409.
[24] Newman, M. E. J., Girvan, M. (2004). Finding and evaluating
community structure in networks. Phys. Rev. E, 69, 026113.
[25] Castellano, C., Fortunato, S., Loreto, V. (2009). Statistical physics of
social dynamics. Rev. Mod. Phys., 81, 591.
[26] Bagrow, J. P., Bollt, E. M. (2005). Local method for detecting
communities. Phys. Rev. E, 72(4), 046108.
[27] Radicchi, F., Castellano, C., Cecconi, F., Loreto, V., Parisi, D.
(2004). Defining and identifying communities in networks. PNAS, 101,
2658–2663.
[28] Deritei, D., Lazar, Zs.I., Papp I., Jarai-Szabo, F., Sumi, R., Varga, L.,
Regan, E., Ercsey-Ravasz, M. (2014). Community detection by graph
Voronoi diagrams. New Journal of Physics, 16, 063007.
[29] Lancichinetti, A., Fortunato, S., Radicchi, F. (2008). Benchmark graphs
for testing community detection algorithms. Phys. Rev. E, 78, 046110.
[30] Bastian, M., Heymann, S., Jacomy, M. (2009). Gephi: an open source
software for exploring and manipulating networks. ICWSM, 8, 361–362.
[31] Adamic, L. A., Glance, N. (2005). The political blogosphere and the
2004 US Election. Proceedings of the WWW-2005 Workshop on the
Weblogging Ecosystem.
[32] Durieux, V., Gevenois, P. A. (2010). Bibliometric indicators: quality
measurements of scientific publication. Radiology, 255(2), 342–351.
[33] Hargens, L. L. (2000). Using the literature: reference networks, reference
contexts, and the social structure of scholarship. Am Sociol Rev, 65,
846–865. [34] Van Raan, A. F. J. (2006). Statistical Properties of Bibliometric
Indicators: Research Group Indicator Distributions and Correlations. J.
Am. Soc. Inf. Sci. Tec., 57, 408–430.
[35] Hutchins, B. I., Yuan, X., Anderson, J. M., Santangelo, G. M. (2016).
Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates
to Measure Influence at the Article Level. PLOS Biology., 14, e1002541.
https://doi.org/10.1371/journal.pbio.1002541.
[36] Gonzalez-Betancor, S. M., Dorta-Gonzalez, P. (2017). An indicator of
the impact of journals based on the percentage of their highly cited
publications. Online Information Review, 41, 398–411.
[37] Tsallis, C., De Albuquerque, M. P. (2000). Are citations of scientific
papers a case of nonextensivity? Eur. Phys. J. B, 13, 777–780.
[38] Lehmann, S., Lautrup, B., Jackson, A. D. (2003). Citation networks in
high energy physics. Phys. Rev. E, 68, 026113.
[39] Brzezinski, M. (2015). Power laws in citation distributions: evidence
from Scopus. Scientometrics, 103, 213–228.
[40] Blondel, V. D., Guillaume, J.-L., Lambiotte, R., Lefebvre E. (2008). Fast
unfolding of communities in large networks. J. Stat. Mech., P10008.
[1] Garfield, E. (1998). The Impact Factor and Using It Correctly. Der
Unfallchirurg, 101(6), 413–414.
[2] Bergstrom, C. T., West, J. D., Wiseman, M. A. (2008). The eigenfactor
metrics. Journal of Neuroscience, 28(45), 11433–11434.
[3] Davis P. M. (2008). Eigenfactor: Does the principle of repeated
improvement result in better estimates than raw citation counts. JASIST,
59(13), 2186–2188.
[4] Bollen, J., Rodriguez, M. A., Van de Sompel, H. (2006). Journal status.
Scientometrics, 69(3), 669–687.
[5] Hirsch, J. E. (2005). An index to quantify an inidividual‘s scientific
research output. PNAS, 102(46), 16569–16572.
[6] Schubert, A., Braun, T. (1996). Cross-field normalization of scientometric
indicators. Scientometrics, 36(3), 311–324.
[7] Radicchi, F., Fortunato, S., Castellano, C. (2008). Universality of citation
distributions: Toward an objective measure of scientific impact. PNAS,
105(45), 17268–17272.
[8] Waltman, L., van Eck, N. J. (2013). Source normalized
indicators of citation impact: an overview of different
approaches and an empirical comparison, Scientometrics, 96, 699,
https://doi.org/10.1007/s11192-012-0913-4.
[9] Bouyssou, D., Marchant, T. (2016). Ranking authors using fractional
counting of citations: An axiomatic approach. Journal of Informetrics,
10(1), 183–199, https://doi.org/10.1016/j.joi.2015.12.006.
[10] Zitt, M., Small, H. (2008) Modifying the journal impact factor by
fractional citation weighting: The audience factor. J. Am. Soc. Inf. Sci.,
59(11), 1856–1860, http://dx.doi.org/10.1002/asi.20880.
[11] Kostoff, R. N. (1997) Citation analysis cross-field normalization: a new
paradigm. Scientometrics., 39(3), 225-230.
[12] Seglen, P. O. (1997). Why the impact factor of journals should not be
used for evaluating research. BMJ, 314(7079), 498–502.
[13] Opthof, T. (1997). Sense and nonsense about the impact factor.
Cardiovascular Research, 33(1), 1–7.
[14] Bornmann, L., Daniel, H.-D. (2008). What do citation counts measure?
A review of studies on citing behavior. Journal of Documentation, 64(1),
45–80.
[15] Web of Science. http://www.webofknowledge.com, Accessed on
07/05/2018. [16] Leydesdorff, L., Wagner, C. S., Bornmann, L. (2017). Betweenness
and diversity in journal citation networks as measures of
interdisciplinarity A tribute to Eugene Garfield. Scientometrics.
https://doi.org/10.1007/s11192-017-2528-2.
[17] Page, L., Brin, S., Motwani, R., Winograd, T. (1999). The PageRank
citation ranking: Bringing order to the Web, Technical Report 1999-66,
Stanford InfoLab, November 1999. http://ilpubs.stanford.edu:8090/422/.
Accesed 21 August 2013.
[18] Papp, I., Ercsey-Ravasz, M., Deritei, D., Sumi, R., J´arai-Szab´o, F.,
Florian, R. V., Cabuz, A. I., L´az´ar, Zs.I. (2013). The P-Index: Hirsch
Index of Individual Publications. Proceedings of ISSI, 2013, 2086–2088.
[19] Waltman, L., van Eck, N. J. (2012), A new methodology for constructing
a publicationlevel classification system of science, J. Am. Soc. Inf. Sci.
Technol., 63, 2378–2392.
[20] Ruiz-Castilloa, J., Waltman, L. (2015), Field-normalized citation impact
indicators using algorithmically constructed classification systems of
science, Journal of Informetrics, 9, 102–117.
[21] ˘ Subelj, L., van Eck, N. J., Waltman, L. (2016), Clustering Scientific
Publications Based on Citation Relations: A Systematic Comparison of
Different Methods. PLOS ONE, 11, e0154404.
[22] van Eck, N. J., Waltman, L. (2017) Citation-based clustering of
publications using CitNetExplorer and VOSviewer. Scientometrics, 111,
1053-1070.
[23] Newman, M. E. J. (2001). The structure of scientific collaboration
networks. Proc. Natl. Acad. Sci. USA, 98(2), 404–409.
[24] Newman, M. E. J., Girvan, M. (2004). Finding and evaluating
community structure in networks. Phys. Rev. E, 69, 026113.
[25] Castellano, C., Fortunato, S., Loreto, V. (2009). Statistical physics of
social dynamics. Rev. Mod. Phys., 81, 591.
[26] Bagrow, J. P., Bollt, E. M. (2005). Local method for detecting
communities. Phys. Rev. E, 72(4), 046108.
[27] Radicchi, F., Castellano, C., Cecconi, F., Loreto, V., Parisi, D.
(2004). Defining and identifying communities in networks. PNAS, 101,
2658–2663.
[28] Deritei, D., Lazar, Zs.I., Papp I., Jarai-Szabo, F., Sumi, R., Varga, L.,
Regan, E., Ercsey-Ravasz, M. (2014). Community detection by graph
Voronoi diagrams. New Journal of Physics, 16, 063007.
[29] Lancichinetti, A., Fortunato, S., Radicchi, F. (2008). Benchmark graphs
for testing community detection algorithms. Phys. Rev. E, 78, 046110.
[30] Bastian, M., Heymann, S., Jacomy, M. (2009). Gephi: an open source
software for exploring and manipulating networks. ICWSM, 8, 361–362.
[31] Adamic, L. A., Glance, N. (2005). The political blogosphere and the
2004 US Election. Proceedings of the WWW-2005 Workshop on the
Weblogging Ecosystem.
[32] Durieux, V., Gevenois, P. A. (2010). Bibliometric indicators: quality
measurements of scientific publication. Radiology, 255(2), 342–351.
[33] Hargens, L. L. (2000). Using the literature: reference networks, reference
contexts, and the social structure of scholarship. Am Sociol Rev, 65,
846–865. [34] Van Raan, A. F. J. (2006). Statistical Properties of Bibliometric
Indicators: Research Group Indicator Distributions and Correlations. J.
Am. Soc. Inf. Sci. Tec., 57, 408–430.
[35] Hutchins, B. I., Yuan, X., Anderson, J. M., Santangelo, G. M. (2016).
Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates
to Measure Influence at the Article Level. PLOS Biology., 14, e1002541.
https://doi.org/10.1371/journal.pbio.1002541.
[36] Gonzalez-Betancor, S. M., Dorta-Gonzalez, P. (2017). An indicator of
the impact of journals based on the percentage of their highly cited
publications. Online Information Review, 41, 398–411.
[37] Tsallis, C., De Albuquerque, M. P. (2000). Are citations of scientific
papers a case of nonextensivity? Eur. Phys. J. B, 13, 777–780.
[38] Lehmann, S., Lautrup, B., Jackson, A. D. (2003). Citation networks in
high energy physics. Phys. Rev. E, 68, 026113.
[39] Brzezinski, M. (2015). Power laws in citation distributions: evidence
from Scopus. Scientometrics, 103, 213–228.
[40] Blondel, V. D., Guillaume, J.-L., Lambiotte, R., Lefebvre E. (2008). Fast
unfolding of communities in large networks. J. Stat. Mech., P10008.
@article{"International Journal of Business, Human and Social Sciences:77912", author = "Levente Varga and Dávid Deritei and Mária Ercsey-Ravasz and Răzvan Florian and Zsolt I. Lázár and István Papp and Ferenc Járai-Szabó", title = "Normalizing Scientometric Indicators of Individual Publications Using Local Cluster Detection Methods on Citation Networks", abstract = "One of the major shortcomings of widely used
scientometric indicators is that different disciplines cannot be
compared with each other. The issue of cross-disciplinary
normalization has been long discussed, but even the classification
of publications into scientific domains poses problems. Structural
properties of citation networks offer new possibilities, however, the
large size and constant growth of these networks asks for precaution.
Here we present a new tool that in order to perform cross-field
normalization of scientometric indicators of individual publications
relays on the structural properties of citation networks. Due to the
large size of the networks, a systematic procedure for identifying
scientific domains based on a local community detection algorithm
is proposed. The algorithm is tested with different benchmark
and real-world networks. Then, by the use of this algorithm, the
mechanism of the scientometric indicator normalization process is
shown for a few indicators like the citation number, P-index and
a local version of the PageRank indicator. The fat-tail trend of the
article indicator distribution enables us to successfully perform the
indicator normalization process.", keywords = "Citation networks, scientometric indicator, cross-field
normalization, local cluster detection.", volume = "12", number = "9", pages = "1190-10", }
