Bayesian Network Model for Students- Laboratory Work Performance Assessment: An Empirical Investigation of the Optimal Construction Approach
There are three approaches to complete Bayesian
Network (BN) model construction: total expert-centred, total datacentred,
and semi data-centred. These three approaches constitute the
basis of the empirical investigation undertaken and reported in this
paper. The objective is to determine, amongst these three
approaches, which is the optimal approach for the construction of a
BN-based model for the performance assessment of students-
laboratory work in a virtual electronic laboratory environment. BN
models were constructed using all three approaches, with respect to
the focus domain, and compared using a set of optimality criteria. In
addition, the impact of the size and source of the training, on the
performance of total data-centred and semi data-centred models was
investigated. The results of the investigation provide additional
insight for BN model constructors and contribute to literature
providing supportive evidence for the conceptual feasibility and
efficiency of structure and parameter learning from data. In addition,
the results highlight other interesting themes.
[1] D. Azzi, I. E. Achumba, V. L. Dunn, and G. A. Chukwudebe,
"Intelligent Performance Assessment of Students- Laboratory Work in a
Virtual Electronic Laboratory Environment", Paper submitted to IEEE
Transactions on Learning Technologies, on 09/Dec./2010, for review.
[2] I. E. Achumba, and D. Azzi, "A Virtual Electronic Laboratory for
Genuine Practical Experiences: Description and Evaluation", Paper
submitted to IEEE Transactions on Learning Technologies, for review.
Revised and resubmitted on 09/Dec./2010.
[3] G. F. Cooper and E. Herskovits, "A Bayesian Method for the Induction
of Probabilistic Networks from Data", Machine Learning, vol. 9, no. 4,
pp. 309-347.
[4] N. Friedman, "The Bayesian Structural EM Algorithm", Proceedings of
the 14th Annual Conference on Uncertainty in Artificial Intelligence
(UAI-98), 1998, pp. 129 - 138.
[5] Y. Pan and E. S. Burnside, "The Effects of Training Parameters on
Learning a Probabilistic Expert System for Mammography",
International Congress Series, vol. 1268, 2004, pp. 1027 - 1032.
[6] M. de Jongh, M. and M. J. Druzdzel, "A Comparison of Structural
Distance Measures for Causal Bayesian Network Models", Recent
Advances in Intelligent Information Systems, 2009, pp. 443-456.
Retrieved September 07, 2010, from
http://iis.ipipan.waw.pl/2009/proceedings/iis09-43.pdf.
[7] L. A. ZADEH, "What is Optimal?", IRE Transactions on Information
Theory, 1958, pp. 3. Retrieved September 08, 2010, from
http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=01057441.
[8] M. J. Druzdzel and H. A. Simon, "Causality in Bayesian Belief
Networks", Proceedings of the Conference on Uncertainty in Artificial
Intelligence, 2006, pp. 3-11.
[9] N. Fenton and M. Neil, "Comment: Expert Elicitation for Reliable
System Design", Statistical Science, vol. 21, no. 4, 2006, pp. 451 - 453.
[10] R. Rajabally, P. Sen, S. Whittle, and J. Dalton, "Aids to Bayesian Belief
Network Construction", Proceedings of the 2nd International IEEE
Conference on Intelligent Systems, vol. 2, 2004, pp. 457 - 461.
[11] B. Das, "Generating Conditional Probabilities for Bayesian Networks:
Easing the Knowledge Acquisition Problem", August 4, 2008. Retrieved
on February 24, 2010, from
http://arxiv.org/ftp/cs/papers/0411/0411034.pdf.
[12] Druzdzel, M. J., & van der Gaag, L. C. (2000). Building Probabilistic
Networks: Where Do the Numbers Come from? (Guest Editors-
Introduction). IEEE Transactions on Knowledge and Data Engineering,
12(4), 1-485.
[13] K. Ng and B. Abramson, "Consensus Diagnosis: A Simulation Study",
IEEE Trans on Systems, Man, and Cybernetics, vol. 22, no. 5, 1992, pp.
916 - 928.
[14] M. J. Druzdzel and A. Onisko, "Are Bayesian Networks Sensitive to
Precision of Their Parameters?", Intelligent Information Systems, 2008,
pp. 35 - 44. Retrieved on March 10, 2011, from
ftp://ftp.pitt.edu/users/d/r/druzdzel/iis08.pdf
[15] P. Myllymaki, "Advantages of Bayesian Networks in Data Mining and
Knowledge Discovery", 2010.
http://www.bayesit.com/docs/advantages.html
[16] G. Heinrich, "Parameter estimation for text analysis", Technical Note
Version 2.4, vsonix GmbH and University of Leipzig, August, 2008.
Retrieved on July 19, 2010 from
http://www.arbylon.net/publications/text-est.pdf.
[17] R. G. Cowell, "Parameter Learning from Incomplete Data for Bayesian
Networks", 1999. Retrieved from
http://www.staff.city.ac.uk/~rgc/webpages/aistats99.pdf
[18] L. Oteniya, "Bayesian belief networks for dementia diagnosis and other
applications: a comparison of hand-crafting and construction using a
novel data driven technique", Unpublished PhD Thesis, Department of
Computing Science, University of Stirling, Stirling, FK9 4LA, Scotland
[19] P. Spirtes, C. Glymour, and R. Scheines, "An algorithm for fast
recovery of sparse causal graphs", Social Science Computer Review,
vol. 9, 1991, pp. 62-72. Retrieved on July 22, 2010, from
http://www.hss.cmu.edu/philosophy/techreports/15_Spirtes.pdf.
[20] F. Sahin, M. C. Yavuz, Z. Arnavut, and O. Uluyol, "Fault diagnosis for
airplane engines using Bayesian networks and distributed particle
swarm optimization", Parallel Computing, vol. 33, no. 2, 2007, pp. 124-
143.
[21] D. M. Chickering, D. Geiger, and D. Heckerman, "Learning Bayesian
Networks: Search Methods and Experimental Results", 1995. Retrieved
from http://research.microsoft.com/enus/
um/people/dmax/publications/aistats95.pdf
[22] D. Heckerman, "A Tutorial on Learning With Bayesian Networks",
Technical Report MSR-TR-95-06, USA: Microsoft Research. Retrieved
from http://research.microsoft.com/pubs/69588/tr-95-06.pdf
[23] W. Lam and F. Bacchus, "Learning Bayesian belief networks: an
approach based on the MDL principle", Computational Intelligence,
vol. 10, 1994, pp. 269-293.
[24] H. Akaike, "A new look at the statistical model identification", IEEE
Transactions on Automatic Control, vol. 19, no. 6, 1974, pp. 716-723.
[25] G. Schwarz, "Estimation the dimension of a model", Annals of Statistics
vol. 6, 1978, pp. 462-464.
[26] D. Heckerman, D. Geiger, and D. M. Chickering, "Learning Bayesian
Networks: The Combination of Knowledge and Statistical Data", vol.
20, 1995, pp. 197-243, Kluwer Academic Publishers, Hingham, MA,
USA.
[27] W. Buntine, "Theory refinement on Bayesian networks", Proceedings of
the 7th Annual Conference on Uncertainty in Artificial Intelligence, Los
Angeles, CA, 1991, pp. 52 - 60.
[28] S. Yang and K. Chang, "Comparison of Score Metrics for Bayesian
Network learning", IEEE Transactions on Systems, Man, and
Cybernetics, Part A: Systems and Humans, vol. 32, no. 3, 2002, pp.
419-428. Retrieved on July 22, 2010 from
http://volgenau.gmu.edu/~kchang/publications/journal_pdf/com_score_
matrics.pdf.
[29] G. F. Cooper, "A Bayesian Method for Learning Belief Networks that
Contain Hidden Variables", AAAI Technical Report WS-93-02, 1993.
Retrieved from http://www.aaai.org/Papers/Workshops/1993/WS-93-
02/WS93-02-011.pdf
[30] M. Forster, and E. Sober, "AIC Scores as Evidence - a Bayesian
Interpretation", 2010. Retrieved from
http://philosophy.wisc.edu/sober/forster%20and%20sober%20AIC%20
Scores%20as%20Evidence%20jan%2028%202010.pdf
[31] D. M. Chickering, D. Geiger, and D. Heckerman, "Learning Bayesian
networks is np-hard", Technical Report MSR-TR-94-17, Microsoft
Research, November, 1994. Retrieved on July 22, 2010 from
http://research.microsoft.com/apps/pubs/default.aspx?id=69598.
[32] R. W. Robinson, "Counting labelled acyclic digraphs", 1973. In F.
Harary (Ed.), New Directions in the Theory of Graphs, New York:
Academic Press, pp. 239-273.
[33] S. Anderson, D. Madigan, and M. Perlman, "A characterization of
markov equivalence classes for acyclic digraphs", Annals of Statistics,
vol. 25, 1997, pp. 505 -541.
[34] M. J. Druzdzel, "GeNIe: A development environment for graphical
decision-analytic models", Proceedings of the Annual Symposium of
the American Medical Informatics Association, Washington, D.C.,
1999, pp. 1206. Retrieved from
http://www.pitt.edu/~druzdzel/psfiles/amia99.pdf
[35] NSC (Norsys Software Corp), Netica-J Reference Manual (Version
3.25), 2008. http://www.norsys.com/netica-j/docs/NeticaJ_Man.pdf
[36] ARG (Automated Reasoning Group) (2004), SamIam.
http://reasoning.cs.ucla.edu/samiam/
[37] D. M. Chickering, "The WinMine Toolkit", Technical Report: MSRTR-
2002-103, October 2002. Retrieved from
http://research.microsoft.com/enus/
um/people/dmax/WinMine/WinMine.pdf
[38] M. Ramoni and P. Sebastiani, "Learning Bayesian networks from
incomplete databases", KMi Technical Report KMi-TR-43, Intelligent
Data Analysis Journal, vol. 2, no. 1, 1997.
[39] J. Pearl and S. Russell, "Bayesian Networks", November 2000.
Retrieved February 12, 2010, from
http://www.cs.berkeley.edu/~russell/papers/hbtnn-bn.ps
[40] MR (Microsoft Research), "WinMine Toolkit Tutorial:, Retrieved on
February 02, 2011, from http://research.microsoft.com/enus/
um/people/dmax/WinMine/Tutorial/Tutorial.html.
[41] BNTT (Bayesian Network Toolbox Tutorial), "How to use the Bayes
Net Toolbox", October, 2007. Retrieved on February 02, 2011 from
http://bnt.googlecode.com/svn/trunk/docs/usage.html
[42] D. Pennock, "Evaluating probabilistic predictions", December, 2006.
Retrieved from http://blog.oddhead.com/2006/12/26/evaluatingprobabilistic-
predictions/
[43] G. Blattenberger and F. Lad, "Separating the Brier Score into
Calibration and Refinement Components: A Graphical Exposition", The
American Statistician, vol. 39, no. 1, 1985, pp. 26-32. Retrieved from
http://www.jstor.org/stable/pdfplus/2683902.pdf
[44] M. Morgan and M. Henrion, Uncertainty: a guide to dealing with
uncertainty in quantitative risk and policy analysis, London: Cambridge
University Press, 1990.
[45] P. Doshi, L. Greenwald, and J. Clarke, "Towards Effective Structure
Learning for Large Bayesian Networks", AAAI Technical Report WS-
02-14, 2002. Retrieved from
http://www.aaai.org/Papers/Workshops/2002/WS-02-14/WS02-14-
003.pdf
[46] NSC (Norsys Software Corp), "Advanced Topics: Testing nets with
Cases", 2008. http://www.norsys.com/tutorials/netica/secD/tut_D2.htm
[47] I. J. Good, "Rational decisions", Journal of the Royal Statistical Society,
vol. 14, 1952, pp. 107-114. In M. Roulston, "The Logarithmic Scoring
Rule a.k.a. "ignorance"", Retrieved from
http://www.cawcr.gov.au/bmrc/wefor/staff/eee/verif/Ignorance.html
[48] F. V. Jenson, Bayesian Networks and Decision Graphs. Springer, USA,
2001.
[49] M. S. Roulston and L. A. Smith, "Evaluating probabilistic forecasts
using information theory", Monthly Weather Review, vol. 130, 2002, pp.
1653-1660.
[1] D. Azzi, I. E. Achumba, V. L. Dunn, and G. A. Chukwudebe,
"Intelligent Performance Assessment of Students- Laboratory Work in a
Virtual Electronic Laboratory Environment", Paper submitted to IEEE
Transactions on Learning Technologies, on 09/Dec./2010, for review.
[2] I. E. Achumba, and D. Azzi, "A Virtual Electronic Laboratory for
Genuine Practical Experiences: Description and Evaluation", Paper
submitted to IEEE Transactions on Learning Technologies, for review.
Revised and resubmitted on 09/Dec./2010.
[3] G. F. Cooper and E. Herskovits, "A Bayesian Method for the Induction
of Probabilistic Networks from Data", Machine Learning, vol. 9, no. 4,
pp. 309-347.
[4] N. Friedman, "The Bayesian Structural EM Algorithm", Proceedings of
the 14th Annual Conference on Uncertainty in Artificial Intelligence
(UAI-98), 1998, pp. 129 - 138.
[5] Y. Pan and E. S. Burnside, "The Effects of Training Parameters on
Learning a Probabilistic Expert System for Mammography",
International Congress Series, vol. 1268, 2004, pp. 1027 - 1032.
[6] M. de Jongh, M. and M. J. Druzdzel, "A Comparison of Structural
Distance Measures for Causal Bayesian Network Models", Recent
Advances in Intelligent Information Systems, 2009, pp. 443-456.
Retrieved September 07, 2010, from
http://iis.ipipan.waw.pl/2009/proceedings/iis09-43.pdf.
[7] L. A. ZADEH, "What is Optimal?", IRE Transactions on Information
Theory, 1958, pp. 3. Retrieved September 08, 2010, from
http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=01057441.
[8] M. J. Druzdzel and H. A. Simon, "Causality in Bayesian Belief
Networks", Proceedings of the Conference on Uncertainty in Artificial
Intelligence, 2006, pp. 3-11.
[9] N. Fenton and M. Neil, "Comment: Expert Elicitation for Reliable
System Design", Statistical Science, vol. 21, no. 4, 2006, pp. 451 - 453.
[10] R. Rajabally, P. Sen, S. Whittle, and J. Dalton, "Aids to Bayesian Belief
Network Construction", Proceedings of the 2nd International IEEE
Conference on Intelligent Systems, vol. 2, 2004, pp. 457 - 461.
[11] B. Das, "Generating Conditional Probabilities for Bayesian Networks:
Easing the Knowledge Acquisition Problem", August 4, 2008. Retrieved
on February 24, 2010, from
http://arxiv.org/ftp/cs/papers/0411/0411034.pdf.
[12] Druzdzel, M. J., & van der Gaag, L. C. (2000). Building Probabilistic
Networks: Where Do the Numbers Come from? (Guest Editors-
Introduction). IEEE Transactions on Knowledge and Data Engineering,
12(4), 1-485.
[13] K. Ng and B. Abramson, "Consensus Diagnosis: A Simulation Study",
IEEE Trans on Systems, Man, and Cybernetics, vol. 22, no. 5, 1992, pp.
916 - 928.
[14] M. J. Druzdzel and A. Onisko, "Are Bayesian Networks Sensitive to
Precision of Their Parameters?", Intelligent Information Systems, 2008,
pp. 35 - 44. Retrieved on March 10, 2011, from
ftp://ftp.pitt.edu/users/d/r/druzdzel/iis08.pdf
[15] P. Myllymaki, "Advantages of Bayesian Networks in Data Mining and
Knowledge Discovery", 2010.
http://www.bayesit.com/docs/advantages.html
[16] G. Heinrich, "Parameter estimation for text analysis", Technical Note
Version 2.4, vsonix GmbH and University of Leipzig, August, 2008.
Retrieved on July 19, 2010 from
http://www.arbylon.net/publications/text-est.pdf.
[17] R. G. Cowell, "Parameter Learning from Incomplete Data for Bayesian
Networks", 1999. Retrieved from
http://www.staff.city.ac.uk/~rgc/webpages/aistats99.pdf
[18] L. Oteniya, "Bayesian belief networks for dementia diagnosis and other
applications: a comparison of hand-crafting and construction using a
novel data driven technique", Unpublished PhD Thesis, Department of
Computing Science, University of Stirling, Stirling, FK9 4LA, Scotland
[19] P. Spirtes, C. Glymour, and R. Scheines, "An algorithm for fast
recovery of sparse causal graphs", Social Science Computer Review,
vol. 9, 1991, pp. 62-72. Retrieved on July 22, 2010, from
http://www.hss.cmu.edu/philosophy/techreports/15_Spirtes.pdf.
[20] F. Sahin, M. C. Yavuz, Z. Arnavut, and O. Uluyol, "Fault diagnosis for
airplane engines using Bayesian networks and distributed particle
swarm optimization", Parallel Computing, vol. 33, no. 2, 2007, pp. 124-
143.
[21] D. M. Chickering, D. Geiger, and D. Heckerman, "Learning Bayesian
Networks: Search Methods and Experimental Results", 1995. Retrieved
from http://research.microsoft.com/enus/
um/people/dmax/publications/aistats95.pdf
[22] D. Heckerman, "A Tutorial on Learning With Bayesian Networks",
Technical Report MSR-TR-95-06, USA: Microsoft Research. Retrieved
from http://research.microsoft.com/pubs/69588/tr-95-06.pdf
[23] W. Lam and F. Bacchus, "Learning Bayesian belief networks: an
approach based on the MDL principle", Computational Intelligence,
vol. 10, 1994, pp. 269-293.
[24] H. Akaike, "A new look at the statistical model identification", IEEE
Transactions on Automatic Control, vol. 19, no. 6, 1974, pp. 716-723.
[25] G. Schwarz, "Estimation the dimension of a model", Annals of Statistics
vol. 6, 1978, pp. 462-464.
[26] D. Heckerman, D. Geiger, and D. M. Chickering, "Learning Bayesian
Networks: The Combination of Knowledge and Statistical Data", vol.
20, 1995, pp. 197-243, Kluwer Academic Publishers, Hingham, MA,
USA.
[27] W. Buntine, "Theory refinement on Bayesian networks", Proceedings of
the 7th Annual Conference on Uncertainty in Artificial Intelligence, Los
Angeles, CA, 1991, pp. 52 - 60.
[28] S. Yang and K. Chang, "Comparison of Score Metrics for Bayesian
Network learning", IEEE Transactions on Systems, Man, and
Cybernetics, Part A: Systems and Humans, vol. 32, no. 3, 2002, pp.
419-428. Retrieved on July 22, 2010 from
http://volgenau.gmu.edu/~kchang/publications/journal_pdf/com_score_
matrics.pdf.
[29] G. F. Cooper, "A Bayesian Method for Learning Belief Networks that
Contain Hidden Variables", AAAI Technical Report WS-93-02, 1993.
Retrieved from http://www.aaai.org/Papers/Workshops/1993/WS-93-
02/WS93-02-011.pdf
[30] M. Forster, and E. Sober, "AIC Scores as Evidence - a Bayesian
Interpretation", 2010. Retrieved from
http://philosophy.wisc.edu/sober/forster%20and%20sober%20AIC%20
Scores%20as%20Evidence%20jan%2028%202010.pdf
[31] D. M. Chickering, D. Geiger, and D. Heckerman, "Learning Bayesian
networks is np-hard", Technical Report MSR-TR-94-17, Microsoft
Research, November, 1994. Retrieved on July 22, 2010 from
http://research.microsoft.com/apps/pubs/default.aspx?id=69598.
[32] R. W. Robinson, "Counting labelled acyclic digraphs", 1973. In F.
Harary (Ed.), New Directions in the Theory of Graphs, New York:
Academic Press, pp. 239-273.
[33] S. Anderson, D. Madigan, and M. Perlman, "A characterization of
markov equivalence classes for acyclic digraphs", Annals of Statistics,
vol. 25, 1997, pp. 505 -541.
[34] M. J. Druzdzel, "GeNIe: A development environment for graphical
decision-analytic models", Proceedings of the Annual Symposium of
the American Medical Informatics Association, Washington, D.C.,
1999, pp. 1206. Retrieved from
http://www.pitt.edu/~druzdzel/psfiles/amia99.pdf
[35] NSC (Norsys Software Corp), Netica-J Reference Manual (Version
3.25), 2008. http://www.norsys.com/netica-j/docs/NeticaJ_Man.pdf
[36] ARG (Automated Reasoning Group) (2004), SamIam.
http://reasoning.cs.ucla.edu/samiam/
[37] D. M. Chickering, "The WinMine Toolkit", Technical Report: MSRTR-
2002-103, October 2002. Retrieved from
http://research.microsoft.com/enus/
um/people/dmax/WinMine/WinMine.pdf
[38] M. Ramoni and P. Sebastiani, "Learning Bayesian networks from
incomplete databases", KMi Technical Report KMi-TR-43, Intelligent
Data Analysis Journal, vol. 2, no. 1, 1997.
[39] J. Pearl and S. Russell, "Bayesian Networks", November 2000.
Retrieved February 12, 2010, from
http://www.cs.berkeley.edu/~russell/papers/hbtnn-bn.ps
[40] MR (Microsoft Research), "WinMine Toolkit Tutorial:, Retrieved on
February 02, 2011, from http://research.microsoft.com/enus/
um/people/dmax/WinMine/Tutorial/Tutorial.html.
[41] BNTT (Bayesian Network Toolbox Tutorial), "How to use the Bayes
Net Toolbox", October, 2007. Retrieved on February 02, 2011 from
http://bnt.googlecode.com/svn/trunk/docs/usage.html
[42] D. Pennock, "Evaluating probabilistic predictions", December, 2006.
Retrieved from http://blog.oddhead.com/2006/12/26/evaluatingprobabilistic-
predictions/
[43] G. Blattenberger and F. Lad, "Separating the Brier Score into
Calibration and Refinement Components: A Graphical Exposition", The
American Statistician, vol. 39, no. 1, 1985, pp. 26-32. Retrieved from
http://www.jstor.org/stable/pdfplus/2683902.pdf
[44] M. Morgan and M. Henrion, Uncertainty: a guide to dealing with
uncertainty in quantitative risk and policy analysis, London: Cambridge
University Press, 1990.
[45] P. Doshi, L. Greenwald, and J. Clarke, "Towards Effective Structure
Learning for Large Bayesian Networks", AAAI Technical Report WS-
02-14, 2002. Retrieved from
http://www.aaai.org/Papers/Workshops/2002/WS-02-14/WS02-14-
003.pdf
[46] NSC (Norsys Software Corp), "Advanced Topics: Testing nets with
Cases", 2008. http://www.norsys.com/tutorials/netica/secD/tut_D2.htm
[47] I. J. Good, "Rational decisions", Journal of the Royal Statistical Society,
vol. 14, 1952, pp. 107-114. In M. Roulston, "The Logarithmic Scoring
Rule a.k.a. "ignorance"", Retrieved from
http://www.cawcr.gov.au/bmrc/wefor/staff/eee/verif/Ignorance.html
[48] F. V. Jenson, Bayesian Networks and Decision Graphs. Springer, USA,
2001.
[49] M. S. Roulston and L. A. Smith, "Evaluating probabilistic forecasts
using information theory", Monthly Weather Review, vol. 130, 2002, pp.
1653-1660.
@article{"International Journal of Information, Control and Computer Sciences:49822", author = "Ifeyinwa E. Achumba and Djamel Azzi and Rinat Khusainov", title = "Bayesian Network Model for Students- Laboratory Work Performance Assessment: An Empirical Investigation of the Optimal Construction Approach", abstract = "There are three approaches to complete Bayesian
Network (BN) model construction: total expert-centred, total datacentred,
and semi data-centred. These three approaches constitute the
basis of the empirical investigation undertaken and reported in this
paper. The objective is to determine, amongst these three
approaches, which is the optimal approach for the construction of a
BN-based model for the performance assessment of students-
laboratory work in a virtual electronic laboratory environment. BN
models were constructed using all three approaches, with respect to
the focus domain, and compared using a set of optimality criteria. In
addition, the impact of the size and source of the training, on the
performance of total data-centred and semi data-centred models was
investigated. The results of the investigation provide additional
insight for BN model constructors and contribute to literature
providing supportive evidence for the conceptual feasibility and
efficiency of structure and parameter learning from data. In addition,
the results highlight other interesting themes.", keywords = "Bayesian networks, model construction, parameterlearning, structure learning, performance index, model comparison.", volume = "5", number = "7", pages = "707-10", }
