Journal of Systems Engineering and Electronics ›› 2019, Vol. 30 ›› Issue (3): 511-524.doi: 10.21629/JSEE.2019.03.09

• Systems Engineering • Previous Articles     Next Articles

Learning Bayesian networks by constrained Bayesian estimation

Xiaoguang GAO*(), Yu YANG(), Zhigao GUO()   

  • Received:2018-10-25 Online:2019-06-01 Published:2019-07-04
  • Contact: Xiaoguang GAO E-mail:cxg2012@nwpu.edu.cn;youngiv@126.com;buckleyguo@mail.nwpu.edu.cn
  • About author:GAO Xiaoguang was born in 1957. She received her Ph.D. degree from the Northwestern Polytechnical University, Xi'an, China in 1989. She is currently a professor in the Department of System Engineering, Northwestern Polytechnical University. She now is the deputy director of Automatic Control Specialized Committee of China Ordnance Industry Association, specialized committee member of China Aviation Society of Weapon System, specialized committee member of Photoelectric Technology of China Astronautical Society. Her research interests include probabilistic graphical models, deep learning, reinforcement learning, advanced control theory and its application in complex systems, attack defense confrontation and effectiveness evaluation of integrated avionics systems, and aviation fire control and operational effectiveness analysis. E-mail:cxg2012@nwpu.edu.cn|YANG Yu was born in 1991. He received his undergraduate degree from Northwestern Polytechnical University, Xi'an, China. He is now a Ph.D. candidate from the Department of SystemS Engineering, Northwestern Polytechnical University. His areas of research include Bayesian networks, data mining, and image recognition. E-mail:youngiv@126.com|GUO Zhigao was born in 1986. He is currently a Ph.D. candidate at the Department of System Engineering, Northwestern Polytechnical University. He has been the author of peer-reviewed publications on International Journal of Approximate Reasoning, Advanced Methodology for Bayesian Networks, and so on. His research interests cover Bayesian networks, model optimization, knowledge and data mining. E-mail:buckleyguo@mail.nwpu.edu.cn
  • Supported by:
    the National Natural Science Foundation of China(61573285);the Innovation Foundation for Doctor Dissertation of Northwestern Polytechnical University, China(CX201619);This work was supported by the National Natural Science Foundation of China (61573285) and the Innovation Foundation for Doctor Dissertation of Northwestern Polytechnical University, China (CX201619)

Abstract:

Bayesian networks (BNs) have become increasingly popular in recent years due to their wide-ranging applications in modeling uncertain knowledge. An essential problem about discrete BNs is learning conditional probability table (CPT) parameters. If training data are sparse, purely data-driven methods often fail to learn accurate parameters. Then, expert judgments can be introduced to overcome this challenge. Parameter constraints deduced from expert judgments can cause parameter estimates to be consistent with domain knowledge. In addition, Dirichlet priors contain information that helps improve learning accuracy. This paper proposes a constrained Bayesian estimation approach to learn CPTs by incorporating constraints and Dirichlet priors. First, a posterior distribution of BN parameters is developed over a restricted parameter space based on training data and Dirichlet priors. Then, the expectation of the posterior distribution is taken as a parameter estimation. As it is difficult to directly compute the expectation for a continuous distribution with an irregular feasible domain, we apply the Monte Carlo method to approximate it. In the experiments on learning standard BNs, the proposed method outperforms competing methods. It suggests that the proposed method can facilitate solving real-world problems. Additionally, a case study of Wine data demonstrates that the proposed method achieves the highest classification accuracy.

Key words: Bayesian networks (BNs), parameter learning, constraints, sparse data