Journal of Systems Engineering and Electronics ›› 2019, Vol. 30 ›› Issue (3): 511524.doi: 10.21629/JSEE.2019.03.09
• Systems Engineering • Previous Articles Next Articles
Xiaoguang GAO*(), Yu YANG(), Zhigao GUO()
Received:
20181025
Online:
20190601
Published:
20190704
Contact:
Xiaoguang GAO
Email:cxg2012@nwpu.edu.cn;youngiv@126.com;buckleyguo@mail.nwpu.edu.cn
About author:
GAO Xiaoguang was born in 1957. She received her Ph.D. degree from the Northwestern Polytechnical University, Xi'an, China in 1989. She is currently a professor in the Department of System Engineering, Northwestern Polytechnical University. She now is the deputy director of Automatic Control Specialized Committee of China Ordnance Industry Association, specialized committee member of China Aviation Society of Weapon System, specialized committee member of Photoelectric Technology of China Astronautical Society. Her research interests include probabilistic graphical models, deep learning, reinforcement learning, advanced control theory and its application in complex systems, attack defense confrontation and effectiveness evaluation of integrated avionics systems, and aviation fire control and operational effectiveness analysis. Email:Supported by:
Xiaoguang GAO, Yu YANG, Zhigao GUO. Learning Bayesian networks by constrained Bayesian estimation[J]. Journal of Systems Engineering and Electronics, 2019, 30(3): 511524.
Table 2
Information on standard BNs"
BN  Node  Arc  Parameter  Constraint 
Andes  223  338  1 157  935 
Win95pts  76  112  574  222 
Hepar2  70  123  1 453  398 
Hailfinder  56  66  2 656  437 
Alarm  37  46  509  194 
Insurance  27  52  984  333 
Boerlage92  23  36  86  143 
Sachs  11  17  178  95 
Asia  8  8  18  18 
Survey  6  6  21  20 
Cancer  5  4  10  12 
Earthquake  5  4  10  10 
Weather  4  4  9  9 
Table 3
Average learning results on standard BNs"
BN  Data  ML  CML  MAP  CBE 
Andes  20^{**}  1.273 (0.059)  0.826 (0.039)  0.238 (0.007)  0.066 (0.001) 
100^{**}  0.841 (0.074)  0.626 (0.039)  0.149 (0.009)  0.049 (0.001)  
500^{**}  0.454 (0.026)  0.353 (0.020)  0.078 (0.003)  0.024 (0.001)  
Win95pts  20^{*}  0.452 (0.090)  0.320 (0.074)  0.244 (0.007)  0.141 (0.001) 
100^{*}  0.595 (0.049)  0.485 (0.041)  0.219 (0.010)  0.133 (0.001)  
500^{*}  0.718 (0.081)  0.643 (0.073)  0.208 (0.013)  0.120 (0.001)  
Hepar2  20^{*}  0.608 (0.096)  0.469 (0.075)  0.176 (0.010)  0.128 (0.001) 
100^{*}  0.699 (0.062)  0.569 (0.056)  0.183 (0.009)  0.124 (0.006)  
500^{*}  0.801 (0.078)  0.652 (0.062)  0.197 (0.011)  0.116 (0.004)  
Hailfinder  20^{**}  0.968 (0.075)  0.814 (0.061)  0.224 (0.006)  0.112 (0.001) 
100^{**}  1.095 (0.051)  0.993 (0.053)  0.267 (0.011)  0.093 (0.001)  
500^{***}  1.142 (0.037)  1.089 (0.034)  0.296 (0.009)  0.065 (0.001)  
Alarm  20^{***}  0.560 (0.061)  0.297 (0.044)  0.264 (0.016)  0.057 (0.002) 
100^{**}  0.448 (0.067)  0.283 (0.052)  0.180 (0.015)  0.046 (0.002)  
500^{**}  0.375 (0.051)  0.267 (0.041)  0.118 (0.011)  0.033 (0.002)  
Insurance  20^{*}  0.735 (0.086)  0.527 (0.062)  0.264 (0.009)  0.135 (0.003) 
100^{*}  0.567 (0.066)  0.443 (0.054)  0.171 (0.009)  0.108 (0.002)  
500  0.357 (0.050)  0.285 (0.032)  0.096 (0.006)  0.075 (0.002)  
Boerlage92  20^{***}  1.908 (0.322)  1.192 (0.200)  0.196 (0.031)  0.020 (0.003) 
100^{***}  0.828 (0.201)  0.560 (0.085)  0.105 (0.020)  0.016 (0.003)  
500^{***}  0.260 (0.124)  0.186 (0.079)  0.035 (0.014)  0.008 (0.002)  
Sachs  20^{**}  1.143 (0.196)  0.775 (0.156)  0.249 (0.019)  0.090 (0.004) 
100^{**}  0.675 (0.123)  0.465 (0.074)  0.158 (0.014)  0.063 (0.003)  
500^{**}  0.384 (0.081)  0.283 (0.056)  0.085 (0.008)  0.038 (0.002)  
Asia  20^{***}  0.647 (0.348)  0.349 (0.218)  0.168 (0.038)  0.023 (0.003) 
100^{***}  0.343 (0.180)  0.266 (0.171)  0.088 (0.029)  0.015 (0.002)  
500^{**}  0.147 (0.119)  0.123 (0.111)  0.035 (0.023)  0.012 (0.003)  
Survey  20^{***}  1.410 (0.550)  0.893 (0.366)  0.141 (0.058)  0.023 (0.007) 
100^{***}  0.724 (0.382)  0.340 (0.190)  0.066 (0.028)  0.015 (0.004)  
500^{**}  0.141 (0.193)  0.052 (0.044)  0.030 (0.026)  0.012 (0.005)  
Cancer  20^{***}  0.368 (0.309)  0.290 (0.283)  0.088 (0.034)  0.015 (0.004) 
100^{***}  0.864 (0.866)  0.358 (0.305)  0.058 (0.045)  0.007 (0.002)  
500^{***}  0.131 (0.161)  0.115 (0.155)  0.021 (0.012)  0.004 (0.003)  
Earthquake  20^{***}  0.618 (0.515)  0.320 (0.267)  0.162 (0.012)  0.011 (0.001) 
100^{***}  1.491 (0.765)  0.687 (0.328)  0.142 (0.061)  0.001 (0.001)  
500^{***}  0.414 (0.492)  0.271 (0.172)  0.073 (0.041)  0.006 (0.004)  
Weather  20^{**}  0.438 (0.283)  0.401 (0.267)  0.056 (0.026)  0.020 (0.005) 
100  0.036 (0.049)  0.031 (0.048)  0.012 (0.010)  0.011 (0.003)  
500  0.014 (0.009)  0.013 (0.009)  0.002 (0.001)  0.003 (0.001) 
Table 4
Constraints for the Wine model"
Node  Constraints  
A  a = 1  a = 2  a = 3 
B  p(b = 1)  p(b = 3)  p(b = 1) 
< p(b = 2)  < p(b = 2)  < p(b = 3)  
< p(b = 3)  < p(b = 1)  < p(b = 2)  
C  p(c = 3)  p(c = 3)  p(c = 1) 
< p(c = 2)  < p(c = 2)  < p(c = 3)  
< p(c = 1)  < p(c = 1)  < p(c = 2)  
D  p(d = 1)  p(d = 3)  p(d = 1) 
< p(d = 3)  < p(d = 1)  < p(d = 3)  
< p(d = 2)  < p(d = 2)  < p(d = 2)  
H  p(h = 1)  p(h = 3)  p(h = 1) > 0.99 
< p(h = 3)  < p(h = 1)  
< p(h = 2)  < p(h = 2)  
K  p(k = 3)  p(k = 1) > 0.9  p(k = 1) 
< p(k = 1)  < p(k = 3)  
< p(k = 2)  < p(k = 2)  
L  p(l = 2) > 0.9  p(l = 3)  p(l = 1) > 0.9 
< p(l = 1)  
< p(l = 2)  
N  p(l = 1)  p(n = 1) > 0.9  p(l = 3) 
< p(l = 3)  < p(l = 2)  
< p(l = 2)  < p(l = 1) 
Table 5
Classification results on Wine data according to four feature combinations"
Node  ML  CML  MAP  CBE 
B, C, G, I, J, K, L, M, N  0.93  0.94  0.95  0.96 
B, C, D, E, F, G, I, K, L, N  0.79  0.79  0.88  0.89 
B, C, D, E, F, G, I, J, K, L, M, N  0.82  0.81  0.92  0.94 
B, C, D, E, F, G, H, I, J, K, L, M, N  0.79  0.74  0.92  0.93 
1  PEARL J. Probabilistic reasoning in intelligent systems:networks of plausible inference. San Mateo: Morgan Kaufmann Publishers, 1988. 
2  DE CAMPOS L M. A scoring function for learning Bayesian networks based on mutual information and conditional independence tests. Journal of Machine Learning Research, 2006, 7, 2149 2187. 
3 
JAWORSKA J, GABBERT S, ALDENBERG T. Towards optimization of chemical testing under REACH:a Bayesian network approach to integrated testing strategies. Regulatory Toxicology and Pharmacology, 2010, 57 (23): 157 167.
doi: 10.1016/j.yrtph.2010.02.003 
4 
VELIKOVA M, VAN SCHELTINGA J T, LUCAS P J F. Exploiting causal functional relationships in Bayesian network modelling for personalised healthcare. International Journal of Approximate Reasoning, 2014, 55 (1): 59 73.
doi: 10.1016/j.ijar.2013.03.016 
5 
SUK H, SIN B K, LEE S W. Hand gesture recognition based on dynamic Bayesian network framework. Pattern Recognition, 2010, 43 (9): 3059 3072.
doi: 10.1016/j.patcog.2010.03.016 
6 
YANG G S, LIN Y, BHATTACHARYA P. A driver fatigue recognition model based on information fusion and dynamic Bayesian network. Information Sciences, 2010, 180 (10): 1942 1954.
doi: 10.1016/j.ins.2010.01.011 
7 
CHAN T U, HART B T, KENNARD M J. Bayesian network models for environmental flow decision making in the Daly River, Northern Territory, Australia. River Research and Applications, 2012, 28 (3): 283 301.
doi: 10.1002/rra.1456 
8 
ZHANG Y Z, QU Y H, WANG J D. Estimating leaf area index from MODIS and surface meteorological data using a dynamic Bayesian network. Remote Sensing of Environment, 2012, 127, 30 43.
doi: 10.1016/j.rse.2012.08.015 
9 
PLANT N G, HOLLAND K T. Prediction and assimilation of surfzone processes using a Bayesian network Part Ⅰ:forward models. Coastal Engineering, 2011, 58 (1): 119 130.
doi: 10.1016/j.coastaleng.2010.09.003 
10  CHANG R, SHOEMAKER R, WANG W. A novel knowledgedriven systems biology approach for phenotype prediction upon genetic intervention. IEEE/ACM Trans. on Computational Biology and Bioinformatics, 2011, 3 (8): 683 697. 
11 
CONSTANTINOU A C, FENTON N, MARSH W, et al. From complex questionnaire and interviewing data to intelligent Bayesian network models for medical decision support. Artificial Intelligence in Medicine, 2016, 67, 75 93.
doi: 10.1016/j.artmed.2016.01.002 
12 
CONSTANTINOU A C, FREESTONE M, MARSH W, et al. Risk assessment and risk management of violent reoffending among prisoners. Expert Systems with Applications, 2015, 42 (21): 7511 7529.
doi: 10.1016/j.eswa.2015.05.025 
13 
YET B, PERKINGS Z, FENTON N, et al. Not just data:a method for improving prediction with knowledge. Journal of Biomedical Informatics, 2014, 48, 28 37.
doi: 10.1016/j.jbi.2013.10.012 
14 
CASTELO R, SIEBES A. Priors on network structures. Biaing the search for Bayesian networks. International Journal of Approximate Reasoning, 2000, 24 (1): 39 57.
doi: 10.1016/S0888613X(99)000419 
15  FENTON N, NEIL M, LAGNADO D A. A general structure for legal aruments about evidence using Baysian networks. Cognitive Science, 2013, 37 (1): 61 102. 
16  DRUDZEL M J. Probabilistic reasoning in decision support systems:from computation to common sense. Pittsburgh, USA: Carnegie Mellon University, 1993. 
17 
BRUNK H D. Maximum likelihood estimates of monotone parameters. Annals of Mathematical Statistics, 1955, 26 (4): 607 616.
doi: 10.1214/aoms/1177728420 
18 
DRUDZEL M J, VAN DER GAAG L C. Building probabilistic networks:where do the numbers come from? IEEE Trans. on Knowledge and Data Engineering, 2000, 12 (4): 481 486.
doi: 10.1109/TKDE.2000.868901 
19  SIJBERS J, DEN DEKKER A J, SCHEUNDERS P, et al. Maximum likelihood estimation of Rician distribution parameters. IEEE Trans. on Medical Imaging, 2011, 17 (3): 357 361. 
20 
FEELDERS A, VAN DER GAAG L C. Learning Bayesian network parameters under order constraints. International Journal of Approximate Reasoning, 2006, 42 (12): 37 53.
doi: 10.1016/j.ijar.2005.10.003 
21  DRUDZEL M J, VAN DER GAAG L C. Elicitation of probabilities for belief networks:combining qualitative and quantitative information. Proc. of the 11th Conference on Uncertainty in Artificial Intelligence, 1995, 141 148. 
22  ZHOU Y, FENTON N, TIMOTHY M H, et al. Probabilistic graphical models parameter learning with transferred prior and constraints. Proc. of the 31st Conference on Uncertainty in Artificial Intelligence, 2015, 972 981. 
23  HECKERMAN D, GEIGER D, CHICHERING D M. Learning Bayesian networks:the combination of knowledge and statistical data. Machine Learning, 1995, 20 (3): 197 243. 
24  CHANG R, WANG W. Novel algorithm for Bayesian paramter learning with informative prior constraints. Proc. of the IEEE International Joint Conference on Neural Networks, 2010, 1 8. 
25  DAVIS P J, RABINOWITZ P. Methods of numerical integration. FENG Z X, WU F L Trans. Beijing: Higher Education Press, 1986: 235 255. 
26  ZHOU Y, FENTON N, ZHOU C. An empirical study of Bayesian network parameter learning with monotonic influence constraints. Decision Support System, 2016, 87 (C): 69 79. 
27 
LIAO W H, JI Q. Learning Bayesian network parameters under incomplete data with domain knowledge. Pattern Recognition, 2009, 42 (11): 3046 3056.
doi: 10.1016/j.patcog.2009.04.006 
28  ALTENDORF E E, RESTIFICAR A, DIETTERICH T. Learning from sparse data by exploiting monotonicity constraints. Proc. of the 21st International Conference on Uncertainty in Artificial Intelligence in Edinburgh, 2005, 18 26. 
29  WITTING F, JAMESON A. Exploiting qualitative knowledge in the learning of conditional probabilities of Bayesian networks. Proc. of the 16th International Conference on Uncertainty in Artificial Intelligence, 2000, 644 652. 
30  NICULESCU R S, MITCHELL T M, RAO R B. A theoretical framework for learning Bayesian networks with parameter inequality constraints. Proc. of the 20th International Joint Conference on Artificial Intelligence, 2007, 155 160. 
31  NICULESCU R S, MITCHELL T M, RAO R B. Bayesian network parameter learning with parameter constraints. Journal of Machine Learning Research, 2006, 7 (3): 1357 1383. 
32  NOCEDAL J, WRIGHT S J. Leastsquares problems. NOCEDAL J, WRIGHT S ed. Numerical optimization. New York: Springer, 2006: 245 269. 
33  DE CAMPOS C, TONG Y, JI Q. Constrained maximum likelihood learning of Bayesian networks for facial action recognition. Proc. of the European Conference on Computer Vision, 2008, 168 181. 
34 
DE CAMPOS C, JI Q. Improving Bayesian network parameter learning using constraints. Proc. of the 19th International Conference on Pattern Recognition, 2018, 1 4.
doi: 10.1109/ICPR.2008.4761287 
35  FEELDERS A, VAN DER GAAG L C. Learning Bayesian network parameters with prior knowledge about contexspecific qualitative influences. Computer Science, 2012, arXiv: 1207.1387. 
36  BRUNK H. Maximum likelihood estimates of monotone parameters. Annals of Mathematical Statistics, 1955, 26 (1): 607 611. 
37 
ZHOU Y, FENTON N, NEIL M. Bayesian network approach to multinomial parameter learning using data and expert judgments. International Journal of Approximate Reasoning, 2014, 55 (5): 1252 1268.
doi: 10.1016/j.ijar.2014.02.008 
38  ZHOU Y, FENTON N, NEIL M. An extended MPLC model for Bayesian network parameter learning with exterior constraints. VAN DER GAAG L, FEELDERS A ed. Probabilistic graphical models. Utrecht, The Netherlands: Springer International Publishing, 2014: 581 596. 
39  LANGSETH H, MARQUEZ D, NEIL M. Fast approximate inference in hybrid Bayesian networks using dynamic discretization. Proc. of the International Work Conference on the Interplay between Natural and Artificial Computation, 2013, 7930 (3): 225 234. 
40 
NEIL M, TAILOR M, MARQUEZ D. Inference in hybrid Bayesian networks using dynamic discretization. Statistics and Computing, 2007, 17 (3): 219 233.
doi: 10.1007/s112220079018y 
41  GAN B S. Numerical integration. GAN B S ed. An isogeometric approach to beam structures. Utrecht, The Netherlands: Springer International Publishing, 2018: 47 59. 
42  BOERLAGE B. Link strength in Bayesian networks. Vancouver, Canada: University of British Columbia, 1992. 
43  SCUTARI M. Bnlearnan R package for Bayesian network learning and inference. http://www.bnlearn.com/bnrepository/. 
44 
KULLBACK S, LEIBLER R A. On information and sufficiency. The Annals of Mathematical Statistics, 1951, 22 (1): 79 86.
doi: 10.1214/aoms/1177729694 
45  KOLLER D, FRIEDMAN N. Probabilistic graphical models:principles and techniques. London: The MIT Press, 2009. 
46  CHEN Y C, WHEELER T A, KOCHENDERFER M J. Learning discrete Bayesian networks from continuous data. Computer Science, 2015, 59, 103 132. 
47  MEEHAN A, DE CAMPOS C. Averaged extended tree augmented naïve classifier. Entropy, 2015, 17 (7): 5085 5100. 
48 
LUIS R, SUCAR L E, MORALE E F. Inductive transfer for learning Bayesian networks. Machine Learning, 2010, 79 (12): 227 255.
doi: 10.1007/s1099400951604 
[1]  Fang YE, Ying MAO, Yibing LI, Xinrui LIU. Target threat estimation based on discrete dynamic Bayesian networks with small samples [J]. Journal of Systems Engineering and Electronics, 2022, 33(5): 11351142. 
[2]  Hongwei ZHANG, Weixin XIE. Constrained auxiliary particle filtering for bearingsonly maneuvering target tracking [J]. Journal of Systems Engineering and Electronics, 2019, 30(4): 684695. 
[3]  Ruohai Di, Xiaoguang Gao, and Zhigao Guo. Learning Bayesian network parameters under new monotonic constraints#br# [J]. Journal of Systems Engineering and Electronics, 2017, 28(6): 12481255. 
[4]  Jiufu Liu, Wenliang Liu, Jianyong Zhou, Yan Sun, and Zhisheng Wang. Improved design of online fault diagnoser for partially observed Petri nets with generalized mutual exclusion constraints [J]. Systems Engineering and Electronics, 2017, 28(5): 971978. 
[5]  Shuai Hao1,*, Yongmei Cheng2, and Xu Ma1. Robust detection algorithm with triple constraints for cooperative target based on spectral residual [J]. Systems Engineering and Electronics, 2017, 28(4): 654. 
[6]  Liang Yang, Wanchun Chen, Xiaoming Liu, and Hao Zhou. Robust entry guidance using multisegment linear pseudospectral model predictive control [J]. Systems Engineering and Electronics, 2017, 28(1): 103. 
[7]  Feng Tan, Mingzhe Hou, and Guangren Duan. Stabilization of linear timevarying systems with state and input constraints using convex optimization [J]. Systems Engineering and Electronics, 2016, 27(3): 649655. 
[8]  Zhen He and Yuping Lu. Constrained cooperative control design for distributed morphing wing systems [J]. Journal of Systems Engineering and Electronics, 2012, 23(4): 588595. 
[9]  Xiongbo Wan, Huajing Fang, and Sheng Fu. Fault detection for networked systems subject to access constraints and packet dropouts [J]. Journal of Systems Engineering and Electronics, 2011, 22(1): 127134. 
[10]  Ling Wang?, Dongmei Fu, Qing Li, and Zhichun Mu. Modelling method with missing values based on clustering and support vector regression [J]. Journal of Systems Engineering and Electronics, 2010, 21(1): 142147. 
[11]  Bo Yang, Huajing Fang, and Hua Wang. Secondorder consensus in networks of dynamic agents with communication timedelays [J]. Journal of Systems Engineering and Electronics, 2010, 21(1): 8894. 
[12]  Zhang Qunliang & Xi Yugeng. Resilient predictive control for a class of uncertain timedelay systems [J]. Journal of Systems Engineering and Electronics, 2007, 18(2): 334340. 
Viewed  
Full text 


Abstract 

