条件随机场介绍(7)—— An Introduction to Conditional Random Fields

参考文献

[1] S.M.AjiandR.J.McEliece,“Thegeneralizeddistributivelaw,”IEEETrans- actions on Information Theory, vol. 46, no. 2, pp. 325–343, 2000.

[2] Y.Altun,I.Tsochantaridis,andT.Hofmann,“HiddenMarkovsupportvector machines,” in International Conference on Machine Learning (ICML), 2003.

[3] G.AndrewandJ.Gao,“Scalabletrainingofl1-regularizedlog-linearmodels,” in International Conference on Machine Learning (ICML), 2007.

[4] L.R.Bahl,P.F.Brown,P.V.deSouza,andR.L.Mercer,“Maximummutual information estimation of hidden Markov model parameters for speech recogni- tion,” in International Conference on Acoustics, Speech, and Signal Processing (ICASSP), vol. 11, pp. 49–52, 1986.

[5] G. H. Bakir, T. Hofmann, B. Sch ?olkopf, A. J. Smola, B. Taskar, and S. V. N. Vishwanathan, eds., Predicting Structured Data. MIT Press, 2007.

[6] T. Berg-Kirkpatrick, A. Bouchard-C?ot ?e, J. DeNero, and D. Klein, “Painless unsupervised learning with features,” in Conference of the North American Chapter of the Association for Computational Linguistics (HLT/NAACL), pp. 582–590.

[7] A. Bernal, K. Crammer, A. Hatzigeorgiou, and F. Pereira, “Global discriminative learning for higher-accuracy computational gene prediction,” PLoS Computational Biology, vol. 3, no. 3, 2007.

[8] D. P. Bertsekas, Nonlinear Programming. Athena Scientific, 2nd ed., 1999.

[9] J. Besag, “Statistical analysis of non-lattice data,” The Statistician, vol. 24, no. 3, pp. 179–195, 1975.

[10] A. Blake, P. Kohli, and C. Rother, eds., Markov Random Fields for Vision and Image Processing. MIT Press, 2011.

[11] D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent Dirichlet allocation,” Journal of Machine Learning Research, vol. 3, p. 993, 2003.

[12] P. Blunsom and T. Cohn, “Discriminative word alignment with conditional random fields,” in International Conference on Computational Linguistics and Annual Meeting of the Association for Computational Linguistics (COLING- ACL), pp. 65–72, 2006.

[13] L. Bottou, “Stochastic gradient descent examples on toy problems,” 2010.

[14] Y. Boykov and M.-P. Jolly, “Interactive graph cuts for optimal boundary & region segmentation of objects in nd images,” in International Conference on Computer Vision (ICCV), vol. 1, pp. 105–112, 2001.

[15] J. K. Bradley and C. Guestrin, “Learning tree conditional random fields,” in International Conference on Machine Learning (ICML), 2010.

[16] R. Bunescu and R. J. Mooney, “Collective information extraction with rela- tional Markov networks,” in Annual Meeting of the Association for Computational Linguistics (ACL), 2004.

[17] R.H.Byrd,J.Nocedal,andR.B.Schnabel,“Representationsofquasi-Newton matrices and their use in limited memory methods,” Mathematical Programming, vol. 63, no. 2, pp. 129–156, 1994.

[18] R.Caruana,“Multitasklearning,”MachineLearning,vol.28,no.1,pp.41–75, 1997.

[19] R. Caruana and A. Niculescu-Mizil, “An empirical comparison of supervisedlearning algorithms using different performance metrics,” Technical Report TR2005-1973, Cornell University, 2005.

[20] H. L. Chieu and H. T. Ng, “Named entity recognition with a maximum entropy approach,” in Conference on Natural Language Learning (CoNLL), pp. 160–163, 2003.

[21] Y. Choi, C. Cardie, E. Riloff, and S. Patwardhan, “Identifying sources of opinions with conditional random fields and extraction patterns,” in Proceed- ings of the Human Language Technology Conference/Conference on Empirical Methods in Natural Language Processing (HLT-EMNLP), 2005.

[22] C.-T. Chu, S. K. Kim, Y.-A. Lin, Y. Yu, G. Bradski, A. Y. Ng, and K. Oluko- tun, “Map-reduce for machine learning on multicore,” in Advances in Neural Information Processing Systems 19, pp. 281–288, MIT Press, 2007.

[23] S. Clark and J. R. Curran, “Parsing the WSJ using CCG and log-linear models,” in Proceedings of the Meeting of the Association for Computational Linguistics (ACL), pp. 103–110, 2004.

[24] T. Cohn, “Efficient inference in large conditional random fields,” in European Conference on Machine Learning (ECML), pp. 606–613, Berlin, Germany, September 2006.

[25] M. Collins, “Discriminative training methods for hidden Markov models: Theory and experiments with perceptron algorithms,” in Conference on Empirical Methods in Natural Language Processing (EMNLP), 2002.

[26] P. J. Cowans and M. Szummer, “A graphical model for simultaneous parti- tioning and labeling,” in Conference on Artificial Intelligence and Statistics (AISTATS), 2005.

[27] K. Crammer, O. Dekel, J. Keshet, S. Shalev-Shwartz, and Y. Singer, “Online passive-aggressive algorithms,” Journal of Machine Learning Research, 2006.

[28] K. Crammer and Y. Singer, “Ultraconservative online algorithms for multi- class problems,” Journal of Machine Learning Research, vol. 3, pp. 951–991, January 2003.

[29] A. Culotta, R. Bekkerman, and A. McCallum, “Extracting social networks and contact information from email and the web,” in First Conference on Email and Anti-Spam (CEAS), Mountain View, CA, 2004.

[30] A. Culotta and A. McCallum, “Confidence estimation for information extrac- tion,” in Human Language Technology Conference (HLT), 2004.

[31] H. Daum ?e III, J. Langford, and D. Marcu, “Search-based structured predic- tion,” Machine Learning Journal, 2009.

[32] H. Daum ?e III and D. Marcu, “Learning as search optimization: Approximate large margin methods for structured prediction,” in International Conference on Machine Learning (ICML), Bonn, Germany, 2005.

[33] T.Deselaers,B.Alexe,andV.Ferrari,“Localizingobjectswhilelearningtheir appearance,” in European Conference on Computer Vision (ECCV), 2010.

[34] J. V. Dillon and G. Lebanon, “Stochastic composite likelihood,” Journal of Machine Learning Research, vol. 11, pp. 2597–2633, October 2010.

[35] G. Elidan, I. McGraw, and D. Koller, “Residual belief propagation: Informed scheduling for asynchronous message passing,” in Conference on Uncertainty in Artificial Intelligence (UAI), 2006.

[36] P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan, “Object detection with discriminatively trained part based models,” IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 2010.

[37] J. Finkel, T. Grenager, and C. D. Manning, “Incorporating non-local infor- mation into information extraction systems by Gibbs sampling,” in Annual Meeting of the Association for Computational Linguistics (ACL), 2005.

[38] J. R. Finkel, A. Kleeman, and C. D. Manning, “Efficient, feature-based, con- ditional random field parsing,” in Annual Meeting of the Association for Computational Linguistics (ACL/HLT), pp. 959–967, 2008.

[39] K. Ganchev, J. Graca, J. Gillenwater, and B. Taskar, “Posterior regulariza- tion for structured latent variable models,” Technical Report MS-CIS-09-16, University of Pennsylvania Department of Computer and Information Science, 2009.

[40] A.E.GelfandandA.F.M.Smith,“Sampling-basedapproachestocalculating marginal densities,” Journal of the American Statistical Association, vol. 85, pp. 398–409, 1990.

[41] S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 6, pp. 721–741, 1984.

[42] N. Ghamrawi and A. McCallum, “Collective multi-label classification,” in Conference on Information and Knowledge Management (CIKM), 2005.

[43] A. Globerson, T. Koo, X. Carreras, and M. Collins, “Exponentiated gradient algorithms for log-linear structured prediction,” in International Conference on Machine Learning (ICML), 2007.

[44] J. Goodman, “Exponential priors for maximum entropy models,” in Proceedings of the Human Language Technology Conference/North American Chapter of the Association for Computational Linguistics (HLT/NAACL), 2004.

[45] J. Graca, K. Ganchev, B. Taskar, and F. Pereira, “Posterior vs parameter sparsity in latent variable models,” in Advances in Neural Information Pro- cessing Systems 22, (Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, and A. Culotta, eds.), pp. 664–672, 2009.

[46] Y. Grandvalet and Y. Bengio, “Semi-supervised learning by entropy mini- mization,” in Advances in Neural Information Processing Systems (NIPS), 2004.

[47] M.L.GregoryandY.Altun,“Usingconditionalrandomfieldstopredictpitch accents in conversational speech,” in Annual Meeting of the Association for Computational Linguistics (ACL), pp. 677–683, 2004.

[48] A.Gunawardana,M.Mahajan,A.Acero,andJ.C.Platt,“Hiddenconditional random fields for phone classification,” in International Conference on Speech Communication and Technology, 2005.

[49] X. He, R. S. Zemel, and M. A. Carreira-Perpin ?i ?an, “Multiscale conditional random fields for image labelling,” in Conference on Computer Vision and Pattern Recognition (CVPR), 2004.

[50] G. E. Hinton, “Training products of experts by minimizing contrastive diver- gence,” Neural Computation, vol. 14, pp. 1771–1800, 2002.

[51] L. Hirschman, A. Yeh, C. Blaschke, and A. Valencia, “Overview of BioCre- AtIvE: critical assessment of information extraction for biology,” BMC Bioin- formatics, vol. 6, no. Suppl 1, no. Suppl 1, 2005.

[52] F.Jiao,S.Wang,C.-H.Lee,R.Greiner,andD.Schuurmans,“Semi-supervised conditional random fields for improved sequence segmentation and labeling,” in Joint Conference of the International Committee on Computational Lin- guistics and the Association for Computational Linguistics (COLING/ACL), 2006.

[53] S. S. Keerthi and S. Sundararajan, “CRF versus SVM-struct for sequence labeling,” Technical report, Yahoo! Research, 2007.

[54] J.KieferandJ.Wolfowitz,“Stochasticestimationofthemaximumofaregres- sion function,” Annals of Mathematical Statistics, vol. 23, pp. 462–466, 1952.

[55] J.-D. Kim, T. Ohta, Y. Tsuruoka, Y. Tateisi, and N. Collier, “Introduction to the bio-entity recognition task at JNLPBA,” in International joint workshop on natural language processing in biomedicine and its applications, pp. 70–75, Association for Computational Linguistics, 2004.

[56] P. Kohli, L. Ladicky, and P. H. S. Torr, “Robust higher order potentials for enforcing label consistency,” International Journal of Computer Vision, vol. 82, no. 3, no. 3, pp. 302–324, 2009.

[57] D. Koller and N. Friedman, Probabilistic Graphical Models: Principles and Techniques. MIT Press, 2009.

[58] F. R. Kschischang, B. J. Frey, and H.-A. Loeliger, “Factor graphs and the sum-product algorithm,” IEEE Transactions on Information Theory, vol. 47, no. 2, no. 2, pp. 498–519, 2001.

[59] T. Kudo, K. Yamamoto, and Y. Matsumoto, “Applying conditional random fields to Japanese morphological analysis,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2004.

[60] A. Kulesza and F. Pereira, “Structured learning with approximate inference,” in Advances in Neural Information Processing Systems, 2008.

[61] S. Kumar and M. Hebert, “Discriminative fields for modeling spatial depen- dencies in natural images,” in Advances in Neural Information Processing Systems (NIPS), 2003.

[62] S. Kumar and M. Hebert, “Discriminative random fields,” International Jour- nal of Computer Vision, vol. 68, no. 2, no. 2, pp. 179–201, 2006.

[63] J. Lafferty, A. McCallum, and F. Pereira, “Conditional random fields: Prob- abilistic models for segmenting and labeling sequence data,” International Conference on Machine Learning (ICML), 2001.

[64] J.Langford,A.Smola,andM.Zinkevich,“Slowlearnersarefast,”inAdvances in Neural Information Processing Systems 22, (Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, and A. Culotta, eds.), pp. 2331–2339, 2009.

[65] T. Lavergne, O. Capp ?e, and F. Yvon, “Practical very large scale CRFs,” in Annual Meeting of the Association for Computational Linguistics (ACL), pp. 504–513, 2010.

[66] Y. Le Cun, L. Bottou, G. B. Orr, and K.-R. Mu ?ller, “Efficient backprop,” in Neural Networks, Tricks of the Trade, Lecture Notes in Computer Science LNCS 1524, Springer Verlag, 1998.

[67] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, November 1998.

[68] Y. LeCun, S. Chopra, R. Hadsell, R. Marc’Aurelio, and F.-J. Huang, “A tutorial on energy-based learning,” in Predicting Structured Data, (G. Bakir, T. Hofman, B. Sch ?olkopf, A. Smola, and B. Taskar, eds.), MIT Press, 2007.

[69] S. Z. Li, Markov Random Field Modeling in Image Analysis. Springer-Verlag, 2001.

[70] W. Li and A. McCallum, “A note on semi-supervised learning using Markov random fields,” 2004.

[71] P. Liang, H. Daum ?e III, and D. Klein, “Structure compilation: Trading struc- ture for features,” in International Conference on Machine Learning (ICML), pp. 592–599, 2008.

[72] P. Liang and M. I. Jordan, “An asymptotic analysis of generative, discrim- inative, and pseudolikelihood estimators,” in International Conference on Machine Learning (ICML), pp. 584–591, 2008.

[73] P. Liang, M. I. Jordan, and D. Klein, “Learning from measurements in expo- nential families,” in International Conference on Machine Learning (ICML), 2009.

[74] C.-J. Lin, R. C.-H. Weng, and S. Keerthi, “Trust region newton methods for large-scale logistic regression,” in Interational Conference on Machine Learn- ing (ICML), 2007.

[75] B. G. Lindsay, “Composite likelihood methods,” Contemporary Mathematics, pp. 221–239, 1988.

[76] Y. Liu, J. Carbonell, P. Weigele, and V. Gopalakrishnan, “Protein fold recog- nition using segmentation conditional random fields (SCRFs),” Journal of Computational Biology, vol. 13, no. 2, no. 2, pp. 394–406, 2006.

[77] D. G. Lowe, “Object recognition from local scale-invariant features,” in Inter- national Conference on Computer Vision (ICCV), vol. 2, pp. 1150–1157, 1999.

[78] D. J. Lunn, A. Thomas, N. Best, and D. Spiegelhalter, “WinBUGS — a Bayesian modelling framework: Concepts, structure, and extensibility,” Statis- tics and Computing, vol. 10, no. 4, no. 4, pp. 325–337, 2000.

[79] D. J. C. MacKay, Information Theory, Inference, and Learning Algorithms. Cambridge University Press, 2003.

[80] R. Malouf, “A comparison of algorithms for maximum entropy parameter estimation,” in Conference on Natural Language Learning (CoNLL), (D. Roth and A. van den Bosch, eds.), pp. 49–55, 2002.

[81] G. Mann and A. McCallum, “Generalized expectation criteria for semi- supervised learning of conditional random fields,” in Proceedings of Associ- ation of Computational Linguistics, 2008.

[82] M. P. Marcus, B. Santorini, and M. A. Marcinkiewicz, “Building a large anno- tated corpus of English: The Penn Treebank,” Computational Linguistics, vol. 19, no. 2, no. 2, pp. 313–330, 1993.

[83] A. McCallum, “Efficiently inducing features of conditional random fields,” in Conference on Uncertainty in AI (UAI), 2003.

[84] A. McCallum, K. Bellare, and F. Pereira, “A conditional random field for discriminatively-trained finite-state string edit distance,” in Conference on Uncertainty in AI (UAI), 2005.

[85] A. McCallum, D. Freitag, and F. Pereira, “Maximum entropy Markov models for information extraction and segmentation,” in International Conference on Machine Learning (ICML), pp. 591–598, San Francisco, CA, 2000.

[86] A. McCallum and W. Li, “Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons,” in Seventh Conference on Natural Language Learning (CoNLL), 2003.

[87] A. McCallum, K. Schultz, and S. Singh, “FACTORIE: Probabilistic program- ming via imperatively defined factor graphs,” in Advances in Neural Informa- tion Processing Systems (NIPS), 2009.

[88] A. McCallum and B. Wellner, “Conditional models of identity uncertainty with application to noun coreference,” in Advances in Neural Information Processing Systems 17, (L. K. Saul, Y. Weiss, and L. Bottou, eds.), pp. 905– 912, Cambridge, MA: MIT Press, 2005.

[89] R. J. McEliece, D. J. C. MacKay, and J.-F. Cheng, “Turbo decoding as an instance of Pearl’s “belief propagation” algorithm,” IEEE Journal on Selected Areas in Communications, vol. 16, no. 2, no. 2, pp. 140–152, 1998.

[90] S. Miller, J. Guinness, and A. Zamanian, “Name tagging with word clus- ters and discriminative training,” in HLT-NAACL 2004: Main Proceedings, (D. Marcu, S. Dumais, and S. Roukos, eds.), pp. 337–342, Boston, Mas- sachusetts, USA: Association for Computational Linguistics, May 2–May 7 2004.

[91] T. P. Minka, “The EP energy function and minimization schemes,” Technical report, 2001.

[92] T. P. Minka, “A comparsion of numerical optimizers for logistic regression,” Technical report, 2003.

[93] T. P. Minka, “Discriminative models, not discriminative training,” Technical Report MSR-TR-2005-144, Microsoft Research, October 2005.

[94] T. P. Minka, “Divergence measures and message passing,” Technical Report MSR-TR-2005-173, Microsoft Research, 2005.

[95] I. Murray, “Advances in Markov chain Monte Carlo methods,” PhD thesis, Gatsby computational neuroscience unit, University College London, 2007.

[96] I. Murray, Z. Ghahramani, and D. J. C. MacKay, “MCMC for doubly- intractable distributions,” in Uncertainty in Artificial Intelligence (UAI), pp. 359–366, AUAI Press, 2006.

[97] A. Y. Ng, “Feature selection, l1 vs. l2 regularization, and rotational invari- ance,” in International Conference on Machine Learning (ICML), 2004.

[98] A. Y. Ng and M. I. Jordan, “On discriminative vs. generative classifiers: A comparison of logistic regression and naive Bayes,” in Advances in Neu- ral Information Processing Systems 14, (T. G. Dietterich, S. Becker, and Z. Ghahramani, eds.), pp. 841–848, Cambridge, MA: MIT Press, 2002.

[99] N. Nguyen and Y. Guo, “Comparisons of sequence labeling algorithms and extensions,” in International Conference on Machine Learning (ICML), 2007.

[100] J. Nocedal and S. J. Wright, Numerical Optimization. New York: Springer-Verlag, 1999.

[101] S. Nowozin and C. H. Lampert, “Structured prediction and learning in computer vision,” Foundations and Trends in Computer Graphics and Vision, vol. 6, no. 3-4, no. 3-4, 2011.

[102] C. Pal, C. Sutton, and A. McCallum, “Sparse forward-backward using minimum divergence beams for fast training of conditional random fields,” in Inter- national Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2006.

[103] J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988.

[104] F. Peng, F. Feng, and A. McCallum, “Chinese segmentation and new word detection using conditional random fields,” in International Conference on Computational Linguistics (COLING), pp. 562–568, 2004.

[105] F. Peng and A. McCallum, “Accurate information extraction from research papers using conditional random fields,” in Human Language Technology Con- ference and North American Chapter of the Association for Computational Linguistics (HLT-NAACL), 2004.

[106] D.Pinto,A.McCallum,X.Wei,andW.B.Croft,“Tableextractionusingcon- ditional random fields,” in ACM SIGIR Conference on Research and Devel- opment in Information Retrieval, 2003.

[107] Y. Qi, M. Szummer, and T. P. Minka, “Bayesian conditional random fields,” in Conference on Artificial Intelligence and Statistics (AISTATS), 2005.

[108] Y. Qi, M. Szummer, and T. P. Minka, “Diagram structure recognition by Bayesian conditional random fields,” in International Conference on Computer Vision and Pattern Recognition, 2005.

[109] A.Quattoni,M.Collins,andT.Darrell,“Conditionalrandomfieldsforobject recognition,” in Advances in Neural Information Processing Systems (NIPS), pp. 1097–1104, 2005.

[110] A. Quattoni, S. Wang, L.-P. Morency, M. Collins, and T. Darrell, “Hidden- state conditional random fields,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007.

[111] L.R.Rabiner,“AtutorialonhiddenMarkovmodelsandselectedapplications in speech recognition,” Proceedings of the IEEE, vol. 77, no. 2, no. 2, pp. 257– 286, 1989.

[112] N. Ratliff, J. A. Bagnell, and M. Zinkevich, “Maximum margin planning,” in International Conference on Machine Learning, July 2006.

[113] M.RichardsonandP.Domingos,“Markovlogicnetworks,”MachineLearning, vol. 62, no. 1–2, no. 1–2, pp. 107–136, 2006.

[114] S. Riezler, T. King, R. Kaplan, R. Crouch, J. T. Maxwell III, and M. John- son, “Parsing the Wall Street Journal using a lexical-functional grammar and discriminative estimation techniques,” in Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2002.

[115] H. Robbins and S. Monro, “A stochastic approximation method,” Annals of Mathematical Statistics, vol. 22, pp. 400–407, 1951.

[116] C. Robert and G. Casella, Monte Carlo Statistical Methods. Springer, 2004.

[117] D.Rosenberg,D.Klein,andB.Taskar,“Mixture-of-parentsmaximumentropy Markov models,” in Conference on Uncertainty in Artificial Intelligence (UAI), 2007.

[118] D. Roth and W. Yih, “Integer linear programming inference for conditional random fields,” in International Conference on Machine Learning (ICML), pp. 737–744, 2005.

[119] C. Rother, V. Kolmogorov, and A. Blake, “Grabcut: Interactive foreground extraction using iterated graph cuts,” ACM Transactions on Graphics (SIG- GRAPH), vol. 23, no. 3, no. 3, pp. 309–314, 2004.

[120] E. F. T. K. Sang and S. Buchholz, “Introduction to the CoNLL-2000 shared task: Chunking,” in Proceedings of CoNLL-2000 and LLL-2000, 2000. See http://lcg- www.uia.ac.be/~erikt/research/np- chunking.html.

[121] E. F. T. K. Sang and F. D. Meulder, “Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition,” in Proceed- ings of CoNLL-2003, (W. Daelemans and M. Osborne, eds.), pp. 142–147, Edmonton, Canada, 2003.

[122] S. Sarawagi and W. W. Cohen, “Semi-Markov conditional random fields for information extraction,” in Advances in Neural Information Processing Sys- tems 17, (L. K. Saul, Y. Weiss, and L. Bottou, eds.), pp. 1185–1192, Cam- bridge, MA: MIT Press, 2005.

[123] K. Sato and Y. Sakakibara, “RNA secondary structural alignment with conditional random fields,” Bioinformatics, vol. 21, pp. ii237–242, 2005.

[124] B. Settles, “Abner: An open source tool for automatically tagging genes, pro- teins, and other entity names in text,” Bioinformatics, vol. 21, no. 14, no. 14, pp. 3191–3192, 2005.

[125] F. Sha and F. Pereira, “Shallow parsing with conditional random fields,” in Conference on Human Language Technology and North American Association for Computational Linguistics (HLT-NAACL), pp. 213–220, 2003.

[126] S. Shalev-Shwartz, Y. Singer, and N. Srebro, “Pegasos: Primal estimated sub- gradient solver for SVM,” in International Conference on Machine Learning (ICML), 2007.

[127] J.Shotton,J.Winn,C.Rother,andA.Criminisi,“Textonboost:Jointappear- ance, shape and context modeling for mulit-class object recognition and seg- mentation,” in European Conference on Computer Vision (ECCV), 2006.

[128] P. Singla and P. Domingos, “Discriminative training of Markov logic net- works,” in Proceedings of the National Conference on Artificial Intelligence, pp. 868–873, Pittsburgh, PA, 2005.

[129] F. K. Soong and E.-F. Huang, “A tree-trellis based fast search for finding the n-best sentence hypotheses in continuous speech recognition,” in International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 1991.

[130] D. H. Stern, T. Graepel, and D. J. C. MacKay, “Modelling uncertainty in the game of go,” in Advances in Neural Information Processing Systems 17, (L. K. Saul, Y. Weiss, and L. Bottou, eds.), pp. 1353–1360, Cambridge, MA: MIT Press, 2005.

[131] I. Sutskever and T. Tieleman, “On the convergence properties of contrastive divergence,” in Conference on Artificial Intelligence and Statistics (AIS- TATS), 2010.

[132] C. Sutton, “Efficient Training Methods for Conditional Random Fields,” PhD thesis, University of Massachusetts, 2008.

[133] C. Sutton and A. McCallum, “Collective segmentation and labeling of distant entities in information extraction,” in ICML Workshop on Statistical Rela- tional Learning and Its Connections to Other Fields, 2004.

[134] C. Sutton and A. McCallum, “Piecewise training of undirected models,” in Conference on Uncertainty in Artificial Intelligence (UAI), 2005.

[135] C. Sutton and A. McCallum, “Improved dynamic schedules for belief propa- gation,” in Conference on Uncertainty in Artificial Intelligence (UAI), 2007.

[136] C. Sutton and A. McCallum, “An introduction to conditional random fields for relational learning,” in Introduction to Statistical Relational Learning, (L. Getoor and B. Taskar, eds.), MIT Press, 2007.

[137] C. Sutton and A. McCallum, “Piecewise training for structured prediction,” Machine Learning, vol. 77, no. 2–3, no. 2–3, pp. 165–194, 2009.

[138] C. Sutton, A. McCallum, and K. Rohanimanesh, “Dynamic conditional random fields: Factorized probabilistic models for labeling and segmenting sequence data,” Journal of Machine Learning Research, vol. 8, pp. 693–723, March 2007.

[139] C. Sutton and T. Minka, “Local training and belief propagation,” Technical Report TR-2006-121, Microsoft Research, 2006.

[140] C. Sutton, K. Rohanimanesh, and A. McCallum, “Dynamic conditional random fields: Factorized probabilistic models for labeling and segmenting sequence data,” in International Conference on Machine Learning (ICML), 2004.

[141] C. Sutton, M. Sindelar, and A. McCallum, “Reducing weight undertraining in structured discriminative learning,” in Conference on Human Language Technology and North American Association for Computational Linguistics (HLT-NAACL), 2006.

[142] B. Taskar, P. Abbeel, and D. Koller, “Discriminative probabilistic models for relational data,” in Conference on Uncertainty in Artificial Intelligence (UAI), 2002.

[143] B. Taskar, C. Guestrin, and D. Koller, “Max-margin Markov networks,” in Advances in Neural Information Processing Systems 16, (S. Thrun, L. Saul, and B. Sch ?olkopf, eds.), Cambridge, MA: MIT Press, 2004.

[144] B. Taskar, D. Klein, M. Collins, D. Koller, and C. Manning, “Max-margin parsing,” in Empirical Methods in Natural Language Processing (EMNLP04), 2004.

[145] B. Taskar, S. Lacoste-Julien, and D. Klein, “A discriminative matching approach to word alignment,” in Conference on Human Language Technol- ogy and Empirical Methods in Natural Language Processing (HLT-EMNLP), pp. 73–80, 2005.

[146] K. Toutanova, D. Klein, C. D. Manning, and Y. Singer, “Feature-rich part-of- speech tagging with a cyclic dependency network,” in HLT-NAACL, 2003.

[147] I. Tsochantaridis, T. Hofmann, T. Joachims, and Y. Altun, “Support vector machine learning for interdependent and structured output spaces,” in Inter- ational Conference on Machine Learning (ICML), ICML ’04, 2004.

[148] P. Viola and M. Narasimhan, “Learning to extract information from semi- structured text using a discriminative context free grammar,” in Proceedings of the ACM SIGIR, 2005.

[149] S. V. N. Vishwanathan, N. N. Schraudolph, M. W. Schmidt, and K. Mur- phy, “Accelerated training of conditional random fields with stochastic meta-descent,” in International Conference on Machine Learning (ICML), pp. 969–976, 2006.

[150] M. J. Wainwright and M. I. Jordan, “Graphical models, exponential fami- lies, and variational inference,” Foundations and Trends in Machine Learning, vol. 1, no. 1-2, no. 1-2, pp. 1–305, 2008.

[151] M.J.Wainwright,“EstimatingthewrongMarkovrandomfield:Benefitsinthe computation-limited setting,” in Advances in Neural Information Processing Systems 18, (Y. Weiss, B. Sch ?olkopf, and J. Platt, eds.), Cambridge, MA: MIT Press, 2006.

[152] M. J. Wainwright, T. Jaakkola, and A. S. Willsky, “Tree-based reparameteri- zation framework for analysis of sum-product and related algorithms,” IEEE Transactions on Information Theory, vol. 45, no. 9, no. 9, pp. 1120–1146, 2003.

[153] H. Wallach, “Efficient training of conditional random fields,” M.Sc. thesis, University of Edinburgh, 2002.

[154] M.WellingandS.Parise,“Bayesianrandomfields:TheBethe-Laplaceapprox- imation,” in Uncertainty in Artificial Intelligence (UAI), 2006.

[155] M. Wick, K. Rohanimanesh, A. Culotta, and A. McCallum, “SampleRank: Learning preferences from atomic gradients,” in Neural Information Process- ing Systems (NIPS) Workshop on Advances in Ranking, 2009.

[156] M. Wick, K. Rohanimanesh, A. McCallum, and A. Doan, “A discriminative approach to ontology alignment,” in International Workshop on New Trends in Information Integration (NTII), 2008.

[157] J. S. Yedidia, W. T. Freeman, and Y. Weiss, “Constructing free energy approximations and generalized belief propagation algorithms,” Technical Report TR2004-040, Mitsubishi Electric Research Laboratories, 2004.

[158] J.S.Yedidia,W.T.Freeman,andY.Weiss,“Constructingfree-energyapprox- imations and generalized belief propagation algorithms,” IEEE Transactions on Information Theory, vol. 51, no. 7, pp. 2282–2312, July 2005.

[159] C.-N. Yu and T. Joachims, “Learning structural svms with latent variables,” in International Conference on Machine Learning (ICML), 2009.

[160] J. Yu, S. V. N. Vishwanathan, S. Gu ?unter, and N. N. Schraudolph, “A quasi- Newton approach to nonsmooth convex optimization problems in machine learning,” Journal of Machine Learning Research, vol. 11, pp. 1145–1200, March 2010.

[161] Y. Zhang and C. Sutton, “Quasi-Newton Markov chain Monte Carlo,” in Advances in Neural Information Processing Systems (NIPS), 2011.

时间: 2024-08-04 04:33:20

条件随机场介绍(7)—— An Introduction to Conditional Random Fields的相关文章

条件随机场介绍(1)—— An Introduction to Conditional Random Fields

条件随机场介绍 原文:An Introduction to Conditional Random Fields 作者: Charles Sutton (School of Informatics, University of Edinburgh, Edinburgh, EH8 9AB, UK)Andrew McCallum (Department of Computer Science, University of Massachusetts, Amherst, MA, 01003, USA)

An Introduction to Conditional Random Fields论文摘要(翻译:Trey;审校:Shooya)

摘要:通常,我们希望预测大量的,正如依赖于其他观察到的变元一样,互相间同样存在依赖的变元.结构化的预测模型,本质上是分类方法和图形化建模的结合,这些预测模型将图形化模型对多变元数据的有效建模能力,以及分类方法通过使用大的输入特征集来作出预测的能力结合起来.本指引讨论用于结构化预测的一个流行的概率模型--条件随机域.条件随机域广泛应用于自然语言处理.计算机视觉,以及生物信息学.我们讨论推理的方法以及条件随机域的参数估计,包括实现大规模条件随机域的实际问题.我们并没有假设读者具备任何图形化建模的预备

a introduction to conditional random fields

1.Structured prediction methods are essentially a combination of classification and graphical modeling. 2.They combine the ability of graphical models to compactly model multivariate data with the ability of classification methods to perform predicti

条件随机场介绍(4)—— An Introduction to Conditional Random Fields

4. 推断 高效的推断算法对条件随机场的训练和序列预测都非常重要.主要有两个推断问题:第一,模型训练之后,为新的输入\(\mathbf{x}\)确定最可能的标记\(\mathbf{y}^* = \arg \max_{\mathbf{y}} p(\mathbf{y}|\mathbf{x})\):第二,如第5部分所述,参数估计常要求计算标记子集上的边缘分布,例如节点的的边缘分布\(p(y_t|\mathbf{x})\)和边上的边缘分布\(p(y_t,y_{t-1}|\mathbf{x})\).这两个

(转)Image Segmentation with Tensorflow using CNNs and Conditional Random Fields

Daniil's blog Machine Learning and Computer Vision artisan. About/ Blog/ Image Segmentation with Tensorflow using CNNs and Conditional Random Fields Tensorflow and TF-Slim | Dec 18, 2016 A post showing how to perform Image Segmentation with a recentl

条件随机场 Conditional Random Fields

简介 假设你有冠西哥一天生活中的照片(这些照片是按时间排好序的),然后你很无聊的想给每张照片打标签(Tag),比如这张是冠西哥在吃饭,那张是冠西哥在睡觉,那么你该怎么做呢? 一种方法是不管这些照片的序列性(照片本来是按照时间排序的),然后给每张图片弄一个分类器.例如,给了你冠西哥一个月的生活照作为训练样本(打了Tag的),你可能就会学习到:早上6点黑乎乎的照片可能就是冠西哥在睡觉:有很多亮色的照片可能就是冠西哥在跳舞:有很多车的照片可能就是冠西哥在飙车. 很明显,照片的序列性包含有很多信息,忽视

条件随机场(Conditional Random Fields)入门篇

1.1 引言 关系数据有两个特征: 1 待建模实体概率不独立.2 待建模的实体往往有很多特征可以帮助分类.例如,分类网页时,网页的文字信息可以提供很多信息,但网页间的超链接也可以帮助分类.[Tasker et al., 2012] 图模型是一种利用实体间概率依赖性的自然形式,通常,图模型用来表示联合分布P(x,y), y 表示的是期望的预测,x则表示可见的待待建模实体,类似于mechine learing的 输入x(i)与类标记y,但是对非独立变量的联合分布建模会很困难,因为根据乘法公式P(x,

NLP —— 图模型(二)条件随机场(Conditional random field,CRF)

本文简单整理了以下内容: (一)马尔可夫随机场(Markov random field,无向图模型)简单回顾 (二)条件随机场(Conditional random field,CRF) 这篇写的非常浅,基于 [1] 和 [5] 梳理.感觉 [1] 的讲解很适合完全不知道什么是CRF的人来入门.如果有需要深入理解CRF的需求的话,还是应该仔细读一下几个英文的tutorial,比如 [4] . (一)马尔可夫随机场简单回顾 概率图模型(Probabilistic graphical model,P

条件随机场(CRF)原理和实现

版权声明:作者:金良山庄,欲联系请评论博客或私信,个人主页:http://www.jinliangxu.com/,CSDN博客: http://blog.csdn.net/u012176591 目录(?)[+] 作者:金良([email protected]) csdn博客: http://blog.csdn.net/u012176591 对数域操作函数 class Logspace: def __init__(self): self.LOGZERO =np.nan def eexp(self,