2016.4.15 近期要读的论文

图像识别

1. Krizhevsky, A., Sutskever, I. & Hinton, G. ImageNet
classification with deepconvolutional neural networks. In Proc. Advances in
Neural InformationProcessing Systems 25 1090–1098 (2012).This report was a
breakthrough that used convolutional nets to almost halvethe error rate for
object recognition, and precipitated the rapid adoption ofdeep learning by the
computer vision community.

2. Farabet, C., Couprie, C., Najman, L. & LeCun, Y. Learning
hierarchical features forscene labeling. IEEE Trans. Pattern Anal. Mach.
Intell. 35, 1915–1929 (2013).

3. Tompson, J., Jain, A., LeCun, Y. & Bregler, C. Joint training
of a convolutionalnetwork and a graphical model for human pose estimation. In
Proc. Advances inNeural Information Processing Systems 27 1799–1807 (2014).

4. Szegedy, C. et al. Going deeper with convolutions. Preprint at
http://arxiv.org/abs/1409.4842 (2014).

使用ReLU从而避免unsupervised
pre-training

28. Glorot, X., Bordes, A. & Bengio. Y. Deep sparse rectifier
neural networks. In Proc.14th International Conference on Artificial
Intelligence and Statistics 315–323(2011).

This paper showed that
supervised training of very deep neural networks is much faster if the hidden
layers are composed of ReLU.

深度网络重燃战火

31. Hinton, G. E. What kind of graphical model is the brain? In
Proc. 19th International Joint Conference on Artificial intelligence
1765–1775 (2005).

32. Hinton, G. E., Osindero, S. & Teh, Y.-W. A fast learning
algorithm for deep belief nets. Neural Comp. 18, 1527–1554 (2006).

This paper introduced a
novel and effective way of training very deep neural networks by pre-training
one hidden layer at a time using the unsupervised learning procedure for
restricted Boltzmann machines.

33. Bengio, Y., Lamblin, P., Popovici, D. & Larochelle, H.
Greedy layer-wise training of deep networks. In Proc. Advances in Neural
Information Processing Systems 19 153–160 (2006).

This report demonstrated
that the unsupervised pre-training method introduced in ref. 32 significantly
improves performance on test data and generalizes the method to other
unsupervised representation-learning techniques, such as auto-encoders.

34. Ranzato, M., Poultney, C., Chopra, S. & LeCun, Y. Efficient
learning of sparse representations with an energy-based model. In Proc.
Advances in Neural Information Processing Systems 19 1137–1144 (2006).

无监督初始化，bp fine-tune

33. Bengio, Y., Lamblin, P., Popovici, D. & Larochelle, H.
Greedy layer-wise trainingof deep networks. In Proc. Advances in Neural
Information Processing Systems 19 153–160 (2006).

This report demonstrated
that the unsupervised pre-training method introduced in ref. 32 significantly
improves performance on test data and generalizes the method to other
unsupervised representation-learning techniques, such as auto-encoders.

34. Ranzato, M., Poultney, C., Chopra, S. & LeCun, Y. Efficient
learning of sparse representations with an energy-based model. In Proc.
Advances in Neural Information Processing Systems 19 1137–1144 (2006).

35. Hinton, G. E. & Salakhutdinov, R. Reducing the
dimensionality of data with neural networks. Science 313, 504–507 (2006).

小数据上采用pre-training + fine-tune进行手写数字识别和行人检测

36. Sermanet, P., Kavukcuoglu, K., Chintala, S. & LeCun, Y.
Pedestrian detection with unsupervised multi-stage feature learning. In Proc.
International Conference on Computer Vision and Pattern Recognition
http://arxiv.org/abs/1212.0142 (2013).

采用gpu进行训练

37. Raina, R., Madhavan, A. & Ng, A. Y. Large-scale deep
unsupervised learning using graphics processors. In Proc. 26th Annual
International Conference on Machine Learning 873–880 (2009).

小数据集上pre-training 防止过拟合

40. Bengio, Y., Courville, A. & Vincent, P. Representation
learning: a review and new perspectives. IEEE Trans. Pattern Anal. Machine
Intell. 35, 1798–1828 (2013).

卷积神经网络

41. LeCun, Y. et al. Handwritten digit recognition with a
back-propagation network. In Proc. Advances in Neural Information Processing
Systems 396–404 (1990).

This is the first paper on
convolutional networks trained by backpropagation for the task of classifying
low-resolution images of handwritten digits.

42. LeCun, Y., Bottou, L., Bengio, Y. & Haffner, P.
Gradient-based learning applied to document recognition. Proc. IEEE 86,
2278–2324 (1998).

This overview paper on the
principles of end-to-end training of modular systems such as deep neural
networks using gradient-based optimization showed how neural networks (and in
particular convolutional nets) can be combined with search or inference
mechanisms to model complex outputs that are interdependent, such as sequences
of characters associated with the content of a document.

视觉神经元启发卷积和池化层

43. Hubel, D. H. & Wiesel, T. N. Receptive fields, binocular
interaction, and functional architecture in the cat’s visual cortex. J.
Physiol. 160, 106–154 (1962).

44. Felleman, D. J. & Essen, D. C. V. Distributed hierarchical
processing in the primate cerebral cortex. Cereb. Cortex 1, 1–47 (1991).

一个研究关于convnet和猴子面对同一个神经元在高层次的表现

45. Cadieu, C. F. et al. Deep neural networks rival the
representation of primate it cortex for core visual object recognition. PLoS
Comp. Biol. 10, e1003963 (2014).

微软进行光学字符识别和手写数字识别

49. Simard, D., Steinkraus, P. Y. & Platt, J. C. Best practices
for convolutional neural networks. In Proc. Document Analysis and Recognition
958–963 (2003).

自然图片中的物体检测

50. Vaillant, R., Monrocq, C. & LeCun, Y. Original approach for
the localisation of objects in images. In Proc. Vision, Image, and Signal
Processing 141, 245–250(1994).

51. Nowlan, S. & Platt, J. in Neural Information Processing
Systems 901–908 (1995).

面部识别

52. Lawrence, S., Giles, C. L., Tsoi, A. C. & Back, A. D. Face
recognition: a convolutional neural-network approach. IEEE Trans. Neural
Networks 8, 98–113(1997).

交通信号识别

53. Ciresan, D.,
Meier, U. Masci, J. & Schmidhuber, J. Multi-column deep neural network for
traffic sign classification. Neural Networks 32, 333–338 (2012).

生物图像切割

54. Ning, F. et
al. Toward automatic phenotyping of developing embryos from videos. IEEE Trans.
Image Process. 14, 1360–1371 (2005).

面部检测、行人检测、躯干检测等

36. Sermanet, P., Kavukcuoglu, K., Chintala, S. & LeCun, Y.
Pedestrian detection with unsupervised multi-stage feature learning. In Proc.
International Conference on Computer Vision and Pattern Recognition
http://arxiv.org/abs/1212.0142 (2013).

50. Vaillant, R., Monrocq, C. & LeCun, Y. Original approach for
the localisation of objects in images. In Proc. Vision, Image, and Signal
Processing 141, 245–250(1994).

51. Nowlan, S. & Platt, J. in Neural Information Processing
Systems 901–908 (1995).

56. Garcia, C.
& Delakis, M. Convolutional face finder: a neural architecture for fast and
robust face detection. IEEE Trans. Pattern Anal. Machine Intell. 26,1408–1423
(2004).

57. Osadchy, M.,
LeCun, Y. & Miller, M. Synergistic face detection and pose estimation with
energy-based models. J. Mach. Learn. Res. 8, 1197–1215 (2007).

58. Tompson, J.,
Goroshin, R. R., Jain, A., LeCun, Y. Y. & Bregler, C. C. Efficient object localization
using convolutional networks. In Proc. Conference on Computer Vision and
Pattern Recognition http://arxiv.org/abs/1411.4280
(2014).

面部识别

59. Taigman, Y.,
Yang, M., Ranzato, M. & Wolf, L. Deepface: closing the gap to human-level
performance in face verification. In Proc. Conference on Computer Vision and
Pattern Recognition 1701–1708 (2014).

dropout

62. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. &
Salakhutdinov, R. Dropout: a simple way to prevent neural networks from
overfitting. J. Machine Learning Res. 15, 1929–1958 (2014).

识别和检测

4. Szegedy, C. et al. Going deeper with convolutions. Preprint at
http://arxiv.org/abs/1409.4842 (2014).

58. Tompson, J., Goroshin, R. R., Jain, A., LeCun, Y. Y. &
Bregler, C. C. Efficient object localization using convolutional networks. In
Proc. Conference on Computer Vision and Pattern Recognition
http://arxiv.org/abs/1411.4280 (2014).

59. Taigman, Y., Yang, M., Ranzato, M. & Wolf, L. Deepface:
closing the gap to human-level performance in face verification. In Proc.
Conference on Computer Vision and Pattern Recognition 1701–1708 (2014).

63. Sermanet, P. et al. Overfeat: integrated recognition,
localization and detection using convolutional networks. In Proc. International
Conference on Learning Representations http://arxiv.org/abs/1312.6229 (2014).

64. Girshick, R., Donahue, J., Darrell, T. & Malik, J. Rich
feature hierarchies for accurate object detection and semantic segmentation. In
Proc. Conference on Computer Vision and Pattern Recognition 580–587 (2014).

65. Simonyan, K. & Zisserman, A. Very deep convolutional
networks for large-scale image recognition. In Proc. International Conference
on Learning Representations http://arxiv.org/abs/1409.1556 (2014).

distributed
representations

21. Bengio, Y., Delalleau, O. & Le Roux, N. The curse of highly
variable functions for local kernel machines. In Proc. Advances in Neural
Information Processing Systems 18 107–114 (2005).

数据分布下的整体架构

40. Bengio, Y., Courville, A. & Vincent, P. Representation
learning: a review and new perspectives. IEEE Trans. Pattern Anal. Machine
Intell. 35, 1798–1828 (2013).

distributed
representations增强泛化能力

68. Bengio, Y. Learning Deep Architectures for AI (Now, 2009).

69. Montufar, G. & Morton, J. When does a mixture of products
contain a product of mixtures? J. Discrete Math. 29, 321–347 (2014).

深度增强表达能力

70. Montufar, G. F., Pascanu, R., Cho, K. & Bengio, Y. On the
number of linear regions of deep neural networks. In Proc. Advances in Neural
Information Processing Systems 27 2924–2932 (2014).

通过局部输入确定下一个输出

71. Bengio, Y., Ducharme, R. & Vincent, P. A neural
probabilistic language model. In Proc. Advances in Neural Information
Processing Systems 13 932–938 (2001). This paper introduced neural language
models, which learn to convert a word symbol into a word vector or word
embedding composed of learned semantic features in order to predict the next
word in a sequence.

非监督学习

91. Hinton, G. E., Dayan, P., Frey, B. J. & Neal, R. M. The
wake-sleep algorithm for unsupervised neural networks. Science 268, 1558–1161
(1995).

92. Salakhutdinov, R. & Hinton, G. Deep Boltzmann machines. In
Proc. International Conference on Artificial Intelligence and Statistics
448–455 (2009).

93. Vincent, P., Larochelle, H., Bengio, Y. & Manzagol, P.-A.
Extracting and composing robust features with denoising autoencoders. In Proc.
25th International Conference on Machine Learning 1096–1103 (2008).

94. Kavukcuoglu, K. et al. Learning convolutional feature
hierarchies for visual recognition. In Proc. Advances in Neural Information
Processing Systems 23 1090–1098 (2010).

95. Gregor, K. & LeCun, Y. Learning fast approximations of
sparse coding. In Proc. International Conference on Machine Learning 399–406
(2010).

96. Ranzato, M., Mnih, V., Susskind, J. M. & Hinton, G. E.
Modeling natural images using gated MRFs. IEEE Trans. Pattern Anal. Machine
Intell. 35, 2206–2222(2013).

97. Bengio, Y., Thibodeau-Laufer, E., Alain, G. & Yosinski, J.
Deep generative stochastic networks trainable by backprop. In Proc. 31st
International Conference on Machine Learning 226–234 (2014).

98. Kingma, D., Rezende, D., Mohamed, S. & Welling, M.
Semi-supervised learning with deep generative models. In Proc. Advances in
Neural Information Processing Systems 27 3581–3589 (2014).

cnn＋rnn使用增强学习进行视觉分类

99. Ba, J., Mnih, V. & Kavukcuoglu, K. Multiple object
recognition with visual attention. In Proc. International Conference on
Learning Representations。http://arxiv.org/abs/1412.7755 (2014).

cnn＋rnn使用增强学习玩游戏

100. Mnih, V. et al. Human-level control through deep reinforcement
learning. Nature518, 529–533 (2015).

rnn learn strategies for
selectively attending to one part at a time

76. Bahdanau, D., Cho, K. & Bengio, Y. Neural machine
translation by jointly learning to align and translate. In Proc. International
Conference on Learning Representations http://arxiv.org/abs/1409.0473 (2015).

86. Xu, K. et al. Show, attend and tell: Neural image caption
generation with visual attention. In Proc. International Conference on Learning
Representations http://arxiv.org/abs/1502.03044 (2015).

rnn关注图片特定位置

102. Vinyals, O., Toshev, A., Bengio, S. & Erhan, D. Show and
tell: a neural image caption generator. In Proc. International Conference on
Machine Learning http://arxiv.org/abs/1502.03044 (2014).

时间： 2024-08-27 02:42:56

2016.4.15 近期要读的论文的相关文章

2016/02/15 codes

return e.addTest = function(a,b){ if(typeof a == "object") for(var d in a )y(a,d)&& e.addTest(d,a[d]); else{a = a.toLowerCase(); if(e[a]!== c)return e; b = typeof b = "function"?b():b, typeof f != "undefined" &&am

2016.4.15 -关于分离和总结

2016.4.15 科比退役了,祝老大走好,作为铁杆科密的我没有选择在社交媒体上去煽情和感伤,我只知道,我喜欢科比,是因为他的精神,现在的我需要传承老大的“凌晨四点的洛杉矶”,在内心和行动上默默坚持,默默努力,前行.用自己的实际行动证明一切,而不是说说. 研究生阶段唯一的好友,张同学今天突然告诉我他要选择转博了,没有一点惊喜,反而有点悲伤,剩下的路又要一个人走了,或许是上天对我的考察,,,终于想通 了,其实我们每个人都有自己的路要走,每个人都只能陪你走一段路程而已,剩下的路,你一个人走,要坚持,

2016.2.15 四旋翼相关资料

--------2016.2.15--------最近在玩儿四旋翼,所以相关的资料我会贴上来供需要的朋友参考,如果遇到了我们趟过的地雷就能够方便一些. 机架:F450我们之后由于要将NVIDIA的开发板和飞行器连在一起,所以从机架的选择上我们要慎重考虑,之前为了方便选的是F330,但是DJI正版的F330停产了,所以taobao的F330进行测试,经过测试发现F330相当脆弱,机架非常容易在后写撞击中折断,所以如果上面搭载NVIDIA的开发板的话,那么这样开发的成本太高,一旦炸机可能机毁板子亡,

2016.4.15 nature deep learning review［1］

今天,我本来想膜一下,所以找到了上古时期发表再nature上的反向传播的论文,但是没看下去...所以,翻出来了15年发表在nature上的deep learning,相当于一个review,来阅读一下,而且感觉引文会比较重要,所以这篇中枢值较高的文献拿来学一学. 相关资料: 英文原文: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.436.894&rep=rep1&type=pdf 中文翻译: http://www.csd

macbook pro 2016 2017 15寸 雷电3 外接显卡 epu 简单教程(不修改UEFI)

雷电3外接显卡效果还不错,但是除了akitio node 其他厂家并不会维护自己的固件来适配新机型,我自己买的mbp 2016 15''就出现了和AORUS Gaming Box 1070不兼容的问题,在引导的时候会在win徽标处转圈卡死,我联系了厂家,客服,给他们打了电话发了邮件,都快变成骚扰了,不过并没有解决问题,得到的回复是问题在解决中,不知道什么时候能解决. 后来在egpu.io上找到了解决方案并尝试成功,效果还不错 https://egpu.io/forums/implementati

2016.7.15.2014noip模拟赛D1（和昨天那个不一样，但同样网上搜不到

1．      合理种植 (plant.pas/.c/.cpp) [问题描述] 大COS在氯铯石料场干了半年,受尽了劳苦,终于决定辞职.他来到表弟小cos的寒树中学,找到方克顺校长,希望寻个活干. 于是他如愿以偿接到了一个任务…… 美丽寒树中学种有许多寒树.方克顺希望校园无论从什么角度看都是满眼寒树,因此他不希望有三棵甚至更多寒树种在一条直线上.现在他把校园里n棵寒树的坐标都给了大COS,让他数出存在多少多树共线情况.(若一条直线上有三棵或以上的树,则算出现一个多树共线情况.) [输入] 输入文

读文章论文

论文 Software defect association mining and defect correction effort prediction Type:缺陷预测 Published in:Software Engineering, IEEE Transactions on (Volume:32 ,  Issue: 2) Abstract: Much current software defect prediction work focuses on the number of de

2016.8.15

今天是8月15号,又下起了倾盆大雨,还好没被淋住,希望不要有一天会变得一切都那么糟糕,说说今天吧.今天呢又被迫重复的学习了一遍SqlServer,一个不喜欢走 曾经走过路的人我又该怎么去安然的听课,虽然我的数据库语法记得不熟悉但是我还是不愿意去学第二遍,不是不愿意学习,而是一种性格,我知道自己很差但是我也得需要一点性格吧,我也常常的去改变自己,可是想的和做的不一样的,就跟写一个程序思路有了可是写起来寸步难行,真的是执盘流泪啊! 有的时候你忘掉一个人,然后多年以后你再次和她联系你会发现,你们互相已

2016/1/15 面向对象练习 1,创建类 属性 方法 构造方法 2，自定义图书类 3温度单位转换工具

1 public class Factory { 2 3 //属性 4 //品牌 5 String pinpai; 6 //设备 7 int machine; 8 //材料 9 double material; 10 //资金 11 double money; 12 //人工 13 int rengong; 14 //产品 15 int goods; 16 17 void product(int xiaohao1){ 18 machine=xiaohao1; 19 System.out.prin