中文第一计算机图形学社区OpenGPU 版权所有2007-2018

 找回密码
 注册

扫一扫,访问微社区

搜索
查看: 2598|回复: 10

如何系统学习ML?

[复制链接]
发表于 2017-5-17 19:53:13 | 显示全部楼层 |阅读模式
  水一贴,<<未来简史>>里说未来99%的人类会被淘汰,为了不被淘汰买了本机器学习书在啃,感觉云里雾里啊。有人介绍下如何系统的学习机器学习么。
发表于 2017-5-18 10:43:47 | 显示全部楼层
摘自http://deeplearning.net/ 上面有很多好东西。

Reading List
List of reading lists and survey papers:

Books
Deep Learning, Yoshua Bengio, Ian Goodfellow, Aaron Courville, MIT Press, In preparation.
Review Papers
Representation Learning: A Review and New Perspectives, Yoshua Bengio, Aaron Courville, Pascal Vincent, Arxiv, 2012.
The monograph or review paper Learning Deep Architectures for AI (Foundations & Trends in Machine Learning, 2009).
Deep Machine Learning – A New Frontier in Artificial Intelligence Research – a survey paper by Itamar Arel, Derek C. Rose, and Thomas P. Karnowski.
Graves, A. (2012). Supervised sequence labelling with recurrent neural networks(Vol. 385). Springer.
Schmidhuber, J. (2014). Deep Learning in Neural Networks: An Overview. 75 pages, 850+ references, http://arxiv.org/abs/1404.7828, PDF & LATEX source & complete public BIBTEX file under http://www.idsia.ch/~juergen/deep-learning-overview.html.
LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. “Deep learning.” Nature 521, no. 7553 (2015): 436-444.
Reinforcement Learning
Mnih, Volodymyr, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. “Playing Atari with deep reinforcement learning.” arXiv preprint arXiv:1312.5602 (2013).
Volodymyr Mnih, Nicolas Heess, Alex Graves, Koray Kavukcuoglu. “Recurrent Models of Visual Attention” ArXiv e-print, 2014.
Computer Vision
ImageNet Classification with Deep Convolutional Neural Networks, Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton, NIPS 2012.
Going Deeper with Convolutions, Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich, 19-Sept-2014.
Learning Hierarchical Features for Scene Labeling, Clement Farabet, Camille Couprie, Laurent Najman and Yann LeCun, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013.
Learning Convolutional Feature Hierachies for Visual Recognition, Koray Kavukcuoglu, Pierre Sermanet, Y-Lan Boureau, Karol Gregor, Michaël Mathieu and Yann LeCun, Advances in Neural Information Processing Systems (NIPS 2010), 23, 2010.
Graves, Alex, et al. “A novel connectionist system for unconstrained handwriting recognition.” Pattern Analysis and Machine Intelligence, IEEE Transactions on 31.5 (2009): 855-868.
Cireşan, D. C., Meier, U., Gambardella, L. M., & Schmidhuber, J. (2010). Deep, big, simple neural nets for handwritten digit recognition. Neural computation, 22(12), 3207-3220.
Ciresan, Dan, Ueli Meier, and Jürgen Schmidhuber. “Multi-column deep neural networks for image classification.” Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE, 2012.
Ciresan, D., Meier, U., Masci, J., & Schmidhuber, J. (2011, July). A committee of neural networks for traffic sign classification. In Neural Networks (IJCNN), The 2011 International Joint Conference on (pp. 1918-1921). IEEE.
NLP and Speech
Joint Learning of Words and Meaning Representations for Open-Text Semantic Parsing, Antoine Bordes, Xavier Glorot, Jason Weston and Yoshua Bengio (2012), in: Proceedings of the 15th International Conference on Artificial Intelligence and Statistics (AISTATS)
Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. Socher, R., Huang, E. H., Pennington, J., Ng, A. Y., and Manning, C. D. (2011a).  In NIPS’2011.
Semi-supervised recursive autoencoders for predicting sentiment distributions. Socher, R., Pennington, J., Huang, E. H., Ng, A. Y., and Manning, C. D. (2011b).  In EMNLP’2011.
Mikolov Tomáš: Statistical Language Models based on Neural Networks. PhD thesis, Brno University of Technology, 2012.
Graves, Alex, and Jürgen Schmidhuber. “Framewise phoneme classification with bidirectional LSTM and other neural network architectures.” Neural Networks 18.5 (2005): 602-610.
Mikolov, Tomas, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. “Distributed representations of words and phrases and their compositionality.” In Advances in Neural Information Processing Systems, pp. 3111-3119. 2013.
K. Cho, B. van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, Y. Bengio. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. EMNLP 2014.
Sutskever, Ilya, Oriol Vinyals, and Quoc VV Le. “Sequence to sequence learning with neural networks.” Advances in Neural Information Processing Systems. 2014.
Disentangling Factors and Variations with Depth
Goodfellow, Ian, et al. “Measuring invariances in deep networks.” Advances in neural information processing systems 22 (2009): 646-654.
Bengio, Yoshua, et al. “Better Mixing via Deep Representations.” arXiv preprint arXiv:1207.4404 (2012).
Xavier Glorot, Antoine Bordes and Yoshua Bengio, Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach, in: Proceedings of the Twenty-eight International Conference on Machine Learning (ICML’11), pages 97-110, 2011.
Transfer Learning and domain adaptation
Raina, Rajat, et al. “Self-taught learning: transfer learning from unlabeled data.” Proceedings of the 24th international conference on Machine learning. ACM, 2007.
Xavier Glorot, Antoine Bordes and Yoshua Bengio, Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach, in: Proceedings of the Twenty-eight International Conference on Machine Learning (ICML’11), pages 97-110, 2011.
R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu and P. Kuksa. Natural Language Processing (Almost) from Scratch. Journal of Machine Learning Research, 12:2493-2537, 2011.
Mesnil, Grégoire, et al. “Unsupervised and transfer learning challenge: a deep learning approach.” Unsupervised and Transfer Learning Workshop, in conjunction with ICML. 2011.
Ciresan, D. C., Meier, U., & Schmidhuber, J. (2012, June). Transfer learning for Latin and Chinese characters with deep neural networks. In Neural Networks (IJCNN), The 2012 International Joint Conference on (pp. 1-6). IEEE.
Goodfellow, Ian, Aaron Courville, and Yoshua Bengio. “Large-Scale Feature Learning With Spike-and-Slab Sparse Coding.” ICML 2012.
Practical Tricks and Guides
“Improving neural networks by preventing co-adaptation of feature detectors.” Hinton, Geoffrey E., et al.  arXiv preprint arXiv:1207.0580 (2012).
Practical recommendations for gradient-based training of deep architectures, Yoshua Bengio, U. Montreal, arXiv report:1206.5533, Lecture Notes in Computer Science Volume 7700, Neural Networks: Tricks of the Trade Second Edition, Editors: Grégoire Montavon, Geneviève B. Orr, Klaus-Robert Müller, 2012.
A practical guide to training Restricted Boltzmann Machines, by Geoffrey Hinton.
Sparse Coding
Emergence of simple-cell receptive field properties by learning a sparse code for natural images, Bruno Olhausen, Nature 1996.
Kavukcuoglu, Koray, Marc’Aurelio Ranzato, and Yann LeCun. “Fast inference in sparse coding algorithms with applications to object recognition.” arXiv preprint arXiv:1010.3467 (2010).
Goodfellow, Ian, Aaron Courville, and Yoshua Bengio. “Large-Scale Feature Learning With Spike-and-Slab Sparse Coding.” ICML 2012.
Efficient sparse coding algorithms. Honglak Lee, Alexis Battle, Raina Rajat and Andrew Y. Ng. In NIPS 19, 2007. pdf
“Sparse coding with an overcomplete basis set: A strategy employed by VI?.” . Olshausen, Bruno A., and David J. Field. Vision research 37.23 (1997): 3311-3326.
Foundation Theory and Motivation
Hinton, Geoffrey E. “Deterministic Boltzmann learning performs steepest descent in weight-space.” Neural computation 1.1 (1989): 143-150.
Bengio, Yoshua, and Samy Bengio. “Modeling high-dimensional discrete data with multi-layer neural networks.” Advances in Neural Information Processing Systems 12 (2000): 400-406.
Bengio, Yoshua, et al. “Greedy layer-wise training of deep networks.” Advances in neural information processing systems 19 (2007): 153.
Bengio, Yoshua, Martin Monperrus, and Hugo Larochelle. “Nonlocal estimation of manifold structure.” Neural Computation 18.10 (2006): 2509-2528.
Hinton, Geoffrey E., and Ruslan R. Salakhutdinov. “Reducing the dimensionality of data with neural networks.” Science 313.5786 (2006): 504-507.
Marc’Aurelio Ranzato, Y., Lan Boureau, and Yann LeCun. “Sparse feature learning for deep belief networks.” Advances in neural information processing systems 20 (2007): 1185-1192.
Bengio, Yoshua, and Yann LeCun. “Scaling learning algorithms towards AI.” Large-Scale Kernel Machines 34 (2007).
Le Roux, Nicolas, and Yoshua Bengio. “Representational power of restricted boltzmann machines and deep belief networks.” Neural Computation 20.6 (2008): 1631-1649.
Sutskever, Ilya, and Geoffrey Hinton. “Temporal-Kernel Recurrent Neural Networks.” Neural Networks 23.2 (2010): 239-243.
Le Roux, Nicolas, and Yoshua Bengio. “Deep belief networks are compact universal approximators.” Neural computation 22.8 (2010): 2192-2207.
Bengio, Yoshua, and Olivier Delalleau. “On the expressive power of deep architectures.” Algorithmic Learning Theory. Springer Berlin/Heidelberg, 2011.
Montufar, Guido F., and Jason Morton. “When Does a Mixture of Products Contain a Product of Mixtures?.” arXiv preprint arXiv:1206.0387 (2012).
Montúfar, Guido, Razvan Pascanu, Kyunghyun Cho, and Yoshua Bengio. “On the Number of Linear Regions of Deep Neural Networks.” arXiv preprint arXiv:1402.1869 (2014).
Supervised Feedfoward Neural Networks
The Manifold Tangent Classifier, Salah Rifai, Yann Dauphin, Pascal Vincent, Yoshua Bengio and Xavier Muller, in: NIPS’2011.
“Discriminative Learning of Sum-Product Networks.“, Gens, Robert, and Pedro Domingos, NIPS 2012 Best Student Paper.
Goodfellow, I., Warde-Farley, D., Mirza, M., Courville, A., and Bengio, Y. (2013). Maxout networks. Technical Report, Universite de Montreal.
Hinton, Geoffrey E., et al. “Improving neural networks by preventing co-adaptation of feature detectors.” arXiv preprint arXiv:1207.0580 (2012).
Wang, Sida, and Christopher Manning. “Fast dropout training.” In Proceedings of the 30th International Conference on Machine Learning (ICML-13), pp. 118-126. 2013.
Glorot, Xavier, Antoine Bordes, and Yoshua Bengio. “Deep sparse rectifier networks.” In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. JMLR W&CP Volume, vol. 15, pp. 315-323. 2011.
ImageNet Classification with Deep Convolutional Neural Networks, Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton, NIPS 2012.
Large Scale Deep Learning
Building High-level Features Using Large Scale Unsupervised Learning Quoc V. Le, Marc’Aurelio Ranzato, Rajat Monga, Matthieu Devin, Kai Chen, Greg S. Corrado, Jeffrey Dean, and Andrew Y. Ng, ICML 2012.
Bengio, Yoshua, et al. “Neural probabilistic language models.” Innovations in Machine Learning (2006): 137-186. Specifically Section 3 of this paper discusses the asynchronous SGD.
Dean, Jeffrey, et al. “Large scale distributed deep networks.” Advances in Neural Information Processing Systems. 2012.
Recurrent Networks
Training Recurrent Neural Networks, Ilya Sutskever, PhD Thesis, 2012.
Bengio, Yoshua, Patrice Simard, and Paolo Frasconi. “Learning long-term dependencies with gradient descent is difficult.” Neural Networks, IEEE Transactions on 5.2 (1994): 157-166.
Mikolov Tomáš: Statistical Language Models based on Neural Networks. PhD thesis, Brno University of Technology, 2012.
Hochreiter, Sepp, and Jürgen Schmidhuber. “Long short-term memory.” Neural computation 9.8 (1997): 1735-1780.
Hochreiter, S., Bengio, Y., Frasconi, P., & Schmidhuber, J. (2001). Gradient flow in recurrent nets: the difficulty of learning long-term dependencies.
Schmidhuber, J. (1992). Learning complex, extended sequences using the principle of history compression. Neural Computation, 4(2), 234-242.
Graves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006, June). Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning (pp. 369-376). ACM.
Hyper Parameters
“Practical Bayesian Optimization of Machine Learning Algorithms”, Jasper Snoek, Hugo Larochelle, Ryan Adams, NIPS 2012.
Random Search for Hyper-Parameter Optimization, James Bergstra and Yoshua Bengio (2012), in: Journal of Machine Learning Research, 13(281–305).
Algorithms for Hyper-Parameter Optimization, James Bergstra, Rémy Bardenet, Yoshua Bengio and Balázs Kégl, in: NIPS’2011, 2011.
Optimization
Training Deep and Recurrent Neural Networks with Hessian-Free Optimization, James Martens and Ilya Sutskever, Neural Networks: Tricks of the Trade, 2012.
Schaul, Tom, Sixin Zhang, and Yann LeCun. “No More Pesky Learning Rates.” arXiv preprint arXiv:1206.1106 (2012).
Le Roux, Nicolas, Pierre-Antoine Manzagol, and Yoshua Bengio. “Topmoumoute online natural gradient algorithm.” Neural Information Processing Systems (NIPS). 2007.
Bordes, Antoine, Léon Bottou, and Patrick Gallinari. “SGD-QN: Careful quasi-Newton stochastic gradient descent.” The Journal of Machine Learning Research 10 (2009): 1737-1754.
Glorot, Xavier, and Yoshua Bengio. “Understanding the difficulty of training deep feedforward neural networks.” Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS’10). Society for Artificial Intelligence and Statistics. 2010.
Glorot, Xavier, Antoine Bordes, and Yoshua Bengio. “Deep Sparse Rectifier Networks.” Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. JMLR W&CP Volume. Vol. 15. 2011.
“Deep learning via Hessian-free optimization.” Martens, James. Proceedings of the 27th International Conference on Machine Learning (ICML). Vol. 951. 2010.
Hochreiter, Sepp, and Jürgen Schmidhuber. “Flat minima.” Neural Computation, 9.1 (1997): 1-42.
Pascanu, Razvan, and Yoshua Bengio. “Revisiting natural gradient for deep networks.” arXiv preprint arXiv:1301.3584 (2013).
Dauphin, Yann N., Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Surya Ganguli, and Yoshua Bengio. “Identifying and attacking the saddle point problem in high-dimensional non-convex optimization.” In Advances in Neural Information Processing Systems, pp. 2933-2941. 2014.
Unsupervised Feature Learning
Salakhutdinov, Ruslan, and Geoffrey E. Hinton. “Deep boltzmann machines.” Proceedings of the international conference on artificial intelligence and statistics. Vol. 5. No. 2. Cambridge, MA: MIT Press, 2009.
Scholarpedia page on Deep Belief Networks.
Deep Boltzmann Machines
An Efficient Learning Procedure for Deep Boltzmann Machines, Ruslan Salakhutdinov and Geoffrey Hinton, Neural Computation August 2012, Vol. 24, No. 8: 1967 — 2006.
Montavon, Grégoire, and Klaus-Robert Müller. “Deep Boltzmann Machines and the Centering Trick.” Neural Networks: Tricks of the Trade (2012): 621-637.
Salakhutdinov, Ruslan, and Hugo Larochelle. “Efficient learning of deep boltzmann machines.” International Conference on Artificial Intelligence and Statistics. 2010.
Salakhutdinov, Ruslan. Learning deep generative models. Diss. University of Toronto, 2009.
Goodfellow, Ian, et al. “Multi-prediction deep Boltzmann machines.” Advances in Neural Information Processing Systems. 2013.
RBMs
Unsupervised Models of Images by Spike-and-Slab RBMs, Aaron Courville, James Bergstra and Yoshua Bengio, in: ICML’2011
Hinton, Geoffrey. “A practical guide to training restricted Boltzmann machines.” Momentum 9.1 (2010): 926.
Autoencoders
Regularized Auto-Encoders Estimate Local Statistics, Guillaume Alain, Yoshua Bengio and Salah Rifai, Université de Montréal, arXiv report 1211.4246, 2012
A Generative Process for Sampling Contractive Auto-Encoders, Salah Rifai, Yoshua Bengio, Yann Dauphin and Pascal Vincent, in: ICML’2012, Edinburgh, Scotland, U.K., 2012
Contracting Auto-Encoders: Explicit invariance during feature extraction, Salah Rifai, Pascal Vincent, Xavier Muller, Xavier Glorot and Yoshua Bengio, in: ICML’2011
Disentangling factors of variation for facial expression recognition, Salah Rifai, Yoshua Bengio, Aaron Courville, Pascal Vincent and Mehdi Mirza, in: ECCV’2012.
Vincent, Pascal, et al. “Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion.” The Journal of Machine Learning Research 11 (2010): 3371-3408.
Vincent, Pascal. “A connection between score matching and denoising autoencoders.” Neural computation 23.7 (2011): 1661-1674.
Chen, Minmin, et al. “Marginalized denoising autoencoders for domain adaptation.” arXiv preprint arXiv:1206.4683 (2012).
Miscellaneous
The ICML 2009 Workshop on Learning Feature Hierarchies webpage has a reading list.
Stanford’s UFLDL Recommended Readings.
The LISApublic wiki has a reading list and a bibliography.
Geoff Hinton has readings NIPS 2007 tutorial.
The LISA publications database contains a deep architectures category.
A very brief introduction to AI, Machine Learning, and Deep Learning in Yoshua Bengio‘s IFT6266 graduate class
Memkite’s deep learning reading list, http://memkite.com/deep-learning-bibliography/.
Deep learning resources page, http://www.jeremydjacksonphd.com/?cat=7
 楼主| 发表于 2017-5-18 22:22:39 | 显示全部楼层
Uling 发表于 2017-5-18 10:43
摘自http://deeplearning.net/ 上面有很多好东西。

Reading List

多谢,够长!
发表于 2017-5-24 14:47:22 | 显示全部楼层
这个板块开的很符合潮流趋势
发表于 2017-5-24 15:19:12 | 显示全部楼层
入门的材料是这样?
发表于 2017-5-26 16:14:48 | 显示全部楼层
正在学习Python,然后基于NumPy学习下算例
发表于 2017-6-3 21:34:21 | 显示全部楼层
这个神经网络平衡球的demo挺有意思:
https://cn.mathworks.com/matlabc ... 71&tab=function
发表于 2017-6-5 16:44:57 | 显示全部楼层
摘取自 新智元公众号(ID:AI_era)
给深度学习从业者的书单
一、关于矩阵或者单变量微积分计算的文献(共5项)
Introduction to Algorithms by Erik Demaine and Srinivas Devadas.
Single Variable Calculus by David Jerison.
Multivariable Calculus by Denis Auroux.
Differential Equations by Arthur Mattuck, Haynes Miller, Jeremy Orloff, John Lewis.
Linear Algebra by Gilbert Strang.

二、基于深度学习的计算理论,学习理论,神经科学等等(共12项)
Introduction to the Theory of Computation by Michael Sipser.
Artificial Intelligence: A Modern Approach by Stuart Russell and Peter Norvig.
Pattern Recognition and Machine Learning by Christopher Bishop.
Machine Learning: A probabilistic perspective by Kevin Patrick Murphy.
CS229 Machine Learning Course Materials by Andrew Ng at Stanford University.
Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto.
  Probabilistic Graphical Models: Principles and Techniques by Daphne Koller and Nir Friedman.
  Convex Optimization by Stephen Boyd and Lieven Vandenberghe.
  An Introduction to Statistical Learning with application in R by Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani.
  Neuronal Dynamics: From single neurons to networks and models of cognition by Wulfram Gerstner, Werner M. Kistler, Richard Naud and Liam Paninski.
  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems by Peter Dayan and Laurence F. Abbott.
  Michael I. Jordan Reading List of Machine Learning at Hacker News.

三、关于深度学习基础知识的文献(共5项)
Deep Learning in Neural Networks: An Overview by Jürgen Schmidhuber.
Deep Learning Book by Yoshua Bengio, Ian Goodfellow and Aaron Courville.
Learning Deep Architectures for AI by Yoshua Bengio.
Representation Learning: A Review and New Perspectives by Yoshua Bengio, Aaron Courville, Pascal Vincent.
Reading lists for new LISA students by LISA Lab, University of Montreal.

四、关于深度学习的教材,实用手册以及有用的软件(共17项)
Machine Learning by Andrew Ng.
Neural Networks for Machine Learning by Geoffrey Hinton.
Deep Learning Tutorial by LISA Lab, University of Montreal.
Unsupervised Feature Learning and Deep Learning Tutorial by AI Lab, Stan     ford University.
CS231n: Convolutional Neural Networks for Visual Recognition by Stanfor         d Uiversity.
CS224d: Deep Learning for Natural Language Processing by Stanford Univer  sity.
Theano by LISA Lab, University of Montreal.
PyLearn2 by LISA Lab, University of Montreal.
Caffe by Berkeley Vision and Learning Center (BVLC) and community contrib    utor     Yangqing Jia.
Torch 7
neon by Nervana.
cuDNN by NVIDIA.
ConvNetJS by Andrej Karpathy.
DeepLearning4j
Chainer: Neural network framework by Preferred Networks, Inc.
Blocks by LISA Lab, University of Montreal.
Fuel by LISA Lab, University of Montreal.

五、关于深度学习和特征学习的文献(共11项)
Automatic Speech Recognition - A Deep Learning Approach by Dong Yu an  d Li Deng (Published by Springer, no Open Access)
Backpropagation Applied to Handwritten Zip Code Recognition by Y. LeCu  n, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard an  d L. D. Jackel.
Comparison of Training Methods for Deep Neural Networks by Patrick O. Glauner.
Deep Learning by Yann LeCun, Yoshua Bengio, Geoffrey Hinton. (NO FREE COPY AVAILABLE)
Distributed Representations of Words and Phrases and their Compositionality by Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado and Jeffrey Dean.
Efficient Estimation of Word Representations in Vector Space by Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean.
Efficient Large Scale Video Classification by Balakrishnan Varadarajan, George Toderici, Sudheendra Vijayanarasimhan, Apostol Natsev.
Foundations and Trends in Signal Processing: DEEP LEARNING — Methods and  Applications by Li Deng and Dong Yu.
From Frequency to Meaning: Vector Space Models of Semantics by Peter D. Turney and Patrick Pantel.
LSTM: A Search Space Odyssey by Klaus Greff, Rupesh Kumar Srivastava, Jan Koutník, Bas R. Steunebrink, Jürgen Schmidhuber.
Supervised Sequence Labelling with Recurrent Neural Networks by Alex Graves.

六、最近必读的关于深度学习领域的最新进展(共332项)
A Convolutional Attention Network for Extreme Summarization of Source Code by Miltiadis Allamanis, Hao Peng, Charles Sutton.
A Deep Bag-of-Features Model for Music Auto-Tagging by Juhan Nam, Jorge Herrera, Kyogu Lee.
A Deep Generative Deconvolutional Image Model by Yunchen Pu, Xin Yuan, Andrew Stevens, Chunyuan Li, Lawrence Carin.
A Deep Neural Network Compression Pipeline: Pruning, Quantization, Huffman Encoding by Song Han, Huizi Mao, William J. Dally.
A Deep Pyramid Deformable Part Model for Face Detection by Rajeev Ranjan, Vishal M. Patel, Rama Chellappa.
A Deep Siamese Network for Scene Detection in Broadcast Videos by Lorenzo Baraldi, Costantino Grana, Rita Cucchiara.
A Hierarchical Recurrent Encoder-Decoder For Generative Context-Aware Query Suggestion by Alessandro Sordoni, Yoshua Bengio, Hossein Vahabi, Christina Lioma, Jakob G. Simonsen, Jian-Yun Nie.
A Large-Scale Car Dataset for Fine-Grained Categorization and Verification by Linjie Yang, Ping Luo, Chen Change Loy, Xiaoou Tang.
A Lightened CNN for Deep Face Representation by Xiang Wu, Ran He, Zhenan Sun.
A Mathematical Theory of Deep Convolutional Neural Networks for Feature    Extraction by Thomas Wiatowski, Helmut Bölcskei.
A Multi-scale Multiple Instance Video Description Network by Huijuan Xu, Subhashini Venugopalan, Vasili Ramanishka, Marcus Rohrbach, Kate Saenko.
A Recurrent Latent Variable Model for Sequential Data by Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron Courville, Yoshua Bengio.
A Restricted Visual Turing Test for Deep Scene and Event Understanding by Hang Qi, Tianfu Wu, Mun-Wai Lee, Song-Chun Zhu.
A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification by Ye Zhang, Byron Wallace.
ABC-CNN: An Attention Based Convolutional Neural Network for Visual Question Answering by Kan Chen, Jiang Wang, Liang-Chieh Chen, Haoyuan Gao, Wei Xu, Ram Nevatia.
Accelerating Very Deep Convolutional Networks for Classification and Detection by Xiangyu Zhang, Jianhua Zou, Kaiming He, Jian Sun.
Accurate Image Super-Resolution Using Very Deep Convolutional Networks by Jiwon Kim, Jung Kwon Lee, Kyoung Mu Lee.
Action Recognition using Visual Attention by Shikhar Sharma, Ryan Kiros, Ruslan Salakhutdinov.
Action Recognition With Trajectory-Pooled Deep-Convolutional Descriptors by Limin Wang, Yu Qiao, Xiaoou Tang.
Action-Conditional Video Prediction using Deep Networks in Atari Games by Junhyuk Oh, Xiaoxiao Guo, Honglak Lee, Richard Lewis, Satinder Singh.
Active Object Localization with Deep Reinforcement Learning by Juan C. Caicedo, Svetlana LazebnikadaQN: An Adaptive Quasi-Newton Algorithm for Training RNNs by Nitish Shirish Keskar, Albert S. Berahas.
Adding Gradient Noise Improves Learning for Very Deep Networks by Arvind Neelakantan, Luke Vilnis, Quoc V. Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, James Martens.
Adversarial Autoencoders by Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow.
Adversarial Manipulation of Deep Representations by Sara Sabour, Yanshuai Cao, Fartash Faghri, David J. Fleet.
All you need is a good init by Dmytro Mishkin, Jiri Matas.
An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition by Baoguang Shi, Xiang Bai, Cong Yao.
Answer Sequence Learning with Neural Networks for Answer Selection in Community Question Answering by Xiaoqiang Zhou, Baotian Hu, Qingcai Chen, Buzhou Tang, Xiaolong Wang.
Anticipating the future by watching unlabeled video by Carl Vondrick, Hamed Pirsiavash, Antonio Torralba.
Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answering by Haoyuan Gao, Junhua Mao, Jie Zhou, Zhiheng Huang, Lei Wang, Wei Xu.
Artificial Neural Networks Applied to Taxi Destination Prediction by Alexandre de Brébisson, Étienne Simon, Alex Auvolat, Pascal Vincent, Yoshua Bengio.
Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering by Huijuan Xu, Kate Saenko.
Ask Me Anything: Dynamic Memory Networks for Natural Language Processing by Ankit Kumar, Ozan Irsoy, Jonathan Su, James Bradbury, Robert English, Brian Pierce, Peter Ondruska, Ishaan Gulrajani, Richard Socher.
Ask Me Anything: Free-form Visual Question Answering Based on Knowledge from External Sources by Qi Wu, Peng Wang, Chunhua Shen, Anton van den Hengel, Anthony Dick.
Ask Your Neurons: A Neural-based Approach to Answering Questions about Images by Mateusz Malinowski, Marcus Rohrbach, Mario Fritz.
Associative Long Short-Term Memory by Ivo Danihelka, Greg Wayne, Benigno Uria, Nal Kalchbrenner, Alex Graves.
AttentionNet: Aggregating Weak Directions for Accurate Object Detection by Donggeun Yoo, Sunggyun Park, Joon-Young Lee, Anthony Paek, In So Kweon.
Attention-Based Models for Speech Recognition by Jan Chorowski, Dzmitry Bahdanau, Dmitriy Serdyuk, Kyunghyun Cho, Yoshua Bengio.

Attention to Scale: Scale-aware Semantic Image Segmentation by Liang-Chieh Chen, Yi Yang, Jiang Wang, Wei Xu, Alan L. Yuille.

Attention with Intention for a Neural Network Conversation Model by Kaisheng Yao, Geoffrey Zweig, Baolin Peng.

AtomNet: A Deep Convolutional Neural Network for Bioactivity Prediction in Structure-based Drug Discovery by Izhar Wallach, Michael Dzamba, Abraham Heifets.


七、数据集(共13项)

Caltech 101 by L. Fei-Fei, R. Fergus and P. Perona.

Caltech 256 by G. Griffin, AD. Holub, P. Perona.

CIFAR-10 by Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton.

CIFAR-100 by Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton.

The Comprehensive Cars (CompCars) dataset by Linjie Yang, Ping Luo, Chen Change Loy, Xiaoou Tang.

Flickr30k by Peter Young, Alice Lai, Micah Hodosh, Julia Hockenmaier.

ImageNet by Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg and Li Fei-Fei.

Microsoft COCO by Microsoft Research.

MNIST by Yann LeCun, Corinna Cortes, Christopher J.C. Burges.

Places by MIT Computer Science and Artificial Intelligence Laboratory.

STL-10 by Adam Coates, Honglak Lee, Andrew Y. Ng.

SVHN by Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, Andrew Y. Ng.

WWW Crowd Dataset by Jing Shao, Kai Kang, Chen Change Loy, and Xiaogang Wang.

八、关于学习深度学习的博客、访谈栏目等等(共4项)

Talking Machines hosted by Katherine Gorman and Ryan Adams.

Machine Learning & Computer Vision Talks by computervisiontalks.

How we’re teaching computers to understand pictures by Fei-Fei Li, Stanford University.

Deep Learning Community

九、亚马逊提供的用于深度学习的公共AMI网络服务(共3项)

DGYDLGPUv4 (ami-ba516ee8) [Based on g2.2xlarge]

DGYDLGPUXv1 (ami-52516e00) [Based on g2.8xlarge]

Caffe/CuDNN built 2015-05-04 (ami-763a331e) [For both g2.2xlarge and g2.8xlarge]

十、实用的深度神经网络—从GPU计算的角度来看(共26项)

Slides(8项)

Introduction

Python Platform for Scientific Computing

Theano Crash Course

Machine Learning Basics

Softmax Regression

Feedforward Neural Networks

Convolutional Neural Networks

Recurrent Neural Networks

Practical tutorials(8项)

Python Warm-up, pre-processing

Feedforward Layer

Softmax Regression

Multi Layer Perceptron Network

Feedforward Model

Auto-encoder

Convolutional Neural Networks

Recurrent Neural Networks

Codes

Telauges (10项)

A new deep learning library for learning DL.

MLP Layers: Tanh Layer, Sigmoid Layer, Identity Layer, ReLU Layer

Softmax Regression

ConvNet layers: Tanh Layer, Sigmoid Layer, Identity Layer, ReLU Layer

Max-Pooling layer

Max-Pooling same size

Feedforward Model

Auto-Encoder Model

SGD, Adagrad, Adadelta, RMSprop, Adam

Dropout


发表于 2017-6-7 05:52:40 | 显示全部楼层
本帖最后由 Uling 于 2017-6-7 05:54 编辑
Joel41 发表于 2017-6-5 16:44
摘取自 新智元公众号(ID:AI_era)
给深度学习从业者的书单
一、关于矩阵或者单变量微积分计算的文献(共 ...

赞,我觉得如果有个toturial啥的手把手的交如何搭最简单的网络,便做小实验边学习概念,可能会比一大堆的文献感觉轻松些。
发表于 2017-6-8 15:19:44 | 显示全部楼层
Uling 发表于 2017-6-7 05:52
赞,我觉得如果有个toturial啥的手把手的交如何搭最简单的网络,便做小实验边学习概念,可能会比一大堆的 ...

嗯嗯,理论还是太大了,很多trick被忽略掉了,尤其像Deep Learning搭网络和参数调优有很多trick,目前就在学习Caffe和Torch,可能还不太熟练,很多选择似乎很考经验。Tutorial的话youtube和google上面其实挺多的,到时候整理一下发上来
发表于 2017-6-9 05:59:41 | 显示全部楼层
Joel41 发表于 2017-6-8 15:19
嗯嗯,理论还是太大了,很多trick被忽略掉了,尤其像Deep Learning搭网络和参数调优有很多trick,目前就 ...

能发上来就牛逼了
您需要登录后才可以回帖 登录 | 注册

本版积分规则

QQ|关于我们|小黑屋|Archiver|手机版|中文第一计算机图形学社区OpenGPU

GMT+8, 2019-1-19 11:17 , Processed in 0.071771 second(s), 19 queries .

Powered by Discuz! X3.4

© 2001-2017 Comsenz Inc.

快速回复 返回顶部 返回列表