awesome-very-deep-learning



awesome-very-deep-learning is a curated list for papers and code about implementing and training very deep neural networks.

Deep Residual Learning

Deep Residual Networks are a family of extremely deep architectures (up to 1000 layers) showing compelling accuracy and nice convergence behaviors. Instead of learning a new representation at each layer, deep residual networks use identity mappings to learn residuals.

Papers

Implementations

  1. Torch by Facebook AI Research (FAIR), with training code in Torch and pre-trained ResNet-18/34/50/101 models for ImageNet: blogcode
  2. Torch, CIFAR-10, with ResNet-20 to ResNet-110, training code, and curves: code
  3. Lasagne, CIFAR-10, with ResNet-32 and ResNet-56 and training code: code
  4. Neon, CIFAR-10, with pre-trained ResNet-32 to ResNet-110 models, training code, and curves: code
  5. Neon, Preactivation layer implementation: code
  6. Torch, MNIST, 100 layers: blogcode
  7. A winning entry in Kaggle‘s right whale recognition challenge: blogcode
  8. Neon, Place2 (mini), 40 layers: blogcode
  9. Tensorflow with tflearn, with CIFAR-10 and MNIST: code
  10. Tensorflow with skflow, with MNIST: code
  11. Stochastic dropout in Keras: code
  12. ResNet in Chainer: code
  13. Stochastic dropout in Chainer: code
  14. Wide Residual Networks in Keras: code
  15. ResNet in TensorFlow 0.9+ with pretrained caffe weights: code

In addition, this code by Ryan Dahl helps to convert the pre-trained models to TensorFlow.

Highway Networks

Highway Networks take inspiration from Long Short Term Memory (LSTM) and allow training of deep, efficient networks (with hundreds of layers) with conventional gradient-based methods

Papers

Implementations

  1. Lasagne: code
  2. Caffe: code
  3. Torch: code
  4. Tensorflow: blogcode

Very Deep Learning Theory

Theories in very deep learning concentrate on the ideas that very deep networks with skip connections are able to efficiently approximate recurrent computations (similar to the recurrent connections in the visual cortex) or are actually exponential ensembles of shallow networks

Papers

时间: 2024-10-12 17:46:11

awesome-very-deep-learning的相关文章

Neural Networks and Deep Learning学习笔记ch1 - 神经网络

近期開始看一些深度学习的资料.想学习一下深度学习的基础知识.找到了一个比較好的tutorial,Neural Networks and Deep Learning,认真看完了之后觉得收获还是非常多的.从最主要的感知机開始讲起.到后来使用logistic函数作为激活函数的sigmoid neuron,和非常多其它如今深度学习中常使用的trick. 把深度学习的一个发展过程讲得非常清楚,并且还有非常多源代码和实验帮助理解.看完了整个tutorial后打算再又一次梳理一遍,来写点总结.以后再看其它资料

Deep Learning Enables You to Hide Screen when Your Boss is Approaching

https://github.com/Hironsan/BossSensor/ 背景介绍 学生时代,老师站在窗外的阴影挥之不去.大家在玩手机,看漫画,看小说的时候,总是会找同桌帮忙看着班主任有没有来. 一转眼,曾经的翩翩少年毕业了,新的烦恼来了,在你刷知乎,看视频,玩手机的时候,老板来了! 不用担心,不用着急,基于最新的人脸识别+手机推送做出的BossComing.老板站起来的时候,BossComing会通过人脸识别发现老板已经站起来,然后通过手机推送发送通知“BossComing”,并且震动告

Spark MLlib Deep Learning Convolution Neural Network (深度学习-卷积神经网络)3.1

3.Spark MLlib Deep Learning Convolution Neural Network (深度学习-卷积神经网络)3.1 http://blog.csdn.net/sunbow0 Spark MLlib Deep Learning工具箱,是根据现有深度学习教程<UFLDL教程>中的算法,在SparkMLlib中的实现.具体Spark MLlib Deep Learning(深度学习)目录结构: 第一章Neural Net(NN) 1.源码 2.源码解析 3.实例 第二章D

Spark MLlib Deep Learning Convolution Neural Network (深度学习-卷积神经网络)3.2

3.Spark MLlib Deep Learning Convolution Neural Network(深度学习-卷积神经网络)3.2 http://blog.csdn.net/sunbow0 第三章Convolution Neural Network (卷积神经网络) 2基础及源码解析 2.1 Convolution Neural Network卷积神经网络基础知识 1)基础知识: 自行google,百度,基础方面的非常多,随便看看就可以,只是很多没有把细节说得清楚和明白: 能把细节说清

Spark MLlib Deep Learning Convolution Neural Network (深度学习-卷积神经网络)3.3

3.Spark MLlib Deep Learning Convolution Neural Network(深度学习-卷积神经网络)3.3 http://blog.csdn.net/sunbow0 第三章Convolution Neural Network (卷积神经网络) 3实例 3.1 测试数据 按照上例数据,或者新建图片识别数据. 3.2 CNN实例 //2 测试数据 Logger.getRootLogger.setLevel(Level.WARN) valdata_path="/use

Deep Learning(深度学习)学习笔记整理系列七

Deep Learning(深度学习)学习笔记整理系列 声明: 1)该Deep Learning的学习系列是整理自网上很大牛和机器学习专家所无私奉献的资料的.具体引用的资料请看参考文献.具体的版本声明也参考原文献. 2)本文仅供学术交流,非商用.所以每一部分具体的参考资料并没有详细对应.如果某部分不小心侵犯了大家的利益,还望海涵,并联系博主删除. 3)本人才疏学浅,整理总结的时候难免出错,还望各位前辈不吝指正,谢谢. 4)阅读本文需要机器学习.计算机视觉.神经网络等等基础(如果没有也没关系了,没

TensorFlow和深度学习新手教程(TensorFlow and deep learning without a PhD)

前言 上月导师在组会上交我们用tensorflow写深度学习和卷积神经网络.并把其PPT的參考学习资料给了我们, 这是codelabs上的教程:<TensorFlow and deep learning,without a PhD> 当然登入须要FQ,我也顺带巩固下,做个翻译.不好之处请包括指正. 当然须要安装python,教程推荐使用python3.假设是Mac,能够參考博主的另外两片博文,Mac下升级python2.7到python3.6, Mac安装tensorflow1.0 好多专业词

deep learning书的阅读

最近坚持读书,虽然大多数读的都是一些闲书,传记.历史或者散文之类的书籍,但是也读了点专业书.闲书是散时间读的,放车里,有时间就拿起来读读,专业书则更多的靠得是专注.因为我给自己的规定是一定时间内读完几本书. deep learning这本书读到第四章了,耐心读完前面,觉得其实我还是蛮有收获的,作者主要是从机器学习的角度来解释说明数学概念,其实对我这种功利心很强的人,读 数学书老想着会对我理解算法有什么帮助,读的过程中还是很顺利的. 本来我是想着练习自己的英语写作能力,边读边背诵呢,但是我还是坚持

(转) Deep learning architecture diagrams

FastML Machine learning made easy RSS Home Contents Popular Links Backgrounds About Deep learning architecture diagrams 2016-09-30 Like a wild stream after a wet season in African savanna diverges into many smaller streams forming lakes and puddles,

(转) Deep Learning in a Nutshell: Reinforcement Learning

Deep Learning in a Nutshell: Reinforcement Learning Share: Posted on September 8, 2016by Tim Dettmers No CommentsTagged Deep Learning, Deep Neural Networks, Machine Learning,Reinforcement Learning This post is Part 4 of the Deep Learning in a Nutshel