---
title: [DLHacks]Squeeze-and-Excitation Networks
tags:  #deep learning  
author: [Deep Learning JP](https://www.docswell.com/user/DeepLearning2023)
site: [Docswell](https://www.docswell.com/)
thumbnail: https://bcdn.docswell.com/page/2JVL9WGM7Q.jpg?width=480
description: 2018/06/18 Deep Learning JP:  http://deeplearning.jp/hacks/
published: June 19, 18
canonical: https://www.docswell.com/s/DeepLearning2023/ZXYGGX-dlhackssqueezeandexcitation-networks
---
# Page. 1

![Page Image](https://bcdn.docswell.com/page/2JVL9WGM7Q.jpg)

Squeeze-and-Excitation
Networks
M2


# Page. 2

![Page Image](https://bcdn.docswell.com/page/5EGXZD8XJL.jpg)

• Title: Squeeze-and-Excitation Networks
• Auther: Jie Hu, Li Shen, Gang Sun
• ILSVRC 2017 image classification
• CVPR 2018 Oral


# Page. 3

![Page Image](https://bcdn.docswell.com/page/4JQ8L1957P.jpg)

• CV
CNN
•
•
• Inception architectures [ICML’15, S.Ioffe et al]
•
•
•
[CVPR’16, S.Bell et al]
[NIPS’16, M.Jaderverg et
al]
•
•


# Page. 4

![Page Image](https://bcdn.docswell.com/page/K742DRXVJ1.jpg)

Squeeze-and-Excitation Networks
SE block
•
• Squeeze
•
•
Excitation
2


# Page. 5

![Page Image](https://bcdn.docswell.com/page/LJ1MZQN4EG.jpg)

Squeeze: Global Information Embedding
•
•
•
filter
global average pooling
→


# Page. 6

![Page Image](https://bcdn.docswell.com/page/GJW39M4ZJ2.jpg)

Excitation: Adaptive Recalibration
•
•2
•
fc
• fc – ReLU – fc – Sigmoid
•
(bottleneck) channel : C -> C/r -> C (r=16)
U
Rescaling


# Page. 7

![Page Image](https://bcdn.docswell.com/page/4EZV9Q2LJ3.jpg)



# Page. 8

![Page Image](https://bcdn.docswell.com/page/Y765K6VMEV.jpg)

224 x 224 image
•
•
Forward (GFLOPS)
backward(GPU) (ms)
inference(cpu)
ResNet-50
~3.86
190
164
SE-ResNet-50
~3.87
209
167
• ResNet-50
, 10%↑ (+ 2.5 million parameters)
•
SE Block
• 4%↑, top1 accuracy
0.1%


# Page. 9

![Page Image](https://bcdn.docswell.com/page/G75YP6DQ74.jpg)

• ImageNet Classification
•
•
•


# Page. 10

![Page Image](https://bcdn.docswell.com/page/9J2G6M8WJR.jpg)

• Pytorch
• https://github.com/moskomule/senet.pytorch
•
v0.4
• Cifar10
• Google Colaboratory


# Page. 11

![Page Image](https://bcdn.docswell.com/page/DEYV9X69EM.jpg)

SE block
•
•
•
•
•