David Evans ARO Workshop on Adversarial Learning Stanford, 14 Sept 2017

54 Pages·2017·25.41 MB·English

by  David Evans

evadeML.org
Shrinking and
Exploring
David Evans
University of Virginia
Adversarial
Search Spaces
ARO Workshop on
Adversarial Learning
Stanford, 14 Sept 2017
Weilin Xu Yanjun Qi
Machine Learning is Eating Computer Science
1
Security State-of-the-Art
Random guessing
attack success Threat models Proofs
probability
information
"𝟏𝟐𝟖
Cryptography 𝟐 theoretic, resource required
bounded
capabilities,
"𝟑𝟐
System Security 𝟐 motivations, common
rationality
Adversarial white-box,
"𝟏𝟏 "𝟔
𝟐 *; 𝟐 rare!
Machine Learning black-box
2
Adversarial Examples
“panda”
+ 0.007 × [𝑛𝑜𝑖𝑠𝑒] =
“gibbon”
Example from: Ian J. Goodfellow, Jonathon Shlens, Christian Szegedy.
Explaining and Harnessing Adversarial Examples. 2014.
3
Adversarial Examples Game
6
Given seed sample, 𝑥, find 𝑥 where:
6
𝑓 𝑥 ≠ 𝑓(𝑥) Class is different (untargeted)
6
𝑓 𝑥 = 𝑡 Class is 𝑡 (targeted)
6
∆ 𝑥, 𝑥 ≤ 𝛿 Difference below threshold
6
∆ 𝑥, 𝑥 is defined in some (simple!) metric space:
𝐿 𝐿 𝐿 𝐿
“norm (# different), norm, norm (“Euclidean”), norm:
@ A B C
4
Detecting
Prediction
0
Model
Adversarial
Examples
S
q
u
e Model Adversarial
e
Prediction
z
e 1
r
1
Yes
S
q
u 𝒇(𝑝𝑟𝑒𝑑 , 𝑝𝑟𝑒𝑑 , … , 𝑝𝑟𝑒𝑑 )
@ A K
e Model
e
z Prediction
e 2
r
2
Input
No
…
Legitimate
S
q
u
e Model’
e
z
Prediction
e
k
r
k
“Feature Squeezing”
𝒙 [0.054, 0.4894, 0.9258, 0.0116, 0.2898, 0.5222, 0.5074, …]
Squeeze: 𝑓 = round(𝑓 ×4)/4
O O
[0.0, 0.5, 1.0, 0.0, 0.25, 0.5, 0.5, …]
6 6
squeeze 𝒙 ≈ squeeze 𝒙 ⟹ 𝑓(squeeze 𝒙 ) ≈ 𝑓(squeeze 𝒙 )
[0.0, 0.5, 1.0, 0.0, 0.25, 0.5, 0.5, …]
Squeeze: 𝑓 = round(𝑓 ×4)/4
O O
6
𝒙
[0.0491, 0.4903, 0.9292, 0.009, 0.2942, 0.5243, 0.5078, …]
6
Example Squeezers
e
m
e
l o
a
r
c
h
s
c
y
o
e
n
r
g o

m
t
i
b
t
- i 3x3 smoothing:
8 b
- Replace with median of pixels and its neighbors
1
Reduce Color Depth Median Smoothing
7
Simple Instantiation
Model
Prediction
0
(7-layer
CNN)
Adversarial
Yes
B
i
t

D
Model
1 e Prediction
p
1
max 𝐿 𝑝 , 𝑝 , 𝐿 𝑝 , 𝑝2 > 𝑡
t
h A @ A A @
-
Input No
M
Prediction
2 e 2
Legitimate
× d Model
2 i
a
n
s
e
800
l
p
m
a
Legitimate
600
x
E

f
o
400
r
e
Adversarial
b
m
threshold = 0.0029
200
u
detection: 98.2%, FP < 4%
N
0
0.0 0.4 0.8 1.2 1.6 2.0
Maximum 𝐿 distance between original and squeezed input
A
9