This document discusses active learning techniques called Deep Badge Active Learning. It proposes using gradient embeddings to represent samples and k-means++ initialization for sample selection. Specifically, it uses the gradient embedding for feature representation, then performs k-means++ initialization to select samples by finding those with the maximum 2-norm and those farthest from existing samples, adding them to the set iteratively. This aims to select a diverse set of samples, similar to how binary search works. The technique could improve over entropy-based and core-set selection approaches for active learning with convolutional neural networks.

1. Deep Badge Active Learning by
Diverse, Uncertain Gradient Lower Bounds
Active Sampling
ICLR 2020

2. Sampling
Passive Sampling : Random
Active Sampling : Use model
=>
Feature Representation
+ Decision Boundary / Uncertainty
+ Select Sample

3. Active Sampling
- Entropy Based : Near the Decision boundary, Uncertainty
- Core-set selection : Farthest Samples, Diversity
ACTIVE LEARNING FOR CONVOLUTIONAL NEURAL
NETWORKS: A CORE-SET APPROACH (ICLR 2018)

4. BADGE
1. Use Gradient Embedding for Feature Representation
2. Use K-means++ init for Sample Selection
=> K-Selection
Set = {}
1. Find Max 2-Norm -> Set add
2. Find Farthest Sample from Set -> Set add
3. Repeat 2
~ Binary Search와 유사하게 작동할 것임

5. Result

6. Result (from SIMILAR Neurips 2021)