# Which Self-Supervised Objective Should You Use? A 5-Question Decision Guide

"Let's do self-supervised pretraining" is easy to say. Picking *which* SSL objective fits your data is where projects go right or wrong. Here's the decision process we use, written as five questions.

### Q1. What modality is your data?

- **Text**: masked-token prediction (BERT-style) or next-token prediction is still the strongest default, because words depend heavily on nearby words.
- **Images**: you're choosing between masked reconstruction (MAE), contrastive view matching (SimCLR) and teacher-student methods (BYOL).
- **Speech**: mask spans in time and predict them (wav2vec 2.0-style). Image-style cropping rarely makes sense for audio.
- **Video and sensors**: temporal order is free supervision. Predict the next interval or match clips from the same sequence.

### Q2. Can you create "safe" augmentations?

Contrastive and teacher-student methods learn that two augmented views of one sample are the same thing. That only works if the augmentation doesn't change what matters.

- Random crop of a holiday photo: safe.
- Random crop of a chest X-ray: may cut out the one lesion that matters.
- Colour jitter on a dermatology image: may erase a diagnostic signal.

**If safe augmentations don't exist, prefer masked modeling.**

### Q3. Are reliable negatives available?

Contrastive losses (InfoNCE, NT-Xent) push different samples apart. But "different sample" isn't always "different class": two photos of different golden retrievers become negatives too. If false negatives are common, or large batches are too expensive, look at:

- **BYOL-style teacher-student**: no negatives; an online network predicts a slowly updated target network.
- **Barlow Twins / VICReg**: prevent collapse with variance and decorrelation terms instead of negatives.

### Q4. What's your compute budget?

- Contrastive methods often want big batches or memory queues, which is expensive on a single GPU.
- MAE only encodes the visible patches (often about 25%), so high mask ratios make pretraining cheaper.
- BYOL removes negatives but keeps two branches and needs careful tuning for stability.

### Q5. What will the encoder be used for?

- **Classification**: globally well-separated embeddings (contrastive methods do well here)
- **Detection / segmentation**: spatial detail (masked reconstruction keeps more of it)
- **Retrieval**: meaningful similarity geometry
- **Forecasting**: temporal and causal structure

### The whole thing as a tree

```
Predictable missing structure in each sample?
├── Yes → Masked modeling (text, images, audio) / temporal prediction (sequences)
└── No → Can you make meaning-preserving views?
         ├── Yes → Reliable negatives? → Contrastive
         │                          → Otherwise teacher-student or VICReg
         └── No → Rethink the objective, or whether SSL fits at all
```

### Don't skip evaluation

Pretraining loss only measures the pretext task. Report a **linear probe** (are features directly usable?), **full fine-tuning** (how well does it adapt?), **k-NN accuracy**, and performance with 1%/10% of labels (the label-efficiency SSL promises).

### Go deeper

This is the decision layer. For how each family works step by step, with walkthroughs of BERT, SimCLR, BYOL, MAE and wav2vec 2.0, read our full guide: [Self-Supervised Learning Explained](https://www.aheadmint.com/articles/self-supervised-learning-explained) on AheadMint.

— The AheadMint Team