# Step 7: Iteration Loop Now you have a trained model and know where it fails. This step shows how to: 1. Run a **Golden QA check** to detect annotation drift 2. Select the **next batch** using a hybrid strategy The hybrid strategy balances: - **30% Coverage** - Diversity sampling to avoid tunnel vision - **70% Targeted** - Samples similar to failures This balance is critical. Only chasing failures creates a model that’s great at edge cases and terrible at normal cases. ## Golden QA Check Before selecting the next batch, verify annotation quality hasn’t drifted. The golden set is a small, carefully reviewed sample we check each iteration. **What to look for:** - Label count distribution staying stable - No unexpected empty samples - Class distribution roughly matching earlier rounds ## Prepare for Next Batch Selection ## Define Acquisition Budget **Batch sizing guidance:** - Size batches to your labeling capacity - For this tutorial, we’ll select ~20% of remaining groups ## Part 1: Coverage Selection (30%) Use ZCore scores computed in Step 3 to select diverse groups from remaining pool. ## Part 2: Targeted Selection (70%) Find groups similar to failures using embedding-based neighbor search. ## Combine and Tag Batch v1 ## The Complete Loop You now have the full iteration recipe: ```none 1. Run Golden QA check (detect drift) 2. Annotate the current batch: - Step 4: 2D detections on left camera - Step 5: 3D cuboids on point cloud 3. Train on all annotated data (Step 6) 4. Evaluate on val set, tag failures 5. Select next batch: 30% coverage + 70% targeted 6. Repeat until stopping criteria ``` ### Stopping Criteria Stop when: - Gains per labeled sample flatten (diminishing returns) - Remaining failures are mostly label ambiguity - Val metrics hit your target threshold ### The 30% Coverage Rule **Don’t skip the coverage budget.** Only chasing failures leads to: - Overfitting to edge cases - Distorted class priors - Models that fail on “normal” inputs Coverage keeps you honest. ## Summary You implemented the iteration loop: - **Golden QA check** to detect annotation drift - **Hybrid acquisition**: 30% coverage + 70% targeted - Tagged `batch:v1` ready for annotation (all slices: left, right, pcd) **Why this works:** - Coverage prevents overfitting to edge cases - Targeting fixes known failures - Golden QA catches annotation drift early - The combination improves faster than either strategy alone **Your turn:** Repeat Steps 4-7 with batch_v1, then batch_v2, etc. --- ## Congratulations! You’ve completed the Full Loop annotation tutorial. You now know how to: 1. **Setup** - Create group-level splits for multimodal data 2. **Select** - Use ZCore for diversity-based sample selection 3. **Annotate 2D** - Label detections on camera images 4. **Annotate 3D** - Label cuboids on point clouds 5. **Train + Evaluate** - Train a model and analyze failures 6. **Iterate** - Use hybrid acquisition to select the next batch This workflow scales from small experiments to production annotation pipelines.