GeoAI & Deep Learning Geospatial Hub
Artificial intelligence engineering for Earth Observation: Segment Anything Model (SAM-Geo), Vision Transformers, U-Net building extraction, YOLOv8 drone detection, and our interactive Tiling & GPU Memory Calculator.
PyTorch ecosystem with native multispectral GeoDataset samplers and spatio-temporal batch iterators.
Zero-shot foundation segmentation coupled with multi-scale Vision Transformer hierarchical feature pyramids.
Compound loss function overcoming 98:2 background-to-building class imbalance in satellite scenes.
Slicing Aided Hyper Inference preserving sub-meter object detection definition on 10K+ orthomosaics.
Deep Learning Architectures for Earth Observation & Satellite Imagery
Geospatial Artificial Intelligence (GeoAI) adapts deep computer vision architectures to process non-RGB multi-spectral bands, sub-meter spatial resolutions, and continental geographic extents. Standard computer vision models trained on natural photographic images (such as ImageNet) make assumptions that fail on aerial imagery: photographic images exhibit perspective foreshortening, horizontal horizon orientation, and 3-channel RGB color spaces. In contrast, Earth Observation imagery is nadir-looking, rotationally invariant, multi-spectral (up to 13 bands in Sentinel-2), and possesses high dynamic radiometric depth (12-bit to 16-bit integer reflectance).
1. Convolutional Neural Networks: U-Net and DeepLabV3+
The U-Net architecture remains a cornerstone of geospatial semantic segmentation:
- Encoder (Contracting Path): Progressively reduces spatial dimensions through strided convolutions and max-pooling while doubling feature channels, extracting high-level semantic context. Modern implementations replace traditional vanilla convolutions with pre-trained backbones (such as ResNet-34, ResNet-50, or EfficientNet-B4).
- Skip Connections: Transfer high-resolution spatial feature maps directly from the encoder levels to the corresponding decoder levels. This preserves sharp geometric boundaries (rooftop edges, agricultural parcel boundaries, roads) that would otherwise be lost during downsampling.
- Decoder (Expansive Path): Upsamples feature representations using transposed convolutions or bilinear interpolation to reconstruct a pixel-level classification mask matching input spatial dimensions.
DeepLabV3+ enhances U-Net by introducing Atrous Spatial Pyramid Pooling (ASPP). By applying dilated convolutions with varying dilation rates ($r = 6, 12, 18$), ASPP captures multi-scale geographic context (such as small single-family homes and sprawling logistics warehouses) without increasing parameter counts.
2. Vision Transformers (ViT) and SegFormer
While standard convolutions possess a localized receptive field ($3 \times 3$ or $5 \times 5$ pixels), Vision Transformers leverage self-attention mechanisms to model global spatial relationships across an entire scene:
Attention(Q, K, V) = softmax((Q * K^T) / sqrt(d_k)) * V
In geospatial tasks, SegFormer employs hierarchical transformer encoders (Mix Transformer, MiT) that output multi-level features without requiring complex positional embeddings. SegFormer handles arbitrary image resolutions and demonstrates superior generalization when transferred between different satellite sensors (such as training on Sentinel-2 and inferring on Landsat 8-9).
Geospatial Foundation Models: SAM-Geo & Prithvi
The paradigm in GeoAI has shifted from training small task-specific models from scratch to fine-tuning large pre-trained Geospatial Foundation Models:
1. Segment Anything Model for Geospatial (SAM-Geo)
Meta's Segment Anything Model (SAM) has been integrated into geospatial workflows through the open-source segment-geospatial library (SAM-Geo). SAM consists of a heavy Vision Transformer image encoder (ViT-H, 636 million parameters), a prompt encoder (accepting points, bounding boxes, or text), and a lightweight mask decoder.
SAM-Geo enables zero-shot instance segmentation directly on georeferenced GeoTIFFs:
- Point Prompts: Clicking on a single solar panel or building automatically segments the contiguous structure.
- Text-Guided Prompts: Coupling SAM with GroundingDINO allows querying satellite scenes with natural language prompts (e.g.,
"solar photovoltaic arrays"or"swimming pools"), generating zero-shot bounding boxes that guide SAM mask generation.
2. NASA & IBM Prithvi Foundation Model
Trained on the Harmonized Landsat-Sentinel (HLS) dataset across the contiguous United States using a Masked Autoencoder (SatMAE) approach, Prithvi models temporal dynamics across multi-spectral channels (Blue, Green, Red, Narrow NIR, SWIR1, SWIR2). Fine-tuning Prithvi requires 10x to 50x less labelled training data to achieve higher F1-scores than classical models in flood extent mapping, wildfire burn scar analysis, and crop classification.
Tiny Object Detection in Drone Imagery with YOLO & SAHI
High-resolution drone mapping produces gigantic orthomosaics ($10,000 \times 10,000$ pixels at $2\text{ cm/px}$ GSD). Attempting to pass this image into a standard YOLOv8 or YOLOv11 object detector requires resizing the image to $640 \times 640$ pixels, reducing a $30 \times 30$ pixel rooftop solar panel to less than $1$ pixel, rendering detection impossible.
Slicing Aided Hyper Inference (SAHI) overcomes this limitation through an intelligent tiled inference pipeline:
- The full orthomosaic is sliced into overlapping tiles (e.g., $512 \times 512$ pixels with 20% horizontal and vertical overlap).
- Each patch is fed into the YOLO detector at native full resolution.
- Detections from all slices are re-projected back into original orthomosaic pixel coordinates.
- Non-Maximum Suppression (NMS) or Non-Maximum Merging (NMM) is executed across tile seams to resolve duplicate detections of objects cut by tile boundaries.
Production PyTorch Pipeline for Satellite Semantic Segmentation
The following standalone Python recipe implements a production training pipeline: custom multi-band GeoTIFF dataset loading, U-Net architecture, compound Dice + Focal Loss, and mixed-precision forward pass:
import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import Dataset, DataLoader
import rasterio
import numpy as np
# 1. Compound Dice + Focal Loss for Imbalanced Geospatial Classes
class DiceFocalLoss(nn.Module):
def __init__(self, alpha=0.25, gamma=2.0, smooth=1e-6):
super().__init__()
self.alpha = alpha
self.gamma = gamma
self.smooth = smooth
def forward(self, pred_logits, targets):
probs = torch.sigmoid(pred_logits)
probs_flat = probs.view(-1)
targets_flat = targets.view(-1)
# Dice Loss
intersection = (probs_flat * targets_flat).sum()
dice_loss = 1.0 - ((2.0 * intersection + self.smooth) /
(probs_flat.sum() + targets_flat.sum() + self.smooth))
# Focal Loss
bce = nn.functional.binary_cross_entropy_with_logits(pred_logits, targets, reduction="none")
p_t = probs * targets + (1.0 - probs) * (1.0 - targets)
focal_loss = (self.alpha * (1.0 - p_t) ** self.gamma * bce).mean()
return dice_loss + focal_loss
# 2. Multi-Spectral Satellite Patch Dataset
class SatellitePatchDataset(Dataset):
def __init__(self, patch_paths, mask_paths):
self.patch_paths = patch_paths
self.mask_paths = mask_paths
def __len__(self):
return len(self.patch_paths)
def __getitem__(self, idx):
# Read 4-channel GeoTIFF (Red, Green, Blue, NIR)
with rasterio.open(self.patch_paths[idx]) as src:
img = src.read() # Shape: (4, H, W)
img = np.nan_to_num(img, nan=0.0).astype(np.float32) / 10000.0 # Normalize reflectance
with rasterio.open(self.mask_paths[idx]) as src:
mask = src.read(1) # Shape: (H, W)
mask = (mask > 0).astype(np.float32)
return torch.tensor(img), torch.tensor(mask).unsqueeze(0)
# 3. Training Execution with Mixed Precision
def train_model(model, dataloader, epochs=25, device="cuda"):
model.to(device)
criterion = DiceFocalLoss()
optimizer = optim.AdamW(model.parameters(), lr=1e-4, weight_decay=1e-2)
scaler = torch.cuda.amp.GradScaler() # FP16 mixed precision
print(f"Commencing training on {device} with mixed precision...")
model.train()
for epoch in range(epochs):
epoch_loss = 0.0
for images, masks in dataloader:
images = images.to(device)
masks = masks.to(device)
optimizer.zero_grad()
with torch.cuda.amp.autocast():
logits = model(images)
loss = criterion(logits, masks)
scaler.scale(loss).backward()
scaler.step(optimizer)
scaler.update()
epoch_loss += loss.item()
print(f"Epoch [{epoch+1}/{epochs}] - Loss: {epoch_loss / len(dataloader):.4f}")
# Example invocation:
# model = CustomUNet(in_channels=4, out_channels=1)
# train_model(model, my_dataloader)
Common Pitfalls and Field Troubleshooting in GeoAI
Deploying machine learning models in operational Earth Observation environments reveals several critical failure modes:
1. Spatial Autocorrelation & Data Leakage
The most common flaw in geospatial academic literature is randomly splitting image patches into training and validation sets. Because adjacent patches share identical weather, sensor angle, and soil properties, the validation score tests memory rather than generalization. Always employ Spatial Block Cross-Validation: train on one geographical county or river basin and validate on an entirely separate geographic region.
2. Tiling Edge Boundary Seams
When a full scene is cut into discrete non-overlapping tiles, convolutional padding causes network predictions to degrade along tile perimeters. When tiles are reassembled, visible grid lines and broken building boundaries appear. Overcome this by using overlapping sliding windows with Gaussian weighting (predicting on $512 \times 512$ patches with $20\%$ overlap and blending overlapping pixels using a 2D Gaussian kernel).
3. Radiometric Domain Shift
A neural network trained on dry summer Mediterranean satellite imagery will experience significant performance degradation if evaluated on wet spring imagery in northern Europe. Mitigate domain shifts by:
- Applying surface reflectance Bottom-of-Atmosphere (BOA) atmospheric corrections prior to model input.
- Integrating calculated invariant spectral indices (such as SUI, NDVI, MNDWI) as auxiliary input bands alongside raw spectral bands.
- Employing heavy photometric augmentations (random gamma adjustments, contrast stretching, and color jitter) during training.
Frequently Asked Questions on GeoAI & Deep Learning
Why is random train/validation pixel splitting invalid for geospatial deep learning datasets?
Tobler's First Law of Geography states that nearby spatial locations are more related than distant locations. When training patches are randomly sampled across the same satellite scene, adjacent training and validation patches share identical lighting, soil types, building architectures, and atmospheric conditions. This spatial autocorrelation causes severe data leakage, resulting in artificially inflated validation scores that fail to generalize to new geographic regions. Spatial block cross-validation with geographic buffer zones is mandatory.
How does Slicing Aided Hyper Inference (SAHI) solve the tiny object detection problem in drone imagery?
High-resolution drone orthomosaics frequently measure 10,000 x 10,000 pixels or larger, while objects of interest (vehicles, solar panel defects, cattle) measure only 20 x 20 pixels. Downsampling the whole image to standard YOLO input sizes (640 x 640) collapses tiny targets into sub-pixel blur. SAHI divides the large orthomosaic into overlapping sliding tiles at native resolution, runs inference on each tile independently, and merges overlapping bounding boxes across tile borders using Non-Maximum Suppression (NMS).
Why do standard Cross-Entropy loss functions fail in building footprint segmentation?
In regional satellite imagery, building footprints typically occupy only 1% to 5% of total landscape pixels, while background forest, agriculture, and water account for 95% to 99%. Under standard Cross-Entropy loss, a model predicting 100% background achieves 95% accuracy while failing completely. Geospatial engineers employ compound loss functions combining Dice Loss and Focal Loss (Loss = Dice + lambda * Focal) to penalize false negatives on rare foreground classes.
What is the primary difference between Segment Anything Model for Geospatial (SAM-Geo) and classical supervised U-Net models?
Classical U-Net models require training on thousands of manually annotated pixel-level masks for a specific target class. SAM-Geo is a foundation model trained on over 1 billion masks that performs zero-shot instance segmentation. By supplying point coordinates, bounding boxes, or text prompts (via GroundingDINO), SAM-Geo immediately segments unfamiliar solar farms, agricultural plots, or water bodies without task-specific retraining.
How does mixed precision (FP16 / BF16) reduce GPU memory consumption during deep learning training?
Standard single-precision floating point (FP32) consumes 4 bytes per model weight, activation, and gradient tensor. Mixed precision (FP16 or Bfloat16) stores weights and activations in 2 bytes while maintaining FP32 master weights for numerical stability. This cuts GPU VRAM consumption by nearly 50%, allowing practitioners to double batch sizes or increase image patch dimensions from 256 to 512 pixels without out-of-memory errors.