At Goddard Space Flight Center, the GSFC Data Science Group has completed the testing for their SatVision Top-of-Atmosphere (TOA) Foundation Model, a geospatial foundation model for coarse-resolution all-sky remote sensing imagery. The team, comprised of Mark Carroll, Caleb Spradlin, Jordan Caraballo-Vega, Jian Li, Jie Gong, and Paul Montesano, has now released their model for wide application in science investigations.
Foundation models can transform the landscape of remote sensing (RS) data analysis by enabling the pre-training of large computer-vision models on vast amounts of remote sensing data. These models can be fine-tuned with small amounts of labeled training and applied to various mapping and monitoring applications. Because most existing foundation models are trained solely on cloud-free satellite imagery, they are limited to applications of land surface or require atmospheric corrections. SatVision-TOA is trained on all-sky conditions which enables applications involving atmospheric variables (e.g., cloud or aerosol).
SatVision TOA is a 3 billion parameter model trained on 100 million images from Moderate Resolution Imaging Spectroradiometer (MODIS). This is, to our knowledge, the largest foundation model trained solely on satellite remote sensing imagery. By including “all-sky” conditions during pre-training, the team incorporated a range of cloud conditions often excluded in traditional modeling. This enables 3D cloud reconstruction and cloud modeling in support of Earth and climate science, offering significant enhancement for large-scale earth observation workflows.
With an adaptable and scalable model design, SatVision-TOA can unify diverse Earth observation datasets and reduce dependency on task-specific models. SatVision-TOA leverages one of the largest public datasets to capture global contexts and robust features. The model could have broad applications for investigating spectrometer data, including MODIS, VIIRS, and GOES-ABI. The team believes this will enable transformative advancements in atmospheric science, cloud structure analysis, and Earth system modeling.
The model architecture and model weights are available on GitHub and Hugging Face, respectively. For more information, including a detailed user guide, see the associated white paper: SatVision-TOA: A Geospatial Foundation Model for Coarse-Resolution All-Sky Remote Sensing Imagery.
![Examples of image reconstruction by SatVision-TOA. Left: MOD021KM v6.1 cropped image chip using MODIS bands [1, 3, 2]. Middle: The same images with randomly applied 8 × 8 mask patches, masking 60% of the original image. Right: The reconstructed images produced by the model, along with their respective Structural Similarity Index Measure (SSIM) scores. These examples illustrate the model’s ability to preserve structural detail and reconstruct heterogeneous features, such as cloud textures and land-cover transitions, with high fidelity.](https://www.nasa.gov/wp-content/uploads/2024/12/reconstruction-v4.png)
Examples of image reconstruction by SatVision-TOA. Left: MOD021KM v6.1 cropped image chip using MODIS bands [1, 3, 2]. Middle: The same images with randomly applied 8×8 mask patches, masking 60% of the original image. Right: The reconstructed images produced by the model, along with their respective Structural Similarity Index Measure (SSIM) scores. These examples illustrate the model’s ability to preserve structural detail and reconstruct heterogeneous features, such as cloud textures and land-cover transitions, with high fidelity.NASA