GigaTIME

Multimodal AI generates virtual population for tumor microenvironment modeling (Cell)

Authors: Jeya Maria Jose Valanarasu*, Hanwen Xu*, Naoto Usuyama*, Chanwoo Kim*, Cliff Wong, Peniel Argaw, Racheli Ben Shimol, Angela Crabtree, Kevin Matlock, Alexandra Q. Bartlett, Jaspreet Bagga, Yu Gu, Sheng Zhang, Tristan Naumann, Bernard A. Fox, Bill Wright, Ari Robicsek, Brian Piening, Carlo Bifulco, Sheng Wang, Hoifung Poon

📄 Paper • Code

What is this model?

• Task: GigaTIME can take in a H&E slide and create a virtual mIF slide across 21 protein channels.
• Inputs: The model processes the H&E slide at a tile size of 256*256. The input is a tensor of shape [B,3,256,256].
• Outputs: The output is the 23 channel spatial proteomics maps (2 are background channels) of shape [B,23,256,256]. The order of protein channels can be found in the code.

Quickstart

Checkout the gigatime_testing.ipynb notebook at Github Repo

Model Uses

Intended Use

The data, code, and model checkpoints are intended to be used solely for (I) future research on pathology AI models and (II) reproducibility of the experimental results reported in the reference paper. The data, code, and model checkpoints are not intended to be used in clinical care or for any clinical decision-making purposes.

Primary Intended Use

The primary intended use is to support AI researchers reproducing and building on top of this work. GigaTIME should be helpful for generating virtual mIF profiles from routine H&E pathology slides.

Out-of-Scope Use

Any deployed use case of the model --- commercial or otherwise --- is out of scope. Although we evaluated the models using a broad set of publicly-available research benchmarks, the models and evaluations are intended for research use only and not intended for deployed use cases.

License Notice

The model is not intended or made available for clinical use as a medical device, clinical support, diagnostic tool, or other technology intended to be used in the diagnosis, cure, mitigation, treatment, or prevention of disease or other conditions. The model is not designed or intended to be a substitute for professional medical advice, diagnosis, treatment, or judgment and should not be used as such. All users are responsible for reviewing the output of the developed model to determine whether the model meets the user’s needs and for validating and evaluating the model before any clinical use.

Citation

@article{valanarasu2025gigatime,
  title={Multimodal AI generates virtual population for tumor microenvironment modeling},
  author={Valanarasu, Jeya Maria Jose and Xu, Hanwen and Usuyama, Naoto and Kim, Chanwoo and Wong, Cliff and Argaw, Peniel and Ben Shimol, Racheli and Crabtree, Angela and Matlock, Kevin and Bartlett, Alexandra Q. and Bagga, Jaspreet and Gu, Yu and Zhang, Sheng and Naumann, Tristan and Fox, Bernard A. and Wright, Bill and Robicsek, Ari and Piening, Brian and Bifulco, Carlo and Wang, Sheng and Poon, Hoifung},
  journal={Cell},
  year={2025},
  publisher={Cell Press}
}