NASA has a massive amount of data and it receives more every day. While some of the data is processed immediately, much of the data is archived to be processed later, sometimes years later. This situation needs to change if researchers are to utilize the data to investigate critical issues with dynamically changing characteristics like global climate change. To increase its ability to process and use this data in a timely manner, NASA’s Marshall Space Flight Center announced a joint development program with IBM Research to process the NASA data using IBM’s foundation AI technology.

To put this task into perspective, the GPT-3 data set, which led to the development of ChatGPT AI platform that recently passed a Wharton MBA exam, represents about 45TB (terabytes) of data. By comparison, NASA estimates that its data set could be upwards of 250PB (petabytes). With 1PB equal to 1,000TB, the NASA data set is over 5,000 times larger than the GPT-3 data set, making this a monumental task, but the benefits could be ground-breaking.

Previously, IBM estimated that 90% of data collected is never used, and in their press invitation, IBM and NASA noted that “Currently, half of all scientific findings come from archived data, which makes it challenging for researchers to study ever-evolving threats such as climate change.” Efficiently mining the enormous amount of archival data needs the power of AI. IBM Research’s massive cloud resources, the collective experiences of the company’s AI experts, and its AI foundation model technology will help NASA filter and analyze earth-science data in days or months rather than years or even decades.