- Astronomy has seen an exponential rise in data collection over the last decade.
- This requires new methods for data analysis, including AI.
- With the launch of new surveys, big data methodology has become a necessity.
A new class of extremely large telescopes has evolved to collect vast amounts of data; The volume of data collected from an entire survey ten years ago can now be collected in one night. One example of these new generation telescopes is fifth Sloan Digital Sky Survey (SDSS-V), launched in 2020 and slated to collect optical and infrared spectra for more than six million objects during its five-year lifetime [1].
Human inspection is wholly inadequate for dealing with millions of pieces of fuzzy and distorted images. Even traditional, centralized data processing systems can’t keep up with the petabytes of astronomical data emerging from new surveys. The solution is a new field called Astro information, which astronomy with machine learning and AI to search for habitable exoplanets, estimate red shifts, and classify galaxies and supernovas [2].
“I think [CNN] will become the norm since future astronomical surveys will produce an enormous quantity of data which will be necessary to inspect. We don’t have enough astronomers to cope with this.”
Data Scientist Carlo Enrico Petrillo [3].
A Brief History of Astronomical AI
Although Artificial Neural Network (ANN) has become a modern-day buzzword, its roots go back more than sixty years. The first neural network applied to a real-world problem was created in the late 1950s by a team of Stanford researchers for the purpose of eliminating phone line echoes [4]. However, it would be another three decades before the tool would be applied to astronomical research.
The first use of ANN in astronomy is attributed to researchers at the University of Arizona’s Steward Observatory, who published a 1990 paper detailing the use of ANN to study galaxy morphological classification (which divides galaxies into groups based on their appearance) and star galaxy separation (which distinguishes stars from galaxies).
The next 15 years saw an explosion of AI methods in astronomy, with ANNs applied to a myriad of fields including object detection to the classification of celestial objects [5]. In 2015, a French research team [6] used Hadoop, Hive, and Google’s MapReduce for Large Synoptic Survey Telescope (LSST) data and query management. The use of MapReduce was a game-changer at the time for two main reasons:
- It negated the need for programming low-level details related to fault tolerance, load balancing, and parallelization.
- One pass algorithms were efficiently implemented, making big data analysis for astronomical data feasible [7].
The following year, a team of U.S. researchers [8] built an Apache Spark-based toolkit called KIRA [9], which was deployed on Amazon’s EC2 cloud. The setup achieved speedups of up to 4.1 times faster over an equivalent C implementation running on the scalable network file system GlusterF. The team also showed that Apache Spark can integrate with pre-existing astronomy image processing libraries, allowing for existing source code to be reused for new analysis pipelines.
Will AI Prove Einstein Wrong?
A key point in Einstein’s theory of general relativity, published in 1915, is that massive space objects distort the fabric of spacetime. Astronomers use the theory, now called gravitational lensing – to learn more about distant celestial bodies. In 2017, a team of European astronomers developed a groundbreaking method to detect gravitational lenses using a convolutional neural network (CNN) –The first time CNN had been used to find “peculiar objects” in an astronomical survey [3].
Einstein’s specific predictions about the existence of gravitational lenses came true. However, his theory of relativity as a whole may be on the scientific chopping block in the not-so-distant future. The European Space Agency’s Euclid mission [11], set to launch later this year, will collect tens of Petabytes of data from around 10 billion sources. Astronomers hope to find out why the universe is expanding. Is there a mysterious, dark force pulling galaxies apart? Or was Einstein’s theory of relatively wrong? We don’t know the answer yet, but if Euclid’s data leads to the nullification of Einstein’s hypothesis, its replacement will be provided not by a new human genius, but by Artificial Intelligence.
References
[1] SDSS-V
[2] Astronomical big data processing using machine learning
[3] Artificial intelligence finds 56 new gravitational lens candidates
[4] History: The 1940’s to the 1970’s
[5] The application of artificial neural networks in astronomy
[6] Benchmarking SQL on mapreduce systems
[7] Google’s mapreduce programming model—revisited
[8] Processing astronomy imagery using big data technology
[9] Github– KIRA
[10] Finding strong gravitational lenses in the Kilo Degree Survey with Convolutional Neural Networks
