Alternative data generally refers to information beyond traditional sources that include corporate financial statements and filings, stock market data, and government reports. Initially used by investment firms, today, alternative data is also actively utilized by government agencies to analyze economic, social, and political environments and formulate policies.
More information than ever before is being digitized, creating new alternative data sources. This article summarizes these sources and provides examples of how governments use alternative data to implement policies aimed at improving social welfare, crime rates, healthcare, and the environment.
Alternative data sources are growing
The use of digital applications to conduct business, governmental, and social transactions — commonly referred to as “digitization” — leads to an increase in the production of public data. As a result, sources of alternative data are growing and presently include:
1. Mobile and desktop applications
Applications are a trove of data revealing insights into consumer preferences and behaviors. Industries that collect data and use this particular source include travel, food delivery, consumer goods, health and fitness, transportation, and entertainment.
2. Credit and debit card transactions
Credit and debit card transactions reveal insights into consumer spending habits, preferences, and disposable income. Common use cases include academic, government, and marketing research.
3. Email receipt transaction data
Similar to credit and debit card transaction data, receipts sent via email to purchasers reveal critical information about consumer behavior and spending patterns.
Subject to privacy laws and regulations, email receipt transaction data is obtained from various sources, including email service providers and third-party data firms. Common use cases include academic, government, and marketing research.
4. Satellite data
Satellites are a valuable way to collect environmental and human behavior data. Examples include measuring and monitoring vegetation, land use, coastline changes, soil moisture, surface elevation, tectonic activity, melting glaciers, temperature changes, and air quality.
5. Low-level drones
Drones equipped with cameras and sensors can capture high-resolution images, videos, and detailed data from the Earth’s surface. Applications include monitoring changes in building infrastructure, vegetation, agriculture, topography, and air quality.
6. Online public resources
The vast expanse of data available on the internet makes online public resources a dominant source of alternative data. Some examples include:
Public databases
Public databases include filings from the European Securities and Markets Authority (ESMA) and Securities Exchange Commission (SEC), global supply chain import and export records, patent and trademark filings, and government contracts. While valuable, these documents are typically challenging to obtain at a large scale due to inaccessible file formats, like PDF.
Web scraping
Publicly available information on social media sites, marketplaces, and other websites can provide a wide range of alternative data. Some examples include:
- Real-time product and service pricing data
- Consumer sentiment data from product reviews
- Stock, bond, and commodity pricing data
- Job market data
- Census data
- Real-time weather data
Online public data is typically obtained through web scraping — the automated gathering of information from the internet.
Web scraping uses scripts and software tools to extract, parse, and store data in a structured format that analysts can read. Examples of data types gathered by web scraping include text, images, and videos. Watch a live demonstration of a web scraper in action to learn more.
Five ways government agencies use alternative data
Government agencies are increasingly unlocking the power of alternative data to gain insights and make policy decisions that can positively impact citizens.
Recent examples include:
1. Economic “nowcasting”
Nowcasting — a portmanteau of “now” and “forecasting” — uses high-frequency data and statistical models to predict the present or near-term future state of the economy.
Alternative data sources such as social media, online job postings, credit card transactions, and satellites typically provide timelier and more relevant information than traditional sources like government reports. As a result, government agencies (e.g., the Central Bank of Japan) are nowcasting indicators like inflation using alternative sources, such as mobility data.
2. Social welfare programs
The Fundación Paraguaya, a self-sustainable NGO, helps people living in poverty by using alternative data to identify their needs and, based on this, provide resources to help them achieve their goals.
As part of its strategy, the foundation extracts data from a dedicated application and web platform to identify economic trends, generate real-time reports, and create “poverty maps.” These activities enable policymakers to develop strategies that channel aid to areas that need more resources.
3. Environmental policy
The European Union’s Copernicus program collects air and water quality data using various methods (like satellites) to inform policy decisions. For example, satellite data collected by researchers helps identify high-pollution areas. Doing so enables policymakers to make regulatory decisions aimed at reducing emissions.
4. Healthcare and transportation planning
In 2018, the government of Singapore launched “Smart Nation,” a program leveraging technology and data aimed at improving the lives of its citizens. Part of the program involved a pilot project called “Smart Elderly Monitoring and Alert System,” which promoted the use of the Healthy 365 App to monitor the health of elderly residents living in public housing.
Another example was the on-demand public bus service trial in 2018 that paired dynamic routing and matching algorithms to optimize public bus services. Using a mobile app, commuters requested pick-ups and drop-offs at any bus stop on routes with unpredictable or low ridership.
5. Crime reduction
The city of New Orleans in the United States launched a program in 2017 called “Operation: Real-Time Crime Center.” The program used alternative data from several sources to identify and respond to crime in real time, including social media, public cameras, and other surveillance technology.
According to a case study published by the agency, the program reduced murders by 46 percent between 2018 and 2019. In addition, gun violence was reduced by 82 percent in one of the most dangerous areas, and city-wide armed robberies dropped by over 30 percent.
Alternative data obtained via web scraping can positively impact social change
Alternative data is obtained from many places; however, the internet remains the largest and most accessible source of information. You can learn more about alternative data scraping, its challenges, and how to overcome them in our whitepaper: Scraping Alternative Data: Technological Challenges to Keep in Mind.
