There are currently over 6,000 job listings on Indeed for “data engineer” in the San Francisco Bay Area. Let’s put this into context:

• There are 3x more listings for data engineer than there are for data scientist

• There are 2x more listings for data engineer than for for data analyst

• In addition, our latest study found that there are only 6500 self-described data engineers on LinkedIn

Data engineering is currently the most in-demand skill set in the world of big data. It also might be the most difficult to come by, and salaries are following this demand. Anecdotally, top data engineering positions at companies like Facebook, Amazon, and Google can earn more than $500k per year. Indeed’s data shows a more modest distribution, but the salaries range well into the six figures:

3 trends driving this demand

Jonathan Coveney, formerly of Stripe now at Google, points to three trends that are currently driving up demand for data engineering talent:

New sophistication in how companies think about data and the people who manage it. “There’s a growing sense today that data isn’t just a byproduct,” Jonathan says, “but rather it’s a core part of what a company does.” In the past, a company might run a one-off analysis of a database log. Today, data engineers own the analysis and organization of that data. They identify with data processes, which leads to a more nuanced understanding of their data architecture.

The growing popularity of “how we built our data infrastructure” blog posts certainly supports Coveney’s observation. Building scalable data systems is viewed as a competitive advantage, and doing this right is becoming a point of engineering pride.

Push towards machine learning. Thanks to machine learning advances, access to proprietary data has become a major competitive advantage for firms in almost all industries. Collecting and making this data available has therefore become a key strategic function.

In our research, we found that “machine learning” is in the top five date engineering skills at companies with less than 1000 employees. Larger companies are focusing data engineering talent on managing BI projects (note the popularity of Oracle in the chart below), while growing companies are investing their data talent into more innovative projects.

Companies building data products. There’s some overlap here with machine learning, but Jonathan uses the example of maps to describe the difference: “The machine learning aspects of maps include things like traffic detection and routing, but the infrastructure of maps relies on managing and organizing massive volumes of data–that’s data engineering.” Today’s tech companies need to play well in both areas.

From tech company to data company

Across all three of the above trends, we see a shift of tech companies becoming data companies. Uber, Airbnb, Facebook (one of the top three employers of data engineers), Jawbone, Spotify – these are tech companies, sure, but they also deal with the management and productization of massive volumes of data.

Data products certainly aren’t going anywhere, and based on some quick searches on Indeed, San Francisco isn’t the only tech hub looking to hire for this skill set:  

In the major tech hubs, data engineer is significantly more in-demand then data scientist (which has been called the “sexiest job of the 21st century”). There’s something big happening here, and it’s going to be interesting to see how these trends play out.

This post originally appeared on The Startup Grind