Summary: Mid-career switching into data science is difficult. But if you already have substantial data base experience becoming a Data Engineer can be a great career by itself, or even the first stepping stone to becoming a Data Scientist.
One of the most common requests we get is for advice on mid-career switching into data science.
About you: You’ve got 5 or 10 years invested in some other line of work. That can vary from sales to engineering to computer science. You have work experience that means you no longer earn an entry level salary. In fact you probably already are making an above average salary.
At your age you’re much more likely to be married and perhaps have even started a family. Oh, and you probably also have a mortgage.
All of which means that no matter how disenchanted you are with your current line of work, the biggest problem is practical – you need to maintain a certain level of financial stability. How do you find enough time to learn the skills to let you compete for jobs in data science?
If this was even four or five years ago the shortage of data scientists was serious enough that employers were willing to look at folks without formal degrees in data science and looked more willingly at those who had done their studies on line, in MOOCs, or boot camps. But the reality is that there are now enough data scientists graduating with Masters or PhDs to pretty much meet the growing demand.
Even with the explosion in DS adoption across industries you can see this in the leveling off of DS salaries. They are still quite substantial and appropriate for the educational investment. But the days of astronomical annual increases and wide spread poaching are behind us.
You can see in salary surveys that DS salaries at pretty much all levels are stable year over year showing that even though demand is increasing, supply is keeping pace.
Not to be a Debbie-downer but if you’re in sales, or have a Masters in Engineering and now design cell phones, to be competitive you’re going to need a Masters in Data Science. Exceptions? Maybe. Probability? Pretty low.
BUT THERE IS AN EXCEPTION
Data Science versus Data Engineering
Now that there is wide spread adoption of AI in all its forms, the practice of data science is much more a team sport with several different specialty roles emerging. One of the fastest growing specialties is Data Engineering and there’s a shortage of them right now.
So here’s where the exception exists. If you have a degree in computer science or even just substantial experience as a DB engineer, DB architect, high-level DBA, or any other role that has you planning, creating, and maintaining large scale data operations, the transition to Data Engineer may be for you.
As with so many things relating to data science jobs there is a lot of smoke around job titles. Also, there’s considerable overlap between the data scientist’s role and the data engineer. Here’s the essence of the data engineer role:
Data Engineers are focused on designing, building, and maintaining the infrastructure and architecture for the data used by data scientists to build their models, and ultimately used in a production environment to make better decisions to increase revenue, decrease cost, and enhance customer experience.
Not too many years ago, data scientists were charged with this task also. But it became apparent that the skills and mindset required were at least slightly different and that efficiency would increase if the roles were separated.
By DataCamp
Too simplify, Data Engineers create the infrastructure and provide the data that Data Scientists need to do their jobs, then make sure that data is available at the quantity and quality to support the predictive models when implemented.
Required Skills
According to a list of requirements developed by Springboard from recent Glassdoor listings:
You probably already have these skills:
- Bachelor’s degree in computer science, statistics, information systems, or another quantitative field
- Five or more years of professional experience or a master’s degree plus three or more years of experience
- Advanced working knowledge of SQL (writing and debugging)
- Experience working with query authoring, relational databases, and a familiarity with a variety of databases
- Experience performing internal and external root cause analysis
- Strong analytical skills when working with unstructured data sets
- Proven experience successfully manipulating, processing, and extracting value from large and disconnected data sets
- Working knowledge of bash scripting and/or JavaScript
- Strong organizational and project management skills
- Experience with automation and configuration management
- Working understanding of code and script (for example, bash, Java, JavaScript, and Python)
- System monitoring, alerting, and dashboarding experience
If you don’t have these you can pick them up pretty rapidly:
- Experience with tools such as Hadoop, Kafka, and Spark
- Experience developing, managing, and optimizing big data architectures and pipelines
- Experience working with MongoDB, PostgreSQL, and Redis
- Experience working with cloud-based data solutions (e.g., AWS, EC2, EMR, RDS, and Redshift)
Salaries and Demand
Right now salaries for mid-career Data Engineers are essentially the same as Data Scientists. Demand is particularly high as there’s a shortage which means your odds of being considered even if you’re light on a few of these skills is higher.
Also, if there’s an on-going AI initiative in your company and you’re already on board, your odds of being given a shot are much higher.
Still want to be a Data Scientist?
Once you are on board and engaged in the data science program you can decide if you want to make the jump. If you’re interacting on a regular basis with data scientists you’ll have a front row seat to a better understanding of what additional skills are required.
If you’re still motivated, and personally I think data scientist do have more fun, then gaining the statistical and machine learning skills via on-line course, MOOC, or boot camp is a much more reasonable approach.
Your Secret Super Power
As a mid-career switcher you have some super powers you might not have considered. That’s basically your industry and process experience.
They say that a good consultant and by extension a good employee needs three areas of competence. The knowledge and experience with data base architecture and maintenance is your foundation skill.
But what you have that other newly minted graduates will not are these two:
- Experience with a specific industry.
- Experience with a specific set of processes. These might be sales, marketing, HR, finance, ERPs, manufacturing, or supply chain, among others.
You recall when you were starting out and you didn’t yet have these experiences. Well now you do. Don’t forget to tell the folks who are hiring you about these. It’s your experience with business and process that makes your data base skills more relevant and more valuable.
Other articles by Bill Vorhies
About the author: Bill is Contributing Editor for Data Science Central. Bill is also President & Chief Data Scientist at Data-Magnum and has practiced as a data scientist since 2001. His articles have been read more than 2.1 million times.
