The new advice today for data scientists is not to become a generalist. You can read recent articles on this topic, for instance here. In this blog, I explain why I believe it should be the opposite. I wrote about this here not long ago, and provide additional arguments in this article, as to why it helps to be a generalist.
Of course, it is difficult, and probably impossible to become a data science generalist just after graduating. It takes years to acquire all the skills, yet you don’t need to master all of them. It might be easier for a physicist, engineer, or biostatistician currently learning data science, after years of corporate experience, than it is for a data scientist with no business experience. Possibly the easiest way to become one is to work for start-up’s or small companies, taking on many hats as you will probably be the only data scientist in your company, and will have to change jobs more frequently than if you work for a big company. To the contrary, for a big company, you are expected to work in a very specialized area, though it does not hurt to be a generalist, as I will illustrate shortly. Being a specialized data scientist could put you on a very predictable path that limits your career growth and flexibility, especially if you want to create your company down the line. Let’s start with explaining what a data science generalist is.
The data science generalist
The generalist has experience working in different roles and different environments, for instance, over a period of 15 years, having worked as a
- Business analyst or BI professional, communicating insights to decision makers, mastering tools such as Tableau, SQL and Excel; or maybe being the decision maker herself
- Statistician / data analyst with expertise in predictive modeling
- Expert in algorithm design and optimization
- Researcher in an academic-like setting, or experience in testing / prototyping new data science systems and proofs of concept (POC)
- Builder / architect: designing APIs, dashboards, databases, and deploying/maintaining yourself some modest systems in production mode
- Programmer (statistical or scientific programmer with exposure to high performance computing and parallel architectures – you might even have designed your own software)
- Consultant, directly working with clients, or adviser
- Manager or director role rather than individual contributor
- Professional with roles in various industries (IT, media, Internet, finance, health care, smart cities) in both big and small companies, in various domains ranging from fraud detection, to optimizing sales or marketing, with proven, measurable accomplishments
In short, the generalist has been involved at one time or another, in all phases of the data science project lifecycle.
The generalist might not command a higher salary, but has more flexibility career-wise. Even in a big company, when downsizing occurs, it is easier for the generalist to make a lateral move (get transferred to a different department), than it is for the “one-trick pony”.
Timing is important too. If you become a generalist at age 50 (as opposed to age 45) it might not help as getting hired becomes more difficult as you get past 45. Still, even if 50 or more, it opens up some possibilities, for instance starting your own business. And if you can prove that you have been consistently broadening your skills throughout your career cycle, as generalists do by definition, it will be easier to land a job, especially if your salary expectations are reasonable, and your health is not an issue for your future employer.
An example
I consider myself to be a generalist, and here, I explained how I became one, and the benefits that it provides both in small and large companies, and especially as an entrepreneur. It is true that there are many areas that I do not master, but as a generalist, you can hire the right employees to do what you are not so good at, or what you enjoy less.
I started my career in the academia, and I was expected to become a university professor. I failed an interview for such a position at Iowa State University around 1997, and realized soon after that there was too much competition for me to succeed in this endeavor. Twenty years later, I think this is the best thing that happened to me: not only do I continue to do research and publish (independently) but I don’t have the pressure to “publish or perish” and can write articles or books, even state-of-the-art, that are accessible to a much larger audience. I had to unlearn how to write esoteric scientific articles in the process. But I recognize that did gain a strong research training during these PhD and post-doc years.
Later on, I worked for a number of Internet companies, between 1998 and 2002, in the volatile bubble years of the Internet. No job was safe, and in the process I had to change jobs multiple times. That was the beginning of my corporate years, and having worked with many smart people in different places, if anything, was the beginning to start having a large network of influential people, and getting noticed. While market volatility scares many people, if you can handle the risks (I was young back then, which makes it easier) it can boost your career and experience as a generalist. I played in the stock market back then, sold my home in the Bay Area at the best time, worked for Visa and Wells Fargo for a while, which further diversified my experience. This would not have been very successful without trusting my gut instincts (what to do, when to sell or buy or where – activities that involve analyzing data but also intuition.)
At Wells Fargo, my biggest contribution was about identifying a big IT issue that made all the Internet logs completely wrong, even though I was hired as a BI analyst. I am still remembered at that bank for fixing this multi-million dollar issue. I could not have done that had I been a pure data scientist working exclusively on the mission I was hired for in the fist place. My previous experience in failed Internet companies is what helped me, as well as asking a salary a little below average to increase my chances of being hired.
At NBCi (at their peak they had 1,600 employees) I was the key guy who helped them NOT sign a multi-million deal with Netscape, based on past experience and analyzing the poor traffic coming from that source. Also, although hired as a BI analyst, I knew how to run scripts (Perl scripts at that time) to extract/summarize data from databases and deliver it to my boss automatically by email, in a way that quickly conveys all the important information he needed about weekly business metrics. Usually, BI people do not know how to code to that level. My boss complained one day that I did not work much (I had automated my job of course) but he still was happy. The time I saved doing the number crunching by automating it, I used it to start my own business.
Later in 2010, I trained BI people to write their own queries in Perl. I coded the low level part, all they had to do is put their SQL queries in a text file at the right place (on the right server) then enter a simple one-line UNIX command to run their query. It saved them a tremendous of time (they used Toad before, a very, very slow process to extract big data out of Oracle databases) and everyone was happy. Note: you need to be a very good friend with your IT team to have them approve this process. Again, despite me being perceived as someone not working a lot (that is, working smart rather than hard) everyone was happy (my boss — the CEO –, BI analysts, and myself.) I also mentioned some Google and Bing API’s available to download large list of keywords with pricing and volume information – a risky move for me as it could have killed my job consisting of building these lists myself manually, even though it requires smart web crawling and intelligent NLP to build these keyword lists (it did not killed my job.) Something even the software engineers (in a search company!) were not aware of. They ended up using my Google/Bing accounts (and even my credit card to pay for the services!) to optimize this process, though after initially using my own Perl code to access the API’s, my software engineer colleagues switched to Python.
I took advantage of this free time to further develop my business, and that was pretty much my last experience working for a company. Along those years, I learned enough to run my business, I was ready to become full-time entrepreneur working from home, which is what I did. My experience with new business models was more extensive than that of MBA graduates from top schools, and I was no longer afraid by competition. (that said, a good MBA degree is great for all the fantastic lifetime connections you will make during your college years.)
Even in personal life, being a generalist helps: buy or sell a home (or investing in general) with better outcomes than average, blending both your data science experience with intuition to maximize results. When we launched DSC, I knew more about tax, legal issues, advertising, marketing, sales, IT (automating processes), editorial process – especially regarding managing the specific business in question – than many who have a degree in those fields. It helped a lot saving costs, not having to hire tons of specialized experts. Along the way, I also learned how to significantly increase my negotiation skills, to the point of being able to purchase homes that few would qualify for (and reduce the price without sounding aggressive), the most recent one resulting in a 7-digit loan, even though these other home buyers had a stronger position than me in the first place (them being a US citizen, not born in poverty, on a payroll, working for some big company, sometimes very well paid, with an official job title – I have none of this, other than provable, decent, stable / increasing, income, as proved by many years of tax returns.)
These are some of the benefits of being a generalist.
To not miss this type of content in the future, subscribe to our newsletter. For related articles from the same author, click here or visit www.VincentGranville.com. Follow me on on LinkedIn, or visit my old web page here.
DSC Resources
- Book and Resources for DSC Members
- Comprehensive Repository of Data Science and ML Resources
- Advanced Machine Learning with Basic Excel
- Difference between ML, Data Science, AI, Deep Learning, and Statistics
- Selected Business Analytics, Data Science and ML articles
- Hire a Data Scientist | Search DSC | Find a Job
- Post a Blog | Forum Questions
