Sign In

Communications of the ACM

ACM Careers

Companies Have Been Building Their Data Teams Backwards

View as: Print Mobile App Share:
data engineer working at a computer

Structuring a data team gives specific roles for each stage of the data process.

Businesses in the U.K. have been building their data teams "backwards" — hiring data scientists before hiring the data engineers who construct and maintain the systems that give data scientists something to analyze.

The work of a data team is usually quite structured: first, data engineers build a data pipeline, then data analysts perform relatively straightforward analysis while data scientists conduct advanced analysis.

U.K. businesses, however, appear to have hired data scientists first, and only started to hire large volumes of data engineers in the last two years.

Between 2012 and 2018, businesses went on a data scientist hiring spree. These new data scientists, however, quickly found themselves confronted with data pipelines that needed work and data quality that needed fixing.

Now the demand for data engineers has outstripped demand for data scientists. The demand also outpaces the supply of talent, which presents a problem.

From Computing
View Full Article


No entries found