The State of AI 2019: Divergence

Chapter 6: The war for talent

While demand for AI professionals exceeds supply, winners and losers are emerging in the war for talent.

Summary

  • Demand for AI talent has doubled in 24 months. There is a gulf between demand and supply, with two roles available for every AI professional.
  • The pool of AI talent remains small. AI demands advanced competencies in mathematics, statistics and programming; AI developers are seven times more likely to have a Doctoral degree than other developers.
  • Supply is increasing – machine learning has become the top emerging field of employment in the United States. Greater supply is being driven by: high pay; the inclusion of AI modules in university computer science courses; companies’ investment in staff training; and AI technology companies ‘pump priming’ the market with free educational resources.
  • Over time, AI tools offering greater abstraction will make AI accessible to less specialised developers.
  • Talent shortages are sustaining high salaries. AI professionals are among the best paid developers and their salaries continue to increase; half enjoyed salary growth of 20% or more in the last three years.
  • Winners and losers are emerging in the war for talent. The technology and financial services sectors are absorbing 60% of AI talent.
  • The ‘brain drain’ from academia to industry is real and will have mixed implications, catalysing AI’s immediate impact while inhibiting teaching and moving value from the public domain to private companies.
  • High job satisfaction is intensifying the war for talent. Three quarters of AI professionals are satisfied in their current role.
  • To optimise hiring and retention, companies should align roles to AI professionals’ primary motivators – learning opportunities, office environment and access to preferred technologies.
  • New practitioners in the field are following sub-optimal paths to employment. Company websites and technology job boards are less effective than engaging with recruiters, friends, family and colleagues, according to those already employed in the field.

Recommendations

Executives

  • To attract AI talent, leverage your advantages as a large company. Offer access to vast data sets, the opportunity for impact at scale and high salaries.
  • Develop best-in-class training to up-skill existing developers.
  • Diversity delivers economic value and competitive advantage. Review the culture in your company, AI team and hiring practices to ensure diversity, representation and inclusion.
  • Collaborate with universities to support your search for talent, strengthen your reputation as a supporter of AI innovation and train colleagues through engagement with university research programmes.

Entrepreneurs

  • Engage with universities, meet-ups and conferences to identify and attract promising candidates before they enter the market.
  • Exploit your advantages as a start-up to attract AI talent. Offer work that can ‘make a difference’, direct impact on product, opportunities for learning, access to preferred technologies and an appealing office environment – in addition to equity.
  • Follow best practices in our AI Playbook (www.mmcventures.com/research) to optimise each stage of your recruitment funnel.
  • Given demand for AI talent, maintain a focus on team satisfaction and retention.

Investors

  • Amidst a competitive market for talent, assess potential investees’ ability to attract and retain high quality AI personnel.
  • Develop a competency in the recruitment of AI talent, including engagement with specialist recruiters, to assist portfolio companies.
  • Understand best practices for every stage of a company’s AI recruitment funnel – and ensure their proliferation across your portfolio.

Policy-makers

  • Develop programs and funding to support education in science, technology, engineering and mathematics (STEM) subjects. Investment in STEM will mitigate talent shortages and empower workers for the age of AI.
  • Enable the next generation of AI academics and mitigate the ‘brain drain’ to industry by providing greater, more accessible grant funding and access to national data sets for the public good.

Explore our AI Playbook, a blueprint for developing and deploying AI, at www.mmcventures.com/research.

AI talent is in high demand

As AI is woven into the fabric of consumer experiences, and corporate adoption of AI extends from early adopters to the early mainstream, demand for developers who can create AI solutions has surged. In the United Kingdom, job listings for AI roles have increased 485% since 2014 (Indeed). A quarter of
companies highlight that lack of available AI talent is a primary inhibitor in their efforts to adopt AI (Gartner).

Growth in demand is accelerating. In the United States, year-on-year growth in AI-related job postings increased from 20% (2016) to 32% (2017) (Indeed). In the last 24 months, AI-related job postings as a proportion of total postings nearly doubled (Fig. 56).

“In the last 24 months, AI-related job postings as a proportion of total postings nearly doubled.”

Fig 56. AI-related job postings as a proportion of the total have doubled in 18 months

Source: Indeed

Supply is increasing…

In the United States, machine learning has become the top emerging field of employment, with ten times the number of individuals listing it as their profession today compared with five years ago (LinkedIn) (Fig. 57). Data science, more broadly, is the second-from-top emerging field of employment, with more than six-fold growth.

Fig 57. In the US, machine learning is the top emerging job

Source: LinkedIn

Supply is increasing as:

  • developers recognise opportunity for challenge and high pay within the field;
  • universities update computer science courses to include AI components and resources. Undergraduate Computer Science courses at the universities of Cambridge, Harvard, MIT, Oxford, Princeton and Stanford all include AI components. In addition, many universities offer free online AI resources, including Stanford’s and Columbia’s ‘Machine Learning’ courses and MIT’s ‘Deep Learning for Self-Driving Cars’ (Fig. 58).
  • large companies invest in training initiatives for staff. Three quarters of large companies are offering some form of inhouse or external training program, with a third providing formal training (Gartner).
  • AI-focused technology companies ‘pump prime’ the market with free educational resources (Fig. 59), including Google’s ‘Machine Learning’ course and NVIDIA’s ‘Fundamentals of Deep Learning for Computer Vision’ resource.
Fig 58. Many universities offer free online AI courses and resources

Source: MIT

Fig 59. Technology companies are offering free educational AI resources

Source: Google, Udacity

…but the pool of AI talent is small

Estimates of the number of global AI developers vary widely, in part depending upon definition. There may be as few as 22,000 highly-trained AI specialists (Element) and up to 300,000 AI researchers and practitioners within broader technical teams (Tencent). AI originated in academia. The advanced mathematics, statistics and computer science required to understand and apply AI required extensive education, limiting the size of the available talent pool. AI developers are highly educated; nearly 60% have a Master’s or Doctoral degree (Fig. 60). AI developers are twice as likely to have a Master’s degree and seven times more likely to have a Doctoral degree than other professional developers (Fig. 61). Two thirds of data scientists believe their university education has been important or very important for their career success (Kaggle).

Fig 60. 60% of AI developers have a Master’s or Doctoral degree

Source: Kaggle

Fig 61. AI developers are seven times more likely to have a Doctoral degree than others

Source: Kaggle, Stackoverflow

In addition to technical skills, increasingly AI practitioners must have:

  • domain knowledge, to interpret data appropriately and provide relevant recommendations;
  • engineering experience, to develop solutions that work in the real world as well as the laboratory;
  • commercial experience, to develop and manage AI teams.

The combination of technical, sector-specific, engineering and commercial competencies required from AI professionals continues to limit the size of the talent pool.

Education and the democratisation of AI will mitigate talent shortages

Over time, a larger talent pool and more accessible AI tools will alleviate much of the shortfall in AI talent – and enable greater realisation of AI’s benefits.

Governments’ investment in education – in science, technology, engineering and mathematics (STEM) subjects – will be vital for countries to broaden their pools of AI talent. The proliferation of AI courses and resources from universities and technology companies, and market demand, will also boost supply.

However, AI will also become accessible to less specialised developers over time. Development environments for new technologies tend towards higher levels of abstraction over time (few developers program in assembly language today). AI will follow this pattern.

Prior to 2000, developing AI required advanced mathematics, sophisticated programming and the specification of algorithms by hand. Successive developments have reduced the burden on developers:

  • Numpy (2005) abstracted portions of required mathematics.
  • Compute Unified Device Architecture (CUDA, 2007) reduced the requirement to code by hand.
  • Python libraries (2010) and TensorFlow (2015) progressively abstracted network development.

Today, Google, Amazon and Microsoft offer AI services that require no implementation knowledge of AI. Developers with limited coding experience can upload data and solve simpler classification problems. While there will remain a large core of highly educated AI developers to progress research,
advanced– and domain-specific AI, we expect the technology to become accessible to a greater proportion of developers over time, expanding the pool of developers who can deploy it.

There is a gulf between demand for AI talent and supply

While supply of AI talent is increasing, demand significantly outstrips supply and will continue to do so in the medium term. There are 2.3 roles available for every suitable candidate (Indeed). “There is a mountain of demand and a trickle of supply” (Chris Nicholson, CEO, Skymind). AI professionals themselves cite lack of available talent as their second-greatest challenge (Fig. 62).

Fig 62. AI professionals cite lack of available talent as their second-greatest challenge

Source: Kaggle

Talent shortages are sustaining high salaries

A shortage of AI developers is driving high salaries in the market. Data scientists and machine learning specialists are among the best paid professional developers (Fig. 63). At the 20 highest-paying companies, salaries for AI engineers average $224,000 (Fig. 64). Leaders in the field command vast sums.

Fig 63. AI professionals are among the highest paid developers

Source: StackOverflow

Fig 64. Salaries for AI engineers average $224,000 at the 20 highest-paying companies

Source: Paysa

AI developers’ salaries are particularly high relative to their level of professional experience. Nearly half of data scientists have under two years of professional experience (Kaggle); nearly three quarters have less than ten. Compared with other developers, data scientists enjoy among the greatest salary premium relative to their level of experience (Fig. 65).

Fig 65. AI professionals are paid highly relative to their level of experience

Source: Stackoverflow

AI salaries continue to increase

Salaries for AI professionals have grown significantly in recent years and continue to increase. Almost all data scientists report increased pay in the last three years; nearly half grew their salaries by 20% or more (Fig. 66).

In the last 12 months, salaries have continued to increase (Fig. 67). This year’s pay dynamic has been more favourable to AI professionals than to many other developers. However, AI professionals are not the only group to enjoy significant year-on-year pay rises. Developers specialising in system administration, embedded applications and enterprise applications all received similar increases. DevOps specialists, who integrate and automate development and operations functions for faster cycles of improvement, are enjoying the greatest average raises.

“Salaries for AI professionals have grown significantly in recent years and continue to increase.”

Fig 66. Most AI professionals' salaries have increased in the last three years

Source: Kaggle

Fig 67. AI professionals’ salaries have increased further in the last 12 months

Source: StackOverflow, MMC Ventures

Winners and losers are emerging in the war for AI talent

Despite AI’s potential to reshape sectors ranging from retail to healthcare, technology and financial services firms are absorbing nearly 60% of AI talent (Fig. 68) (Burtch Works). 44% of data scientists are employed in the Technology sector – more than in the healthcare, consulting, marketing, retail, academia and Government sectors combined. Financial services, with a 14% share of data scientists, is a distant second.

Within the Technology sector, the world’s largest technology companies – including Amazon, Apple, Facebook, Google, IBM and Microsoft – are consolidating much of the available talent. Amazon, Microsoft and Apple combined are estimated to be investing $620m in AI talent (Paysa).

44%

of data scientists are employed in the Technology sector – more than in the Healthcare, Consulting, Marketing, Retail, Academia and Government sectors combined.

Source: Burtch Works

Fig 68. Technology and financial services firms are absorbing nearly 60% of AI talent (distribution of data scientists by sector)

Source: The Burtch Works Study – May 2018. N=2,212

The technology and financial services sectors are emerging winners in the war for AI talent – and creating virtuous cycles to extend their leadership. In addition to absorbing the greatest share of data scientists today, technology and financial services companies are planning to increase their investment in AI by the greatest proportion in the next three years (McKinsey Global Institute). Technology and financial service companies are prioritising AI, committing resources and building network effects around people and data to establish and extend leadership in the field.

Conversely, select sectors – including retail and consulting – are lagging, both in their ability to attract AI talent today and in their investment for the future. While many sectors, including retail and consulting, offer numerous prediction and optimisation problems well suited to AI, and large data sets to train AI algorithms effectively, the emerging gulf between winners and losers in the war for AI talent is likely to widen.

The ‘brain drain’ from academia is real

The perceived ‘brain drain’ from academia to industry is real – and will have mixed implications. While alternative surveys suggest that up to 15% of data scientists currently work in academia (Kaggle), many are leaving for roles in global technology companies. A three– to five–fold increase in salary, vast data sets for analysis and access to greater hardware resources attract many. Between 2006 and 2014, the proportion of AI research publications including an author with corporate affiliation increased from approximately 2% to nearly 40% (The Economist). Talent has continued to migrate to industry. In the UK, in the last 18 months several leading AI researchers have moved to industry to accept senior roles at Uber, Amazon and Google.

In industry, AI experts are freed from the burden of securing research grants, may innovate faster, and can catalyse AI’s immediate impact on the world. However, their migration has drawbacks – including fewer teachers to train the next generation of practitioners, a concentration of expertise and experience in a small number of companies, and reduced sharing of ideas. National talent working for the public good is becoming overseas resource for private gain – with international implications. The field of AI itself arose from academic experimentation. If we lose the next generation of academics, “in the end, society will suffer” (Maja Pantic, Professor of Affective and Behavioural Computing, Imperial College London).

High job satisfaction is intensifying the war for talent

Competition for AI talent is fierce, not simply because supply is limited. Three quarters of AI developers are content with their current roles, rating their job satisfaction 6 out of 10 or better (Fig. 69).

To optimise hiring and retention, companies should align roles to AI professionals’ primary motivators. To developers, opportunities for learning and professional development, the office environment in which they will be working, and the technologies (languages and frameworks) they will be using are more important than money (Fig. 70).

Fig 69. Three quarters of AI developers are satisfied (6 out of 10 or better) with their current roles

Source: Kaggle

Fig 70. Learning, office environment and the technologies they will use are AI developers’ primary motivators

Source: Kaggle

Large companies seeking to attract AI talent should: take advantage of their ability to pay high salaries and offer job security; highlight the large data sets they have for analysis and the learning opportunities these will provide; emphasise the impact AI developers will have given the companies’ large customer bases; and offer their AI professionals extensive hardware and software resources. Large companies should seek to mitigate likely concerns regarding agility, autonomy and freedom to publish.

Startups and scale-ups cannot, and need not, compete with the pay offered by large companies. Startups should market to candidates: the intellectual and technical challenges they can provide and associated learning opportunities; an engaging office environment; impressive job titles; a greater opportunity to impact product; increased autonomy; faster cycles of innovation; and greater freedom to publish. Startups should address probable concerns regarding pay by highlighting the large, long-term financial rewards they can offer through equity awards.

“Start-ups and scale-ups cannot, and need not, compete with the pay offered by large companies.”

New practitioners are following sub-optimal paths to employment

The pathways into AI employment – company websites and technology job boards – prioritised by those entering the field are among the least effective (Fig. 71). People successfully employed in AI highlight that engagement with recruiters, friends, family and colleagues is the most fruitful route into the industry.

Fig 71. The most effective route into AI work is engagement with recruiters

Source: Kaggle