Hiring Data and AI Engineers at Scale
I interview data and AI engineering candidates regularly in a large healthcare organization. Over time, I’ve found that the difference between strong and average candidates is not usually a single technical skill. It is a combination of fundamentals, character, systems thinking, and how they approach real-world scenarios.
​
We operate in a Microsoft-heavy environment, so tools like Azure, Fabric, and related data platforms are part of the landscape. But tools are not what determine success. The real signal comes from how engineers think, communicate, and operate inside complex systems.
​
When I evaluate candidates, I categorize skills into three columns: nice to have skills, fairly necessary skills, and non-negotiable traits. The importance of this categorization comes from the fact that hiring mistakes almost never come down to a single specific skill; they are usually caused by gaps in judgment, communication, or engineering fundamentals.
​
While this list is not exhaustive or universal, my general criteria are outlined below.
​
Nice to have skills: useful acceleration, not core requirements
​
There are technologies that help engineers move faster in our environment, but I often treat them as accelerators rather than prerequisites.
​
Experience with Microsoft Fabric or Microsoft Foundry can be helpful because they reduce onboarding time in our data ecosystem. Familiarity with Delta-style architectures and PySpark are also useful because they often show up in large-scale data processing workflows. Azure experience is naturally beneficial in a Microsoft-based infrastructure, especially when working with data services, pipelines, and deployment patterns.
​
On the AI side, understanding model selection and basic tradeoffs between approaches is valuable, particularly when engineers are contributing to applied machine learning systems rather than purely data engineering work.
​
However, I have never made a hiring decision based on these skills alone. These are learnable (provided the candidate is teachable, but more on that later) for a strong engineer. I have seen candidates without any of these tools become highly effective within months when they have the right fundamentals. The important thing is that they have familiarity with the main pillars, described below.
​
Fairly necessary skills: the foundation of real engineering work
​
This is where hiring criteria starts to become serious. These skills are not optional in a meaningful engineering role, especially in a healthcare environment where reliability and correctness matter.
​
SQL is essential because most real-world data work still depends on understanding, shaping, and validating structured data. Python is equally important because it is the primary language for automation, data processing, and system integration in our stack. I've seen engineers with other object-oriented programming skills succeed, but I always like to see some Python experience on a CV.
​
Automation is a core expectation because manual processes do not scale in healthcare systems. Engineers must be able to design repeatable, reliable workflows rather than one-off analyses.
​
Version control and SDLC understanding are critical; I do not expect my team to build isolated scripts, but to contribute to systems that need to be maintained, audited, and improved over time. If a candidate does not understand how code moves from development to production, that represents a significant gap.
​
Risk management matters more than many candidates expect. In healthcare, data systems influence operational and clinical decisions. Engineers need to think about what happens when data is missing, incorrect, delayed, or misunderstood.
​
API integration is also important because modern data systems rarely exist in isolation. Engineers need to understand how systems communicate and what failure modes look like when those integrations break.
​
Finally, overall systems thinking is a major differentiator. Strong candidates should be able to reason through how data flows through multiple layers, how decisions are made downstream, and how one change can have unexpected effects elsewhere in the organization.
​
Non-negotiable traits: what actually determines success
​
The most important part of hiring is not always technical, but behavioral and cognitive. Many skills can be taught, but these traits are typically personal come down to the values an individual shows up with on their first day at work.
​
Teachability is the first non-negotiable trait. The technology landscape changes constantly. If someone is not able to learn quickly, accept feedback, and adapt, they will plateau regardless of their current skill level.
​
Communication is equally important. Engineers in our environment do not work in isolation. They collaborate with analysts, clinicians, product teams, and leadership. If someone cannot clearly explain what they are doing and why, their technical ability becomes less useful.
​
Problem solving and critical thinking are core requirements as well. Candidates must be able to work through ambiguity, break down unfamiliar problems, and make reasonable decisions with incomplete information. In healthcare data, perfect information is rare.
​
Professionalism and self-motivation are absolutely critical. Much of our engineering work involves long-running problems, incremental improvements, and ownership over systems that do not always have clear boundaries.
​
While a specific tech stack is not always a deal-breaker, engineering knowledge is still essential at a baseline level. Candidates should understand how systems are built, tested, deployed, and maintained. This does not mean knowing every tool, but it does mean understanding engineering principles.
​
Enthusiasm is often underestimated. Engineers who care about the work tend to learn faster, engage more deeply with problems, and improve the systems around them.
​
Ethical awareness is particularly important in AI and healthcare. Data and AI systems can influence real human outcomes. Candidates need to show awareness of privacy, bias, and the consequences of incorrect or misleading outputs. Having the right values on display goes a long way in this line of work.
​
Time management and the ability to follow instructions are practical but critical. In large organizations, misalignment on priorities or failure to execute requirements correctly can create significant downstream issues.
​
Nail the Interview
​
After countless interviews, I’ve learned that strong hiring decisions are rarely about finding someone who already knows every tool in the stack. Rather, they are about identifying people who can grow into complexity, think clearly under constraints, and operate responsibly in systems where the cost of mistakes is real.
In a Microsoft-based data and AI environment, tools like Azure, Fabric, and PySpark help engineers move faster. But the real long-term success signal is whether someone can understand systems, communicate effectively, and consistently make sound engineering decisions.
​
The best engineers are not defined by what they already know. They are defined by how they show up. And on each new employee's first day, I take the team out to lunch. On that day, I want to be absolutely confident when I say, "This is our newest engineer. I really think you're going to like them and I'm thrilled to have them here with us today."