The Hard Parts of Being an AI Engineer | by Minh Le Duc | Jul, 2025

In fact, you’ll often be in charge of building a service where AI is just one piece of the puzzle. More often than not, you’ll find yourself dealing with data pipelines, deployment workflows, API integrations, or debugging someone else’s legacy code.

Building and maintaining a reliable system is just as important — if not more so — than building a cool model. You might spend hours figuring out why a GPU job fails silently or why the service times out under load, rather than tuning your Transformer.

Note: A new-hard service may be less pain-in-the-*ss than debugging legacy code.

Even though foundation models, LLMs, and diffusion networks are getting a lot of attention, the reality is that many practical problems don’t need anything that fancy.

In my case, I’ve still used XGBoost well after BERT became popular — because it fit the business requirements better, especially with limited data and tight latency limits.

If you’re working on embedded AI, a simpler, more lightweight model that ensures accuracy and small size might be the smarter choice.

Sometimes, choosing a tried-and-true approach over the latest trend is the right call — and that’s completely valid.

Note: I used the word “use”, not “update”. The choice of updating latest knowledge in this field is up to you.

One of the hardest things for me was accepting that I can’t know it all. AI moves fast, and while you’re learning about one thing, ten new papers just came out.

Your product manager expects you to explain LLM pricing models, while your teammate asks about ONNX optimization — and oh, by the way, can you also build that dashboard for the weekly report?

Being an AI engineer can feel like being stretched thin between research, engineering, product, and ops. It’s humbling.

The model is just a means to an end. If it doesn’t get shipped, monitored, and improved over time, it’s just another notebook collecting dust.

In many cases, your biggest value comes from translating business problems into data problems — and then translating your solution back into business language.

The real work starts after training.

Finally, you will realise that, maybe just some rule-based + some API to “prettify” the end result is enough. You can deliver something, and your customers get those things.