Why developers are becoming the AI implementation powerhouse
Learn how a developer can benefit from learning about data science and expanding the scope of his or her role.
I always thought application developers and software engineers would be great in implementing data science and AI. They can code, they know how to think logically, and they are driven to solve difficult problems. We also see skilled developers working in the marketplace, while data scientists and machine learning engineers are in short supply comparatively speaking. With automated AI lifecycle management tools and the proliferation of cloud AI environments, today’s developers are given an easier onramp to get data science into their skillset arsenal.
Upkar Lidder has transitioned from a full stack developer to a developer advocate who is focused on data science and AI. We had a conversation on how a developer can benefit from learning about data science and expanding the scope of his or her role.
Tell me more about your background. How did you get into data science? I joined IBM in 2014 as a developer, and I’ve been a software developer for over 15 years. I worked as a software engineer for an IBM product division, and I worked in support as well. Then, I moved to IBM Global Business Services (GBS) to develop software for various clients including banking and education clients.
As part of the GBS assignment, I worked on a project for a university where I applied Watson Education. We built apps to help students succeed by predicting what kind of help each individual needs and providing personalized recommendations. Based on what the students did or their aptitude, using Watson Discovery, we were sending content such as articles and books that were personalized to them to retain them and ultimately help them succeed. There was a data science team within this project at GBS, and I ended up not doing application development, but instead moved to data engineering. My job was to make sure that all of the data was flowing and feeding the models that the data science team was building.
I saw firsthand the impact that data scientists can have on the society. Rather than me staying outside looking in, I wanted to partake in making the society better — in this case, helping one student at a time.
Was that the first time you learned about data science? Actually, no. I took a data mining class in college. I thought it was interesting, but I chose software engineering because software engineering jobs were more available at the time. I really liked math, statistics, and algebra. We did not even call it data science, actually. We focused on patterns and heuristics. Data science and AI were academic when I started 12 years ago, and were very research-centric.
I started my career at IBM writing code, structuring problems, and building software and solutions. In 2011, I got my MBA while working at IBM. Then, I had various jobs. In 2018, I became a developer advocate with the primary goal of developer education. Now, I specialize in data and AI for developers.
From a developer perspective, what interests you most now? Deep learning is the first thing that comes to mind. Deep neural networks are now being used to solve real-world problems. Another area is bias, or AI fairness. Even spreading the awareness of the risk is a good thing. AutoAI can create the model pipelines with the click of a button, but is it fair? It may be highly accurate at that point in time, but we need to think through the impact that it has on people’s lives or how the accuracy will change over time.
No-code or low-code development tools give you a great starting point. You may have to tweak and customize them to extend them, but it’s a good way to get started. Defaults may become better over time so that you don’t have to customize too much in the future.
What challenges have you faced in managing the continuous integration and continuous deliver (CICD) pipelines? The biggest challenges have been when pipelines were not fully operationalized and automated. You always end up trying to put out fires or figure out how not to damage existing models. And we still need to get the new projects complete without impacting it. How do you get new work done without messing up the existing work, such as the rolling update of Kubernetes? How do we do A/B testing? I still remember when clients would upload the file at midnight and when it reached the automation/process, one Excel file might have a bad character, and the whole process was shut down. It could impact everyone – the data science team and the app team. One simple mistake could stop the process. This is why we need to think through the entire lifecycle from the app, data, and model perspectives. It’s important to integrate ModelOps with the DevOps lifecycle.
You are a software engineer writing code, and it gives a distinct advantage. What would you say to aspiring developers and software engineers with an interest in data science? Every developer should learn the basics. But, a little knowledge can be potentially dangerous. So you cannot just implement any code. There is a saying: all models are bad, but some are useful. AutoAI can do a lot, but you need intuition and experience to make it all work while using this kind of automation tool.
If you are in a start-up, it makes sense to cross train to expand your areas of responsibility. If you are working for a larger company, many of the problems are already taken care of by others. I feel that sometimes it makes more sense for me to use my curiosity to build models as opposed to natural language processing and image recognition where you just use the models through transfer learning. Data science has so many trials and errors. You’ll try and fail many times. That’s how you become an experienced data scientist and build your intuition.
Developers are becoming a major force for AI implementation. Do you have some final advice for a developer who is interested in data science? When you are building a website or writing an app, you are trying to accomplish something. If you are always curious about how users interact with the data, for example, what happened, what are they trying to accomplish, and what might happen, you have a great fit for putting data science into your skills arsenal. Let your curiosity lead you. But know that with great power, there comes a great responsibility as well.