Yes, it’s very difficult to get everything perfect the first time! The more iterations of the process, the better the final pipeline will be. I think it’s important for both customers and machine learning beginners to understand. We usually only present the final notebook or model and don’t show all the previous iterations and methods we tried that didn’t work.

This can give a false impression that data scientists know what we are doing the whole time and complete everything in one go. A more realistic image would be showing all the notebooks with hundreds of cells where we’re trying to figure out how to solve the problem.