Overview

Skill Level: Any Skill Level

Evaluate your project based on the business objectives that you put together.

Ingredients

This week’s recipe has no code. Instead, we are going to discuss how to approach the evaluation of your predictive maintenance effort. The technical or machine learning success criteria has already been evaluated during the modelling phase. The overall project evaluation needs to be done based on the business objectives that you put together. These results comprise the final model or models and any findings from the process.

 

Step-by-step

  1. Looking at the results

    During the evaluation phase, you put together your official assessment of whether the project met the business objectives. It’s important to include your stakeholders in this assessment. At the end, you’ll want to write up a summary of the models you experimented with that satisfy the business goals for the project. Some of the questions you will also want to answer include:

    • Are the results clearly presented for a general business audience?
    • Are there any interesting findings that you want to highlight? Did a lot of the data that was available turn out not to be of much use? Are there other types of data that should be collected?
    • Can you articulate which models performed best and how useful they were in terms to delivering the business goals
    • Do the model results deliver the business objectives?
    • Has this process raised other questions and how might the business understand and act on these questions?
  2. Looking at the process

    It’s important to review the process itself so that future predictive maintenance exercises can learn from and improve on previous projects.

    It helps to document each phase in the project and identify opportunities for improvement. Some of the questions you will want to answer for each phase include:

     

    • How valuable was the phase and how much did it contribute to the overall results?
    • Are there opportunities to improve the phase?
    • What mistakes were made during this phase? How can they be avoided in future exercises.
    • Did any activities take a lot time of time and prove ultimately fruitless? How can we identify these activities earlier to avoid wasted time?
    • Any big surprises crop up? How might these surprises be identified sooner?
    • Were there better ways of doing any of the particular phases that could be used in future projects? One thing I always like to do is to keep code snippets for functions that I might reuse in future projects.
  3. Next steps

    Basically, you have two options here:

    1. Iterate on the models. Another modelling iteration, based on what you learned from the previous iteration, might get you the results you need to deploy.
    2. Deploy. If the models are good enough, you can begin to integrate into your business process or system. If they are not good enough, prepare  a report for stakeholders. We’ll talk a bit more about that in our next recipe.

     

Join The Discussion