Once the data preparation phase is complete, its time to move on to the fun part of the CRISP-DM framework: modeling. Here you’ll chose which modeling technique to use, create some tests to assess the accuracy of your model, build the model, and then assess the model using the tests you created.
CRISP-DM in depth: data preparation
After developing business understanding and data understanding, the next big objective in the CRISP-DM methodology is to prepare the data for modelling and analysis. This involves selecting, cleaning and transforming the data which will be used for the project. While this isn’t flashy work, it typically accounts for 60% to 80% of the effort for a project.
CRISP-DM in depth: data understanding
Having developed business understanding and a deep knowledge of the problem you are trying to solve, the next step in the CRISP-DM framework is to develop that same level of understanding around the data itself. This step isn’t analysis, but rather looking at the structure and shape of the data in order to determine what information is available and how to go about building your analysis.
CRISP-DM in depth: business understanding
When using the CRISP-DM framework, the first step in the data mining process is to develop your business understanding. This stage of the process is about gaining knowledge of the business, the issues they face, opportunities for improvement, their objectives, their constraints and creating your project plan.
The Cross-Industry Standard Process for Data Mining
While analysis tools and algorithms have evolved at a rapid pace, the overall business process for analytics has remained remarkably stable. One seminal work on the analytic process is IBM’s Cross-Industry Standard Process for Data Mining (CRISP-DM). At over 20 years old, it remains a relevant and useful tool for describing the overall data science workflow.