Hands Experience Cleaning Data. Hands-on experience cleaning data produces great wisdom, or so said John Sumser.
er. The process of cleaning data is not just about refining raw numbers, it’s about gaining valuable insights and learning from our imperfections.
I recall being part of a committee-led data collection effort several years ago. Our goal was to simplify the data collection process and minimize data cleaning at the end. We succeeded in creating tools that generated easy-to-collect data, but in the process, we lost sight of what really mattered – gaining real insights.
- Simplifying data collection can lead to overlooking important aspects of the data.
- Limiting the scope of questions can make the data confirm existing views, rather than revealing alternative perspectives.
Data isn’t like other natural resources that gain value with processing. While we tend to think that refining data is necessary, it’s actually the imperfections and inconsistencies that provide valuable information.
In fact, the process of cleaning data is where real discovery happens. We learn about the nature of our investigation, the imperfections of our questions, and the things that data sources want to tell us.
Hands-on experience cleaning data produces great wisdom. The material that we shed in the cleaning process creates the first layer of real insight. The problem is that tidy processes are inexpensive and efficient, but the things we want to learn may not be.
We need to shift our approach to data collection and focus on gaining real insights. This means embracing the imperfections and inconsistencies in our data, rather than trying to simplify and refine it to fit our preconceived notions.
So, the next time you’re faced with a data collection effort, remember John Sumser’s wise words: “In the process of cleaning the data, great discoveries are made about the nature of the investigation.”
Data Cleaning is just the beginning of the story, not the end.
As John Sumser said, “Hands-on experience cleaning the data produces great wisdom.” It’s time to start embracing the wisdom of messy data and real learning.
It’s time to shift our approach and focus on gaining real insights, rather than just collecting easy-to-summarize data.
Learn more about the power of data cleaning and the importance of gaining real insights.
