It’s easy to think that if you just knew the statistics better, data analysis wouldn’t be as difficult.
It is true that more statistical knowledge is always useful. But I found that statistical knowledge is only part of the problem.
Another key part is developing data analysis skills. These skills are applicable to all types of analysis. It doesn’t matter what statistical method or software you use. So even if you never need more statistical sophisticated analysis than a t-test, developing these skills will make your job easier.
Statistical knowledge is necessary even to start developing these skills. And as you develop skills, you will find that the statistics become more understandable. But more statistical knowledge will not replace. So what are these important skills?
1. Planning for data analysis
Like most projects, data analysis projects are more efficient and have fewer problems if you have a plan. The plan requires you to think through important decisions ahead of time that will take a long time to repeat.
Proper statistical analysis depends on the research question, research design, variables, and any problems with the data. Considering how they work together before you start can save you a lot of time and headaches in your analysis.
2. Data analysis project management
Even if you are the only one involved in data analysis, you still have to manage the project itself. You need to keep track of files, allocate enough time for each step, and find the resources you need.
3. Cleaning, encoding, formatting and structuring data.
Have you heard the term GIGO? Garbage in, garbage out. When analyzing data, the data itself is important. And they must be clean.
And once they are clean, you will need to encode and format the variables. You then need to properly structure them for planned data analysis.
This step often takes much longer and requires more skills in statistical software than the data analysis itself.
4. Carrying out analyzes promptly.
There is a specific order in which the stages of analysis are performed, and decisions need to be made at each stage.
If you do them out of order or encounter an obstacle you can’t solve, the analysis will be slower and more tedious. More importantly, you may be wrong.
5. Checking Assumptions and Eliminating Violations
Yes, each statistical test and model has its own assumptions. Thus, the content of this skill will depend on what you are doing. But the general approach to testing assumptions is the same. And you also need a lot of ability to read uncertain situations and draw conclusions.
6. Recognizing and resolving data problems
Real data is messy data.
Real data has problems that make analysis difficult. Outliers, small sample sizes, and truncated distributions occur in all types of datasets. Recognizing when a data problem occurs, knowing if it’s serious enough to cause problems, and knowing what to do about it are important skills.
7. Finding problematic results and troubleshooting
Sometimes you get weird results – really weird results – despite cleaning the data, checking assumptions, and looking for problems with the data.
There are a lot of possible reasons: typos in the data set, software failures, missing data.
It requires the ability to recognize when something is wrong and how to investigate the problem and its solutions.
8. Interpretation, presentation and communication of results.
This set of skills may be the most important of all. This includes interpreting the results and recording them in a way that the audience understands. This requires the creation of useful, appropriate and accurate graphs and tables.
It also means knowing how to make your statistical program do the hard work for you. If you can create tables the way you want, you won’t have to spend hours reformatting them.
How to develop these skills
No data analyst has these skills at the start, no matter how many statistics classes they have taken. There is only one way to develop these skills: experience in analyzing real data sets is required.
But developing skills will be easier if you have specific training in those skills and someone to guide you as you gain experience. Consider yourself a student of a skilled craft. Yes, you can do it yourself. But you will improve your skills faster and get better at your job along the way with some guidance.
