How to measure anything.

By Douglas W. Hubbard

I need to re-read this at some point. It's a useful book. Much of it covers how to practically use slightly advanced statistics.

It is better to be approximately right than exactly wrong. When you want to measure something, ask what decisions the measurements will drive. When you answer that question, you will probably discover that you don't need as much accuracy as you think.

Measure in terms of ranges, say the 90% confidence interval, and not by specific numbers. Specifics hide the amount of error you have and give you a false sense of accuracy.

Try guessing a 90% confidence interval by asking yourself two questions. What number is the answer probably larger than, but possibly not? What number is the answer probably less then, but probably not?

If you have 5 samples, the 90% confidence interval is between the largest and smallest sample.

If you have 8 samples, the 90% confidence interval is between the 2nd largest and the 2nd smallest sample.

Beware of using weighted scores to make decisions. Your biases can compound instead of cancel out. For exmaple, four one star movies do not provide the same happieness as as one four star movie.