Does a display that reads 23.47 offer more precision than a display that reads 23.5? The addition of extra digits might make your instrument seem more accurate, but the additional digits are only helpful if the instrument reacts properly to the measurand.
For example, a digital thermometer capable of resolving 0.1 degree steps has better resolution than a thermometer that can only resolve 1 degree steps. A multimeter that resolves resistance to the nearest ohm has different resolution than a multimeter that resolves resistance to 0.1 ohm. Fine resolution allows you to measure small changes, but it doesn’t necessarily mean the measurement is accurate. It is possible to have great resolution on a device that has significant error.
To check the resolution of your instrument, set up a circuit with a resistor. Choose the appropriate resistance scale and press the probes firmly against the resistor. Take multiple measurements without moving the resistor or altering your connection. If the reading stays the same, your instrument has good repeatability.
Compare the readings from your instrument to the nominal resistance and tolerance of the resistor. A large deviation does not indicate that the instrument is inaccurate. It indicates that the resistor differs from its nominal value by more than its stated tolerance. You cannot assume that the resistor is wrong and the instrument is right just because there is a large deviation between the two values.
To measure the accuracy of your device, you must use a reference standard, which should be a known source, known condition, or known instrument. For example, if you place a temperature sensor in a known temperature condition and the sensor consistently reads 2 degrees higher than the temperature condition, the sensor is repeating well, but the instrument is not accurate.
If the readings jump up and down for no apparent reason, then the repeatability is bad. But before you blame the sensor, check the entire measurement chain. Poor repeatability could be caused by anything from loose connections, noise, moving the probes, poor power supply, or a changing ambient temperature. Always note your equipment settings and test conditions for each measurement so that repeat measurements are valid.
There are many ways that accuracy, repeatability, and resolution can coexist. It is possible to have a highly repeatable set of measurements that are all wrong. Another possibility is that the measurements are scattered but the average is near the correct value. It is also possible to have a highly resolved instrument whose measurements vary slightly due to noise. This means the least significant digit might vary, and the additional digits are meaningless. The appearance of increased precision can hide these differences and lead you to believe that your display is accurate.
When taking measurements, always keep in mind the following:
- What is the minimum increment of my device?
- How accurate is my instrument compared to a reference?
- How repeatable are my measurements?
By keeping these three measurements separate, you will be able to better understand datasheets, more easily diagnose problems, and have better results.