100% Employee Owned, Founded 1954

Search
Close this search box.

Metrology Terminology: Accuracy, Precision, Resolution

Contact Today   

In metrology, using the correct terminology can be critical in getting the measurement information required. Three terms that are often incorrectly used interchangeably are accuracy, precision, and resolution.

When speaking about the accuracy of a measurement, you are referring to the data’s correctness.  This means that an accurate instrument would provide measurements closest to the actual value or standard.  Precision, on the other hand, is the ability of a device to provide you with consecutive measurements that are close to one another.  An instrument can give extremely precise measurements that are not at all correct. Precision is directly related to the repeatability and reproducibility of your measurements.

Accuracy

Accuracy

Precision

Precision

Accuracy & Precision

Accuracy/Precision

If an instrument were accurate but not precise, data collected using the device would hover close to the true value, but there will be disagreement between the individual readings.  Conversely, a device that is precise but not accurate will give you very repeatable measurements that are not necessarily correct.  Quality measurement equipment should give you readings that are both accurate and precise, meaning the measurement data is close to the true value and is highly repeatable and reproducible.

Another term often confused with accuracy and precision is resolution, which is the smallest change that can be measured with the instrument. Also referred to the least significant digit or LSD. In scales and balances this would be referred to as divisions or counts.  For example, the smallest unit change you can see in a floor scale might be a pound, while a balance could measure weight in milligrams. The floor scale has comparably low resolution to the balance, but for the application it is not necessary to know the weight of a pallet down to the milligram.

A common misconception is that a device with high resolution is very precise and accurate.  This isn’t necessarily true. Though resolution does directly relate to precision, it does not guarantee accuracy.  A balance could have very high resolution, while giving inaccurate or unreliable measurements.  Attempting to alter the settings of a scale to make it have a higher resolution than for which it has been NTEP approved can have a serious impact on its performance.  Referred to as stretched resolution, these scales are set up in such a way that the parameters are beyond those stated on their Certificate of Conformance (COC).  Because the scale is not intended to be used in this way, you cannot ensure that it will stay within calibration for any length of time, and users should know that altering scale in this way does not improve accuracy.

It’s critical when installing and calibrating scales, that each component of the weighing system is evaluated to make sure it will meet your accuracy, precision and resolution requirements. This is also generally true for any piece measurement equipment; ensuring that it meets the needs of your specific application is essential.  If you are unsure about what level of accuracy, precision, or resolution that you need, our team of measurement experts are here to help. Contact us today to find the right equipment for your application.

See how our precision measurement team can help improve quality, increase efficiency, and reduce risk.

Contact our Team

Hang Tight! We're Searching... Searching... Searching...

We’re looking through thousands of pages to find the most relevant information.

In the meantime, enjoy these fun facts…

Did you know… Cross Company is an ESOP (Employee Stock Ownership Plan). Our ESOP started in 1979 and as of 2006, we are 100% employee-owned! Learn more about our ESOP and how that benefits both team members and our customers.
Did you know... the precision measurement group at Cross was founded in 1939 by our current CEO's grandfather, Jim King. That's a whole lot of calibration!
Did you know... A fingerprint weighs about 50 micrograms. We know, we weighed it! The residue left from a finger can actually make a difference in weight results which is why we wear gloves when we calibrate weights. For reference, a sheet of paper is about 4.5 grams, that’s 4.5 million micrograms.
Did you know… Cross Company has grown significantly since our start in 1954. Over the years we've acquired 26 companies! Today, our five groups have expertise in everything from industrial automation to precision measurement, and industry knowledge going all the way back to 1939.