Free engineering calculators and unit convertersceo@instrumentationblog.in
Calibration · Glossary

What is Precision?

spread of repeated readingsCalibration

Precision is how closely repeated measurements of the same quantity agree with each other, regardless of the true value.

Advertisement

Precision Explained

A precise instrument gives tightly grouped readings; an accurate one gives readings centred on the true value. Calibration corrects bias but cannot remove poor precision.

Precision is described by repeatability and reproducibility.

Advertisement

Calculators for Precision

CalculatorRepeatability CalculatorFree online calculator with formula and worked example.CalculatorMeasurement Uncertainty CalculatorFree online calculator with formula and worked example.

Free Downloads

Free downloadMeasurement Uncertainty Budget Excel SheetFree download in Excel and PDF.

Frequently Asked Questions

What is Precision?

Precision is how closely repeated measurements of the same quantity agree with each other, regardless of the true value.

Can an instrument be precise but wrong?

Yes, with a constant offset; calibration fixes that.

Which matters more for control?

Repeatability matters most for control; accuracy matters for accounting and safety limits.

You May Also Like

Learn more on the blog

Instrumentation calibration and troubleshooting guide

A practical guide on instrumentationblog.in with more detail on precision and how it is applied on real plants.

Read Full Article

Related Terms

Calibration termRepeatabilityRepeatability is the closeness of successive readings of the same input under the same conditions, same…Calibration termReproducibilityReproducibility is the closeness of results for the same quantity when conditions change, such as…Calibration termAccuracyAccuracy is how close a measurement is to the true value; in specifications it is the maximum expected error.

More Calibration Terms

Calibration termHysteresisHysteresis is the difference between readings at the same input when approached from below (upscale) and…Calibration termLinearityLinearity is how closely an instrument output follows a straight line between input and output; non…Calibration termZero and Span ErrorZero and span error describe an instrument whose reading is offset at the bottom of the range (zero) or…Calibration termMeasurement UncertaintyMeasurement uncertainty is a quantified range around a result within which the true value is expected to…Calibration termType A and Type B UncertaintyType A and Type B uncertainty are the two ways of evaluating uncertainty: Type A from statistics of…Calibration termCoverage FactorCoverage factor k multiplies the combined standard uncertainty to give an expanded uncertainty at a…

All calibration terms · Full glossary A to Z

Advertisement