Calibration Curve (Linear Regression) Calculator
Paste reference values and instrument readings. The calculator fits a least squares straight line, shows the slope and offset (gain and zero error), R², residuals and the formula to correct readings.
- b = (nΣxy − ΣxΣy) ÷ (nΣx² − (Σx)²) = 1.0014
- a = (Σy − bΣx) ÷ n = 0.062
- Residual = reading − fitted value
Residuals
| Reference | Reading | Fitted | Residual | Error |
|---|---|---|---|---|
| 0 | 0.05 | 0.062 | −0.012 | 0.05 |
| 25 | 25.12 | 25.097 | 0.023 | 0.12 |
| 50 | 50.1 | 50.132 | −0.032 | 0.1 |
| 75 | 75.21 | 75.167 | 0.043 | 0.21 |
| 100 | 100.18 | 100.202 | −0.022 | 0.18 |
Least Squares Line
b = (nΣxy − ΣxΣy) ÷ [nΣx² − (Σx)²], a = (Σy − bΣx) ÷ n
R² = 1 − Σ(y − ŷ)² ÷ Σ(y − ȳ)²
x is the reference (standard) value and y the instrument reading. The slope b shows the gain error, the offset a the zero error, and the residuals the nonlinearity left after correcting gain and zero. The correction inverts the fitted line.
Worked example
For the five points shown, the fit is reading = 0.062 + 1.0014 × reference: a gain error of +0.14 % and a zero error of +0.062, with residuals within ±0.043.
Frequently Asked Questions
What does the slope tell me?
The gain (span) error. A slope of 1.002 means the instrument reads 0.2 % too much per unit of input.
What do residuals show?
How far each point is from the fitted line, which is the nonlinearity after zero and span are corrected.
Can I use this for sensor coefficients?
For linear sensors, yes. For RTDs and thermocouples use the Callendar–Van Dusen or polynomial fits instead.
Learn more on the blog
Instrumentation calibration and troubleshooting guide
A practical, field focused guide on instrumentationblog.in that explains the theory behind this calculator and how engineers apply it on real plants.
Read Full Article