BioDeviceHub

Calibration Error Calculator

Compare an instrument's readings with a reference at several points, showing the error as a percentage of reading and of full scale and whether each is within tolerance.

Formula

error=reading−reference\text{error} = \text{reading} - \text{reference}
% of reading=errorreference×100\%\ \text{of reading} = \dfrac{\text{error}}{\text{reference}}\times 100
% of full scale=errorfull scale×100\%\ \text{of full scale} = \dfrac{\text{error}}{\text{full scale}}\times 100
limit=tolerance %×full scale, or×reading, or an absolute value\text{limit} = \text{tolerance}\,\%\times\text{full scale},\ \text{or}\times\text{reading, or an absolute value}
reference\text{reference}
the true value from a traceable standard
reading\text{reading}
what the instrument under test shows
full scale\text{full scale}
the instrument's full-scale span

How it works

A calibration check compares the instrument with a reference at several points across its range. The error at each point is the reading minus the reference, and it is expressed against the reading or the full-scale span, which are different things: an error that is small relative to full scale can be large relative to a small reading.

Manufacturers state tolerances in one of those ways, and the choice matters at the bottom of the range. The table shows each point with its limit and whether it passes; the largest error and the count within tolerance are summarised above it.

Worked example

Five points from 0 to 200, with a tolerance of 0.5% of full scale (full scale 200).

  1. Limit = 0.5% × 200 = 1.0 at every point.
  2. The errors are +0.1, +0.4, +0.6, −0.8 and −1.1.

Four of five points pass. The reading at 200 is out by −1.1 (−0.55% of full scale) and fails.

These are the values the calculator opens with, so you can check its output against this example.

Assumptions

  • The reference is much more accurate than the instrument, with a test uncertainty ratio of at least 4 : 1 and traceable to a national standard.
  • Conditions are as the specification requires.

Common mistakes

  • Testing only at one point.
  • Applying a percentage-of-reading limit at zero, where it gives a limit of zero.
  • Using a reference whose own uncertainty is not small against the tolerance.

Related tools

Sensor Sensitivity CalculatorSlope, offset and non-linearity from calibration pointsOpen calculatorAccuracy and Precision CalculatorBias, standard deviation, %RSD and total error from repeatsOpen calculatorPercent Error CalculatorPercent error of a measured value against an accepted oneOpen calculator

Related equipment

Service documentation, failure modes and parts for the instruments this calculation is used with.