Instruments, Calibration & Measurement

Learn instrument and measurement fundamentals, including range, accuracy, precision, resolution, calibration, verification, zero, span, drift, standards, measurement error, and operator troubleshooting.

Reliable treatment decisions depend on reliable measurements. Operators use instruments to measure pH, dissolved oxygen, conductivity, turbidity, chlorine residual, temperature, pressure, flow, and many other process variables.

An instrument can display a precise-looking number and still be wrong. Operators should understand measurement range, accuracy, precision, calibration, verification, drift, and common sources of error.

What Is Measurement?

Measurement is the process of assigning a numerical value to a physical or chemical property using an appropriate method or instrument.

Examples include:

7.2 pH;
2.1 mg/L dissolved oxygen;
0.18 NTU turbidity;
1.5 mg/L chlorine residual;
62 psi pressure.

Measurement Units

A number without the correct unit can be misleading.

Common operator units include:

mg/L;
µg/L;
NTU;
psi;
gpm;
MGD;
°F;
°C;
mV;
µS/cm.

Instrument Range

Range is the interval of values an instrument is designed to measure.

For example, a pressure transmitter may have a range of:

0 to 100 psi

A sample or process condition outside the instrument range may produce unreliable or invalid results.

Choose an Appropriate Range

An instrument with a very wide range may not provide the best resolution for a narrow operating range.

For example, measuring 0 to 10 psi with a 0 to 1,000 psi instrument may provide less useful detail than using an instrument designed for the lower range.

Accuracy

Accuracy describes how close a measured value is to the true or accepted value.

If a reference standard is 100.0 units and the instrument reads 99.8, the measurement is close to the accepted value.

Precision

Precision describes how closely repeated measurements agree with each other.

An instrument can be precise without being accurate.

Example of Precision Without Accuracy

Suppose the true value is 10.0 mg/L.

Repeated readings are:

11.8 mg/L;
11.9 mg/L;
11.8 mg/L.

The results are precise because they agree closely, but they are inaccurate because they are consistently high.

Resolution

Resolution is the smallest change an instrument can display or distinguish.

For example, one instrument may display temperature to:

1°F;

while another displays:

0.1°F.

Greater displayed resolution does not automatically mean greater accuracy.

Sensitivity

Sensitivity describes how much instrument output changes when the measured condition changes.

A highly sensitive instrument produces a noticeable response to a relatively small change in the measured variable.

Repeatability

Repeatability describes how consistently an instrument produces similar results under the same conditions over a short period.

Measurement Bias

Bias is a systematic tendency for measurements to be consistently high or low compared with the accepted value.

Possible causes include:

incorrect calibration;
contaminated standard;
damaged sensor;
incorrect method;
instrument drift.

Random Error

Random error causes measurements to vary unpredictably around the expected value.

Possible causes include:

sample variability;
electrical noise;
analyst technique;
small environmental changes.

Systematic Error

Systematic error produces a consistent bias in one direction.

Examples include:

incorrect calibration factor;
wrong reagent concentration;
incorrect instrument scaling.

Calibration

Calibration establishes or adjusts the relationship between instrument response and known reference values.

Calibration helps ensure that displayed or reported values correspond correctly to the measured condition.

Calibration Standards

Calibration uses standards or references with known values.

Examples include:

pH buffer solutions;
conductivity standards;
turbidity standards;
known chlorine standards;
pressure references.

Single-Point Calibration

A single-point calibration uses one known reference value.

This may be appropriate for some instruments and checks, depending on the method and manufacturer instructions.

Two-Point Calibration

A two-point calibration uses two reference values and can establish both:

offset;
slope or span response.

Multi-Point Calibration

Some analytical instruments use several standards across the expected measurement range.

This helps define the relationship between:

instrument response;
known concentration.

Zero

Zero refers to the instrument response at the low end or baseline condition.

A zero error shifts the measurement away from the correct baseline.

Span

Span is the difference between the upper and lower values of a measurement range.

For a 0 to 100 psi transmitter:

Span = 100 - 0 = 100 psi

Zero Error

If an instrument reads 5 psi when the true pressure is 0 psi, it has a zero or offset error.

Span Error

If an instrument is correct near zero but increasingly inaccurate toward the upper end of the range, the span or slope may be incorrect.

Calibration Verification

Calibration verification checks whether an existing calibration is still acceptable.

Verification may use a known standard without changing the instrument.

Calibration Versus Verification

Calibration establishes or adjusts instrument response.

Verification checks performance against a known reference.

These terms should not be treated as identical.

As-Found Condition

As-found data show instrument performance before adjustment.

This information can reveal:

drift;
bias;
how far the instrument moved from acceptable performance.

As-Left Condition

As-left data show instrument performance after calibration, adjustment, or repair.

Why As-Found Data Matter

If an instrument is adjusted before the original reading is recorded, useful evidence about its previous condition is lost.

Instrument Drift

Drift is gradual change in instrument response over time.

Possible causes include:

sensor aging;
fouling;
electronic component changes;
temperature;
chemical exposure.

Calibration Frequency

Calibration frequency may depend on:

instrument type;
manufacturer recommendations;
method requirements;
regulatory requirements;
historical stability;
importance of the measurement.

More Frequent Calibration May Be Needed When

instrument drift increases;
sensor fouling is frequent;
measurement is critical;
process conditions are severe;
QC checks frequently fail.

Standards Must Be Reliable

Calibration is only as reliable as the standard used.

Check standards for:

expiration;
storage conditions;
contamination;
correct concentration;
traceability where required.

Do Not Return Used Standard to the Original Container

Returning used calibration solution can contaminate the remaining standard and affect future calibrations.

Temperature and Calibration

Temperature can affect:

instrument response;
chemical equilibrium;
standard value;
sample properties.

Calibration procedures should account for temperature where required.

Field Instruments

Common field instruments include:

portable pH meters;
DO meters;
conductivity meters;
chlorine analyzers;
turbidity meters;
temperature probes.

Laboratory Instruments

Laboratory instruments may include:

spectrophotometers;
balances;
incubators;
titration equipment;
bench-top meters.

Online Process Instruments

Online instruments continuously or frequently measure process conditions.

Examples include:

DO probes;
pH probes;
chlorine analyzers;
turbidimeters;
conductivity sensors.

Online and Laboratory Measurements May Differ

Possible reasons include:

different sample locations;
different measurement times;
sample changes during transport;
instrument error;
calibration differences.

Compare Measurements Correctly

When comparing instruments, confirm that both are measuring:

the same sample;
at approximately the same time;
using compatible units;
under appropriate conditions.

Reference Instruments

A calibrated portable or laboratory instrument can sometimes be used to check an online instrument.

The reference instrument should itself be known to be in acceptable condition.

Sensor Fouling

Sensor fouling can cause:

slow response;
low reading;
high reading;
unstable reading.

Possible fouling materials include:

biofilm;
scale;
solids;
grease;
chemical precipitates.

Cleaning Sensors

Cleaning should follow manufacturer and facility procedures.

Aggressive cleaning can damage:

electrodes;
membranes;
optical surfaces;
coatings.

Response Time

Response time is how quickly an instrument reacts to a change in the measured condition.

Slow response can be caused by:

fouling;
aging sensor;
plugged sample line;
slow sample flow;
instrument design.

Stable Reading

Some instruments require time to stabilize after:

being turned on;
being moved to a new sample;
calibration;
temperature change.

Do not record a result before the method or instrument indicates an acceptable stable condition.

Sample Temperature

Sample temperature can affect:

pH;
DO;
conductivity;
reaction rates;
sensor response.

Automatic Temperature Compensation

Some instruments apply automatic temperature compensation to sensor response.

This does not mean that the chemical properties of the sample become equivalent to another temperature.

Instrument Scaling

Transmitters and control systems must use the correct scaling relationship.

For example, a 4-20 mA signal may represent:

0 to 100 psi

4-20 mA Scaling Example

Suppose:

4 mA = 0 psi;
20 mA = 100 psi;
measured signal = 12 mA.

First determine the fraction of signal span:

(12 - 4) ÷ (20 - 4) = 8 ÷ 16 = 0.50

Then apply the measurement span:

0.50 × 100 psi = 50 psi

Therefore, 12 mA represents approximately 50 psi.

Incorrect Scaling

If the transmitter is configured for 0 to 100 psi but the PLC interprets the signal as 0 to 200 psi, the displayed result will be incorrect even if the transmitter itself is working properly.

Instrument Noise

Electrical or process noise can cause unstable readings.

Possible causes include:

poor grounding;
electrical interference;
air bubbles;
turbulent flow;
loose wiring;
process fluctuations.

Frozen Reading

A reading that remains exactly constant while the process changes may indicate:

failed sensor;
communication problem;
frozen software value;
plugged sample line.

Impossible Reading

A measurement outside the physically possible range should trigger immediate verification.

Possible causes include:

instrument failure;
wrong units;
scaling error;
data-entry error.

Out-of-Range Reading

If a sample exceeds the validated instrument or method range, it may require:

dilution;
different range;
another approved method.

Dilution and Measurement

If a sample is diluted before analysis, the final reported result must account for the dilution factor.

A simple relationship is:

Original Concentration = Measured Concentration × Dilution Factor

Dilution Example

A sample is diluted 1 part sample to a final total of 10 parts.

The instrument reads:

4.2 mg/L

Dilution factor:

Original concentration:

4.2 × 10 = 42 mg/L

Balances

Laboratory balances measure mass.

Reliable balance use requires:

stable surface;
clean weighing area;
appropriate calibration or verification;
protection from drafts where necessary.

Volumetric Measurement

Liquid volume may be measured with:

pipettes;
burettes;
volumetric flasks;
graduated cylinders.

The correct device depends on the accuracy required.

Read the Meniscus Correctly

For many aqueous measurements in transparent glassware, volume is read at the appropriate part of the liquid meniscus according to laboratory procedure.

Eye position should be level with the reading to reduce parallax error.

Parallax Error

Parallax error occurs when a scale is viewed from an incorrect angle.

This can affect:

graduated cylinders;
burettes;
analog gauges.

Titration Measurement

Titration accuracy depends on correct:

sample volume;
reagent strength;
burette reading;
endpoint detection.

Endpoint

The endpoint is the observed point at which the titration is considered complete according to the method.

Overshooting the endpoint can produce an incorrect result.

Colorimetric Measurement

Colorimetric methods determine concentration from the intensity of a developed color.

Performance can be affected by:

dirty sample cells;
scratched cells;
bubbles;
sample color;
turbidity;
incorrect reaction time.

Sample Cell Orientation

Some photometric instruments are sensitive to cell condition and orientation.

Follow the instrument procedure consistently.

Blanking an Instrument

A blank may be used to establish the baseline response of an analytical instrument before measuring samples.

Turbidity Measurement

Turbidity instruments measure light scattering caused by particles.

Measurement can be affected by:

bubbles;
dirty sample cells;
scratches;
settling;
instrument calibration.

Conductivity Measurement

Conductivity depends strongly on:

dissolved ions;
temperature.

Conductivity meters may report values normalized to a reference temperature depending on instrument settings.

pH Measurement

Reliable pH measurement requires:

clean electrode;
proper storage;
fresh calibration buffers;
adequate stabilization time.

Dissolved Oxygen Measurement

DO measurement can be affected by:

sensor fouling;
temperature;
pressure;
membrane or optical-cap condition;
calibration.

Chlorine Measurement

Chlorine residual can change rapidly after sampling.

Prompt measurement is important because:

chlorine continues reacting;
residual can decrease during storage.

Instrument Maintenance Records

Useful instrument records include:

instrument ID;
calibration date;
standard used;
as-found result;
as-left result;
maintenance performed;
person performing the work.

Traceability

Measurement traceability means the reported value can be connected to:

instrument;
calibration;
reference standard;
sample;
analyst;
original record.

Comparing Instrument Results

If two instruments disagree, check:

same sample;
same time;
same units;
calibration status;
temperature;
sensor condition;
method differences.

Do Not Assume the Newer Instrument Is Correct

Instrument age alone does not establish measurement accuracy.

The correct result should be determined using:

known standards;
calibration;
independent verification;
method requirements.

Measurement Uncertainty

No real measurement is perfectly exact.

Measurement uncertainty reflects the fact that small variation can come from:

instrument performance;
standards;
sampling;
analyst technique;
environmental conditions.

Do Not Report False Precision

If a method supports a result to the nearest 0.1 mg/L, reporting many additional decimal places does not make the measurement more accurate.

Field Check Versus Formal Calibration

An operator may perform a quick comparison with a portable instrument to identify a suspicious online reading.

This field check can be useful, but it does not necessarily replace the formal calibration procedure required for the instrument.

When a Measurement Looks Wrong

Check:

sample or process condition;
instrument range;
units;
calibration status;
sensor cleanliness;
temperature;
signal and scaling;
recent maintenance.

Use Independent Evidence

A questionable measurement should be compared with related process information.

Examples include:

flow meter versus tank-level change;
DO meter versus blower airflow;
pressure transmitter versus local gauge;
online pH versus portable pH meter;
chlorine analyzer versus grab test.

Common Instrument and Measurement Mistakes

Assuming more displayed decimal places mean greater accuracy.
Confusing accuracy with precision.
Using an instrument outside its measurement range.
Ignoring as-found calibration data.
Using expired or contaminated standards.
Ignoring sensor fouling or drift.
Recording a result before the instrument stabilizes.
Comparing measurements from different locations or times as if they were identical.
Forgetting to apply a dilution factor.
Using incorrect signal scaling.
Ignoring units.
Accepting an impossible reading without verification.

A Practical Instrument Check

Confirm the parameter and units.
Confirm the expected measurement range.
Inspect the sensor or instrument condition.
Review calibration date.
Verify with an appropriate standard or reference.
Review temperature effects where relevant.
Check signal and scaling for online instruments.
Compare with an independent measurement.
Document calibration or corrective action.
Confirm acceptable operation before returning the instrument to normal use.

A Practical Unexpected-Reading Review

Ask whether the value is physically reasonable.
Confirm the units.
Repeat the measurement when appropriate.
Check instrument range and calibration.
Inspect for fouling, bubbles, contamination, or damage.
Compare with historical trends.
Compare with related process data.
Use an independent instrument or method when practical.
Correct the instrument or process problem identified.

What to Remember for the Exam

Instrument range is the interval of values an instrument is designed to measure.
Accuracy is closeness to the true or accepted value.
Precision is agreement among repeated measurements.
Resolution is the smallest change an instrument can display or distinguish.
More displayed digits do not automatically mean greater accuracy.
Bias is a systematic tendency for results to be high or low.
Calibration establishes or adjusts the relationship between instrument response and known standards.
Calibration verification checks whether an existing calibration remains acceptable.
As-found data describe instrument condition before adjustment.
As-left data describe instrument condition after calibration or repair.
Instrument drift is gradual change in measurement response over time.
Calibration standards must be clean, valid, correctly stored, and appropriate for the method.
Sensor fouling can cause slow, unstable, high, or low readings.
Instrument response time should be considered before recording a result.
Incorrect 4-20 mA scaling can produce a wrong displayed value even when the transmitter is working correctly.
Measurements above the validated range may require dilution or another approved method.
When a sample is diluted, the dilution factor must be applied to the measured result.
Parallax error can occur when analog scales or volumetric glassware are viewed from the wrong angle.
Questionable measurements should be compared with standards, independent instruments, and related process data.
Good measurement records support calibration history, traceability, and reliable process decisions.