How I test a lens 29 April 2018

The Sony RX10Mk4 has an excellent 25x zoom lens which compares favourably with primes at every focal length.

One of the  bugbears of modern photography is sample variation in lens quality.

I am aware from my own experience and from published reviews and reports that all types of camera lenses fixed and interchangeable, budget and premium, from all makers, are subject to bad copies and variable production quality.

I assume the reason for this might be that under pressure to get product out the door at a low price point the makers are unable to test each item before it is shipped. If they did test each lens the process itself would be expensive and the number of rejects would make the exercise even more expensive, thus increasing the price of each item.

Therefore lens buyers like you and me find ourselves involuntarily co-opted into  lens testing whether we like it or not.

The chart referred to in the text. With a good lens I can read every word on the chart at most focal lengths and the best apertures. Note: reproduction on this blog leads to substantial loss of detail from the original.

How can I tell  If the lens is of good quality ?

Look at it    Inspect for general condition including any dings, marks, scratches or defects.  In particular look into the lens using  bright collimated light.  Direct sunlight works fine but a torch with focussed beam is also good. Carefully clean the front element and rear element if accessible before doing this. This is a good way to see if there are any foreign bodies lurking inside there. Note that one or a few small dust spots inside a lens will usually have no detectable effect on optical characteristics and can be ignored.

Check that the lens mount if accessible is secure and not loose. Check the lens cap and hood.

Shake it  Lenses with a built in optical image stabiliser will usually rattle when shaken while powered off. The rattle should cease when the system is powered on.

Passive operation  Check that the zoom, focus and aperture rings turn smoothly without crunching noises, binding or excessive free play. Check that an interchangeable lens mounts easily on a compatible body without binding or excessive free play.

Active operation  Check that autofocus, manual focus and stabiliser functions are working properly. Check that all lens based controls are operating properly.

The cork board. With a good lens I can read all the fine print everywhere on the board including the edges.

  

Optical  I use two different chart style test subjects for indoor-close-to-the-camera testing and a specifically chosen scene for outdoor distant-from-the-camera testing.  After that I go forth and take lots of photos in a variety of conditions.

I have found that some lenses can be quite sharp close in but not at a distance from the subject so I test for both.

For each of the chart/landscape tests I have the camera on tripod, timer delay, low ISO, autofocus, stabiliser off, RAW + JPG finest available.

I set Aperture Priority autofocus and shoot at the widest aperture available at each focal length then stop down 1/3 for each subsequent exposure up to f5.6 or f8 depending on the sensor size of the camera in use.

Anybody can set up standard test subjects similar to these. Most do not and end up plaintively posting on user forums “should my lens be sharper than this…” and similar querulous enquiries.

This type of testing does not give me a “universal” result which can be expressed as line pairs per image height or something like that.

However having used these test subjects for some years I have a very good idea what to expect from a good lens with each subject.

The first test subject is 9 pages (each the same, via photocopy) of classified advertisements from a local newspaper pinned to a 120x80 cm piece of plywood with a backing border for rigidity.  This becomes a type of chart. I have learned from experience that a very good lens will allow me to read all the words right to the edges of the frame at most focal lengths and apertures. Any decentering is easily seen as unsharpness on one side of the frame.

The page chart also allows me to evaluate contrast, distortion, color fringing and other aberrations. It can tell me how the lens responds to stopping down from the widest aperture. Some need the aperture to be closed down one or two stops for best optical performance but others show  loss of sharpness when the aperture is closed down, even a little. I have encountered several lenses from Panasonic which exhibited this behaviour.

I can also use the chart to assess if shutter shock is present.

Almost any camera/lens combination can deliver decently sharp results in the center of the frame. If the frame centers are not sharp  I run a separate check for focus accuracy and consistency. DSLRs are more prone to inaccurate focus than other camera types. Some offer fine focus adjustment for each lens to correct for this.

Next I photograph the cork board on which are pinned a variety of bits and pieces with variable font size, variable contrast and lightness. A good lens will allow me to read the fine print at the edges of the frame at the optimum aperture and focal length. Higher quality lenses are better able to resolve low contrast detail.

This is the landscape scene . It has lots of fine foliage, architectural elements and industrial features. I can stand in one place and find plenty of detail to test focal lengths from 24-1200mm.

Then I go out and photograph the landscape scene. As this changes in appearance with different atmospheric and lighting conditions and different work underway at the marina it can only compare one camera/lens with another at the same time. However that can be very revealing and useful.

While I am outdoors and the sun is shining I take a variety of photos against the light with the sun in frame and varying amounts out of the frame. I note the type and degree of the flares which result.

Then I photograph backlit foliage to see if this will provoke chromatic aberration, purple fringing or some type of flare which it often does. Some lens/sensor/processor combinations are much worse than others in this regard.

I also check if the camera can focus reliably when presented with backlit foliage or other types of multiple bright lights. Some do very badly in this situation.

I look at  my general subject photos to evaluate the character of the out of focus parts of the image (bokeh). Some lenses are prone to double imaging and other undesirable characteristics in out of focus areas.  

After evaluating all the resulting photos on screen I have pretty good understanding of the capabilities of the tested lens(es). I know the lens strengths and weaknesses and  best and worst characteristics.

Comment   I have found excellent and execrable lenses at every price point in the range and from every maker. I have found zooms that are better than primes and vice versa. I have discovered cheap kit lenses that test better than high priced premium optics and vice versa.

I have learned the hard way (nobody gives me anything to test) that the only way to evaluate a lens is to test it myself.

If I get a bad copy and there have unfortunately been many over the years then all the published MTF and LPPIH charts and test information tell me absolutely nothing.

Recommendation   I strongly recommend that camera/lens buyers systematically test all their lenses. With practice it can be done in a day.

Some hints:

Pictures of brick walls do not provide the required information neither do pictures of a pet cat or random snaps of the back yard.

A systematic process is required and is easy enough to do.