Join the conversation
Subcribe to comment rss feedThoughts on Judging Photography Competitions
Tim Parkin
Amateur Photographer who plays with big cameras and film when in between digital photographs.
I’ve written quite a lot about competitions in the past but being as we’re so close to the opening of submissions for the Natural Landscape Photography Awards, I wanted to go into a bit more detail about the process of judging, the potential pitfalls and how we’ve tried to mitigate some of them.
The Goals of the Competition
The first question any competition organisers should be asking themselves is “What is the goal of the competition?”. Sadly, there are many competitions that seem to exist as a marketing exercise and/or a way of making money for the organisers. We’ll try to ignore those and give our imaginary organisers the benefit of the doubt. Most competition organisers will say the purpose is “to select and showcase the best images”... as if that gets us any closer to a useful answer. The problem is that the ‘best images’ approach can lead to all sorts of creative cul-de-sacs and in most cases, the result is ‘the most wow’ images. This can end up with a very tiring and homogenous collection of classically sublime mountain, aerial and nightscape images with a few oddball compositions and lucky shots (or composites) to fill them out.
When we were planning our competition, we were keen for it to be a tool to showcase an inspiring and varied collection of images that represented some of the most interesting and creative work we were seeing online and in person. We wanted to somehow use the competition to curate a virtual exhibition that we all would love to see.
Obviously, exhibitions are hard to organise and expensive (and they don’t have a huge reach) but the next best thing, in my mind at least, is a well-produced fine art book. Therefore, we decided our goal was, essentially, to brief the judges to not necessarily find the ‘best’ or ‘most impactful’ images but to select a range of about 100 or so photographs that really represent the most inspiring work submitted. Along the way, we would obviously have to choose a set of category winners and an overall grand winner but every image in that final set should be able to stand, not just on its own but alongside all the rest of the works selected.
Who are the judges
Who should judge? Tough question. As a type of art, landscape photography is inherently subjective. Yes, there are often images that most people will like more than other images but there are also sometimes images it’s almost impossible to say they are ‘better’ or ‘worse’ than others. These often inspire really strong feelings in at least some viewers. This leads to a discussion around whether it’s better that a small number of people love an image or that everybody likes an image – see our ‘scoring’ discussion for more on that.
Most people who engage in an activity at a certain level are interested in the opinions of their peers (if they want to be judged at all). For example, if I wanted someone to pick out some landscape photography books for me to buy, I’d nearly always choose other landscape photographers, and if those photographers just happened to be widely respected for their aesthetic judgement and skills, all the better.
When we were looking at which judges to ask, we had a few criteria. Some of those were pragmatic, for instance it would be great if they had a high profile to help promote the competition in this first year. But they also had to represent a wide array of styles and practices if possible.
We also wanted judges who will have seen a lot of photographs and know about the compositional and post-processing cliches, the iconic, over photographed locations, etc. so that they could recognise originality and creativity - and also notice flaws in post processing and composition when they saw them.
Judging criteria
Again, from a distance, this looks like an easy one. Just pick the best images. However, the best according to who? Are the judges choosing based on their own subjective tastes or are they trying to second guess an audience’s tastes?
When I’ve been asked to judge competitions before, I’ve had to work these criteria out for myself. My own tastes can be somewhat idiosyncratic and perhaps change depending on what I’ve been looking at recently or the types of work I’ve been interested in creating recently. So, in most cases, I’ve tried to be a little more objective. However, I also know I’ve been hired because of who I am and so trying to be completely objective would deny some of my own personal experiences.
There are some general decisions to be made about what aspects of photography are to be judged. Is originality to be rewarded? (and, conversely, are cliche’s to be avoided)? This may mean that some photographs, although quite impressive, might be passed by for ones that show novel composition, original subject matter or surprising use of light and form.
Other criteria would be around how judges should balance subject and moment vs composition and visual balance etc. i.e. an amazing photograph can be about the right place and the right time. It can also about the photographer having worked hard to create a stunning compositional structure with a subject matter that is usually difficult to work with.
Because our competition is about photography and the photographer, we’ll be briefing the judges to put more attention on the qualities the photographer brings to the work e.g. composition, the moment of capture (through lighting), balance, form, etc.
Categories
The use of categories in competitions is often a knee jerk reaction. Instead of paying much thought to them, the categories are chosen because ‘that’s what everyone else does’.
We’ve used categories as a way to separate and isolate types of image that have unique attributes that make them likely to be over or under-represented in the competition.
It was quite clear from having judged many previous competitions that aerial and night photography tended to be heavily represented. In particular, aerial photography shows such a ‘surprising’ view of the landscape that they tend to stand out when shown along with typical landscape views (and tend to still look ‘OK’ when over processed!).
The ability of the camera to represent the night sky in ways that we don’t tend to see with the naked eye also means that they stand out when judging competitions and hence these were separated out into a separate category.
We also wanted to ensure that intimate and abstract photographs had more of a chance of being seen and hence decide that a dedicated category for these would make sense.
We considered categories for other ‘niche’ subjects, film photography, environmental, etc. but we thought it would make more sense to have the option to recognise and draw attention to these in the final selection rather than try to separate them out.
Using scores
Many competitions ask judges to score each entrant using a variety of different scales. And from the outside looking in, it seems an apparently sensible approach. After all, you compare two images and one gets a higher score than the other, how can that go wrong? However, when you’re hours or days into a judging process, can you really remember what a score of 65 meant two days ago vs a score of 69 today? Inevitably there will be scoring ‘drift’ and then the ability to assess the comparative quality of images goes out of the window.
Scoring can still be useful though, but the judges need something simple to remember whilst doing so. I would imagine most people can only hold a simple set of scores in their head, perhaps from 0=reject, 1=maybe, 2=OK, 3= good, 4=very good to 5=possible winner.
Given a 0-5 or 1-5 scale, it would be fairly straightforward to say all of the 4s and 5s (or 3s, 4s and 5s) go through to the second - depending on how many are in each category. (See ‘pre-judging’ for more information).
There is another possible problem with scoring though. Let’s consider the fact that some images really resonate with a few judges but don’t really do much for others. What happens when we compare these with images ones that just get given a ‘very good’ by everybody. Which one should be better?
If we look at the scores, we might end up with the following (presuming four judges).
This is the resonating image:
5,5,2,2
This is the very good image:
4,4,4,4
The resonating image gets 14 points but the very good image gets 16 points so obviously the very good image does better? Well, not so fast. Art is subjective and I might prefer to see an image that really resonates with a few judges than see the generally ‘pretty good’ image. There must be something about that image and I want to know what it is - hence a scoring system needs to consider more than just the ‘average’.
Judging System
There is a reason we don’t use web-based tools to edit and post-process our photographs. They’re nearly all interminably slow, even on a fast internet connection. And try it on a flaky connection and it becomes a real pain.
Judging images is really just a form of editing and we have some great tools available for this already. I think most will be familiar with Adobe Lightroom and will agree that, when it's working OK, it’s simple to skip backwards and forward in Library mode, picking and rejecting images, giving them star rankings, reassessing, browsing through full pages of thumbnails and then having quick peeks at 100% views.
The judging process in a competition should feel the same way and we plan on distributing a local copy of the images to our judges in a Lightroom Catalog and then letting them grade and sort the images locally whereupon they can send a Lightroom Catalog back to us for further assessment.
Pre-Judging
The pre-judging of competitions is often a place where great photography gets rejected. This isn’t always true but it can sometimes seem that way when you compare the range of images that pass through to second rounds vs those that get rejected.
It used to be fairly common that pre-judging was done by either a different set of less well-known judges or sometimes by the organisers of the competition (in some cases the organisers' secretaries, PAs or interns).
When people see the almost random collection of images that get rejected or pass through to the next round, it often creates a quite visceral loss of faith in competitions.
I’ve spent quite a while considering this issue and, unfortunately, there is no ‘perfect’ answer. In an ideal world, every final round judge would look at every single entrant. However, I can guarantee that many potential judges would baulk at the idea of looking at every image in a large competition (even with significant remuneration!).
We have spent quite some time trying to come up with a compromise that on the one hand ensures that the judges get a chance to see everyone’s images but also making sure they could spend more time assessing images that stand a greater chance of making the final selections.
The way we have decided to do this is for the organisers (Alex Nail, Matt Payne, Rajesh Jyothiswaran and myself, Tim Parkin) to pre-sort all of the images using the 0-5 scoring system mentioned above. We would combine those scores and then present this pre-sorted Lightroom Catalog to the judges.
The judges can then focus most of their attention on the 3’s and 4’s while still being able to scan through the 2’s (and sanity check our 1’s) and finally seeing if they want to demote any of our 5’s.
We will brief the judges that this pre-sorting is only there to help them focus their time and not to influence their final choices. They still have all the images available to reassess and they can score any of them as they wish.
Nepotism
“But the judges will just choose one of their friends!”. It’s a fair accusation, after all, the great photographers are all friends with each other and have a desire to excluded everybody else from the industry and to share the profits between them.
Well - perhaps not. There may be a few people who are friends with each other, unsurprisingly as they tend to get invited to the same events. However, we have seen competitions where the winners have had a relationship of some sort with one or more of one of the commended or winners and so we have introduced a voluntary scheme where if a person recognises the photograph and has a relationship with the person in some way, they should recuse themselves from judging. This should only be an issue if they are related or close friends or have a business relationship (if a photographer is just regular workshop client, this should not be a problem but we will ask the judges to make us aware of this).
In Person vs Remote Judging
There is a lot to be said for remote judging. It makes the best use of time for all involved. It can spread the load across multiple days, allowing judges to take rests when they’re feeling a bit jaded or can’t concentrate appropriately. It also gives the judges a bit of flexibility when they can do the judging. However, there is so much to be said for discussing things in person. If you’ve ever sat and chatted about a book with a friend, you’ll know that each of you brings different knowledge to the conversation and the combined assessment is more than just the average of two different opinions.
I’m heavily influenced by my experiences with the wildlife photographer of the year awards where the final rounds are judged in person over a series of days. The resulting conversations are fascinating and each judge may have specialist or local knowledge that the others don’t have. The group often adapts its position on certain photos based on this information.
We would like for our judges to be able to meet in person for a few days for our awards but with our judges donating their time and with restrictions around Covid, we have compromised on a final day of judging in person after the sifting has been done remotely, to ensure that we have agreement on our winning and commended images.
Managing a Group of Judges
Well, it’s not quite as bad as herding cats, but managing a set of independent, professional photographers should not be underestimated. Some may be quite outspoken, others may be quiet and withdrawn. Left to their own devices it would be possible if perhaps not likely, that the discussion, and hence the conclusions, could be over-influenced by one or more members of the group.
A good chairperson can mitigate most of these issues. They can keep the judging on a timetable, ensure each person’s voice is heard, prompt/remind judges of the criteria they are judging by, etc. The chairperson shouldn’t be making or influencing decisions but should be able to ensure things proceed along a timely and appropriate path.
And that's about it for my thoughts on judging. I think it pays to be as open as possible about your processes rather than have a 'black box' approach and if you have any suggestions for things I may have missed, which aren't clear or just general questions about the competition, please let me know in the comments below.