What do we want?

Hi folks,
more than two weeks have passed since I wrote my last post. The Beginners Contest I announced last time is set up by now, you will find it on the website of the German puzzle community Logic Masters. Meanwhile, I have been planning a new article. Since this is kind of a delicate matter, I have been thinking about how to phrase the following without offending anyone.

There is a question on my mind. What is a good puzzle? In the course of time I have followed a couple of discussions on various platforms which address this topic in different ways. The question is rarely ever presented in its basic form; instead, people are discussing evaluation systems, puzzle events and what kind of puzzles are suitable for them, why some puzzles catch more spotlight than others, etc.

A few months ago I wrote an article on this site regarding “perfect” puzzles. More precisely, I gave a list of criteria which a puzzle should strive to satisfy in order to achieve the highest marks. Well, that was mostly a theoretical reflection; in practice, one cannot simply go over a checklist to determine whether a puzzle creation is nice and beautiful or perhaps less so.

One thing to keep in mind is that puzzles are usually created for a specific purpose (e.g. a contest). Therefore, when the quality of the puzzle is to be judged, it must be measured against the purpose. This may sound trivial, but it is not. Quite often, people try to evaluate a puzzle without any regard to the context.

Something else worth considering is that the survey of opinions and evaluations can be heavily biased. When interpreting feedback on a puzzle, one should have a closer look at who was in a position to give that feedback. Again, this is something that is easily overlooked. Let me present a couple of “exhibits”.

Tom Collyer, not only one of the best solvers of the UK but also Competition Director for various national and international events in the past, drafted a report “What is a Sudoku?” over the last year or so. This report can be found in the depths of the WPF homepage, for example (and I am sure in many other places, too). He also published some afterthoughts on his own Blog.

The reports covers some technical issues, and since I do not care for Sudoku as much as many others do, I skipped some parts of them. However, Tom also relates the title to two other questions, namely “What should the WSC look like” and “Who is the WSC for”, in short “What do we want”. I like in particular how he hints at the mentality of the top level solvers who are trying to answer these questions, and at the bias that comes from the feedback on this level alone.

Let me give you a simplified view of things. There is something like an “elite” among puzzle solvers, consisting essentially of people taking part in puzzle events on at least national level. As far as logical puzzles in general go, I am not sure if there is much to see beyond this elite right now. But when it comes to Sudokus, I am pretty sure. There are, probably literally, millions of Sudoku solvers who have had an occasional go at Sudokus and who do not belong to this elite at all.

I am under the impression that the feedback about Sudoku events comes mostly from the above elite group. To be honest, I am not sure whether anyone outside even knows about the report and, if so, has raised his voice to be included in such a survey. So when we ask about the purpose of WSC-like events and their target, we must acknowledge that the answers mostly come from people who have already been there.

Many puzzle websites that publish puzzles, either on a regular basis (perhaps daily) or using contributions from community members, come with some kind of evaluation system. For instance, Gareth Moore’s PuzzleMix (a platform I learned about at the beginning of the Corona lockdown) allows the solvers to evaluate the difficulty of puzzles, at least the ones that are publicly available.

I have not done that so far because I first wanted to get a feeling for the difficult standards on this site. By now I can only wonder, though. On a scale that uses the terms Easy, Mild, Moderate and Hard (and there may be others in between?), I have yet to find to a puzzle judged as Hard, and very few are considered even Moderate. Now, I have my ups and downs like everybody else, but I frequently encounter puzzles that take me 20 minutes and more to solve them, and they are still labeled Mild at best.

Since I have freely chosen not to take part in the evaluation process myself, I can hardly complain. Still, since one must apparently solve the puzzles before one can evaluate them, there is some obvious bias here. There are of course good reasons to install an evaluation or scoring system that requires the solving of the puzzles first. Basically, one wants to ensure that only puzzlers who have actually spent time working on a puzzle can give any marks. But this comes at a price.

The German Puzzle Portal is another good example. Here there are two categories, Difficulty and a general Rating. But again, one must solve the puzzles before one can evaluate them. As said before, there is nothing a priori wrong with this. But there are several issues which have kept me occupied lately.

First, people typically do not want to give bad marks in a community like this. By giving negative evaluations one puts oneself offside, in a manner of speaking, especially if one is to first to do it, so people are typically very hesitant about it. (This is not my own discovery.) I have observed this with myself, and I am sure others feel similarly.

In a recent discussion in the German forum it was noted that 99% of all entries in the Puzzle Portal have a Rating of above 50%. I am barely active there, but when I pick a puzzle, it is usually one I have a reasonable expectation of liking. So for me there is some bias from the start. (Yes I know, some people are trying to solve every single puzzle in the Portal, but I would expect a significant number of solvers to elect a similar approach.)

A second issue is what the general Rating stands for at all. Personally I like to think of it as “beauty”, but since the purpose of this second scoring category is not defined anywhere, the one word is as good as the other. Every solver can decide for himself what the general Rating means for him and evaluate the puzzle accordingly.

For some people there seems a be a strong connection between beauty and difficulty; for others, this is a rather incidental correlation. Some people like all puzzles which are eccentric, spectacular, in any way out of the ordinary (either regarding the rules or just visually). I have actually seen puzzlers clap their hands at the sight of puzzles with a striking visual theme, even if the solving path was rocky, to put it mildy. Unless a “beauty standard” is established – and I am not sure it is a good idea – the significance of the rating is limited even further.

This brings us back to the purpose of puzzles which can be found in the Puzzle Portal or on similar platforms, i.e. the “What do we want” question. Personally I think that the focus on Logical Puzzles should generally lie on the solving logic, but in reality there is no such requirement, and I cannot claim that my view should be universally accepted.

Some time ago I learned of a YouTube channel called Cracking the Cryptic (it popped up on my YouTube start page one day) hosted by one Simon Anthony who solves Sudokus on screen and explains his steps in real time. Since I was curious, I watched one video where he solved a Sudoku with just four givens but several additional rules so that the solution became unique.

That was no doubt an interesting construction, but I did not consider it a good advertisement for Sudokus (or more generally Logical Puzzles). The short version is that it was not a Sudoku to attract people, but rather a creation to please the aforementioned elite. But then, I do not know what the intention of the channel is, hence once again I am not in a position to judge.

It is easy to be blinded by a strong visual theme, which can be said to include the presence of extremely few clues/givens. In my experience, this often comes at the cost of good solving logic. (I am far from immune to this. On the contrary, when I create a puzzle I often find myself trying to inject some theme, even when my initial motivation clearly lies somewhere else, as in the Classics Collection.)

It is not entirely impossible to combine visual beauty with nice solving logic. The website Grandmaster Puzzles by former Sudoku World Champion Thomas Snyder features many excellent handcrafted puzzles on its blog (which, by the way, has recently announced its reopening). For example, a week ago Thomas posted a TomTom puzzle which I liked a lot, because the theme goes hand in hand with a pleasant solve right up to the end.

I cannot answer the question “What do we want?” for the puzzle community in total, but I can answer it for myself. This is the kind of puzzle which I would like to see more. Please note that this puzzle has nothing spectacular about it. But it has what a good puzzle needs: good starting points, a clean solving path and a nice layout on top of it. I do not know how much time Thomas puts in a creation like this (or his other contributors, for that matter), but it was worth every second of it.

As said before, when it comes to contests, the difficulty level must match the target group in addition to the criteria already mentioned. (I sincerely hope that the puzzles I designed for the Beginners Contest are suitable for actual beginners.)

There are some particular cases where I consider the target group and the suitability issue as a matter of major importance, such as the online qualifiers for the German Puzzle Championship. This event is an opportunity for attracting new members to our community; unfortunately, I felt in a couple of cases over the years that the puzzles were once again primarily attractive to the puzzle elite and not so much to newcomers.

Someone pointed out that things stay the way they are because the feedback is biased in favor of the current state of things, especially since it is frequently difficult to find a team of authors for our Championships, and there is a lot of truth in it. I should note that in my opinion this applies equally to many other events as well, like the GP series.

What we have here is just another “What do we want” phenomenon. Should the feedback be positive and encouraing, to keep everyone happy, or should it be honest, even at the risk of offending some people? It is said that objective and fair criticism should be allowed, but we all know that in practice it does not work out as simple as that. People take offense easily. Like I said at the beginning, a very delicate topic.

In the end it comes down to this: People enjoy to be critical, but they do not like their own work to be the subject of criticism. This observation is obviously not limited to the puzzle community, and it is not limited to our current puzzle activities either – I do not want to give the impression that our community stands at any particular crossroads right now. However, the matter of bias is something to always keep in mind, especially if we are confronted with an overwhelmingly positive feedback.

As far as I am concerned, I intend to continue the Classics Collection after the Beginners Contest, that is, to design more standard puzzles which are less than spectacular (even if a theme makes its way into the puzzles every now and then). This is my goal for the time being, and I hope I will find the time to keep putting it into action.

6 replies on “What do we want?”

Thanks for this highly interesting and thought-provoking discussion!

Regarding your point on Cracking the Cryptic, I absolutely agree that puzzles with nice logic (and not necessarily as much aesthetic) are a better advertisement for puzzles. However, most of their videos do indeed feature these kinds of puzzles. The videos with few givens are the ones that go viral, since someone who has never seen the channel or tried logic puzzles before is more likely to be enticed by a lack of givens (which is noteworthy at a glance) than excellent logic. I think it’s great that they do these kinds of puzzles once in a while, as a reasonable fraction of those who saw the videos would begin to realise the logical beauty of puzzles, whereas they would never have started watching CtC otherwise.

You also reference Thomas Snyder’s excellent TomTom puzzle; I’m certain that we don’t see these puzzles often because they are really hard to make! Most authors can only achieve one of two things: a nice layout or nice logic. It takes a world class author to consistently combine these two aspects.

There is much here worthy of much wider discussion and consideration. I’m fighting hard to resist the temptation to add another contribution here, because for the time being I feel worn out from devoting myself to the puzzling community, and so I am taking an indefinite break from blogs, forums, e-mails, chats, discussions etc, as well as registering a score in any competition. There are a number of reasons for doing this, but one of which relates to the way you’ve described the subject of rocking the boat and criticism. As you say, taking criticism can be a delicate issue, but on the other hand speaking out and giving criticism can also be very wearing and isolating.

Anyhow, I’ll permit myself this particular personal indiscretion and limit myself to the following two points.

1) Context/Purpose is very important, perhaps even for elite solvers. There are some puzzles I enjoy solving against the clock, but might not seem as interesting in my own time. Vice versa there are plenty of puzzles (many of which feature on cracking the cryptic) that I’d find very stressful to solve against the clock, but would happily keep me occupied in the absence of any kind of time pressure.

2) To continue Freddie’s point, I would go further and say it is much easier for an author to impose a nice visual theme onto a particular puzzle, than it is to make sure a puzzle solves nicely. I think this kind of thing is compounded by authors trying to imitate the same kinds of things they’ve seen other authors do in other contests, and perhaps it is fair to say that a nice visual theme is an easy way in which an aspiring author might hope to distinguish themselves.

When it comes to what do *I* want, I am completely with you in this respect – I very strongly prefer to see puzzles which solve nicely with a good solving flow, even if this means that the appearance is not particularly striking (GM Puzzles is by no means perfect in this regard by the way, although it is probably better than most). I also take particular delight in good puzzles of an “established” genre. I also tend to think that it is unfortunate to see quite so many arbitrary complex variations and combinations of rules when it appears more regard has been paid to the pursuit of novelty for its own sake than it has to whether the puzzle is particularly nice to solve. Each to their own, I suppose.

I suppose the last thing to say is that when a lot of the authors are doing all of this more or less for free, then it’s just that little bit harder to complain about it – the polite thing instead is to just keep quiet.

Thank you both for your comments.

My judgment of Cracking the Cryptic may have been harsh. The other day I watched a video featuring a classic Sudoku. It was actually a very nice construction, and the illustration of the solving path was also decent, but the presenter kind of spoiled it by repeatedly describing the puzzle as absolutely amazing, incredible, fantastic, etc. I found the accumulation of superlatives irritating, but then, I do not consider myself part of the target group.

What you say about puzzle creation and visual themes is certainly right. My personal experience is this: When I place some clues at the beginning which have the potential for certain themes, there is a tremendous temptation to continue with a visual setup later, instead of adding different clues which preserve a nice solving path but destroy the theme. And it takes a lot of discipline to go in the direction of solving logic anyway. I imagine this is rather typical for puzzle authors (of any level).

Tom, it’s a pity that have withdrawn (at least for now) from your various roles in the puzzle community, but I understand your reasons, and I believe I share some of them, especially after the labours from the WSPC last year. One reason I have revived this blog is so that I can inject my own views – both on actual puzzles and many puzzle-related topics – without any direct confrontation, or any long-term obligation for that matter.

I, like you, dislike Simon’s excessive use of superlatives; I would prefer the logic to speak for itself. However, as you say, CtC caters towards a more general audience, and most people seem to like it. They have recently been favoured by the YouTube algorithm, it is unlikely that they will change anything anytime soon for fear of upsetting the balance.

I appreciate to read your thoughts on your blog.
I disagree on one point here: even if I’m not a fanatic of CtC, for reasons already mentioned, I don’t think puzzles presented are “creations to please the elite”. To be more precise, my point of view regarding sudoku is that creations made for elite competitions are suitable for lot of people outside elite, but probably not in a competition environment.

In my opinion, that’s a frequent misunderstanding of puzzle community to think there are only elite (as you describe it) and beginners. I mean, I don’t know for general puzzling, but what shows CtC is that there is a huge potential of democratization of the sudokus we solve in competition. Below the CtC videos, I see a lot of comments of people having solved the puzzle, even if they spent 30 minutes, 1 hour, 2 hours. In a non-competitive environment time doesn’t count. These people are not beginner, I would say more precisely they are “non-competitive experienced players”.

Hi Fred,
sorry for the delayed response. You are certainly right that my classification of solvers is extremely superficial and does not do the wide range of puzzle friends justice.

I watched a few more CtC videos recently. Since they were all about Sudokus and variants, I still consider myself prejudiced, but I am realizing that my initial judgment was too harsh, and that the format is actually quite suitable to attract new members to the puzzle community.

Perhaps it would be possible to produce more videos in a similar fashion with other puzzle styles, and in particular featuring very easy puzzles, in order to further promote logical puzzles in general.

Leave a Reply

Your email address will not be published. Required fields are marked *