This article was first published in September 2022 on the website of OSINT Fr.
A Few Tips for Building a (Good) CTF
OSINT CTFs (Capture The Flag) are fun challenges that showcase and improve the open-source intelligence (OSINT) techniques of the participants, as well as those of the people who read the write-ups afterwards (although, of course, nothing beats hands-on practice). In practical terms, these CTFs are driven by questions, where each correct answer unlocks the next one. The TraceLabs challenges, which aim to help locate missing persons, are the exception.
A number of very high-quality CTFs have appeared recently: Hexa OSINT, created by Sopra Steria for La Fabrique Défense, Stranger Case by ESNA, the UYBHYS CTF, put together by members of OSINT-FR, the Bleuet de France and the Hunt by AEGE, and many more. I also had the opportunity to help create the OSINT 317 challenge for the 2021 and 2022 editions of the European Cyber Cup, held at the Forum International de la Cybersécurité (FIC). After (very) many hours spent working on these two projects (several dozen, not counting the fact that you often think about them outside of that time), which received very positive feedback, I would like to share a few tips on what it takes to build a good CTF, in the hope of inspiring new vocations and/or making organisers' lives easier.
1. Be Consistent
Consistency is, in my view, the key element of a good CTF, because everything flows from it. Indeed, the aim of the challenge is to create a fictional investigation and, in any investigation, the investigator comes up against people who act according to a certain logic.
The first step is therefore to write a scenario with an overall storyline and a thorough description of the characters, making sure they maintain consistent digital behaviour across the various platforms on which they have avatars. To this end, it is advisable to keep an up-to-date Excel document with all the details relating to the avatars (date of birth, social media accounts, passwords, etc.).
The scenario is then broken down into questions in chronological order: you have to guide participants step by step (one of the limitations of classic CTFs, in fact, since a real investigation is marked by uncertainty). A collaborative tool such as Miro can help with this. In the example below, you need to understand that solving question 1) unlocks question 2) of the "Ian" hunt and the first question of the "Debbie" hunt (which lets participants work on several questions in parallel).
It goes without saying that the solution to each question must be consistent. If two avatars communicate discreetly, you might use steganography. Using that technique to hide an avatar's date of birth, on the other hand, would not make sense (and would therefore be almost impossible for participants to find).
2. Encourage Thinking and Analysis
The nightmare, when you create a challenge meant to last 24 hours, is that it gets wrapped up in just a few hours. You therefore need to design questions that are complex enough to be workable while still pushing participants to rack their brains and form research hypotheses. But how do you create good questions?
The method used for the 317 challenges consists of keeping an up-to-date table of all the techniques you would like to use to solve the puzzles. Among the most classic: pivoting from a username, using archives, geolocation, and so on. This means staying current with techniques, in order to remain innovative and creative. If the scenario is well defined and each character has well-populated social media accounts, blogs, etc., all that remains is to ask: what information do we want to reveal, and by what technique?
Finally, the questions need to be tested by a team placed in the same conditions as the participants. This step reveals not only the level of difficulty but also any errors, whether in the wording of the questions or even in the answers themselves. Ideally, this testing phase should finish a week before the challenge.
3. Create as Much Noise as Possible
The main difficulty of a real investigation lies in the large number of false leads and the near-infinite number of research hypotheses: an individual may be completely invisible on the web (though you still have to be sure of that!) or, on the contrary, have dozens of accounts, be active on forums, blogs, and so on. And in that case, you have to manage to extract every element of interest. Because it can never be said enough: finding an account/blog is worthless in itself; what matters is analysing it thoroughly and collecting every element.
Applied to a CTF, this means you need to create noise, false leads. You have to "bring the characters to life" by populating their accounts, so that all the articles, tweets, blog posts, etc., are not all published on the same day. If one of the avatars has an Ask FM account with only a single post on it, the clue is gathered far too quickly. Likewise, if you want to hide a clue on a Facebook account, you need to publish plenty of posts to force participants to read everything, as they would in a genuine investigation. These posts must fit within the overall consistency of the scenario, especially if the characters interact with one another. Here again, this requires organisation to define "who" publishes what and when, particularly for the important messages. If a team is in charge of organising the CTF, I recommend that a given character's posts always be written by the same person, so as to keep the tone, the wording, and so on, consistent. You therefore need to define the roles from the very outset of the CTF and stick to them. Everyone must publish regularly (depending on their character's profile), in order to hide the key messages within the mass of posts.
This also brings us to the limits of the exercise: creating a scenario and characters from scratch cuts us off from everything that is a legal document (commercial registers, databases, etc.). You can always improvise (for example, asking for the name of the founder of a restaurant where one of the avatars supposedly had lunch), but this has its limits.
4. Challenge Your Ideas
Another tip for producing a good CTF: constantly challenge your ideas. Much like the writers' room used to write TV series, creating a CTF feeds off everyone's ideas. It sometimes happens that one person comes up with an idea, which is enriched by another, then refined by a third. Regular meetings not only make it possible to track the progress of the overall project, but also offer opportunities for this clash of ideas. It is worth recalling, incidentally, that ideas should not emerge simply because the writers get together; rather, everyone should turn up with lines of work to pursue, otherwise things risk descending into meeting-for-meeting's-sake.
In the end, without a sound methodology, you can waste a great deal of time and energy. Building a CTF therefore calls for organisation, cooperation and, of course, a solid level of OSINT, so that the challenge lives up to the participants' expectations.