Running SIGPwny's First Recruiting CTF

As part of improving the University of Illinois' security club, SIGPwny, the senior and lead members decided to host a capture the flag event near the start of the fall semester to recruit new students, particularly freshmen, to the club.

We had improved attendance and member involvement the previous year primarily because of a changed meeting format, expansion of club activities, and delegation of work to additional members. Additional membership allows us to teach security to more people and acquire resources that increase what we can do. Reaching freshmen is essential because it will enable them to develop into excellent and knowledgeable leaders for the club, ensuring the club will be well-run in the future. It may also lead them to discover a passion for security, creating more talent for the security community.

Hosting any CTF is no easy feat, but ours was especially challenging since it was intended to be large-scale (100+ students), in-person, and easily accessible to first-time players. Even though we knew it would be a difficult undertaking, and it did end up taking a lot of hard work from many members, we felt it would all be worth it in the end.

First, we needed sponsors. Although all the members did work for free and we used open-source software on our existing infrastructure, we needed money for food and prizes, like any other computer science competition at our school. Fortunately, we could leverage the connections of students and alumni to obtain sponsorships that gave us the requisite cash to run the event well. We invited security engineers to help us walk around and teach the students. The engineers were great since they could help beginners with challenges, test our problems to see if they worked as intended, and expose our students to job opportunities in security.

Logistically, there were many things we had to worry about since we were hosting this in person. The time, date, and location of the event were crucial. We decided that the weekend afternoon near the start of the fall semester was ideal. Students would have learned about our club through an open house, wouldn't have much homework or any exams, and likely would be awake and free to work.

Our best options for a room location were in the CS or ECE buildings. We managed to secure the main lecture hall in our CS building, which was great since there would be ample space. However, since "the architects just screwed up," in the words of one CS professor, this room had very few outlets. Fortunately, we obtained many power strips right before the event to help deal with the issue and let people hack for hours on end.

The food of choice for any CS event is pizza since everyone loves pizza and you can get it delivered to you quickly. We used the ACM discount to order a large number of pizzas shortly before the event, but with enough lead time for the order. There was one wave of pizzas at the start and another in the middle. We also provided cookies (including some in Illini blue and orange) and soda. The prizes were ordered online and picked up before we distributed them.

The CTF was five hours, which provided enough time to solve many challenges. We decided to have short presentations of around ten minutes each for our main CTF categories: web, pwn/reverse engineering, crypto, recon, and forensics. Since our regular meetings are a short presentation followed by work on challenges, it was like five meetings in one event. These presentations were intended to teach students some of the basics of each category and help them solve the challenges.

Since there were beginners, and we wanted to encourage learning, we needed the members who were running the CTF and the security engineers to help people through problems. In our usual meetings, we have fewer people trying to solve the challenges, and collaboration among everyone is encouraged. Because of this and the more intimate environment, we can help everyone without much issue.

However, since we were at a larger scale than usual and there were different categories that not every member knew well, I thought using our school's website for queues would be nice. We had mixed thoughts on this since we believed walking up to people or responding to raised hands would be best. We were able to create the queue and start it before the event. We briefly mentioned it in one of the introduction slides but didn't put the queue up on the large projected screen since someone scrapped the idea shortly after.

Students were still using the queue, however, so we responded to requests. It worked well overall, especially since members knew which questions they could answer. Sometimes we couldn't figure out where the person in the room was given their description, and we couldn't follow up with a question. It is worth trying again in the future for large-scale events.

We needed to decide if we would allow teams, and if so, what would be the maximum team size allowed? There were two opposing forces at play here. As a club, we care about learning and want members to work together to solve problems. This mindset is behind why we help members extensively during a regular meeting and encourage collaboration then. However, we can't allow unlimited collaboration for this event since we wouldn't have a fair way of handing out prizes.

If people participated by themselves, prize distribution would be more straightforward, but there would be no one to work with on challenges. If there were relatively large team sizes, it would be challenging to figure out how to split a physical prize with different people on the team. Fixing this issue would require every team member to get a prize, resulting in just a few groups winning anything. Since we wanted some teamwork but many teams to win and participants to get exposure to different categories, we decided to let people work as partners.

There remained another crucial issue with teams and prize distribution. We intended the CTF to be for beginners, but we also wanted more advanced students to play. We wanted to ensure that beginners had a fair chance at earning prizes and that we prioritized when appropriate, so we split the teams into beginner and advanced categories. If either player on a team was advanced, then the team was considered advanced.

Advanced players were those who completed our school's main security course or the primary systems courses in CS or ECE, attended several SIGPwny meetings, or were otherwise advanced at either their own or the event leads' discretion. During the event, we noticed a couple of beginner teams at the top in the beginning. After talking with them, we put them in the advanced category since they had experience. The top four beginner and advanced teams were awarded prizes in a back-and-forth manner, with the best beginner team getting to pick first. Each team member could choose whatever they wanted from the available prizes. We had a few extra prizes that we awarded to the fifth and sixth-best beginner teams.

We needed to do some marketing for our event to ensure people would come, so we posted about the event on Facebook, Twitter, and Reddit. We also printed lovely flyers and posted them in various buildings near the engineering quad. The most important aspects of our event that would get people to come were free food, prizes, and the beginner-friendly environment, so all of our ads mentioned them. The event was advertised as a "HACKathon" since more people were familiar with that term than "capture the flag," and it's one of the only hackathons with actual hacking.

Operationally, we needed to create and host challenges. Most of the problems were required to be accessible to first-time players, but we also wanted more difficult ones that advanced players could try. We had the typical CTF categories, but pwn and reverse engineering were one combined category since they are hard for new players. We also added recon and shell, since they are more accessible, and a scavenger hunt category with flags hidden inside the room and building to motivate people to move around.

Club members submitted challenges to a Git repository, and we tried to track them on a spreadsheet. We could have done a better job of having a standardized challenge format and keeping track of all the challenges on the spreadsheet. Some challenges repeated the solve techniques of other problems, which wasn't a big deal for a beginner CTF, but we should coordinate each category's challenges next time. I think we had a solid mix of each challenge type, but we could have made the number of problems per category more balanced.

An unexpected difficulty was deciding on the points for each challenge and a scoring system. There's the traditional CTF point system where every problem has a static point value, but there's another system where the point value can decrease based on the number of solves. There are some advantages to the latter model, namely that point values adjust nicely if one challenge is more straightforward than anticipated. However, we chose static because I think it would be confusing and demoralizing for beginners to see their team losing points in the middle of the competition. They should be worried about the challenges, not continually monitoring their score.

It was hard to come up with reasonable point values for each challenge because we designed them for beginners, and we had many problems to compare each challenge against. Ultimately, we just reviewed each challenge and put what we thought was okay. The unsolved challenges got 4x the points halfway through the competition to encourage solving them, but that definitely shouldn't be done in the future. A few crypto problems were likely unsolved because of how many crypto challenges we created. Teams needed to solve all of them to earn a top spot since they were now worth a ton of points.

For our infrastructure, we ran everything off of one physical server. We used two virtual machines, one running the CTFd site and another hosting the challenges, typically web and pwn. Docker was used extensively since it makes infrastructure more flexible. CTFd can easily be installed using Docker and just requires modification of the Nginx reverse proxy for our specific configuration. Our challenge server had another reverse proxy to proxy requests to the particular web app or pwn process.

Challenges each ran in their own Docker container. It's easier to write problems if you know the execution environment. There's also no need to worry about typical security configurations since you shouldn't be able to break out of a standard container. If we decide to use a different server in the future or want to share the challenge, it's much easier to deploy again. Containers are also easier to maintain and set up compared to virtual machines or physical servers.

Despite all the logistical and operational challenges, the CTF went great overall. We had 112 players, and it seemed like most people stayed for the entire event, which is awesome. There was plenty of food, and I even got to take home a pizza. The security engineers were impressed it went so well, especially since this was our first event of its kind, and undergrads organized it.

We solicited some feedback about the event on a Google Forms questionnaire via email and a Discord announcement. Respondents overwhelmingly said they enjoyed the competition, thought the prizes were cool, and found the club members and engineers helping to be useful. About half agreed the presentations were useful, and many had neutral thoughts. I agree that they could be improved, and it's worth asking for more detailed feedback on that. Players generally thought the prize distribution was fair, and the event length was about right.

I thought the event length was solid since there were always plenty of challenges to work on, but we were pretty tired by the end, so I don't believe we should extend it. The favorite CTF categories were actually split pretty evenly, so it's nice we had every category. Someone thought it wasn't beginner-friendly enough since there was no cross-team collaboration, and some of the assistance wasn't specific enough. These are valid points, and we should consider standardizing how we assist players.

In conclusion, I'm happy we had this event. People seemed to enjoy it, and there were a lot of thankful comments as feedback, which made our effort worth it. All the members who helped run the event learned a ton too. Interest in the club and meeting attendance is at an all-time high, so we're able to teach many more people now. SIGPwny is in an excellent state, and I know all will be well when I graduate.