Program Annotations Affecting the Decompiler

The entry point address for a function plays a pivotal role for analysis using the Decompiler. Ghidra generally associates a formal Function Symbol and an underlying Function object at this address, which are the key elements that need to be present to trigger decompilation (see Functions). The Function object stores the function body, parameters, local variables, and other information critical to the decompilation process.

Function Symbols and Function objects are generally created automatically by a Ghidra analyzer when initially importing a binary executable and running Auto Analysis. If necessary, however, a user can manually create a Function object from a Listing window by using the Create Function command (pressing the 'F' key), when the cursor is placed on the function's entry point (see Create Function).

When a function is created, Ghidra stores its function body as a set of addresses in the Program database. This formal function body delineates the function from all the other kinds of data within the Program and lets Ghidra immediately link addresses in the middle of the function to the entry point and the Function object. Decompiler windows in particular use the formal function body to know which function to decompile in response to a navigation event to an arbitrary address.

The Decompiler does not use the formal function body when it computes control flow; it recomputes its own idea of the function body starting from the entry point it is handed. If the formal function body was created manually, using a selection for instance, or in other extreme circumstances, the Decompiler's view of the function body may not match the formal view. This can lead to confusing behavior, where clicking in a Decompiler window may unexpectedly navigate the window away from the function.

Control-flow behavior for a machine instruction is generally determined by its underlying p-code (see P-code Control Flow), but this can be changed by applying a Flow Override. A Flow Override maintains the overall semantics of a branching instruction but changes how the branch is interpreted. For instance, a JMP instruction, which traditionally represents a branch within a single function, can be overridden to represent a call to a new function. Flow Overrides are applied by Analyzers or manually by the user.

The Decompiler automatically incorporates any relevant Flow Overrides into its analysis of a function. This can have a significant impact on results. The types of possible Flow Overrides include:

The Decompiler collects and displays comments associated with any address in the formal function body currently decompiling. The comments are integrated line by line into the decompiled code, and an individual comment is displayed on the line before the line of code incorporating the instruction associated with the comment's address.

Because a single line of code typically encompasses multiple machine instructions, there is a possibility that multiple comments at different addresses apply to the same line. In this case, the Decompiler displays each comment on its own line, in address order, directly before the line of code.

Because the output of the Decompiler can be a heavily transformed version compared to the original machine instructions, it is possible that individual instructions no longer have explicit tokens representing them in the output. Comments attached to these instruction will still be displayed in the Decompiler output with the closest associated line of code, usually within the same basic block.

By default, the Decompiler displays only the Pre comments within the body of the function. It also displays Plate comments, but only if they are attached to the entry point of the function. In this case, they are displayed first in the Decompiler output, along with WARNING comments, before the function declaration. Other comment types can be configured to be part of Decompiler output by changing the Decompiler display options (see Display <kind-of> Comments).

	Unlike a Listing window, the Decompiler does not alter how a comment is displayed based on its type. All enabled types of comment are displayed in the same way, on a separate line before the line of code associated with the address.

The Decompiler may decide as part of its analysis that individual basic blocks are unreachable and not display them in the output. In this case, any comments associated with addresses in the unreachable block will also not be displayed.

The Decompiler can generate internal warnings during its analysis and will incorporate them into the output as comments in the same way as the user-defined comments described above. They are not part of Ghidra's comment system, however, and cannot be edited. They can be distinguished from normal comments by the word 'WARNING' at the beginning of the comment.

Ghidra provides various ways that a name and other attributes can be ascribed to a variable. These break up roughly into global variables, defined directly on memory in the Program image, and variables that are local to a function.

Global variable annotations are created from the tool by applying a data-type to a memory location in a Listing window, either by invoking a command from the Data pop-up menu or by dragging a data-type from the Data Type Manager window directly onto the memory location. Refer to the documentation:

Local variables annotations are created from the Listing using various editor dialogs. See, in particular:

The Decompiler window also provides numerous ways of annotating variables, both local and global. In particular, see the commands:

Ghidra maintains its own symbol table that supports namespaces and function scopes, and variable names are automatically incorporated into this. In order to widely accommodate different use cases, Ghidra's symbol table has extremely lax naming rules. Ghidra may allow names that conflict with the stricter rules of the language the Decompiler is attempting to produce. The Decompiler does not currently have an option that checks for this. Users should be aware of:

Ghidra provides extensive support for naming and describing data-types that are tailored for the Program being analyzed. Data-types that are explicitly part of a variable annotation are, to the extent possible, automatically incorporated into the Decompiler's analysis.

	      text[iVar1] = 'a';           // Assigning a character to a variable index of the array
	      cVar2 = text[7];             // Reading a fixed element from an array
	      text._8_2_ = 0x7677;         // Auto-generated token indicating multiple elements are assigned at once
	    

	      struct1.a = 1;                // Assigning an integer to a field named "a"
	      fVar1 = ptr->b;               // Reading field "b" through a pointer
	      struct1._20_4_ = 0xff00ff00;  // Auto-generated name for assigning multiple fields at once
	    

              ADJ(structoffptr)->field1 = 2;  // Accessing the underlying structure's field
              iVar1 = *ADJ(intoffptr);        // Accessing the underlying integer data-type
              ADJ(arrayoffptr)[4] = iVar2;    // Accessing the underlying array
            

Forcing Data-types

The Decompiler performs type propagation as part of its analysis on functions. Data-type information is collected from variable annotations and other sources, which is then propagated via data flow throughout the function to other variables and constants where the data-type may not be immediately apparent.

With few exceptions, a variable annotation is forcing on the Decompiler in the sense that the storage location being annotated is considered an unalterable data-type source. During type propagation, the data-type may propagate to other variables, but the variable representing the storage location being annotated is guaranteed to have the given name and that data-type; it will not be overridden.

	Users should be aware that variable annotations are forcing on the Decompiler and may directly override aspects of its analysis. Because of this, variable annotations are the most powerful way for the user to affect Decompiler output, but setting an incomplete or incorrect data-type as part of an annotation may produce poorer Decompiler output.

The major exception to forcing annotations is if the data-type in the annotation is undefined. Ghidra reserves specific names to represent formally undefined data-types, such as:

• undefined1
• undefined2
• undefined4
• undefined8

These allow annotations to be made even when the user doesn't have information about a variable's data-type. The number in the name only specifies the number of bytes in the variable.

The Decompiler views a variable annotation with an undefined data-type only as an indication of what name should be used if a variable at that storage address exists. The data-type for the variable is filled in, using type propagation from other sources.

For annotations that specifically label a function's formal parameters or return value, the Signature Source also affects how they're treated by the Decompiler. If the Signature Source is set to anything other than DEFAULT, there is a forced one-to-one correspondence between variable annotations and actual parameters in the Decompiler's view of the function. This is stronger than just forcing the data-type; the existence or nonexistence of the variable itself is forced by the annotation in this case. If the Signature Source is forcing and there are no parameter annotations, a void prototype is forced on the function.

A forcing Signature Source is set typically if debug symbols for the function are read in during Program import (IMPORTED) or if the user manually edits the function prototype directly (USER_DEFINED).

If an annotation and the Signature Source force a parameter to exist, specifying an undefined data-type in the annotation still directs the Decompiler to fill in the variable's data-type using type propagation. The same holds true for the return value; an undefined annotation fixes the size of the return value, but the Decompiler fills in its own data-type.

	The Decompiler may still use an undefined data-type to label a variable, even after type propagation. If a variable is simply copied around within a function and there are no other substantive operations or annotations on the variable, the Decompiler may decide the undefined data-type is appropriate.

Every variable annotation is associated with a single storage location, where the value of the variable is stored during execution: generally a register, stack location, or an address in the load image of the Program. The storage location does not necessarily hold the value for that variable at all points of execution, and it is possible for the variable value to be held in different storage locations at different points of execution. The set of execution points where the storage location does hold the variable value is called the annotation scope; this is distinct from, but influenced by, the scope of the variable itself. The different types of storage location are listed below.

Ghidra records a Signature Source for every function, indicating the origin of its prototype information. This is similar to the Symbol Source attached to Ghidra's symbol annotations (see the documentation for Filtering in the Symbol Table). The possible types are:

Upon import of the Program, if there are debugging symbols available, Ghidra will build annotations of the function's parameters and set the Symbol Source type to IMPORTED. Otherwise, it will generally be set to DEFAULT.

However, Ghidra adjusts the Signature Source for a function if there is any change to the prototype. In particular, if the user adds, removes, or edits variable annotations for the function's parameters or return value, Ghidra automatically converts the Signature Source to be USER_DEFINED.

If the Signature Source is set to anything other than DEFAULT, the function's prototype information is forcing on the Decompiler (see the discussion in Forcing Data-types).

The input parameter and return value annotations of the function prototype, like any variable annotations, can be forcing on the Decompiler (see the complete discussion in Forcing Data-types). But keep in mind:

	The input parameters and return value are all forced on the Decompiler as a unit based on the Signature Source. They are all forced if the type is set to anything other than DEFAULT; otherwise none of them are forced.

If the function prototype's annotations are not forcing, the Decompiler will attempt to discover the parameters and return value using the calling convention. The prototype model underlying the calling convention dictates which storage locations can be considered as parameters and their formal ordering.

If there are parameter or return value annotations that do not agree with the calling convention that has been set, the function prototype is said to be using custom storage. Using the Function Editor Dialog for instance, any storage location can be specified as a parameter, and a completely custom prototype can be built for the function.

The Decompiler will disregard the calling convention's rules in this situation and use the custom storage locations for parameters and the return value. Other aspects of the calling convention, like the unaffected list, will still be used.

The constant mutability setting indicates that values within the memory region are read-only and don't change during Program execution. If a read-only variable is accessed in a function being analyzed by the Decompiler, its constant value, if present in the Program's load image, replaces the variable within data flow for the function. The Decompiler may propagate the constant and fold it in to other operations, which can have a substantial impact on the final output.

The volatile mutability setting indicates that values within the memory region may change unexpectedly, even if the code currently executing does not directly write to it. Accessing a variable within a volatile region, either reading or writing, can have other side-effects on the machine state, and it cannot in general be treated as normal variable. If a volatile variable is accessed in a function being analyzed by the Decompiler, each access is expressed as a copy statement on its own line, separated from other expressions, so that the its position within the code and any sequence of accesses is clearly indicated. Any access, either read or write, will always be displayed, even if the value is not directly used by the function. The token representing the variable will be displayed using the Special color, highlighting that the access is volatile (See Color for <token>).

	  X = SREG;              // Reading volatile SREG
	  DAT_mem_002b = 0x20;   // Writing volatile DAT
	

Ghidra can create an association between a name and a constant, called an equate. An equate is a descriptive string that is intended to replace the numeric form of the constant, and equates across the entire Program can be viewed from the Equates Table.

An equate can be applied to a machine instruction with a constant operand by using the Set Equate menu from a Listing window. If the Decompiler successfully follows the operand to a matching constant, the equate's name is displayed as part of the Decompiler's output as well as in any Listing window. A transformed operand is displayed as an expression, where the transforming operation is applied to the equate symbol representing the original constant.

Alternatively, an equate can be applied directly to a constant from a Decompiler window using its Set Equate... menu. The constant may or may not have a corresponding instruction operand but will be displayed in Decompiler output using the descriptive string.

Ghidra can apply a format conversion to any integer constant that is displayed in Decompiler output.

A conversion can be applied to the machine instruction containing the constant as an operand using the Convert menu option from a Listing window. If the Decompiler successfully traces the operand to a matching constant, the format conversion is applied in the Decompiler output as well as in the Listing window.

Alternately, a conversion can be applied directly to an integer constant in a Decompiler window using its Convert menu option. The constant may or may not have a corresponding instruction operand but is displayed in Decompiler output using the conversion.

Conversions applied by the Decompiler are currently limited to:

If necessary, a header matching the format is prepended to the representation string, either "0b", "0x" or just "0". A conversion will not switch the signedness of the constant; the signed or unsigned data-type associated with the constant, as determined by analysis, is preserved. If the constant is negative, with a signed data-type, the representation string will always start with a '-' character.

There is a special class of registers called context registers whose values have a different affect on analysis and decompilation than described above.

	Context registers are inputs to the disassembly decoding process and directly affect which machine instructions are created.

The value in a context register is examined when Ghidra decodes machine instructions from the underlying bytes in the Program. A specific value generally corresponds to a specific execution mode of the processor. The ARM processor T bit, for instance, which selects whether the processor is executing ARM or THUMB instructions, is modeled as a context register in Ghidra. The same set of bytes in the Program can be decoded to machine instructions in more than one way, depending on context register values.

Bytes are typically decoded once using context register values established at the time of disassembly. From Ghidra's more static view of execution, a context register holds only a single value at any point in the code, but the same context register can hold different values for different regions of code. Setting a new value on a region of the Program will affect any subsequent disassembly of code within that region.

If a context register value is changed for a region that has already been disassembled, in order to see the affect of the change, the machine instructions in the region need to be cleared, and disassembly needs to be triggered again (see the documentation on the Clear Plugin).

Values for a context register are set in the same way as for any other register, using the Set Register Values... command described above. Within the Register Manager window, context registers are generally grouped together under the contextreg pseudo-register heading. For details about how context registers are used in processor modeling, see the document "SLEIGH: A Language for Rapid Processor Specification."

Because context registers affect machine instructions, they also affect the underlying p-code and have a substantial impact on decompilation. Although details vary by processor, context register values are typically established during the initial import and analysis of a Program and aren't changed frequently.