Wednesday, November 21, 2012

The Future of Automated Malware Generation

By Stephan Chenette @StephanChenette

This year I gave a series of presentations on "The Future of Automated Malware Generation". This past week the presentation finished its final debut in Tokyo on the 10th anniversary of PacSec.

Hopefully you were able to attend one of the following conferences where it was presented:

If you weren't able to attend one of the above mentioned conferences, I've embedded the slides (below) and will go through the details in this blog (please refer to the slides for additional information).

Motivation / Intro

Much of this presentation was inspired by a number of key motivations:

  1. Greg Hoglund's talk at Blackhat 2010 on malware attribution and fingerprinting
  2. The undeniable steady year by year increase in malware, exploits and exploit kits
  3. My unfinished attempt in adding automatic classification to the cuckoo sandbox
  4. An attempt to clear up the perception by many consumers and corporations that many security products are resistant to simple evasion techniques and contain some "secret sauce" that sets them apart from their competition
  5. The desire to educate consumers and corporations on past, present and future defense and offense techniques
  6. Lastly to help reemphasize the philosophy that when building or deploying defensive technology it's wise to think offensively...and basically try to break what you build
Since the point of the talk is the future of automated malware generation, I'll start with explaining the current state of automated malware generation, and then I'll move to reviewing current defenses found in most products today. 

Given enough time, resources and skill-set, every defense technique can be defeated, to prove this to you I'll share some of the associated offensive techniques. I will then discuss new defense technologies that you'll start to hear more about and then, as has been the cycle in any war, to each defense will come a new offensive technique. So I will then discuss the future of automated malware generation. This is a long blog, but I hope you find it interesting!

Current State of Automated Malware Generation

Automated Malware Generation centers on Malware Distribution Networks (MDNs).

MDNs are organized, distributed networks that are responsible for the entire exploit and infection vector.

There are several players involved:

  • Pay-per-install client - organizations that write malware and gain a profit from having it installed on as many machines as possible
  • Pay-per-install services - organizations that get paid to exploit and infect user machines and in many cases use pay-per-install affiliates to accomplish this
  • Pay-per-install affiliates - organizations that own a lot of  infrastructure and processes necessary to compromise web legitimate pages, redirect users through traffic direction services (TDSs), infect users with exploits (in some cases exploit kits) and finally, if successful, download malware from a malware repository.
Figure: Blackhole exploit kit download chain
Source: Manufacturing Compromise: The Emergence of Exploit-as-a-Service 

There are a number of different types of malware repositories, some that contain the same binary for the life-time of a particular attack campaign, some that periodically update or repackage the binary to avoid and evade simple detection techniques, and polymorphic/metamorphic repositories that produce a unique sample for each user request. More complex attacks generally involve the latter.

Figure: Basic Break-down of Malware Repository Types

Current State of Malware Defense

Most Security desktop and network products on the market today use the following techniques to detect malware:

  • hashes - cryptographic checksums of either the entire malware file or sections of the file, in some cases these could include black-listing and white-listing
  • signatures - syntactical pattern matching using conditional expressions (in some cases format-aware/contextual)
  • heuristics - An expression of characteristics and actions using emulation, API hooking, sand-boxing, file anomalies and/or other analysis techniques
  • semantics - transformation of specific syntax into a single abstract / intermediate representation to match from using more abstract signatures and heuristics

EVERY defense technique can be broken - with enough time, skill and resources.

In the above defensive techniques:

  • hash-based detection can be broken by changing the binary by a single byte
  • signature-based detection be broken using syntax mutation
    • Garbage Code Insertion e.g. NOP, “MOV ax, ax”, “SUB ax 0”
    • Register Renaming e.g. using EAX instead of EBX (as long as EBX isn't already being used)
    • Subroutine Permutation - e.g. changing the order in which subroutines or functions are called as long as this doesn't effect the overall behavior
    • Code Reordering through Jumps e.g. inserting test instructions and conditional and unconditional branching instructions in order to change the control flow
    • Equivalent instruction substitution e.g. MOV EAX, EBX <-> PUSH EBX, POP EAX
  • heuristics-based detection can be broken by avoiding the characteristics the heuristics engine is using or using uncommon instructions that the heuristics engine might be unable to understand in it's emulator (if an emulator is being used)
  • semantics-based detection can be broken by using techniques such as time-lock puzzle (semantics-based detection are unlikely to be used at a higher level such as network defenses due to performance issues) also because implementation requires extensive scope there is a high likelihood that not all cases have been covered. Semantic-based detection is extremely difficult to get right given the performance requirements of a security product.
There are a number of other examples where defense techniques were easily defeated by proper targeted research (generally speaking). Here is a recent post by Trail of Bits only a few weeks ago [Trail of Bits Blog] in their analysis of ExploitSheild's exploitation prevention technology. In my opinion the response from Zero Vulnerability Labs was appropriate [Zero Vulnerability Labs Blog], but it does show that a defense technique can be broken by an attacker if that technology is studied and understood (which isn't that complicated to figure out).

Malware Trends

Check any number of reports and you can see the rise in malware is going up (keep in mind these are vendor reports and have a stake in the results, but being that there really is no other source for the information we'll use them as the accepted experts on the subject) [Symantec] [Trend] [McAfee] [IBM X-Force] [Microsoft] [RSA]

The increase in malware samples has also been said of mobile malware [F-Secure Mobile Threat Report].

Since the rise of malware can't be matched by continually hiring another analyst to analyze malware (this process has its limitations) security companies deploy high-interaction and low-interaction sandboxes. These sandboxes run the malware, analyze its behavior and attempt to trigger various heuristics that will auto-classify the malware by hash. If it's not able to auto-classify then typically the malware is added to a suspicious bucket for a malware analyst to manually review...thus malware analysts are bottle necks in the process of preemptive malware classification.

In addition, a report from Cisco last year found that 33% of Web malware encountered was zero-day malware not detectable by traditional signature-based methodologies at the time of encounter [Cisco 2011 4Q Global Threat Report]

33%!! -- Obviously means there is work to be done on the detection/defense side of the fence.

So how can the security industry use automatic classification? Well, in the last few years a data-driven approach has been the obvious step in the process.

The Future of Malware Defense

With the increase in more malware, exploits, exploit kits, campaign-based attacks, targeted attacks, the reliance on automation will heave to be the future. The overall goal of malware defense has been to a larger degree classification and to a smaller degree clustering and attribution.

Thus statistics and data-driven decisions have been an obvious direction that many of the security companies have started to introduce, either by heavily relying on this process or as a supplemental layer to existing defensive technologies to help in predictive pattern-based analysis and classification.

Where statistics is a discipline that makes you understand data and forces decisions based on data, machine learning is where we train computers to make statistical decisions on real-time data based on inputted data.

While machine learning as a concept has been around for decades, it's only more recently that it's being used in web filtering, data-leakage prevention (DLP), and malware content analysis.

Training machine learning classifiers involves breaking down whatever content you want to analyze e.g. a network stream or an executable file into "features" (basically characteristics).

For example historically certain malware has:
  • No icon
  • No description or company in resource section
  • Is packed
  • Lives in windows directory or user profile

Each of the above qualities/characteristics can be considered "features". Once the defensive technology creates a list of features, it then builds a parser capable of breaking down the content to find those features. e.g. if the content is a PE WIN32 executable, a PE parser will be necessary. The features would include anything you can think of that is characteristic of a PE file.

The process then involves training a classifier on a positive (malicious) and negative (benign) sample set. Once the classifier is trained it can be used to determine if a future unknown sample is benign or malicious and classify it accordingly.

Let me give you a more detailed example: If you've ever played around with malicious PDFs you know there are differences between the structure of a benign PDF and a malicious PDF.

Here are some noteworthy characteristics in the structure of a PDF (FireEye Blog/Presentation - Julia Wolf):
  • Compressed JavaScript
  • PDF header location  e.g %PDF  - within first 1024 bytes
  • Does it contain an embedded file (e.g. flash, sound file)
  • Signed by a trusted certificate
  • Encoded/Encrypted Streams e.g. FlatDecode is used quite a lot in malicious PDFs
  • Names hex escaped
  • Bogus xref table
All the above are features that can be used to feed the classifier during training against benign and malicious sample sets (check out "Scoring PDF structure to detect malicious file" from my friend Rodrigo Montoro (YouTube)

There are two open-source projects that I want to mention using machine learning to determine if a file is malicious:

PDF-XRay from Brandon Dixon:

An explanation of how it works from the pdf-xray site is as follows:

Adobe Open Source Malware Classification Tool by Karthik Raman/Adobe

Details (from website): Perform quick, easy classification of binaries for malware analysis.
Published results: 98.21% accuracy, 6.7% false positive rate
7 features = DebugSize, ImageVersion, IatRVA, ExportSize, ResourceSize, VirtualSize2, NumberOfSections

Personal remarks: This tool is a great proof of concept, but my results weren't as successful as Karthik's  results which I'm told were only on binaries that were not packed, my sample set included packed, unpacked, and files that had never been packed.

Shifting away from analysis of files, we can also attempt to classify shellcode on the wire from normal traffic. Using marchov chains which is a discipline of Artificial Intelligence, but in the realm of natural language processing, we can determine and analyze a network stream of instructions to see if the sequence of instructions are likely to be exploit code.

The below example is attempting to show that most exploit code (shellcode) follows a basic skeleton, be it a decoder loop, decoding a payload and then jumping to that payload or finding the delta, getting the kernel32 imagebase, resolving the addresses for GetProcAddress and LoadLibraryA, calling various functions and finally executing the rest of your payload. 

There are a finite set of published methods to do this, and if you can use semantics, you can further limit the possible sequences and determine if the network stream are instructions and further if those instructions are shellcode.

The Future of Automated Malware Generation

In many cases the path of attack and defense techniques follows the same story of cat and mouse. Just like Tom and Jerry, the chase continues forever, in the context of security, new technology is introduced, new attacks then emerge and in response new countermeasures are brought in to the detection of those attacker's game can come to an end IF they makes a mistake, but whereas cyber-criminal organizations can claim a binary 0 or 1 success or failure, defense can never really claim a victory over all it's attackers. It's a "game" that must always continue.

That being said you'll hear more and more products and security technologies talk about machine learning like it's this unbeatable new move in the game....granted you'll hear it mostly from savvy marketing, product managers or sales folks. In reality it's another useful layer to slow down an attacker trying to get to their end goal, but it's by no means invincible.

Use of machine learning  can be taken circumvented by an attacker in several possible ways:
  • Likelihood of false positives / false negatives due to weak training corpus 
  • Circumvention of classification features
  • Inability to parse/extract features from content
  • Ability to poison training corpus
Let's break down each of those points, because if the next stage of defense will increasingly include machine learning, then attackers will be attempting to include various evasion techniques to avoid this new detection technique.

Likelihood of false positives / false negatives due to weak training corpus
If the defense side creates models based on a small sample set or a sample set that doesn't represent a diverse enough sample set than the model will be too restrictive and thus have false negatives. If a product has too many false-positives, users won't trust it, and if given the choice ignore the results. Products that typically have too many false positives will be discontinued. Attackers can benefit from a weak training corpus by using less popular techniques/vulnerabilities that most likely haven't been used in training and won't be caught by the classifier.

If the defense creates models based only on malicious files and not enough benign files then there will be tons of false positives. Thus, if the attacker models their files to look more representative of good files, there will be a higher likelihood that the acceptable threshold to mitigate false positives will allow the malicious file through.

Circumvention of classification features
At the start of this blog I mentioned that I'm currently attempting to add automatic classification to the cuckoo sandbox, which is an open source behavioral analysis framework. If I were to add such code, it would be open source and any techniques including features would be exposed. Thus, all an attacker would have to do is read my source code, and avoid the features; this is also true for any product that an attacker can buy or demo. They could either read the source code or reverse engineer the product and see which features are being used and attempt to trick the classification algorithm if the threshold/weights/characteristics can be determined. 

Inability to parse/extract features from content
Classification using machine learning is 100% reliant on the fact that the features can be extracted from the content and feed to the classification algorithm, but what if the executable is a .NET binary (Japanese Remote Control Virus) and the engine can't interpret .NET binaries, or if the  format changes, or gets updated e.g. PDF 2.0. For each of these changes, a parser must be built, updated and shipped out. Attackers have the advantage of a window of time between product updates, or again with proper research, an understanding that certain products simply can't handle a particular format in order to extract features.

Ability to poison training corpus
Training a machine learning classifier involves training the algorithm against a known malicious set and a known benign set. If an attacker were able to poison either set, the results and final classification determination would be flawed. This can occur numerous ways. For example: the attacker releases a massive set of files onto the Internet in the off chance that a security product company will use it as its main source of samples, or they poison a number of known malware behavior frameworks such as VirusTotal or malwr, that share samples with security companies, with bogus malware. This scenario is unlikely, because most companies wouldn't rely on one major source for all their testing, but still worth mentioning.


In reality we haven't yet seen malware that contains anti machine learning classification or anit-clustering techniques. What we have seen is more extensive use of on-the-fly symmetric-key encryption where the key isn't hard-coded in the binary itself, but uses something unique about the target machine that is being infected. Take Zeus for example that makes use of downloading an encrypted binary once the machine has been infected where the key is unique to that machine, or Gauss who had a DLL that was encrypted with a key only found on the targeted user's machine.

What this accomplishes is that the binary can only work the intended target machine, it's possible that an emulator would break, but certainly sending it off to home-base or the cloud for behavioral and static analysis will fail, because it simply won't be able to be decrypted and run.

Most defensive techniques if studied, targeted and analyzed can be evaded -- all it takes is time, skill and resources. Using Machine learning to detect malicious executables, exploits and/or network traffic are no exception. At the end of the day it's important that you at least understand that your defenses are penetrable, but that a smart layered defense is key, where every layer forces the attackers to take their time, forces them to learn new skills and slowly gives away their resources, position and possibly intent -- hopefully giving you enough time to be notified of the attack and cease it before ex-filtration of data occurs. What a smart layered defense looks like is different for each network depending on where your assets are and how your network is set up, so there is no way for me to share a one-size fits all diagram, I'll leave that to you to think about.

Useful Links:
Coursera - Machine Learning Course
CalTech - Machine Learning Course
PyML (
Milk (
Shogun ( Code is in C++ but it has a python wrapper.
MDP ( Python library for data mining
PyBrain (
Orange ( Statistical computing and data mining
scikit-learn ( Numpy / Scipy / Cython implementations for major algorithms + efficient C/C++ wrappers
Monte ( a software for gradient-based learning in Python
Rpy2 ( Python wrapper for R

About Stephan
Stephan Chenette has been involved in computer security professionally since the mid-90s, working on vulnerability research, reverse engineering, and development of next-generation defense and attack techniques. As a researcher he has published papers, security advisories, and tools. His past work includes the script fragmentation exploit delivery attack and work on the open source web security tool Fireshark.

Stephan is currently the Director of Security Research and Development at IOActive, Inc.
Twitter: @StephanChenette

Wednesday, November 7, 2012

Hacking an Android Banking Application

Guest post by Juliano Rizzo @julianor

This analysis of a mobile banking application from X bank illustrates how easily anyone with sufficient knowledge can get install and analyze the application, bypassing common protections.

1. Installing and unpacking the application

Only users located in Wonderland can install the X Android application with Google Play, which uses both the phone’s SIM card and IP address to determine the location of the device. To bypass this limitation, remove the SIM card and reset the phone to factory defaults.

Complete the initial Android setup with a Wonderland IP address, using an L2TP VPN service (PPTP encryption support is broken). If Google Play recognizes the device as located in Wonderland, install the application. Once installed in a rooted phone, copy the application APK from /data/app/

These are some of the many reversing tools for unpacking and decompiling the APK package:


In this example, the code was decompiled using jd-gui after converting the apk file to jar using the dex2jar tool. The source code produced by jd-gui from Android binary files is not perfect, but is a good reference. The output produced by the baksmali tool is more accurate but difficult to read. The smali code was modified and re-assembled to produce modified versions of the application with restrictions removed. A combination of decompiled code review, code patching, and network traffic interception was used to analyze the application.

2. Bypassing the SMS activation

The application, when started for the first time, asks for a Wonderland mobile number. Activating a new account requires an activation code sent by SMS. We tested three different ways to bypass these restrictions, two of which worked. We were also able to use parts of the application without a registered account.

2.1 Intercepting and changing the activation code HTTPS request

The app GUI only accepts cell phone numbers with the 0X prefix; always adding the +XX prefix before requesting an activation code from the web service at

Intercepting the request and changing the phone number didn't work, because the phone prefix is verified on the server side.

2.2 Editing shared preferences

The app uses the “Shared Preferences” Android service to store its minimal configuration. The configuration can be easily modified in a rooted phone by editing the file


Set the "activated" preference to "true" like this:

<?xml version='1.0' encoding='utf-8' standalone='yes' ?>
<boolean name="welcome_screen_viewed " value="true" />
<string name="msisdn">09999996666</string>
<boolean name="user_notified" value="true" />
<boolean name="activated " value="true" />
<string name="guid" value="" />
<boolean name="passcode_set" value="true" />
<int name="version_code" value="202" />

2.3 Starting the SetPasscode activity

The Android design allows the user to bypass the normal startup and directly start activities exported by applications. The decompiled code shows that the SetPasscode activity can be started after the activation code is verified. Start the SetPasscode activity using the “am” tool as follows:

From the adb root shell:

#am start -a android.intent.action.MAIN -n

3. Intercepting HTTPS requests

To read and modify the traffic between the app and the server, perform a SSL/TLS MiTM attack. We weren’t able to create a CA certificate and install it using the Android's user interface with the Android version we used for testing. Android ignores CA certificates added by the user. Instead, we located and then modified the app's http client code to make it accept any certificate. Then we installed the modified APK file on the phone. Using iptables on the device, we could redirect all the HTTPS traffic to an MiTM proxy to intercept and modify requests sent by the application.

4. Data storage
The app doesn't store any data on external storage (SD card), and it doesn't use the SQLlite database. Preferences are stored on the "Shared Preferences" XML file. Access to the preferences is restricted to the application. During a review of the decompiled code, we didn't find any evidence of sensitive information being stored in the shared preferences.

5. Attack scenario: Device compromised while the app is running.

The X android application doesn't store sensitive information on the device. In the event of theft, loss or remote penetration, the auto lockout mechanism reduces the risk of unauthorized use of the running application. When the app is not being used or is running in background, the screen is locked and the passcode must be entered to initiate a new session. The HTTPS session cookie expires after 300 seconds. The current session cookie is removed from memory when the application is locked.

5.1 Attacker with root privileges
An attacker with root access to the device can obtain the GUID and phone number  from the unencrypted XML configuration file; but without the clear-text or encrypted passcode, mobile banking web services cannot be accessed. We have discovered a way to “unlock” the application using the Android framework to start the AccountsFragmentActivity activity, but if the web session has already expired, the app limits the attacker to a read-only access. The most profitable path for the attacker at this point is the running process memory dump, as we explain in the next section.

5.2 Memory dump analysis.

An attacker who gets root access can acquire the memory of a running Android application in several different ways. “Acquisition and Analysis of Volatile Memory from Android Devices,” published in the Digital Investigation Journal, describes some of the different methods.
We used the ddms (Dalvik Debug Monitor Services) tool included with the Android SDK to acquire the memory dump. The Eclipse Memory Analyzer tool with Object Query Language support is one of the most powerful ways to explore the application memory. The application passcodes are short numeric strings; a simple search using regular expressions returns too many occurrences. The attacker can't try every occurrence of numeric strings in the process memory as a passcode for the app web services because the account is blocked after a few attempts. The HTTPS session cookies are longer strings and easier to find.

By searching for the prefix “JSESSION,” the attacker can easily locate the cookies when the application is running and active. However, the cookies are removed from memory after a timeout. The ActivityTimeoutTracker function calls the method clear() of the HashMap used to store the cookies. The cookie HashMap is accessed through the singleton class com.X.a.a.e.b.


Reviewing the decompiled code, we located a variable where the passcode is stored before being encrypted to perform the session initiation request. The field String f of the class com.X.a.a.d.a contains the passcode, and it is never overwritten or released. The references to the instance prevent the garbage collection of the string. Executing the OQL query “SELECT toString(object.f) FROM com.X.a.a.d.a object” inside the Eclipse Memory Analyzer is sufficient to reliably locate the passcode in the application memory dump.

Although developers tried to remove information from the memory to prevent this type of attack, they left the most important piece of information unprotected.

6. Attack scenario: Perform MiTM using a compromised CA
The banking application validates certificates against all the CA certificates that ship with Android. Any single compromised CA in the system key store can potentially compromise communication between the app and the backend. An active network attacker can hijack the connection between the mobile app and the server to impersonate the user.

Mobile apps making SSL/TLS connections to a service controlled by the vendor don’t need to trust Certificate Authorities signatures. The app could therefore implement certificate "pinning" or distribute a signing certificate created by the vendor.

The authentication protocol is vulnerable to MiTM attacks. The app's authentication protocol uses the RSA and 3DES algorithms to encrypt the passcode before sending it to the server. After the user types the passcode and presses the "login" button, the client retrieves an RSA public key from the server without having to undergo any authenticity check, which allows for an MiTM attack. We were able to implement an attack and capture the passcodes from the app in our testing environment. Although the authentication protocol implemented by the app is insecure, attacks are prevented by the requirement of SSL/TLS for every request. Once someone bypasses the SSL/TLS certificate verification, though, the encryption of the passcode doesn't provide extra protection.

7. Attack scenario: Enumerate users

The web service API allows user enumeration. With a simple JSON request, attackers can determine if a given phone number is using the service, and then use this information to guess passwords or mount social engineering attacks.


About Juliano

Juliano Rizzo has been involved in computer security for more than 12 years, working on vulnerability research, reverse engineering, and development of high quality exploits for bugs of all classes. As a researcher he has published papers, security advisories, and tools. His recent work includes the ASP.NET “padding oracle” exploit, the BEAST attack, and the CRIME attack. Twitter: @julianor

Thursday, November 1, 2012

iOS Security: Objective-C and nil Pointers

By Shaun Colley

iOS devices are everywhere now. It seems that pretty much every other person has one…an iPhone, iPad or iPod touch - and they're rivaled in popularity only by Android devices.

If you do secure code review, chances are that with the explosion in the number of iOS apps, you may well have done a source code review of an iOS app, or at least played around with some Objective-C code. Objective-C can be a little strange at first for those of us who are used to plain C and C++ (i.e. all the square brackets!), and even stranger for Java coders, but after a while most of us realise that the standard iOS programming environment is packed with some pretty powerful Cocoa APIs, and Objective-C itself actually has some cool features as well. The runtime supports a number of quirky features as well, which people tend to learn shortly after the standard "Hello, World!" stuff…

Objective-C brings a few of its own concepts and terminology into the mix as well. Using Objective-C syntax, we might call a simple method taking one integer parameter like this:

 returnVal = [someObject myMethod:1234];

In other object-oriented languages like C++ and Java we'd normally just refer to this as "calling a member function" or "calling a method", or even "calling a method on an object". However, Objective-C differs slightly in this respect and as such, you do not call methods - instead, you "send a message to an object". This concept as a whole is known as 'message passing'.

The net result is the same - the 'myMethod' method associated with someObject's class is called, but the semantics in how the runtime calls the method is somewhat different to how a C++ runtime might.

Whenever the ObjC compiler sees a line of code such as "[someObject myMethod]", it inserts a call to one of the objc_msgSend(_XXX) APIs with the "receiver" (someObject) and the "selector" ("myMethod:") as parameters to the function. This family of functions will, at runtime, figure out which piece of code needs to be executed bearing in mind the object's class, and then eventually JMPs to it. This might seem a bit long-winded, but since the correct method to call is determined at runtime, this is part of how Objective-C gets its dynamism from.

The call above may end up looking something roughly like this, after the compiler has dealt with it:

 objc_msgSend(someObject, "myMethod:", 1234);

The version of objc_msgSend that is actually called into depends on the return type of the method being called, so accordingly there are a few versions of the interface in the objc_msgSend family.
For example, objc_msgSend() (for most return types), objc_msgSend_fpret() (for floating point return values), and objc_msgSend_stret(), for when the called method returns a struct type.

But what happens if you attempt to message a nil object pointer? Everyone who plays around with Objective-C code long enough soon realises that calling a method on a nil object pointer - or, more correctly, "messaging" a nil object pointer - is perfectly valid. So for example:

someObject = nil;
[someObject myMethod];

is absolutely fine. No segmentation fault - nothing. This is a very deliberate feature of the runtime, and many ObjC developers in fact use this feature to their advantage. You may end up with a nil object pointer due to an object allocation failure (out of memory), or some failure to find a substring inside a larger string, for example…


 MyClass myObj = [[MyClass alloc] init]; // out-of-memory conditions give myObj == nil

 In any case, however an object pointer got to be nil, there are certain coding styles that allow a developer to use this feature perfectly harmlessly, and even for profit. However, there are also ways that too-liberal use of the feature can lead to bugs - both functionally and security-wise.

One thing that needs to be considered is, what do objc_msgSend variants return if the object pointer was indeed found to be nil? That is, we have have

 myObj = nil;
 someVariable = [myObj someMethod];

What will someVariable be equal to? Many developers assume it will always be some form of zero - and often they would be correct - but the true answer actually depends on the type of value that someMethod is defined to return.  Quoting from Apple's API documentation:

- If the method returns any pointer type, any integer scalar of size less than or equal to sizeof(void*), a float, a double, a long double, or a long long, then a message sent to nil returns 0.

- If the method returns a struct, as defined by the OS X ABI Function Call Guide to be returned in registers, then a message sent to nil returns 0.0 for every field in the struct. Other struct data types will not be filled with zeros.

- If the method returns anything other than the aforementioned value types, the return value of a message sent to nil is undefined.

The second line above looks interesting. The rule on the second line deals with methods that return struct types, for which the objc_msgSend() variant called in these cases will be the objc_msgSend_stret() interface.. What the above description is basically saying is, if the struct return type is larger than the width of the architecture's registers (i.e. must be returned via the stack), if we call a struct-returning method on a nil object pointer, the ObjC runtime does NOT guarantee that our structure will be zeroed out after the call. Instead, the contents of the struct are undefined!

When structures to be "returned" are larger than the width of a register, objc_msgSend_stret() works by writing the return value into the memory area specified by the pointer passed to objc_msgSend_stret(). If we take a look in Apple's ARM implementation of objc_msgSend_stret() in the runtime[1], which is coded in pure assembly, we can see how it is indeed true that the API does nothing to guarantee us a nicely 0-initialized struct return value:

 * struct_type    objc_msgSend_stret(id    self,
 *                SEL    op,
 *                    ...);
 * objc_msgSend_stret is the struct-return form of msgSend.
 * The ABI calls for a1 to be used as the address of the structure
 * being returned, with the parameters in the succeeding registers.
 * On entry: a1 is the address where the structure is returned,
 *           a2 is the message receiver,
 *           a3 is the selector

    ENTRY objc_msgSend_stret
# check whether receiver is nil
    teq     a2, #0
    bxeq    lr

If the object pointer was nil, the function just exits…no memset()'ing to zero - nothing, and the "return value" of objc_msgSend_stret() in this case will effectively be whatever was already there in that place on the stack i.e. uninitialized data.

Although I'll expand more later on the possible security consequences of getting
undefined struct contents back, most security people are aware that undefined/uninitialized data can lead to some interesting security bugs (uninitialized pointer dereferences, information leaks, etc).

So, let's suppose that we have a method 'myMethod' in MyClass, and an object pointer of type MyClass that is equal to nil, and we accidentally attempt to call
the myMethod method on the nil pointer (i.e. some earlier operation failed), we have:

 struct myStruct {
       int myInt;
     int otherInt;
     float myFloat;
     char myBuf[20];

 [ … ]

 struct myStruct returnStruct;

 myObj = nil;
 returnStruct = [myObj myMethod];

 Does that mean we should definitely expect returnStruct, if we're running on our ARM-based iPhone, to be full of uninitialized junk?

 Not always. That depends on what compiler you're using, and therefore, in pragmatic terms, what version of Xcode the iOS app was compiled in.

 If the iOS app was compiled in Xcode 4.0 or earlier, where the default compiler is GCC 4.2[2], messaging nil with struct return methods does indeed result in undefined structure contents, since there is nothing in the runtime nor the compiler-generated assembly code to zero out the structure in the nil case.

However, if the app was compiled with LLVM-GCC 4.2 (Xcode 4.1) or Apple LLVM (circa Xcode 4.2), the compiler inserts assembly code that does a nil check followed by a memset(myStruct, 0x00, sizeof(*myStruct)) if the object pointer was indeed nil, adjacent to all objc_msgSend_stret() calls.

Therefore, if the app was compiled in Xcode 4.1 or later (LLVM-GCC 4.2 or Apple LLVM), messaging nil is *guaranteed* to *always* result in zeroed out structures upon return - so long as the default compiler for that Xcode release is used.. Otherwise, i.e. Xcode 4.0, the struct contents are completely undefined.

These two cases become apparent by comparing the disassemblies for calls to objc_msgSend_stret() as generated by 1) GCC 4.2, and 2) Apple LLVM. See the IDA Pro screen dumps below.

 Figure 1 - objc_msgSend_stret() with GCC 4.2

Figure 2 - objc_msgSend_stret() with Apple LLVM

 Figure 1 clearly shows objc_msgSend_stret() being called whether the object pointer is nil or not, and upon return from the function memcpy() is used to copy the "returned"  struct data into the place we asked the structure to be returned to, i.e. our struct on stack. If the object pointer was nil, objc_msgSend_stret() just exits and ultimately this memcpy() ends up filling our structure with whatever happened to be there on the stack at the time…

In Figure 2, on the other hand, we see that the ARM 'CBZ' instruction is used to test the object pointer against 0 (nil) before the objc_msgSend_stret() call, with the memset()-to-0 code path instead being taken if the pointer was indeed nil. This guarantees that in the case of the objective pointer being nil, the structure will be completely zeroed.

Thus, summed up, any iOS applications released before July 2011 are extremely likely to be vulnerable, since they were almost certainly compiled with GCC. Apps built with Xcode 4.1 and up are most likely not vulnerable. But we have to bear in mind that a great deal of developers in real-world jobs do not necessarily update their IDE straightaway, regularly, or even at all (ever heard of corporate policy?). By all accounts, it's probable that vulnerable apps (i.e. Xcode 4.0) are still being released on the App Store today.

 It's quite easy to experiment with this yourself with a bit of test code. Let's write some code that demonstrates the entire issue. We can define a class called HelloWorld, and the class contains one method that returns a 'struct teststruct' value; and the method it simply puts a ton of recognisable data into an instance of 'teststruct', before returning it. The files in the class definition look like this:


#import "hello.h"

@implementation HelloWorld

- (struct teststruct)sayHello
   // NSLog(@"Hello, world!!\n\n");
    struct teststruct testy;
    testy.testInt = 1337;
    testy.testInt2 = 1338;
    testy.inner.test1 = 1337;
    testy.inner.test2 = 1337;
    testy.testInt3 = 1339;
    testy.testInt4 = 1340;
    testy.testInt5 = 1341;
    testy.testInt6 = 1341;
    testy.testInt7 = 1341;
    testy.testInt8 = 1341;
    testy.testInt9 = 1341;
    testy.testInt10 = 1341;
    testy.testFloat = 1337.0;
    testy.testFloat1 = 1338.1;
    testy.testLong1 = 1337;
    testy.testLong2 = 1338;
    strcpy((char *)&testy.testBuf, "hello world\n");
    return testy;



#import <Foundation/Foundation.h>

@interface HelloWorld : NSObject {
    // no instance variables

// methods
- (struct teststruct)sayHello;


struct teststruct {
    int testInt;
    int testInt2;
    struct {
        int test1;
        int test2;
    } inner;
    int testInt3;
    int testInt4;
    int testInt5;
    int testInt6;
    int testInt7;
    int testInt8;
    int testInt9;
    int testInt10;
    float testFloat;
    float testFloat1;
    long long testLong1;
    long long testLong2;
    char testBuf[20];

We can then write a bit of code in main() that allocates and initializes an object of class HelloWorld, calls sayHello, and prints the values it received back. Then, let's set the object pointer to nil, attempt to call sayHello on the object pointer again, and then print out the values in the structure that we received that time around. We'll use the following code:

#import <UIKit/UIKit.h>
#import <malloc/malloc.h>
#import "AppDelegate.h"
#import "hello.h"
#import "test.h"
#include <stdio.h>
#include <stdlib.h>

int main(int argc, char *argv[])
    struct teststruct testStructure1;
    struct teststruct testStructure2;
    struct teststruct testStructure3;
    struct otherstruct otherStructure;
    HelloWorld *hw = [[HelloWorld alloc] init];
    TestObj *otherObj = [[TestObj alloc] init];
    testStructure1 = [hw sayHello];
    /* what did sayHello return? */
    NSLog(@"\nsayHello returned:\n");
    NSLog(@"testInt = %d\n", testStructure1.testInt);
    NSLog(@"testInt = %d\n", testStructure1.testInt2);
    NSLog(@"testInt = %d\n", testStructure1.testInt3);
    NSLog(@"testInt = %d\n", testStructure1.testInt4);
    NSLog(@"testInt = %d\n", testStructure1.testInt5);
    NSLog(@"testInt = %d\n", testStructure1.testInt6);
    NSLog(@"testInt = %d\n", testStructure1.testInt7);
    NSLog(@"testInt = %d\n", testStructure1.testInt8);
    NSLog(@"testInt = %d\n", testStructure1.testInt9);
    NSLog(@"testInt = %d\n", testStructure1.testInt10);
    NSLog(@"testInt = %5.3f\n", testStructure1.testFloat);
    NSLog(@"testInt = %5.3f\n", testStructure1.testFloat1);
    NSLog(@"testInt = %d\n", testStructure1.testLong1);
    NSLog(@"testInt = %d\n", testStructure1.testLong2);
    NSLog(@"testBuf = %s\n", testStructure1.testBuf);
    /* clear the struct again */
    memset((void *)&testStructure1, 0x00, sizeof(struct teststruct));
    hw = nil;  // nil object ptr
    testStructure1 = [hw sayHello];  // message nil
    /* what are the contents of the struct after messaging nil? */
    NSLog(@"\n\nafter messaging nil, sayHello returned:\n");
    NSLog(@"testInt = %d\n", testStructure1.testInt);
    NSLog(@"testInt = %d\n", testStructure1.testInt2);
    NSLog(@"testInt = %d\n", testStructure1.testInt3);
    NSLog(@"testInt = %d\n", testStructure1.testInt4);
    NSLog(@"testInt = %d\n", testStructure1.testInt5);
    NSLog(@"testInt = %d\n", testStructure1.testInt6);
    NSLog(@"testInt = %d\n", testStructure1.testInt7);
    NSLog(@"testInt = %d\n", testStructure1.testInt8);
    NSLog(@"testInt = %d\n", testStructure1.testInt9);
    NSLog(@"testInt = %d\n", testStructure1.testInt10);
    NSLog(@"testInt = %5.3f\n", testStructure1.testFloat);
    NSLog(@"testInt = %5.3f\n", testStructure1.testFloat1);
    NSLog(@"testInt = %d\n", testStructure1.testLong1);
    NSLog(@"testInt = %d\n", testStructure1.testLong2);
    NSLog(@"testBuf = %s\n", testStructure1.testBuf);


OK - let's first test it on my developer provisioned iPhone 4S, by compiling it in Xcode 4.0 - i.e. with GCC 4.2 - since that is Xcode 4.0's default iOS compiler. What do we get?

2012-11-01 21:12:36.235 sqli[65340:b303]
sayHello returned:
2012-11-01 21:12:36.237 sqli[65340:b303] testInt = 1337
2012-11-01 21:12:36.238 sqli[65340:b303] testInt = 1338
2012-11-01 21:12:36.238 sqli[65340:b303] testInt = 1339
2012-11-01 21:12:36.239 sqli[65340:b303] testInt = 1340
2012-11-01 21:12:36.239 sqli[65340:b303] testInt = 1341
2012-11-01 21:12:36.240 sqli[65340:b303] testInt = 1341
2012-11-01 21:12:36.241 sqli[65340:b303] testInt = 1341
2012-11-01 21:12:36.241 sqli[65340:b303] testInt = 1341
2012-11-01 21:12:36.242 sqli[65340:b303] testInt = 1341
2012-11-01 21:12:36.243 sqli[65340:b303] testInt = 1341
2012-11-01 21:12:36.244 sqli[65340:b303] testInt = 1337.000
2012-11-01 21:12:36.244 sqli[65340:b303] testInt = 1338.100
2012-11-01 21:12:36.245 sqli[65340:b303] testInt = 1337
2012-11-01 21:12:36.245 sqli[65340:b303] testInt = 1338
2012-11-01 21:12:36.246 sqli[65340:b303] testBuf = hello world

2012-11-01 21:12:36.246 sqli[65340:b303]

after messaging nil, sayHello returned:
2012-11-01 21:12:36.247 sqli[65340:b303] testInt = 1337
2012-11-01 21:12:36.247 sqli[65340:b303] testInt = 1338
2012-11-01 21:12:36.248 sqli[65340:b303] testInt = 1339
2012-11-01 21:12:36.249 sqli[65340:b303] testInt = 1340
2012-11-01 21:12:36.249 sqli[65340:b303] testInt = 1341
2012-11-01 21:12:36.250 sqli[65340:b303] testInt = 1341
2012-11-01 21:12:36.250 sqli[65340:b303] testInt = 1341
2012-11-01 21:12:36.251 sqli[65340:b303] testInt = 1341
2012-11-01 21:12:36.252 sqli[65340:b303] testInt = 1341
2012-11-01 21:12:36.252 sqli[65340:b303] testInt = 1341
2012-11-01 21:12:36.253 sqli[65340:b303] testInt = 1337.000
2012-11-01 21:12:36.253 sqli[65340:b303] testInt = 1338.100
2012-11-01 21:12:36.254 sqli[65340:b303] testInt = 1337
2012-11-01 21:12:36.255 sqli[65340:b303] testInt = 1338
2012-11-01 21:12:36.256 sqli[65340:b303] testBuf = hello world

Quite as we expected, we end up with a struct full of what was already there in the return position on the stack - and this just happened to be the return value from the last call to sayHello. In a complex app, the value would be somewhat unpredictable.

And now let's compile and run it on my iPhone using Xcode 4.5, where I'm using its respective default compiler - Apple LLVM. The output:

2012-11-01 21:23:59.561 sqli[65866:b303]
sayHello returned:
2012-11-01 21:23:59.565 sqli[65866:b303] testInt = 1337
2012-11-01 21:23:59.566 sqli[65866:b303] testInt = 1338
2012-11-01 21:23:59.566 sqli[65866:b303] testInt = 1339
2012-11-01 21:23:59.567 sqli[65866:b303] testInt = 1340
2012-11-01 21:23:59.568 sqli[65866:b303] testInt = 1341
2012-11-01 21:23:59.569 sqli[65866:b303] testInt = 1341
2012-11-01 21:23:59.569 sqli[65866:b303] testInt = 1341
2012-11-01 21:23:59.570 sqli[65866:b303] testInt = 1341
2012-11-01 21:23:59.571 sqli[65866:b303] testInt = 1341
2012-11-01 21:23:59.572 sqli[65866:b303] testInt = 1341
2012-11-01 21:23:59.572 sqli[65866:b303] testInt = 1337.000
2012-11-01 21:23:59.573 sqli[65866:b303] testInt = 1338.100
2012-11-01 21:23:59.574 sqli[65866:b303] testInt = 1337
2012-11-01 21:23:59.574 sqli[65866:b303] testInt = 1338
2012-11-01 21:23:59.575 sqli[65866:b303] testBuf = hello world

2012-11-01 21:23:59.576 sqli[65866:b303]

after messaging nil, sayHello returned:
2012-11-01 21:23:59.577 sqli[65866:b303] testInt = 0
2012-11-01 21:23:59.577 sqli[65866:b303] testInt = 0
2012-11-01 21:23:59.578 sqli[65866:b303] testInt = 0
2012-11-01 21:23:59.578 sqli[65866:b303] testInt = 0
2012-11-01 21:23:59.579 sqli[65866:b303] testInt = 0
2012-11-01 21:23:59.579 sqli[65866:b303] testInt = 0
2012-11-01 21:23:59.580 sqli[65866:b303] testInt = 0
2012-11-01 21:23:59.581 sqli[65866:b303] testInt = 0
2012-11-01 21:23:59.581 sqli[65866:b303] testInt = 0
2012-11-01 21:23:59.582 sqli[65866:b303] testInt = 0
2012-11-01 21:23:59.582 sqli[65866:b303] testInt = 0.000
2012-11-01 21:23:59.673 sqli[65866:b303] testInt = 0.000
2012-11-01 21:23:59.673 sqli[65866:b303] testInt = 0
2012-11-01 21:23:59.674 sqli[65866:b303] testInt = 0
2012-11-01 21:23:59.675 sqli[65866:b303] testBuf =

Also just as we expected; the Apple LLVM built version gives us all zeroed struct fields, as the compiler-inserted memset() call guarantees us a zeroed struct when we message nil.

Now, to be pragmatic, what are some potential security consequences of us getting junk, uninitialized data back, in real-world applications?

One possible scenario is to consider if we had a method, say, returnDataEntry, that, for example, returns a struct containing some data and a pointer. We could make the scenario more detailed, but for argument's sake let's just assume the structure holds some data and a pointer to some more data.

Consider the following code fragment, in which the developer knows they'll receive a zeroed structure from returnDataEntry if the someFunctionThatCanFail()
call fails:

 struct someData {
    int someInt;
    char someData[50];
    void *myPointer;

[ … ]

 - (struct someData)returnDataEntry

    struct someData myData;
    memset((void *)&myData, 0x00, sizeof(struct someData)); /* zero it out */

    if(!someFunctionThatCanFail()) {  /* can fail! */
            /* something went wrong, return the zeroed struct */
        return myData;

    /* otherwise do something useful */
    myData = someUsefulDataFunction();
    return myData;

In the error case, the developer knows that they can check the contents of the struct against 0 and therefore know if returnDataEntry ran successfully.


  myData = [myObj returnDataEntry];
  if(myData.myPointer == NULL) {
        /* the method failed */

  /* otherwise, use the data and pointer */

 However, if we suppose that the 'myObj' pointer was nil at the time of the returnDataEntry call, and our app was built with a vulnerable version of Xcode, the returned structure will be uninitialized, and myData.myPointer could be absolutely anything, so at this point, we have a dangling pointer and, therefore, a security bug.

Equally, what if some method is declared to return a structure, and that data is later sent to a remote server over the network? A scenario like this could easily result in information leaks, and it's easy to see how that's bad.

Lastly, which is also quite interesting, let's consider some Cocoa APIs that take structs and process them. We'll take a bog standard structure - NSDecimal, for example. The NSDecimal structure is defined as:

typedef struct {
    signed   int _exponent:8;
    unsigned int _length:4;     // length == 0 && isNegative -> NaN
    unsigned int _isNegative:1;
    unsigned int _isCompact:1;
    unsigned int _reserved:18;
    unsigned short _mantissa[NSDecimalMaxSize];
} NSDecimal;

It's pretty obvious by those underscores that all fields in NSDecimal are 'private' - that is, they should not be directly used or modified, and their semantics are subject to change if Apple sees fit. As such, NSDecimal structures should only be used and manipulated using official NSDecimal APIs. There's even a length field, which could be interesting.

The fact that all fields in NSDecimal are documented as being private[3] starts to make me wonder whether the NSDecimal APIs are actually safe to call on malformed NSDecimal structs. Let's test that theory out.

Let's assume we got a garbage NSDecimal structure back from messaging a nil object at some earlier point in the app, and then we pass this NSDecimal struct to Cocoa's NSDecimalString() API. We could simulate the situation with a bit of code like this:

  NSDecimal myDecimal;
   /* fill the two structures with bad data */
   memset(&myDecimal, 0x99, sizeof(NSDecimal));

   NSLocale *usLocale = [[NSLocale alloc] initWithLocaleIdentifier:@"en_US"];

   NSDecimalString(&myDecimal, usLocale);

What happens?

If we quickly just run this in the iOS Simulator (x86), we crash with a write access violation at the following line in NSDecimalString():

<+0505>  mov    %al,(%esi,%edx,1)

(gdb) info reg esi edx
esi            0xffffffca    -54
edx            0x38fff3f4    956298228

Something has clearly gone wrong here, since there's no way that address is going to be mapped and writable…

It turns out that the above line of assembly is part of a loop which uses length values derived from
the invalid values in our NSDecimal struct. Let's set a breakpoint at the line above our crashing line, and see what things look like at the first hit of the
breakpoint, and then, at crash time.

0x008f4275  <+0499>  mov    -0x11c(%ebp),%edx
0x008f427b  <+0505>  mov    %al,(%esi,%edx,1)

(gdb) x/x $ebp-0x11c
0xbffff3bc:    0xbffff3f4

So 0xbffff3f4 is the base address of where the loop is copying data to. And after the write AV, i.e. at crash time, the base pointer looks like:

(gdb) x/x $ebp-0x11c
0xbffff3bc:    0x38fff3f4

Thus after a little analysis, it becomes apparent that the root cause of the crash is stack corruption - the most significant byte of the base destination address is being overwritten (with a 0x38 byte) on the stack during the loop. This is at least a nods towards several Cocoa APIs not being designed to deal with malformed structs with "private" fields. There are likely to be more such cases, considering the sheer size of Cocoa.

Although NSDecimalString() is where the crash occurred, I wouldn't really consider this a bug in the API per se, since it is well-documented that members of NSDecimal structs
are private. This could be considered akin to memory corruption bugs caused by misuse of strcpy() - the bug isn't really in the API as such - it's doing what is was designed to do - it's the manner in which you used it that constitutes a bug.

Interestingly, it seems to be possible to detect which compiler an app was built with by running a strings dump on the Info.plist file found in an app's IPA bundle.

Apple LLVM

sh-3.2# strings Info.plist | grep compiler


sh-3.2# strings Info.plist | grep compiler


sh-3.2# strings Info.plist | grep compiler

What are the take home notes here? Basically, if you use a method that returns a structure type, check the object against nil first! Even if you know YOU'RE not going to be using a mid-2011 version
of Xcode, if you post your library on GitHub or similar, how do you know your code is not going to go into a widely used banking product, for example? - the developers for which may still be using a slightly older version of Xcode, perhaps even due to corporate policy.

It'd therefore be a decent idea to include this class of bugs on your secure code review checklist for iOS applications.

Thanks for reading.





P.S. I've heard that Objective-C apps running on PowerPC platforms can also be vulnerable to such bugs - except with other return types as well, such as float and long long. But I can't confirm that, since I don't readily have access to a system running on the PowerPC architecture.