Decrypting HTTPS traffic on Android the easiest way
I recently wanted to analyse traffic from certain mobile apps for a research project and found that intercepting HTTPS traffic on Android isn’t straightforward (certainly for modern versions). During my own research on the subject, I wanted to share my attempts and findings, given there’s a lot of older information around that might not work anymore or lead you down a dead end. I hope any readers find it useful and informative.
Modern Android versions make it harder to do TLS inspection
Since Android 7 you can no longer install system CA certificates onto an Android device “easily”. That’s not to say you can’t, but measures were implemented to prevent it. In addition to this from Android 7 (Nougat) onwards, mobile apps would also only trust CA certificates in the system certificate store and not the user level, unless you specifically configured an app to allow other user certificates, which other than your own app you might have built, this would be unlikely to be configured on any app from the Google Play Store.
The summary of these changes meant you basically needed root access and a writeable /system path to place any required CA certificate with the correct hash name into the system store to overcome the change. This has been the generally accepted solution for years. With root access you can either make your /system partition writable to write whatever CA certificate required, or mount a temporary read-write filesystem over the top of a read-only directory e.g. /system/etc/security/cacerts, copy the original contents and add your own alongside, which is referred to as injecting.
However with Android 14 on the horizon, there’s a further big change around this coming which is something to be aware of. The HTTP ToolKit blog provides some key information and alternatives, but the typical direct root access and writing to a directory is not going to work going forward, so it is something to be bear in mind when Android 14 is released officially.
Why would you want to do HTTPS interception?
A good question! From an Android perspective the reasons to do HTTPS inspection could be:
- Reverse engineering/researching.
- Validating/verification of a mobile app behaviour.
- More visibility into what an app is doing on a network.
HTTPS interception often opens the debate around security and its benefits and drawbacks, but from this perspective, I am approaching it from an ethical/research point of view, not with the intention of compromising someone’s privacy or data.
Emulating Android devices with Android Studio
As you need root access for access to the system certificate store, Android emulators are the easiest way to do HTTPS interception. Android Studio provides the ability to run emulated Android devices through the Android Virtual Device (AVD) Manager. It allows you to quickly emulate different device definitions (hardware profiles) and API levels/Android versions (system images). I’d recommend using an emulator device for HTTPS interception, given the requirements to achieve it will require root access and the installation of a normally not trusted system CA certificate in the system store in some form, having this on your personal device or a device which has other data on it could be somewhat of a security issue, compared to throw away emulator images which you can easily provision, wipe or delete with a click.
Important: For any HTTPS interception project, remember to use system images that do not include the Google Play Store, these system images use a clean image and are locked down to match a normal OEM device so no root access is allowed, which is required for adding any system CA certificate being added.
Injecting the system CA certificate into an Android device with HTTP ToolKit
Like many things related to Android or Linux, there are different tools for the job, there’s multiple HTTP proxy solutions out there such as mitmproxy or Charles Proxy, but one solution stands out as being possibly the lowest barrier to entry regarding typical HTTPS inspection requirements which is HTTP ToolKit.
Going back to the two approaches around the system CA certificate problem, injection is the alternative approach where you technically don’t modify the original filesystem, but instead mount a temporary filesystem over the top of something like a directory you want to modify. This is the method HTTP ToolKit uses to inject its system CA certificate, in order to intercept, decrypt and provide HTTPS inspection functionality.
Some of the key benefits of HTTP ToolKit over similar projects are:
- Automated setup for a variety of applications including Android emulators through
adb(Android Debug Bridge). Essentially providing a one-click man-in-the-middle solution. - HTTP ToolKit makes use the fake VPN approach which allows selective capture of traffic from certain apps. Because of this approach, you do not need to set any proxy settings on the device. This also gets around one common problem of mobile applications ignoring proxy settings (because they certainly can!).
- A friendly UI to view traffic, inspect headers and data. HTTP ToolKit provides a lot of background information around the data intercepted, with formatting and even syntax highlighting where available.
- Scriptable tests/automation features for end-to-end testing or mock tests, with HTTP rewriting capabilities.
- Provides a companion app installed on the target Android device with configurable options to control what traffic is intercepted, with an application list as well as port rules.
The steps to get setup with Android Studio and HTTP ToolKit are:
- Within Android Studio, open the Virtual Device Manager, choose the device definition required (again do not use any Play Store definition). For the system image use the recommended images and select the required Android version required that supports the app you are interested in doing HTTPS interception on, this could be Android 11, 12, 13 etc. If you haven’t ever used a system image before, you may need to download it first, which you can do by clicking the download icon next to the release name. For the configuration just leave everything as default.
- In the Device Manager window, click the play icon to boot up the emulator. Wait for the emulator to start.
- In HTTP ToolKit, go to Intercept and find “Android Device via ADB”, this option should be available when an emulator is running. If it is greyed out, the emulator running might not be ready, wait a bit longer.
- For first time use you’ll need to accept HTTP ToolKit to act as a VPN (the way it intercepts data) as well as trust the system CA certificate injected.
- When using the ADB one-click option, HTTP ToolKit will set all this up for you, so it’s just a case of accepting a couple of prompt messages, a companion app on the emulator device to control the behaviour/confirm things are working.
You can adjust which apps have their traffic captured, allowing you to have selective control through the companion app. This is useful to keep traffic/noise to a minimum if you are interested in a specific mobile app for analysis or inspection, rather than receiving all traffic from the device.
Within the “View” tab in HTTP ToolKit you can then analyse the traffic being intercepted. Here’s an example capture of Google Chrome and the BBC Sport website, running on an emulated Android 12 device.
This view allows for detailed inspection of all requests captured through HTTP ToolKit. One of the major downsides perhaps is saving captures is not possible unless you have a paid subscription, where as other tools like mitmproxy, do not restrict this functionality behind paid features.
Certificate pinning
On the screenshot of the HTTP ToolKit Android app, you might have noticed a note of “Traffic from all intercepted applications will be available, except those using explicit certificate pinning”. What is that?
Certificate pinning is an additional measure of trust related to SSL/TLS certificates and prevents the problem of “trust assumptions” as well as potentially hostile environments where this type of interception/inspection is designed to undermine privacy or compromise security. It is not exclusive to Android but some Android apps use certificate pinning to prevent the exact HTTPS interception demonstrated. The outcome of this is regardless of a system CA certificate being installed, some Android apps will simply refuse to transmit data over what would be deemed a compromised connection, thus defeating any HTTPS inspection attempts.
The good news is, on average most Android apps likely do not implement certificate pinning, but some high profile apps do, notable examples include Facebook, Twitter/X, Instagram, banking apps and others. For web developers, you may remember HTTP Public Key Pinning (HPKP), which is the same concept. HPKP is now considered deprecated, given the high risks of essentially bricking a website application and its associated domain for little benefit. You only have to read the very honest account of what happened at Smashing Magazine to see how dangerous it can be when not configured correctly and events like this likely contributed to its ultimate demise.
It is not impossible to defeat or workaround certificate pinning either, but the methods to do it, start to go down more complicated avenues of command line tools. HTTP ToolKit also has some detailed information on the subject, but it is straying away from the simple one-click approach outlined. However for those interested, read more about certificate pinning on the HTTP ToolKit blog. Be mindful that some app developers have gone further by implementing their own version of TLS, further thwarting any HTTPS decryption. Some developers just don’t want you knowing how their APIs work!
Final thoughts
For your next HTTPS interception project, why not give HTTP ToolKit a try? While not the only tool for the job, from all the guides and resources I’ve read on Android and HTTPS interception, I think HTTP ToolKit is one of the easiest tools to use to get setup. It is also not exclusively just for Android and can work for other projects too with a variety of preconfigured one-click setups.
Granted, it does have some limitations and some features are behind a paid subscription which might be a deal breaker, but give it a try and see what you think!
