androidhooker / hooker Goto Github PK

Hooker is an opensource project for dynamic analyses of Android applications. This project provides various tools and applications that can be use to automaticaly intercept and modify any API calls made by a targeted application.

License: GNU General Public License v3.0

Java 49.03% Shell 1.24% Python 49.73%

hooker's Introduction

Hooker: Automated Dynamic Analysis of Android Applications

About Hooker

Functional Description

Hooker is an opensource project for dynamic analyses of Android applications. This project provides various tools and applications that can be use to automatically intercept and modify any API calls made by a targeted application.

It leverages Android Substrate framework to intercept these calls and aggregate all their contextual information (parameters, returned values, ...). Collected information can either be stored in a ElasticSearch or in JSON files.

A set of python scripts is also provided to automatize the execution of an analysis to collect any API calls made by a set of applications.

Disclaimer

Android-Hooker is a proof of concept relying on the Substrate framework. That means Hooker cannot work if Substrate is not correctly installed on your device. For the moment, the authors have successfully installed Substrate on devices with Android versions 4.1 and 4.2. If you know how to install Substrate on higher versions, please let us know by email at [email protected] and will be glad to integrate this into the project.

Technical Description

Hooker is made of multiple modules:

APK-instrumenter is an Android application that must be installed prior to the analysis on an Android device (for instance, an emulator).
hooker_xp is a python tool that can be use to control the android device and trigger the installation and stimulation of an application on it.
hooker_analysis is a python script that can be use to collect results stored in the elasticsearch database.
tools/APK-contactGenerator is an Android application that is automatically installed on the Android device by hooker_xp to inject fake contact informations.
tools/apk_retriever is a Python tool that can be use to download APKs from various online public Android markets.
tools/emulatorCreator is a collection of scripts that can be use to prepare an emulator.

More Information

Website: https://github.com/AndroidHooker/hooker
FAQ is available here
Bug Tracker: Bug and feature requests are organized in GitHub Issues
Email: [email protected]
Twitter: Follow authors account (@Tibapbedoum and @Lapeluche)

Getting Started

We developped Hooker using a Debian 64-bits system and as so, it may fail to execute properly on other systems due to improper paths or parameters. Your help to identify those incompatibilities is highly appreciated. Please report an issue in our Bug Tracker if you meet any error while using it.

In order to use Hooker you need at least one server on which you've installed:

python 2.7,
elasticsearch 1.7,
kibana 4.1,
Android 4.1 and 4.2,
androguard 1.9.

Setup your ElasticSearch Host

This step is related to the ElasticSearch installation. Please download and follow ElasticSearch online documentation: http://www.elasticsearch.org/overview/elkdownloads/. You can either install the elasticsearch on a single host or deploy a cluster of elasticsearch nodes.

Setup Android SDK

You can download Android bundle here. If you want to use the Hooker install script, you have to:

Make sure to set your ANDROID_HOME environment variable: $ export ANDROID_HOME=/path/to/your/sdk/folder
Download SDK APIs from your SDK manager.

Installation

An install script is provided to help you build and install all necessary dependances. If you want to use this script, make sure you have the following dependances:

# openjdk-7-jdk, ant, python-setuptools (just apt install them)

When you are all set, run install script in the Hooker root directory:

$ ./install.sh

Build your reference Android Virtual Device (AVD)

Check that you have available targets: $ $ANDROID_HOME/tools/android list target.
Launch the automatic script for an easier installation: cd tools/emulatorCreator && python HookerInstaller.py -s SDK_PATH -a Hooker_test -t ANDROID_TARGET -d AVD_DIRECTORY,
When python logs tell you so:
- Open SuperSU app, click on "Continue" to update SU binary, choose the "Normal" installation mode, wait a bit. Click on "OK" (NOT "Reboot"!) and exit the application.
- Open Substrate app, click "Link Substrate Files", allow Substrate, and reclick again on "Link Substrate Files".
- Install APK-instrumenter APK with ADB.
- Click on "Restart System (Soft)" when the Substrate application pop up.
- Wait for the system to restart and disable the lockscreen security: Menu > System Settings > Security > Screen lock > None
- Close your emulator.
If you don't want to use the automatic script, you'll have to remember that:
- Hooker needs an SD card to work properly,
- Hooker needs to have snapshot enable. Careful if you use android-studio to create your AVD: there is a bug (or feature, dunno) which makes it difficult to use snapshots...

For your interest, you can checkout a video of how to prepare an emulator here

Configure the host where Hooker is executed

If your elasticsearch host is on a different host than your android emulator, you will need to redirect traffic throw network. In order to do this, you can use socat:

$ socat -s -v TCP4-LISTEN:9200,fork,ignoreeof,reuseaddr TCP4:192.168.98.11:9200,ignoreeof

If you have an error concerning OpenGLES emulation (Could not load OpenGLES emulation library), you have to edit your ldconfig (as root):

# echo "/path/to/your/sdk/tools/lib" > /etc/ld.so.conf.d/android.conf
# ldconfig

Play HOOKER

Checkout this video to watch a demo on how to run a manual experiment to analyse one specific application.

Playing with real devices

If you want to use Hooker on real devices, please read first the specific README.

Setup your configuration file

If you want to make a manual analysis, copy file hooker_xp/sampleManualAnalysis.conf,
If you want to make an automatic analysis, copy file hooker_xp/sampleAutomaticAnalysis.conf,
If you want to make an analysis on real devices, copy one of the *RealDevice* configuration files,
Depending on your system configuration, customize the different parameters declared in retained configuration file. Sample configuration files are verbose++, so please read comments,
In relation with previous steps, you need to specify the path to your reference AVD you just built. As the comments explain it, just put the path + name of AVD, i.e. without the .avd extension.

Start ElasticSearch and Kibana

Hooker uses ElasticSearch to store events and Kibana as a frontend to analyse theses. In order to help you analyze applications, we've pushed a Kibana dashboard example in the directory tools/kibana-dashboard. In order to use it, you'll need to run a first experiment and then import the filekibana-export.json. To import a dashboard, you have to go to the URL http://localhost:5601 and:

Go to Settings -> Objects,
Click on Import,
Select the available dashboard.

Run your Experiment

Python experiment script is in hooker_xp directory:

$ cd hooker_xp && python hooker_xp.py -c yourAnalysisConfigurationFile.conf

You should have python logs explaining you what is going on.

Contributing

We would be delighted if you could help us improve this work. Please use github features to provide your bugfixes and improvements.

Authors and Sponsors

The Hooker project has been initiated by Georges Bossert and Dimitri Kirchner. Both work for AMOSSYS, a French IT security company http://www.amossys.fr.

License

This software is licensed under the GPLv3 License. See the LICENSE file in the top distribution directory for the full license text.

hooker's People

Contributors

Stargazers

Watchers

Forkers

gongsunjinqian crudbug misoag test123asa bincker terry2012 winsun tempbottle chubbymaggie vijayendra-g melbshark bkerler askk jeetak47 thelongestusername 44kksharma vaginessa minakov jianjia codecrack3 tamerplatform mallocnode jokerni moonight10 avalz killvxk kernux gaurav9991 majestyhao wangying1357 greenplusdata shuixi2013 nkafei android-sec bigfool k5h4tr1y4 unvsoft rubyzy futurelighthouse iaskwho linux-embedded islandone qiyeboy zekn jackywei1228 gfxox3jj3cqr0rmy 309746069 hhhaiai liyungui apcwowo iamckn-mobile hackertool droid-sec ylyking csky6688 bjzz mdeous cumtqiao drizzlerisk yege0201 timzone 100325128 wushenwu tangsilian jirufeng pangpangcc jijicanyu idkwim xmlxin unforeseenocean olivierh59500 kernullist secpersu bensonchen nexthacker panaki2601 primekun gwef h0ne coderbert zhanglang001 chago gjbae lhoo gg-coder889 focuspadding wyd20162016 0x7678 shantanu561993 cuiopen balrampandey19 shang159 alirezachegini st-rnd zsdlove netstu hohjgdhsj ohsorry shlomiag1 ohyeah521

hooker's Issues

Get Permissions list

Using Android Hooker, Is it possible to get requested permissions list in one file?

how to see result log saved data in Elasticsearch Cluster?

After successfully run hooker_xp.py, I got "The event information of experiment '99ab8cf3c7e126b7600931860dfe16f9' has successfuly been inserted into ES cluster." message.

Now, how can I see these cluster? Does t creates any log file?

Result Analysis

For some reasons actual data is not getting stored in my database,only their datatype are getting stored up on the elastic search database,even though the dynamic analysis ran successfully.
(Also what's up with the events.logs file ,where is it stored ? )

Can I somehow get the result of the analysis in a .txt file or is there some other way to extract the information out of the dynamic analysis done by the tool ?

Thanks in Advance :)

In SharedPreferencesHooker.java, " android.content.SharedPreferences.Editor" not works!

In SharedPreferencesHooker.java, it was ussless to hook android.content.SharedPreferences.Editor. Because Editor is interface, which can not be hooked.
So we shoud hook it's implementation, which is android.app.SharedPreferencesImpl$EditorImpl, and it works!

parameter type "[B" out of memory

In "Hooker.java, getStringRepresentationOfAttribute(Object arg)" . If the parameter type of arg is "[B", the length arg array may too big ,which may cause out of memory. When I print the logcat, the element of "[B" is not printed completed, so I think it is out of memory. I think we should check the type of arg when it is "[B" type, which may contains too many elements.

Error happened when the "experiment" inserting into ES cluster

Hello:
when I run python hooker_xp.py -c myAnalysis.conf, the Error happened as follow :

[MainProcess/Reporter.py:69/INFO] 101: No APK found associated with IDXP 4ca6b8a3e75b46c7b3acb747756a5792
[MainProcess/Es.py:359/DEBUG] 111: The APK '/home/ximi/DroidBox_4.1.1/MaliciousTest_For4.0.apk' has successfuly been inserted into ES cluster.
[MainProcess/Reporter.py:74/INFO] 112: APK /home/ximi/DroidBox_4.1.1/MaliciousTest_For4.0.apk with ID e5777ad1-f8d3-44d9-a3ef-2294c5a5e409 has been assiocated to IDXP in ES cluster
[MainProcess/Es.py:113/DEBUG] 122: The experiment '4ca6b8a3e75b46c7b3acb747756a5792' has successfuly been inserted into ES cluster.

--------------------------------------Error at here--------------------------------------

[MainProcess/hooker_xp.py:70/ERROR] 124: Traceback (most recent call last):
File "hooker_xp.py", line 67, in
main()
File "hooker_xp.py", line 61, in main
analysis.start()
File "/home/ximi/Dynamic_Analysis_of_APP/hooker-master/hooker_xp/hooker_xp/ManualAnalysis.py", line 83, in start
Analysis.reportEvent(self.reporter, idXp, "Analysis", "Executing static analysis on {0}".format(analyzedAPKFile))
File "/home/ximi/Dynamic_Analysis_of_APP/hooker-master/hooker_xp/hooker_xp/analysis/Analysis.py", line 85, in reportEvent
reporter.reportEvent(idXp, sourceEvent, actionEvent, paramsEvent)
File "/home/ximi/Dynamic_Analysis_of_APP/hooker-master/hooker_xp/hooker_xp/report/Reporter.py", line 92, in reportEvent
self.es.insertExperimentSteps(idXp, relTime, emulator=sourceEvent, description=actionEvent)
File "build/bdist.linux-x86_64/egg/hooker_common/elasticsearch/Es.py", line 131, in insertExperimentSteps
raise Exception("Experiment with ID '{0}' doesn't exists, create it before inserting a static event.".format(idXp))
Exception: Experiment with ID '4ca6b8a3e75b46c7b3acb747756a5792' doesn't exists, create it before inserting a static event.

As I saw the "Es.py" script, It seems that it can't find the Experiment in the second time.
I wonder if you can help me
Thanks in advance

No reporters may lead to OutOfMemoryErrors

if no reporters are avaialable, intercepted events are never deleted which can lead to an OutOfMemoryExceptions.
This must be fixed.

Elastic search and Kibana get data from which file

@Tibap
I would like to know from which file you get all data into Elasticsearch head and Kibana.
I searched but get binary files.

ERROR: No module named colorama

Reported by @Ratibhan

cd hooker_xp && python hooker_xp.py -c /home/ratibhan/hooker-master/hooker_xp/sampleAutomaticAnalysis.conf

I have completed all steps, but during execution last command given in README.MD, i got following error, Please tell me the way to resolve it?

ERRROR: No module named colorama

Link Substrate Files failed

everytime Iam trying to "Link Substrate Files" in the substrate app Iam getting these error message:

"Shell Status 1: java.lang.SecurityException: Permission Denial: broadcast asks to run as user -1 but is calling from user 0; this requires Android.permission.INTERACT_ACROSS_USERS_FULL or Android.permission.INTERACT_ACROSS_USERS"

any idea how to fix it?

events.logs not found

I would like to know where events.logs file is stored.

RequestError: TransportError elasticsearch

I have followd the step given in readme,

I am getting this error.
My machine is Ubutnu 64bit.
I am using latest hooker version.

Error : -------------------------------------------------------------------------------------------------
[MainProcess/MainThread/DEBUG] 239: Traceback (most recent call last):
File "hooker_xp.py", line 70, in
main()
File "hooker_xp.py", line 53, in main
analysis = ManualAnalysis(commandLineParser)
File "/home/mobisec/Downloads/hooker-master/hooker_xp/hooker_xp/ManualAnalysis.py", line 51, in init
super(ManualAnalysis, self).init(MainConfiguration.build(commandLineParser), ReportingConfiguration.build(commandLineParser), )
File "/home/mobisec/Downloads/hooker-master/hooker_xp/hooker_xp/analysis/Analysis.py", line 53, in init
self.reporter = Reporter(self.reportingConfiguration)
File "/home/mobisec/Downloads/hooker-master/hooker_xp/hooker_xp/report/Reporter.py", line 51, in init
self.initializeReporters()
File "/home/mobisec/Downloads/hooker-master/hooker_xp/hooker_xp/report/Reporter.py", line 110, in __initializeReporters
self.es = Es([{"host":self.reportingConfiguration.elasticsearchIp, 'port':self.reportingConfiguration.elasticsearchPort}])
File "build/bdist.linux-x86_64/egg/hooker_common/elasticsearch/Es.py", line 57, in __init
self.__initializeConnection()
File "build/bdist.linux-x86_64/egg/hooker_common/elasticsearch/Es.py", line 369, in __initializeConnection
self.__createHookerIndex()
File "build/bdist.linux-x86_64/egg/hooker_common/elasticsearch/Es.py", line 389, in __createHookerIndex
self.esInstance.indices.create(index=Es.ES_INDEX_NAME_HOOKER, body=Es.__getBodyIndex())
File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/utils.py", line 69, in _wrapped
return func(_args, params=params, *_kwargs)
File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/indices.py", line 102, in create
params=params, body=body)
File "/usr/local/lib/python2.7/dist-packages/elasticsearch/transport.py", line 307, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.py", line 89, in perform_request
self._raise_error(response.status, raw_data)
File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/base.py", line 105, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
RequestError: TransportError(400, u'mapper_parsing_exception')

androidTemporaryPath=/tmp/android/

Path to androguard framework

androguardPath=/opt/mobisec/Android/androguard

type of device (real or emulated)

device=emulated

Analysis Configuration

[analysis]

type of the analysis (manual or automatic)

type=automatic

name of the analysis

name=Sample Automatic Analysis

directory in which APKs

apks=../tools/sampleApps/

number of emulators

in manual mode, only a single emulator is used

in automatic mode, one APK = one Emulator

maxNumberOfEmulators=2

applications to install and execute before starting the experiment

[application filename MUST equals their main activity name]

prepareApks=../tools/APK-contactGenerator/com.amossys.hooker.generatecontacts.ImportContacts.apk

scenario

IMPORTANT: In your scenario, you cannot have a stimulate action separated from an execute action

scenario=execute,stimulate,reboot,execute,stimulate,externalStimulation

output directory

outputdirectory=/tmp/outdir/

Reporting Configuration

[reporting]

activate or not the reporting of events in an elasticsearch database

elasticsearch_mode=true

if elasticsearch activated, specify the ES IP address

elasticsearch_ip=127.0.0.1

if elasticsearch activated, specify the ES IP port number

elasticsearch_port=9200

elasticsearch index

elasticsearch_index=hooker_test

elasticsearch doctype for events

elasticsearch_doctype=event

activate or not the creation of a text report containing all the captured events

AVD Snapshot Unresponsive/Offline

After creating and prepping the Android AVD, attempting to launch the AVD from the saved snapshot results in a frozen AVD. The home screen loads fine, but the device is seen as ofline and I'm unable to interact with the screen within the AVD.

I've scoured forums and Google for potential solutions, but have found none.

Help to get APK name also in events.logs

Hello,
I'm getting package-name, method-name, and class-name in events.logs file.
But I'm not getting apk-name in it.

Can somebody help me?

Request for help to understand prepareApks in SampleAutomaticAnalysis.conf

After running install.sh which internally calls "ant debug" which generates "ImportContacts-debug.apk"
inside bin folder of "APK-contactGenerator" dir. And sampleAutomaticAnalysis.conf file has "prepareApks=../tools/APK-contactGenerator/com.amossys.hooker.generatecontacts.ImportContacts.apk" variable defined.
#1 : where can i find "com.amossys.hooker.generatecontacts.ImportContacts.apk"?
#2: if user has to define that variable , what should i define? I tried defining " ImportContacts-debug.apk" as "prepareApks" variable but code crashes. Can you please help? Error for #2

Starting: Intent { cmp=/.ImportContacts-debug }
[MainProcess/MainThread/INFO] 88772: Analyzing directory: ../tools/sampleApps/
[MainProcess/MainThread/DEBUG] 88772: All APKs have been pushed to the analyzing queue, sleeping 5 secs...
Error type 3
Error: Activity class {/.ImportContacts-debug} does not exist.
[MainProcess/MainThread/INFO] 93776: Analyzing directory: ../tools/sampleApps/

Error device not found when I try to start AVD

I am getting the following error when i run cd tools/emulatorCreator && python HookerInstaller.py -s SDK_PATH -a Hooker_test -t ANDROID_TARGET -d AVD_DIRECTORY command

my command is :

python HookerInstaller.py -s /home/sharp/Android/Sdk -a Hooker4.2 -t 2 -d /home/sharp/.android/avd

[MainProcess/HookerInstaller.py:289/ERROR] 495: Device Hooker4.2 not found.

Please help

Kibana Dashboard

Hello,
The data collected into ES, for some reason is not loaded into Kibana.
Importing the dashboard (tools/kibana-dashboard/kibana-export.json) as described in the section "Start ElasticSearch and Kibana" always show the message

"Saved Objects: Could not locate that index-pattern (id: hooker_test)"
In the top of Kibana Settings page

May this warning be the problem?

This is just a piece of data seen under ES (http://localhost:9200/_plugin/head/):

{

    "_index": "hooker_test",
    "_type": "static",
    "_id": "AVPIqrs0HrDCae4vI9ZR",
    "_score": 1,
    "_source": {
        "Android Version Code": "57",
        "PackageName": "com.code.spirity",
        "Main Activity": "com.code.spirity.ui.tour.SplashActivity",
        "Libraries": [ ],
        "Filename": "/home/wagner/droid/analysis/new/base.apk",
        "Activities": [
            {
                "Activity": "com.code.spirity.ui.tour.SplashActivity"
            }
            ,
            {
                "Activity": "com.code.spirity.ui.tour.TourActivity"
            }
            ,
...

I mean, it seems that everything is ok with Hooker execution and data collection, the problem is apparently with Kibana's Dashboard.
Do you have a clue on how to solve that?
Thanks in advance.

Broken Pipe Error

During the following steps of installation:

You then need to install application APK-instrumenter on your reference AVD:

Launch your new AVD with: Save to snapshot option checked,
Install the application using adb $ $ANDROID_HOME/platform-tools/adb install APK-instrumenter/bin/ApkInstrumenterActivity-debug.apk

When the application is installed, open substrate app and click Restart System (Soft). You can then close your AVD.

When i click on substrate app the following error appears:
java.io.IOException: write failed: EPIPE (Broken pipe)

Can you plz help me out to avoid this error?

The similar error comes when i trying to run the hooker.
[PoolWorker-3/MainThread/ERROR] 9689369: Cannot stop the AVD.
[PoolWorker-2/MainThread/ERROR] 9689369: Traceback (most recent call last):
File "/home/ahmad/hooker-master/hooker-master/hooker_xp/hooker_xp/AutomaticAnalysis.py", line 74, in executeExperiment
apkToAnalyze = listOfAPKs.get() # 0xFFFF
File "", line 2, in get
File "/home/ahmad/anaconda/lib/python2.7/multiprocessing/managers.py", line 758, in _callmethod
conn.send((self._id, methodname, args, kwds))
IOError: [Errno 32] Broken pipe
[PoolWorker-3/MainThread/ERROR] 9689369: Exception while executing an experiment : [Errno 32] Broken pipe

Loop (and error) when running with real device

Hi,

First, thank you for developing this promising tool and I have been following it for a while, and it is wonderful to see you guys are supporting real device now!

However, I ran into a problem while trying to run it. It first says I don't have APK instrumenter installed, but I do (see last phone screenshot). Then it kept going even after that error, until it stuck in the loop in the end (see Terminal screenshot). I can provide more information as needed, and thank you in advance for any help!

dynamic analysis

I want to make dynamic analysis to detect malware android applications so I want a report from dynamic analysis . Is hooker give me this report for every apk file? if yes , plz send me sample from result report